ChatGPT解决这个技术问题 Extra ChatGPT

Add a volume to Docker, but exclude a sub-folder

Supposed I have a Docker container and a folder on my host /hostFolder. Now if I want to add this folder to the Docker container as a volume, then I can do this either by using ADD in the Dockerfile or mounting it as a volume.

So far, so good.

Now /hostFolder contains a sub-folder, /hostFolder/subFolder.

I want to mount /hostFolder into the Docker container (whether as read-write or read-only does not matter, works both for me), but I do NOT want to have it included /hostFolder/subFolder. I want to exclude this, and I also want the Docker container be able to make changes to this sub-folder, without the consequence of having it changed on the host as well.

Is this possible? If so, how?

It does work via tmpfs: stackoverflow.com/a/71911280/3419751

k
kernix

Using docker-compose I'm able to use node_modules locally, but ignore it in the docker container using the following syntax in the docker-compose.yml

volumes:
   - './angularApp:/opt/app'
   - /opt/app/node_modules/

So everything in ./angularApp is mapped to /opt/app and then I create another mount volume /opt/app/node_modules/ which is now empty directory - even if in my local machine ./angularApp/node_modules is not empty.


It works but i cannot remove these dirs inside container For example: i need to replace /opt/app/node_modules/ with another directory with same name. Error happens: 'volume busy'
Please note that order is important: if your image had files under /opt/app/node_modules/ they will be ignored if you mount it after the parent directory. For exemple, if you want to include modules in the image, but be able to refresh it later, you would list volumes in this order: - /opt/app/node_modules/ then the parent - './angularApp:/opt/app'
Don't forget the trailing slash! eg. /opt/app/node_modules doesn't work, it will be overwritten by ./angularApp/node_modules
thanks! You saved my day! In case of using simple docker, not compos it would look like: -v $(pwd):/build/ -v /build/node_modules
I found the document, docs.docker.com/storage/volumes/….If you start a container which creates a new volume, as above, and the container has files or directories in the directory to be mounted (such as /app/ above), the directory’s contents are copied into the volume.
N
Nate T

If you want to have subdirectories ignored by docker-compose but persistent, you can do the following in docker-compose.yml:

volumes:
  node_modules:
services:
  server:
    volumes:
      - .:/app
      - node_modules:/app/node_modules

This will mount your current directory as a shared volume, but mount a persistent docker volume in place of your local node_modules directory. This is similar to the answer by @kernix, but this will allow node_modules to persist between docker-compose up runs, which is likely the desired behavior.


Named volumes are a superior approach. Protip: make sure you aren't using a hyphenated volume name. You will spend a small portion of your life in a debug nightmare, only to find out that you were once again fooled by a ridiculous nuance.
"Note that docker-compose down will kill these persistent volumes." This is not true in Docker for Mac v 17.0.5+ (and possibly older versions). docker-compose down will remove the containers, but the volume will persist until you run something like docker system prune
In case someone is interestead to understand why using the node_modules: named volume "ignores" the /app/node_modules in the docker container, may find this post usefull How docker handles multiple mount types?
I don't know whether I'm doing something wrong, but if I do this, start the container and then run npm install on the HOST machine, it changes the node_modules inside the CONTAINER too. I can't imagine why anyone would want to do that, but still I'm looking for a cleaner separation.
@Renra had the exact same issue, my host machine would constantly write over node_modules even though I had the named volume... caused all sorts of incompatibility issues. Gave up on fixing it and worked around it by only copying my src folder instead of the whole repo
s
squid_ink

For those trying to get a nice workflow going where node_modules isn't overridden by local this might help.

Change your docker-compose to mount an anonymous persistent volume to node_modules to prevent your local overriding it. This has been outlined in this thread a few times.

services:
  server:
    build: .
    volumes:
      - .:/app
      - /app/node_modules

This is the important bit we were missing. When spinning up your stack use docker-compose -V. Without this if you added a new package and rebuilt your image it would be using the node_modules from your initial docker-compose launch.

    -V, --renew-anon-volumes   Recreate anonymous volumes instead of retrieving
                               data from the previous containers.

The note about --renew-anon-volumes is an important addition. There are cases where you want to mount your source code folder and exclude the node_modules folder by mounting it as an anonymous volume. The flag avoids the re-using the previously created anonymous volume so that changes to package.json will reflect correctly.
When using this approach, an empty node_modules directory is created on the host. How can we avoid that?
F
Frank Wong

To exclude a file, use the following

volumes:
   - /hostFolder:/folder
   - /dev/null:/folder/fileToBeExcluded

Perfect solution for what I needed (to NOT map node_modules up to the host because symlink stuff broke on a Windows host). Pointing to /dev/null did not work, but creating a blank directory and mapping it to that worked perfectly and solved the issues I was having - thanks for this idea.
This doesn't work in Docker for Mac 18.09.0: Cannot start service xxx: OCI runtime create failed: container_linux.go:348: starting container process caused "process_linux.go:402: container init caused \"rootfs_linux.go:58: mounting \\\"/dev/null\\\" to rootfs \\\"/var/lib/docker/overlay2/d15ed56ad020d408c63a1f6a6365dbb88d5d3b78a4605980d3faa9861102ef21/merged\\\" at [...] unknown: Are you trying to mount a directory onto a file (or vice-versa)? Check if the specified host path exists and is the expected type. Adding a trailing slash didn't help.
I can confirm that it does not work either on Ubuntu 16.04 with Docker 18.09.3.
That doesn't work for me. All it does is copy /dev/null into the container, as a device file.
Please note this works only for excluding FILES! /dev/null is a file and so should be the target, the error message is quite clear on this: Are you trying to mount a directory onto a file (or vice-versa)? Check if the specified host path exists and is the expected type. This is not exactly what the OP was looking for, but it was very useful for me, since I needed to exclude a single file.
D
DS.

With the docker command line:

docker run \
    --mount type=bind,src=/hostFolder,dst=/containerFolder \
    --mount type=volume,dst=/containerFolder/subFolder \
    ...other-args...

The -v option may also be used (credit to Bogdan Mart), but --mount is clearer and recommended.


A
Adrian Mouat

First, using the ADD instruction in a Dockerfile is very different from using a volume (either via the -v argument to docker run or the VOLUME instruction in a Dockerfile). The ADD and COPY commands just take a copy of the files at the time docker build is run. These files are not updated until a fresh image is created with the docker build command. By contrast, using a volume is essentially saying "this directory should not be stored in the container image; instead use a directory on the host"; whenever a file inside a volume is changed, both the host and container will see it immediately.

I don't believe you can achieve what you want using volumes, you'll have to rethink your directory structure if you want to do this.

However, it's quite simple to achieve using COPY (which should be preferred to ADD). You can either use a .dockerignore file to exclude the subdirectory, or you could COPY all the files then do a RUN rm bla to remove the subdirectory.

Remember that any files you add to image with COPY or ADD must be inside the build context i.e. in or below the directory you run docker build from.


learn something new every day :-) the COPY and ADD work from the build context (not the host). Would it be possible to use a volume to mask the subFolder? The OP wants to mount the /hostFolder (I think), and make /hostFolder/subFolder disconnected. If there was a VOLUME for each, but only the hostFolder was 'mounted' (-v), would that isolate the subFolder changes to the container?
I'm not sure, I've never tried nesting volumes. I'm not convinced it's a good idea, but I'd have to look into it.
I can't seem to find an example anywhere of what to put in a .dockerignore file to exclude a sub directory (or even a directory). Given that is the stated purpose of the file, it's quite odd.
once you add any files, they still end up in some layers. So a remove of them will not help you hide them if security sensitive or reduce image size (because they will be in the underlying layers). There are options to squash the layers. One is github.com/goldmann/docker-squash and another is using oc ex dockerbuild from the openshift project.
D
Daenor

for the people who also had the issue that the node_modules folder would still overwrite from your local system and the other way around

volumes:
  node_modules:
services:
  server:
    volumes:
      - .:/app
      - node_modules:/app/node_modules/

This is the solution, With the trailing / after the node_modules being the fix.


So this just maps your local node_modules folder to be inside /app/node_modules folder in the container? So inside the container, we have /app/node_modules/node_modules? Why?
h
holdbar

Looks like the old solution doesn't work anymore(at least for me). Creating an empty folder and mapping target folder to it helped though.

volumes:
   - ./angularApp:/opt/app
   - .empty:/opt/app/node_modules/

Doesn't work for me either with docker-compose 1.25.0-rc3, file syntax v3.
E
Emmario Delar

To exclude a mounted file contained in the volume of your machine, you will have to overwrite it by allocating a volume to this same file. In your config file:

services:
  server:
    build : ./Dockerfile
    volumes:
      - .:/app

An example in you dockerfile:

# Image Location
FROM node:13.12.0-buster
VOLUME /app/you_overwrite_file