profile
viewpoint

Ask questionsRUN --mount=type=cache should inherit ownership/permissions from mountpoint

When using RUN --mount=type=cache, the uid/gid of the mount defaults to 0:0 (root), even if the mountpoint is an existing directory with different permissions.

While it's possible to manually override the uid/gid for the mount, I think it would make sense to default to the permissions of the mountpoint (if exists), and otherwise fall back to the current behaviour (root).

Example / reproduction steps:

# syntax=docker/dockerfile:1.1.3-experimental

FROM busybox

ARG UID=1000
ARG GID=1000

RUN addgroup -g ${GID} foo
RUN adduser -D -u ${UID} -G foo foo
RUN mkdir -p /mycache && chown -R "${UID}:${GID}" /mycache

# this works (as we're root):
RUN mkdir -p /mycache/root-nomount/bar

# this works (we're still root):
RUN --mount=type=cache,target=/mycache \
        mkdir -p /mycache/root-withmount/bar

USER ${UID}:${GID}

# without `--mount`, this works, because `/mycache` was created with
# the correct permissions:
RUN mkdir -p /mycache/foo-nomount/bar

# with `--mount`, this works if we manually set the uid/gid, but this does not
# work well (currently) with dynamically set UID/GID
RUN --mount=type=cache,target=/mycache,uid=1000,gid=1000 \
        mkdir -p /mycache/foo-withmount-manually/bar

# but this fails:
#
# > [stage-0 9/9] RUN --mount=type=cache,target=/mycache         mkdir -p /mycache/foo-withmount/bar:
#15 0.607 mkdir: can't create directory '/mycache/foo-withmount/': Permission denied
RUN --mount=type=cache,target=/mycache \
        mkdir -p /mycache/foo-withmount/bar
DOCKER_BUILDKIT=1 docker build .

[+] Building 11.9s (15/15) FINISHED                                                                                                                                                           
 => [internal] load build definition from Dockerfile                                                                                                                                     0.1s
 => => transferring dockerfile: 1.07kB                                                                                                                                                   0.0s
 => [internal] load .dockerignore                                                                                                                                                        0.1s
 => => transferring context: 2B                                                                                                                                                          0.0s
 => resolve image config for docker.io/docker/dockerfile:1.1.3-experimental                                                                                                              1.4s
 => docker-image://docker.io/docker/dockerfile:1.1.3-experimental@sha256:888f21826273409b5ef5ff9ceb90c64a8f8ec7760da30d1ffbe6c3e2d323a7bd                                                2.8s
 => => resolve docker.io/docker/dockerfile:1.1.3-experimental@sha256:888f21826273409b5ef5ff9ceb90c64a8f8ec7760da30d1ffbe6c3e2d323a7bd                                                    0.0s
 => => sha256:888f21826273409b5ef5ff9ceb90c64a8f8ec7760da30d1ffbe6c3e2d323a7bd 2.03kB / 2.03kB                                                                                           0.0s
 => => sha256:ee85655c57140bd20a5ebc3bb802e7410ee9ac47ca92b193ed0ab17485024fe5 527B / 527B                                                                                               0.0s
 => => sha256:80b5f664ac0c5f6b89608f7b0afd9548ac5b2d4cd631b7b723d9f2edca8676d9 897B / 897B                                                                                               0.0s
 => => sha256:73e11b97eeae87ce062a224a9ff2d7f8a919d3b50e014a3f33a79b0c219a7003 8.82MB / 8.82MB                                                                                           2.3s
 => => extracting sha256:73e11b97eeae87ce062a224a9ff2d7f8a919d3b50e014a3f33a79b0c219a7003                                                                                                0.3s
 => [internal] load metadata for docker.io/library/busybox:latest                                                                                                                        0.0s
 => [stage-0 1/9] FROM docker.io/library/busybox                                                                                                                                         0.0s
 => => resolve docker.io/library/busybox:latest                                                                                                                                          0.0s
 => [internal] settings cache mount permissions                                                                                                                                          0.1s
 => [stage-0 2/9] RUN addgroup -g 1000 foo                                                                                                                                               0.9s
 => [stage-0 3/9] RUN adduser -D -u 1000 -G foo foo                                                                                                                                      0.9s
 => [stage-0 4/9] RUN mkdir -p /mycache && chown -R "1000:1000" /mycache                                                                                                                 0.8s
 => [stage-0 5/9] RUN mkdir -p /mycache/root-nomount/bar                                                                                                                                 1.0s
 => [stage-0 6/9] RUN --mount=type=cache,target=/mycache         mkdir -p /mycache/root-withmount/bar                                                                                    0.8s
 => [stage-0 7/9] RUN mkdir -p /mycache/foo-nomount/bar                                                                                                                                  0.8s
 => [stage-0 8/9] RUN --mount=type=cache,target=/mycache,uid=1000,gid=1000         mkdir -p /mycache/foo-withmount-manually/bar                                                          0.8s
 => ERROR [stage-0 9/9] RUN --mount=type=cache,target=/mycache         mkdir -p /mycache/foo-withmount/bar                                                                               0.7s
------                                                                                                                                                                                        
 > [stage-0 9/9] RUN --mount=type=cache,target=/mycache         mkdir -p /mycache/foo-withmount/bar:
#14 0.543 mkdir: can't create directory '/mycache/foo-withmount/': Permission denied
------
failed to solve with frontend dockerfile.v0: failed to solve with frontend gateway.v0: rpc error: code = Unknown desc = failed to build LLB: executor failed running [/bin/sh -c mkdir -p /mycache/foo-withmount/bar]: runc did not terminate sucessfully
moby/buildkit

Answer questions tonistiigi

This isn't really possible as uid/gid is part of the cache. Therefore it can't be different for same mount appearing multiple times. Another issue is that USER can be username that only makes sense in the relation of /etc/passwd that probably is not on the mount. Looking that up is slow(and actually not possible as all mounts are prepared independently) and again may be different for different contexts.

Related questions

Cannot build from local image with buildctl (OCI Worker) hot 32
php build fails when cache is enabled hot 14
RUN --mount=type=cache causes the whole build context to be loaded in hot 12
Cannot build from local image with buildctl (OCI Worker) hot 10
rootless image didn't work on OKD (OpenShift) 3.11 hot 7
ssh agent only forwarded from keys on Windows hot 7
`httpProxy` support hot 6
Attempt to mount an overlay layer that is already in-use hot 6
Image id shown as missing with docker history hot 6
Image id shown as missing with docker history hot 6
rootless image didn't work on OKD (OpenShift) 3.11 hot 6
Documentation claims that --mount works with 18.06 hot 6
Evaluate running BuildKit with UML
Cannot build from local image with buildctl (OCI Worker)
Documentation claims that --mount works with 18.06 hot 4
Github User Rank List