Ask questionsdind-rootless image needs overlay detection
I've been working on getting docker:dind-rootless working on CentOS 7.7 and ran into an issue because of overlay detection. The docker:dind-rootless image apparently assumes the host OS supports overlay (there's even a TODO) which then causes the image to fail when launching child containers with a "operation not permitted" referencing the overlay mount command.
My workaround is passing --storage-driver=vfs on the docker run but I believe that the image should auto-detect overlay support or have documentation about the above flag.
Answer questions
Caligatio
It's definitely something unique to the image vs an underlying dockerd problem (I think).
Relevant log entries from running ./dockerd-rootless.sh --experimental directly on host (note the use of vfs):
ERRO[2019-10-22T07:53:59.008188371+01:00] 'overlay' not found as a supported filesystem on this host. Please ensure kernel is new enough and has overlay support loaded. storage-driver=overlay2
ERRO[2019-10-22T07:53:59.009369254+01:00] AUFS cannot be used in non-init user namespace storage-driver=aufs
ERRO[2019-10-22T07:53:59.014333070+01:00] 'overlay' not found as a supported filesystem on this host. Please ensure kernel is new enough and has overlay support loaded. storage-driver=overlay
INFO[2019-10-22T07:53:59.032246241+01:00] Loading containers: start.
WARN[2019-10-22T07:53:59.035343935+01:00] Running modprobe bridge br_netfilter failed with message: modprobe: ERROR: could not insert 'br_netfilter': Operation not permitted
insmod /lib/modules/3.10.0-1062.4.1.el7.x86_64/kernel/net/bridge/br_netfilter.ko.xz
, error: exit status 1
INFO[2019-10-22T07:53:59.119941796+01:00] Default bridge (docker0) is assigned with an IP address 172.17.0.0/16. Daemon option --bip can be used to set a preferred IP address
INFO[2019-10-22T07:53:59.155279439+01:00] Loading containers: done.
INFO[2019-10-22T07:53:59.165889078+01:00] Docker daemon commit=9013bf583a graphdriver(s)=vfs version=19.03.4
Relevant log entries from running docker run -ti --privileged docker:dind-rootless --experimental on a rootful docker daemon (note use of overlay):
ERRO[2019-10-22T06:56:41.322548316Z] AUFS cannot be used in non-init user namespace storage-driver=aufs
WARN[2019-10-22T06:56:41.325611279Z] [graphdriver] WARNING: the overlay storage-driver is deprecated, and will be removed in a future release
INFO[2019-10-22T06:56:41.358526463Z] Loading containers: start.
WARN[2019-10-22T06:56:41.390562547Z] Running modprobe bridge br_netfilter failed with message: Device "bridge" does not exist.
bridge 151336 2 br_netfilter,ebtable_broute
stp 12976 1 bridge
llc 14552 2 bridge,stp
Device "br_netfilter" does not exist.
br_netfilter 22256 0
bridge 151336 2 br_netfilter,ebtable_broute
modprobe: can't change directory to '/lib/modules': No such file or directory
, error: exit status 1
INFO[2019-10-22T06:56:41.479545182Z] Default bridge (docker0) is assigned with an IP address 172.17.0.0/16. Daemon option --bip can be used to set a preferred IP address
INFO[2019-10-22T06:56:41.509871117Z] Loading containers: done.
INFO[2019-10-22T06:56:41.537544463Z] Docker daemon commit=9013bf583a graphdriver(s)=overlay version=19.03.4
There are enough moving/experimental parts here that there might be something else going on. Any ideas? I'm happy to experiment.