/test pull-cri-containerd-node-e2e

/test pull-cri-containerd-node-e2e

Yes, seems mistake. Opened a PR to revert this: https://github.com/containerd/cri/pull/1584

Reverts containerd/cri#1530 , seems merged by mistake

@giuseppe PTAL?

Previously, {"type":"bind"} without {"options": ["bind"]} was failing with ENODEV.

See https://github.com/containers/podman/issues/7652

Is there a reason that the config generator is not adding the bind option to mounts? a hypothetical filesystem called bind couldn't be mounted with this change.

This sounds too much hypothetical

Dear runc community,

I am new to runc/containers and I am getting the error below while trying to buld an image. I am building the image using buildkit inside a container managed by k8s. I already posted this issue in buildkit slack channel and they refered me to runc.

This is the Dockerfile I am using:

FROM python:3

RUN pip install pytest

And this is the error I am getting:

# /tmp/bin/buildctl build --frontend dockerfile.v0 --local context=. --local dockerfile=. --output type=image,name=masber/hello-world:{{ gitresult.after }},push=true
[+] Building 5.2s (4/5)                                                                                                                                                                                    
 => [internal] load build definition from Dockerfile                                                                                                                                                  0.2s
 => => transferring dockerfile: 75B                                                                                                                                                                   0.0s
 => [internal] load .dockerignore                                                                                                                                                                     0.3s
 => => transferring context: 2B                                                                                                                                                                       0.0s
 => [internal] load metadata for docker.io/library/python:3                                                                                                                                           1.9s
 => CACHED [1/2] FROM docker.io/library/python:3@sha256:e9b7e3b4e9569808066c5901b8a9ad315a9f14ae8d3949ece22ae339fff2cad0                                                                              0.0s
 => => resolve docker.io/library/python:3@sha256:e9b7e3b4e9569808066c5901b8a9ad315a9f14ae8d3949ece22ae339fff2cad0                                                                                     0.0s
 => [2/2] RUN pip install pytest                                                                                                                                                                      2.9s
ERRO[3425] /moby.buildkit.v1.Control/Solve returned error: runc did not terminate successfully
executor failed running [/bin/sh -c pip install pytest]
github.com/moby/buildkit/solver/llbsolver/ops.(*execOp).Exec
	/src/solver/llbsolver/ops/exec.go:732
github.com/moby/buildkit/solver.(*sharedOp).Exec.func1
	/src/solver/jobs.go:688
github.com/moby/buildkit/util/flightcontrol.(*call).run
	/src/util/flightcontrol/flightcontrol.go:121
sync.(*Once).doSlow
	/usr/local/go/src/sync/once.go:66
sync.(*Once).Do
[+] Building 5.2s (5/5) FINISHED                                                                                                                                                                           
 => [internal] load build definition from Dockerfile                                                                                                                                                  0.2s
 => => transferring dockerfile: 75B                                                                                                                                                                   0.0s
 => [internal] load .dockerignore                                                                                                                                                                     0.3s
 => => transferring context: 2B                                                                                                                                                                       0.0s
 => [internal] load metadata for docker.io/library/python:3                                                                                                                                           1.9s
 => CACHED [1/2] FROM docker.io/library/python:3@sha256:e9b7e3b4e9569808066c5901b8a9ad315a9f14ae8d3949ece22ae339fff2cad0                                                                              0.0s
 => => resolve docker.io/library/python:3@sha256:e9b7e3b4e9569808066c5901b8a9ad315a9f14ae8d3949ece22ae339fff2cad0                                                                                     0.0s
 => ERROR [2/2] RUN pip install pytest                                                                                                                                                                2.9s
------
 > [2/2] RUN pip install pytest:
#5 2.861 No help topic for 'run'
------
error: failed to solve: rpc error: code = Unknown desc = executor failed running [/bin/sh -c pip install pytest]: runc did not terminate successfully

I am hoping someone could help me to understand why of this error, what it means and how can I debug further in order to fix it or work around it.

thank you very much

duplicate: https://github.com/moby/buildkit/issues/1690

@thaJeztah @StefanScherer @tiborvass PTAL

Description

Apparently systemd_cgroup = true in plugins."io.containerd.grpc.v1.cri" is no longer supported by containerd 1.3.7. Enabling it causes containerd to skip loading the cri plugin:

containerd[17742]: time="2020-09-18T10:46:45.630684350Z" level=warning msg="failed to load plugin io.containerd.grpc.v1.cri" error="invalid plugin config: `systemd_cgroup` only works for runtime io.containerd.runtime.v1.linux"

Steps to reproduce the issue:

1. containerd config default > /etc/containerd/config.toml
2. sed -i -e 's/systemd_cgroup = false/systemd_cgroup = true/' /etc/containerd/config.toml
3. start containerd

Describe the results you received:

It starts, but fails to load the cri plugin:

containerd[17742]: time="2020-09-18T10:46:45.630684350Z" level=warning msg="failed to load plugin io.containerd.grpc.v1.cri" error="invalid plugin config: `systemd_cgroup` only works for runtime io.containerd.runtime.v1.linux"

This in turn makes kubeadm unhappy:

error execution phase preflight: [preflight] Some fatal errors occurred:
[ERROR CRI]: container runtime is not running: output: time="2020-09-18T10:47:07Z" level=fatal msg="getting status of runtime failed: rpc error: code = Unimplemented desc = unknown service runtime.v1alpha2.RuntimeService"

Describe the results you expected:

If systemd_cgroup = true is no longer valid, then systemd_cgroup = false should not be printed in the containerd config default output. Showing a version there implies that it can be edited.

I think I understand where this is coming from (straight dump of the in-memory struct), but it's still not nice.

Output of containerd --version:

containerd containerd.io 1.3.7 8fba4e9a7d01810a393d5d25a3621dc101981175

systemd_cgroup is for io.containerd.runtime.v1.linux runtime, which is deprecated but still supported.

To enable systemd cgroup for the current default runtime io.containerd.runc.v2, please see https://github.com/containerd/containerd/issues/4203#issuecomment-651532765

Probably your runc is too old, make sure to have v1.0.0-rc10 or later at least. (rc92 is recommended)

@giuseppe Have you seen this?

Describe the bug

passing stdin to az container exec fails with ENOTTY (termios.error: (25, 'Inappropriate ioctl for device'))

Command Name az container exec

Errors:

(25, 'Inappropriate ioctl for device')
Traceback (most recent call last):
python3.8/site-packages/knack/cli.py, ln 215, in invoke
    cmd_result = self.invocation.execute(args)
cli/core/commands/__init__.py, ln 654, in execute
    raise ex
cli/core/commands/__init__.py, ln 718, in _run_jobs_serially
    results.append(self._run_job(expanded_arg, cmd_copy))
cli/core/commands/__init__.py, ln 711, in _run_job
    six.reraise(*sys.exc_info())
...
cli/command_modules/container/custom.py, ln 623, in container_exec
    _start_exec_pipe(execContainerResponse.web_socket_uri, execContainerResponse.password)
cli/command_modules/container/custom.py, ln 657, in _start_exec_pipe
    oldtty = termios.tcgetattr(sys.stdin)
termios.error: (25, 'Inappropriate ioctl for device')

To Reproduce:

echo "echo foo" | az container exec -g $MY_RESOURCE_GROUP --name $MY_DEBIAN_CONTAINER --exec-command "/bin/bash"

Expected Behavior

The stdin content should be passed to the process, and "foo" should be printed.

Environment Summary

macOS-10.15.6-x86_64-i386-64bit
Python 3.8.5
Installer: HOMEBREW

azure-cli 2.11.1

Additional Context

https://github.com/Azure/azure-cli/issues/13352 ("az container exec in azure devops throws (25, 'Inappropriate ioctl for device')") has the same error code but reproduction steps are different.

<!--Please don't remove this:--> <!--auto-generated-->

CentOS 7 requires running echo “user.max_user_namespaces=10000” > /etc/sysctl.d/42-rootless.conf and sysctl --system as root

@thaJeztah Graph driver PR has been here, but we haven't yet reached consensus on how to integrate containerd storage.

https://github.com/moby/moby/pull/41002

Updated the image, thanks @vrothberg

Contribution is wanted.

A workaround is like this, but this is insecure because it needs --privileged

$ docker run -it --rm -v /somewhere/rootfs:/mnt -w /mnt --privileged alpine chroot . sh -ec "mount -t proc none /proc && exec sh"

Sorry I don't have

Description runsc flags panics with "panic: invalid ref leak mode '\x00'"

Steps to reproduce

$ runsc flags
  -TESTONLY-test-name-env string
        TEST ONLY; do not ever use! Used for automated tests to improve logging.
  -TESTONLY-unsafe-nonroot
        TEST ONLY; do not ever use! This skips many security measures that isolate the host from the sandbox.
  -allow-flag-override
        allow OCI annotations (dev.gvisor.flag.<name>) to override flags for debugging.
  -alsologtostderr
        send log messages to stderr.
  -cpu-num-from-quota
        set cpu number to cpu quota (least integer greater or equal to quota value, but not less than 2)
  -debug
        enable debug logging.
  -debug-log string
        additional location for logs. If it ends with '/', log files are created inside the directory with default names. The following variables are available: %TIMESTAMP%, %COMMAND%.
  -debug-log-fd int
        file descriptor to write debug logs to.  If set, the 'debug-log-dir' flag is ignored. (default -1)
  -debug-log-format string
        log format: text (default), json, or json-k8s. (default "text")
  -file-access value
        specifies which filesystem to use for the root mount: exclusive (default), shared. Volume mounts are always shared.
  -fsgofer-host-uds
        allow the gofer to mount Unix Domain Sockets.
  -fuse
        TEST ONLY; use while FUSE in VFSv2 is landing. This allows the use of the new experimental FUSE filesystem.
  -gso
        enable hardware segmentation offload if it is supported by a network device. (default true)
  -log string
        file path where internal debug information is written, default is stdout.
  -log-fd int
        file descriptor to log to.  If set, the 'log' flag is ignored. (default -1)
  -log-format string
        log format: text (default), json, or json-k8s. (default "text")
  -log-packets
        enable network packet logging.
  -net-raw runsc exec
        enable raw sockets. When false, raw sockets are disabled by removing CAP_NET_RAW from containers (runsc exec will still be able to utilize raw sockets). Raw sockets allow malicious containers to craft packets and potentially attack the network.
  -network value
        specifies which network to use: sandbox (default), host, none. Using network inside the sandbox is more secure because it's isolated from the host network.
  -num-network-channels int
        number of underlying channels(FDs) to use for network link endpoints. (default 1)
  -oci-seccomp
        Enables loading OCI seccomp filters inside the sandbox.
  -overlay
        wrap filesystem mounts with writable overlay. All modifications are stored in memory inside the sandbox.
  -overlayfs-stale-read
        assume root mount is an overlay filesystem (default true)
  -panic-log string
        file path were panic reports and other Go's runtime messages are written.
  -panic-log-fd int
        file descriptor to write Go's runtime messages. (default -1)
  -panic-signal int
        register signal handling that panics. Usually set to SIGUSR2(12) to troubleshoot hangs. -1 disables it. (default -1)
  -platform string
        specifies which platform to use: ptrace (default), kvm. (default "ptrace")
  -profile
        prepares the sandbox to use Golang profiler. Note that enabling profiler loosens the seccomp protection added to the sandbox (DO NOT USE IN PRODUCTION).
  -qdisc value
        specifies which queueing discipline to apply by default to the non loopback nics used by the sandbox. (default fifo)
panic: invalid ref leak mode '\x00'

goroutine 1 [running]:
gvisor.dev/gvisor/pkg/refs.(*LeakMode).String(0xc000280580, 0xf5d6e0, 0xc000280580)
        pkg/refs/refcounter.go:267 +0x121
flag.isZeroValue(0xc0000c59c0, 0x10b62ea, 0x8, 0x10e8551)
        GOROOT/src/flag/flag.go:458 +0x104
flag.(*FlagSet).PrintDefaults.func1(0xc0000c59c0)
        GOROOT/src/flag/flag.go:521 +0x20b
flag.(*FlagSet).VisitAll(0xc0000c2120, 0xc00009bc68)
        GOROOT/src/flag/flag.go:388 +0x61
flag.(*FlagSet).PrintDefaults(0xc0000c2120)
        GOROOT/src/flag/flag.go:504 +0x4e
github.com/google/subcommands.(*flagger).Execute(0xc0000c6000, 0x127e0a0, 0xc0000b0000, 0xc000193800, 0xc00026dce0, 0x2, 0x2, 0x7fdafbcd47d0)
        external/com_github_google_subcommands/subcommands.go:367 +0x333
github.com/google/subcommands.(*Commander).Execute(0xc0000c6000, 0x127e0a0, 0xc0000b0000, 0xc00026dce0, 0x2, 0x2, 0x0)
        external/com_github_google_subcommands/subcommands.go:200 +0x2f9
github.com/google/subcommands.Execute(...)
        external/com_github_google_subcommands/subcommands.go:481
main.main()
        runsc/main.go:230 +0x13ca
```

**Environment**

*   `runsc -version`
```console
runsc version release-20200907.0-157-gca3087472050
spec: 1.0.1-dev
```
*   `git describe` (if built from source)
```console
release-20200907.0-157-gca3087472
```

Description

Loopback address should be 127.0.0.1/8, not 127.0.0.1/32 .

Steps to reproduce

$ docker run --rm --runtime runsc alpine ip addr show dev lo
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/32 scope global dynamic 

OTOH runc works as expected

$ docker run --rm --runtime runc alpine ip addr show dev lo
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever

Environment

Please include the following details of your environment:

• runsc -version

$ runsc --version
runsc version release-20200907.0-157-gca3087472050
spec: 1.0.1-dev

• docker version or docker info (if available)

$ docker version
Client:
 Version:           20.03.0-dev
 API version:       1.41
 Go version:        go1.13.15
 Git commit:        e0eba83bd
 Built:             Wed Sep 16 10:08:53 2020
 OS/Arch:           linux/amd64
 Context:           default
 Experimental:      true

Server:
 Engine:
  Version:          dev
  API version:      1.41 (minimum version 1.12)
  Go version:       go1.13.15
  Git commit:       f99814d749
  Built:            Wed Sep 16 10:07:13 2020
  OS/Arch:          linux/amd64
  Experimental:     true
 containerd:
  Version:          v1.4.0-81-g373cbc2a
  GitCommit:        373cbc2a7f5469b2a833660ba2f474cf4f947d32
 runc:
  Version:          1.0.0-rc92+dev
  GitCommit:        892477ca26638b214dc79150a09ba3565b93137d
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0

• uname -a

• git describe (if built from source)

Connecting to ACI and Kubernetes could be easily supported using az container exec and kubectl exec. -v can be implemented using sshfs (without ssh), -p can be implemented using socat.

Google Cloud Run cannot be supported probably, as it lacks shell access

https://github.com/AkihiroSuda/sshocker/blob/3144baeef5209e6cb6428a0cd48aa8fb26a2218e/pkg/reversesshfs/reversesshfs.go#L74-L81

We should use https://pkg.go.dev/github.com/pkg/[email protected]#NewRequestServer to prevent the remote host from accessing unexpected local directories (if sshfs had a bug)

Fix #4

Currently a shell can start before the mounts are ready

Fix #4

Currently a shell can start before the mounts are ready

It might be useful if multiple remote hosts can be connected to an ad-hoc VPN like this:

(terminal1)$ sshocker network create foo
(terminal1)$ sshocker run --network foo --ip 192.168.42.1011/24 host1
(terminal2)$ sshocker run --network foo --ip 192.168.42.102/24 host2

This could be easily implemented using vdeplug4, though it is not apt-gettable.

Unlike github.com/pkg/sftp, SFTPGo is GPLv3-licensed and hard to combine with other libraries 😞

What's current status?

https://github.com/containerd/containerd/pull/4505 introduced a configuration knob for plugins.overlayfs.root_path.

We should do this for all the plugins.

• [x] aufs: https://github.com/containerd/containerd/pull/4533
• [x] btrfs
• [X] devicemapper: https://github.com/containerd/containerd/pull/3022
• [x] native
• [X] overlay: https://github.com/containerd/containerd/pull/4505
• [x] zfs: https://github.com/containerd/containerd/pull/4534

Thanks

@kolyshkin ptal

PR: https://github.com/containers/podman/pull/7698

@vrothberg Could you push the image to quay?

Fix #7618

This bumps up ROOTLESS_CNI_INFRA_VERSION to 2

This happen when lo interface inside the container isn't brought up for some reason. So rootlesskit and slirp4netns are probably innocent :)

@vrothberg @giuseppe @rhatdan PTAL?

Seems hanging here: https://github.com/containers/podman/blob/852943516606f32ccc2406f41bcf3df42d7c622c/vendor/github.com/rootless-containers/rootlesskit/pkg/port/builtin/parent/tcp/tcp.go#L58

Hi. Cross-posting the bug, which is tracked during build progress for cri-o :

github.com/containerd/continuity/fs mkdir -p $WORK/b407/ cd $WORK /usr/local/bin/llvm-goc -fgo-importcfg=/dev/null -c -x c - -o /dev/null || true mkdir -p $WORK/b407/importcfgroot/github.com/containerd/continuity ln -s /home/oceanfish81/.cache/go-build/01/0149872ebed362b2db5d5cb500fe4c36dc53aa0eeaf43445d2a1d57f87797f22-d $WORK/b407/importcfgroot/github.com/containerd/continuity/libsysx.a mkdir -p $WORK/b407/importcfgroot/github.com/pkg ln -s /home/oceanfish81/.cache/go-build/5c/5c5c0c7d8f5175eae2b4d2a083bbfa586296699b6d6e847a6034fb77f0283e5b-d $WORK/b407/importcfgroot/github.com/pkg/liberrors.a mkdir -p $WORK/b407/importcfgroot/github.com/sirupsen ln -s /home/oceanfish81/.cache/go-build/92/92e22833f5517c1584404320f391ae330ef4ec12a7ee701ea0863735c4e5a39e-d $WORK/b407/importcfgroot/github.com/sirupsen/liblogrus.a mkdir -p $WORK/b407/importcfgroot/golang.org/x/sync ln -s /home/oceanfish81/.cache/go-build/b4/b4dade4823a139177beb6326f044ad405152a51915db5eff2d1f1100f377d538-d $WORK/b407/importcfgroot/golang.org/x/sync/liberrgroup.a mkdir -p $WORK/b407/importcfgroot/golang.org/x/sys ln -s /home/oceanfish81/.cache/go-build/79/79581a3b9031e51313108a803ec4be9cd5e789dcf66ef4dbe3af40cfbd47664a-d $WORK/b407/importcfgroot/golang.org/x/sys/libunix.a cd /home/oceanfish81/cri-o/vendor/github.com/containerd/continuity/fs /usr/local/bin/llvm-goc -c -O2 -g -m64 -fdebug-prefix-map=$WORK=/tmp/go-build -gno-record-gcc-switches -fgo-pkgpath=github.com/containerd/continuity/fs -o $WORK/b407/go.o -I $WORK/b407/importcfgroot ./copy.go ./copy_linux.go ./diff.go ./diff_unix.go ./dtype_linux.go ./du.go ./du_unix.go ./hardlink.go ./hardlink_unix.go ./path.go ./stat_linuxopenbsd.go ./time.go github.com/containerd/continuity/fs vendor/github.com/containerd/continuity/fs/dtype_linux.go:61:26:

error: reference to undefined identifier 'syscall.DT_UNKNOWN'

Is it possible to rule out the cause of this, by expressing the algorithm without the usage of a pre-compiled constant (to calculate in another way) ?

Ivan

Can we update the README to clarify the current minimum kernel requirement? 3.10? (Does it work without Red Hat patches?)

Shouldn't be required even for rootful, I guess

Thanks for reporting, what's your distro?

Could you open an issue?

By "this will not work", do you just mean the warning isn't printed, or is something actually non-functional?

done

Please revendor github.com/containerd/aufs

This issue can be closed?

gVisor has non-trivial issues on performance and compatibility, so runtime: gvisor is mostly expected to set only for non-DB containers.

This proposal is for supporting Kata and gVisor, not for supporting GPUs

Please sign and squash commits

Available now https://download.docker.com/linux/centos/8/x86_64/stable/Packages/

<!-- If you are reporting a new issue, make sure that we do not have any duplicates already open. You can ensure this by searching the issue list for this repository. If there is a duplicate, please close your issue and add a comment to the existing issue instead.

If you suspect your issue is a bug, please edit your issue description to include the BUG REPORT INFORMATION shown below. If you fail to provide this information within 7 days, we cannot debug your issue and will close it. We will, however, reopen it if you later provide the information.

For more information about reporting issues, see https://github.com/moby/moby/blob/master/CONTRIBUTING.md#reporting-other-issues

GENERAL SUPPORT INFORMATION

The GitHub issue tracker is for bug reports and feature requests. General support for docker can be found at the following locations:

• Docker Support Forums - https://forums.docker.com
• Slack - community.docker.com #general channel
• Post a question on StackOverflow, using the Docker tag

General support for moby can be found at the following locations:

• Moby Project Forums - https://forums.mobyproject.org
• Slack - community.docker.com #moby-project channel
• Post a question on StackOverflow, using the Moby tag

BUG REPORT INFORMATION

Use the commands below to provide key information from your environment: You do NOT have to include this information if this is a FEATURE REQUEST -->

Description

<!-- Briefly describe the problem you are having in a few paragraphs. -->

Steps to reproduce the issue: 1. 2. 3.

Describe the results you received:

Describe the results you expected:

Additional information you deem important (e.g. issue happens only occasionally):

Output of docker version:

(paste your output here)

Output of docker info:

(paste your output here)

Additional environment details (AWS, VirtualBox, physical, etc.):

@tiborvass @thaJeztah PTAL

All the current fields in .linux.resources.unified overrides the v1 config (.linux.resources.{memory,memorySwap}).

We should ensure that we can set unified fields that do not override v1.

#144

In containerd, there is a size limit for label size (4096 chars). If an image has many layers, containerd.io/snapshot/remote/stargz.layers will hit the limit of label size and the remote snapshot preparation will fail. This commit fixes this by limiting the size of the label.