Ask questions`kind` should be able to start the control plane when `/dev/kmsg` is not available
<!-- Please use this template while reporting a bug and provide as much info as possible. Not doing so may result in your bug not being addressed in a timely manner. Thanks!-->
kind is not able to start the control plane (with the provided node images) when
/dev/kmsg is not available in the node container.
kind then fails with:
✗ Starting control-plane 🕹️ Error: failed to create cluster: failed to init node with kubeadm: exit status 1
Checking on the node, I can see the following error:
# docker exec -ti kind-control-plane sh -c 'journalctl -u kubelet | grep -m1 -C3 kmsg' Jun 26 08:18:59 kind-control-plane kubelet: I0626 08:18:59.522248 375565 kuberuntime_manager.go:205] Container runtime containerd initialized, version: 1.2.6-0ubuntu1, apiVersion: v1alpha2 Jun 26 08:18:59 kind-control-plane kubelet: I0626 08:18:59.525786 375565 server.go:1083] Started kubelet Jun 26 08:18:59 kind-control-plane kubelet: I0626 08:18:59.526254 375565 server.go:144] Starting to listen on 0.0.0.0:10250 Jun 26 08:18:59 kind-control-plane kubelet: F0626 08:18:59.532066 375565 kubelet.go:1405] Failed to start OOM watcher open /dev/kmsg: no such file or directory Jun 26 08:18:59 kind-control-plane kubelet: I0626 08:18:59.535478 375565 server.go:350] Adding debug handlers to kubelet server. Jun 26 08:18:59 kind-control-plane systemd: kubelet.service: Main process exited, code=exited, status=255/EXCEPTION Jun 26 08:18:59 kind-control-plane systemd: kubelet.service: Failed with result 'exit-code'.
For the node images I bake myself, I have the following dirty hack in place:
kind build node-image ...
ExecStartPrefor the kubelet which just does something like
[ -e /dev/kmsg ] || ln -s /dev/console /dev/kmsg
kind create cluster ... successfully starts the control plane.
What you expected to happen:
kind should be able to bring up the control plane, at least when
/dev/console is available.
How to reproduce it (as minimally and precisely as possible):
Run kind on an environment without
/dev/kmsg, eg. concourse (via kind-on-c for example) and with the 1.15 node image provided by kind.
Anything else we need to know?:
I currently (need to dig deeper) have no idea what the downsides are when linking
/dev/console or what to do when
/dev/console is not available either (when would that be the case). My tests seem to suggest that the cluster works just fine when doing this linking. Would the
kind maintainers be open for a PR which sets such a
ExecStartPre (or makes
/dev/kmsg available via some other means for that matter)?
kind version: (use
kind version v0.4.0
Kubernetes version: (use
v1.15.0 (the version that is baked into
Docker version: (use
docker info): ``
Containers: 1 Running: 1 Paused: 0 Stopped: 0 Images: 1 Server Version: 18.09.6 Storage Driver: overlay2 Backing Filesystem: extfs Supports d_type: true Native Overlay Diff: true Logging Driver: json-file Cgroup Driver: cgroupfs Plugins: WARNING: bridge-nf-call-iptables is disabled WARNING: bridge-nf-call-ip6tables is disabled Volume: local Network: bridge host macvlan null overlay Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog Swarm: inactive Runtimes: runc Default Runtime: runc Init Binary: docker-init containerd version: bb71b10fd8f58240ca47fbb579b9d1028eea7c84 runc version: 2b18fe1d885ee5083ef9f0838fee39b62d653e30 init version: fec3683 Security Options: apparmor seccomp Profile: default Kernel Version: 4.15.0-50-generic Operating System: Alpine Linux v3.10 (containerized) OSType: linux Architecture: x86_64 CPUs: 4 Total Memory: 15.66GiB Name: dd95e405-f2d2-4d82-54b3-36f4d1287a53 ID: YMS4:B7WR:ZESV:VMM6:5CE3:OMGK:7U2Q:3GAS:EGEL:RPTL:35MW:EFPF Docker Root Dir: /scratch/docker Debug Mode (client): false Debug Mode (server): false Registry: https://index.docker.io/v1/ Labels: Experimental: false Insecure Registries: 127.0.0.0/8 Live Restore Enabled: false Product License: Community Engine
OS (e.g. from
NAME="Alpine Linux" ID=alpine VERSION_ID=3.10.0 PRETTY_NAME="Alpine Linux v3.10" HOME_URL="https://alpinelinux.org/" BUG_REPORT_URL="https://bugs.alpinelinux.org/"
Answer questions hoegaarden
maybe related: https://github.com/kubernetes-sigs/kind/issues/455