profile
viewpoint

Ask questionsgo panic on intial rke up

RKE version: v.0.2.6

Docker version: (docker version,docker info preferred)

Client:
 Debug Mode: false

Server:
 Containers: 0
  Running: 0
  Paused: 0
  Stopped: 0
 Images: 1
 Server Version: 18.09.8
 Storage Driver: overlay2
  Backing Filesystem: xfs
  Supports d_type: true
  Native Overlay Diff: true
 Logging Driver: json-file
 Cgroup Driver: cgroupfs
 Plugins:
  Volume: local
  Network: bridge host macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
 Swarm: inactive
 Runtimes: runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: 894b81a4b802e4eb2a91d1ce216b8817763c29fb
 runc version: 425e105d5a03fabd737a126ad93d62a9eeede87f
 init version: fec3683
 Security Options:
  seccomp
   Profile: default
 Kernel Version: 3.10.0-957.1.3.el7.x86_64
 Operating System: CentOS Linux 7 (Core)
 OSType: linux
 Architecture: x86_64
 CPUs: 2
 Total Memory: 1.758GiB
 Name: ip-10-99-76-224.ec2.internal
 ID: CMSC:I77F:3I5T:Z64I:XGJS:O43D:4RZY:WCT2:2M6G:GHZX:LAHZ:I3T6
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 Registry: https://index.docker.io/v1/
 Labels:
 Experimental: false
 Insecure Registries:
  127.0.0.0/8
 Live Restore Enabled: false
 Product License: Community Engine

Operating system and kernel: (cat /etc/os-release, uname -r preferred)

$ cat /etc/os-release 
NAME="CentOS Linux"
VERSION="7 (Core)"
ID="centos"
ID_LIKE="rhel fedora"
VERSION_ID="7"
PRETTY_NAME="CentOS Linux 7 (Core)"
ANSI_COLOR="0;31"
CPE_NAME="cpe:/o:centos:centos:7"
HOME_URL="https://www.centos.org/"
BUG_REPORT_URL="https://bugs.centos.org/"

CENTOS_MANTISBT_PROJECT="CentOS-7"
CENTOS_MANTISBT_PROJECT_VERSION="7"
REDHAT_SUPPORT_PRODUCT="centos"
REDHAT_SUPPORT_PRODUCT_VERSION="7"

Type/provider of hosts: (VirtualBox/Bare-metal/AWS/GCE/DO) AWS EC2 instances

cluster.yml file:

addon_job_timeout: 60
ssh_agent_auth: true
nodes:
    - address: ec2-xx-xx-xx-xx.compute-1.amazonaws.com
      user: centos
      role:
        - controlplane
        - etcd
        - worker

Steps to Reproduce:

rke up

Results:

INFO[0000] Initiating Kubernetes cluster                
INFO[0000] [state] Possible legacy cluster detected, trying to upgrade 
INFO[0000] [reconcile] Rebuilding and updating local kube config 
INFO[0000] Successfully Deployed local admin kubeconfig at [./kube_config_cluster.yml] 
INFO[0001] [state] Fetching cluster state from Kubernetes 
INFO[0031] Timed out waiting for kubernetes cluster to get state 
WARN[0031] Failed to fetch state from kubernetes: Timeout waiting for kubernetes cluster to get state 
INFO[0031] [dialer] Setup tunnel for host [ec2-xx-xx-xx-xx.compute-1.amazonaws.com] 
INFO[0031] [state] Fetching cluster state from Nodes    
INFO[0032] [state] Successfully started [cluster-state-deployer] container on host [ec2-xx-xx-xx-xx.compute-1.amazonaws.com] 
INFO[0033] [certificates] Getting Cluster certificates from Kubernetes 
WARN[0034] Failed to fetch certs from kubernetes: Get https://ec2-xx-xx-xx-xx.compute-1.amazonaws.com:6443/api/v1/namespaces/kube-system/secrets/kube-ca?timeout=30s: dial tcp xx.xx.xx.xx:6443: connect: connection refused 
INFO[0034] [certificates] Fetching kubernetes certificates from nodes 
INFO[0034] [certificates] Successfully started [cert-fetcher] container on host [ec2-xx-xx-xx-xx.compute-1.amazonaws.com] 
INFO[0035] Successfully Deployed state file at [./cluster.rkestate] 
INFO[0035] [certificates] Generating CA kubernetes certificates 
INFO[0035] [certificates] Generating Kubernetes API server aggregation layer requestheader client CA certificates 
INFO[0035] [certificates] Generating Node certificate   
INFO[0035] [certificates] Generating etcd-ec2-xx-xx-xx-xx.compute-1.amazonaws.com certificate and key 
INFO[0035] [certificates] Generating Kubernetes API server certificates 
INFO[0036] [certificates] Generating Service account token key 
INFO[0036] [certificates] Generating Kube Controller certificates 
INFO[0036] [certificates] Generating Kube Proxy certificates 
INFO[0036] [certificates] Generating Kubernetes API server proxy client certificates 
INFO[0036] [certificates] Generating Kube Scheduler certificates 
INFO[0036] [certificates] Generating admin certificates and kubeconfig 
INFO[0036] Successfully Deployed state file at [./cluster.rkestate] 
INFO[0036] Building Kubernetes cluster                  
INFO[0036] [dialer] Setup tunnel for host [ec2-xx-xx-xx-xx.compute-1.amazonaws.com] 
INFO[0037] [network] No hosts added existing cluster, skipping port check 
INFO[0037] [certificates] Deploying kubernetes certificates to Cluster nodes 
INFO[0043] [reconcile] Rebuilding and updating local kube config 
INFO[0043] Successfully Deployed local admin kubeconfig at [./kube_config_cluster.yml] 
INFO[0044] [certificates] Successfully deployed kubernetes certificates to Cluster nodes 
INFO[0044] [reconcile] Reconciling cluster state        
INFO[0044] [reconcile] Check etcd hosts to be deleted   
panic: runtime error: invalid memory address or nil pointer dereference [recovered]
	panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0xd7998d]

goroutine 1 [running]:
github.com/rancher/rke/vendor/github.com/urfave/cli.HandleAction.func1(0xc000654ca0)
	/go/src/github.com/rancher/rke/vendor/github.com/urfave/cli/app.go:472 +0x278
panic(0x1362220, 0x236c070)
	/usr/local/go/src/runtime/panic.go:513 +0x1b9
github.com/rancher/rke/vendor/k8s.io/client-go/util/cert.EncodeCertPEM(0x0, 0xc000505ad0, 0x1755c40, 0x23a8c00)
	/go/src/github.com/rancher/rke/vendor/k8s.io/client-go/util/cert/pem.go:69 +0x4d
github.com/rancher/rke/cluster.reconcileEtcd(0x17783a0, 0xc00003a0d0, 0xc00017e000, 0xc000312000, 0xc00055ef00, 0x0, 0x0)
	/go/src/github.com/rancher/rke/cluster/reconcile.go:175 +0x130
github.com/rancher/rke/cluster.ReconcileCluster(0x17783a0, 0xc00003a0d0, 0xc000312000, 0xc00017e000, 0x0, 0x0, 0x152d7f6, 0xb, 0x0, 0x0, ...)
	/go/src/github.com/rancher/rke/cluster/reconcile.go:43 +0x1ab
github.com/rancher/rke/cmd.ClusterUp(0x17783a0, 0xc00003a0d0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x152d7f6, 0xb, 0x0, ...)
	/go/src/github.com/rancher/rke/cmd/up.go:133 +0x557
github.com/rancher/rke/cmd.clusterUpFromCli(0xc0001052c0, 0x0, 0x0)
	/go/src/github.com/rancher/rke/cmd/up.go:242 +0x582
reflect.Value.call(0x12f8540, 0x15d6b40, 0x13, 0x1525e6b, 0x4, 0xc0004d2c40, 0x1, 0x1, 0xc0004de000, 0x1, ...)
	/usr/local/go/src/reflect/value.go:447 +0x449
reflect.Value.Call(0x12f8540, 0x15d6b40, 0x13, 0xc0004d2c40, 0x1, 0x1, 0x5, 0x4, 0xc000061030)
	/usr/local/go/src/reflect/value.go:308 +0xa4
github.com/rancher/rke/vendor/github.com/urfave/cli.HandleAction(0x12f8540, 0x15d6b40, 0xc0001052c0, 0x0, 0x0)
	/go/src/github.com/rancher/rke/vendor/github.com/urfave/cli/app.go:481 +0x1fb
github.com/rancher/rke/vendor/github.com/urfave/cli.Command.Run(0x15258be, 0x2, 0x0, 0x0, 0x0, 0x0, 0x0, 0x153740e, 0x14, 0x0, ...)
	/go/src/github.com/rancher/rke/vendor/github.com/urfave/cli/command.go:186 +0x8f6
github.com/rancher/rke/vendor/github.com/urfave/cli.(*App).Run(0xc000001800, 0xc00000a080, 0x2, 0x2, 0x0, 0x0)
	/go/src/github.com/rancher/rke/vendor/github.com/urfave/cli/app.go:235 +0x52e
main.mainErr(0x175e520, 0xc00000c018)
	/go/src/github.com/rancher/rke/main.go:56 +0xf9e
main.main()
	/go/src/github.com/rancher/rke/main.go:19 +0x3e
rancher/rke

Answer questions superseb

This is fixed in master (https://github.com/rancher/rke/pull/1462), either there is a kubeconfig and no cluster.rkestate causing the issue. If this happens in using a directory with only cluster.yml and a fresh node, let me know and please provide exact steps to reproduce.

useful!
source:https://uonfu.com/
Github User Rank List