Ask questionsIssues encountered deploying to OpenStack
1. What kops version are you running? The command kops version, will display
this information.
Version 1.12.0-alpha.1 (git-d44c7fed9)
2. What Kubernetes version are you running? kubectl version will print the
version if a cluster is running or provide the Kubernetes version specified as
a kops flag.
1.11.7
3. What cloud provider are you using? openstack
4. What commands did you run? What is the simplest way to reproduce this issue?
kops create cluster --cloud openstack \
--name sd-dev-k8s.zedev.net \
--zones nova \
--network-cidr 192.168.220.0/24 \
--master-count 3 \
--node-count 3 \
--master-size m1.medium \
--node-size m1.xlarge.mem \
--topology private \
--bastion \
--ssh-public-key ~/.ssh/id_rsa.pub \
--networking weave \
--os-ext-net ze-public1 \
--kubernetes-version 1.11.7 \
--image container-linux-1967.6.0
5. What happened after the commands executed? First failure I encountered:
I0222 17:21:09.900518 7176 create_cluster.go:1456] Using SSH public key: /home/ubuntu/.ssh/id_rsa.pub
W0222 17:21:10.990238 7176 create_cluster.go:713] Running with masters in the same AZs; redundancy will be reduced
error populating configuration: error loading config file: open /home/ubuntu/.openstack/config: no such file or directory
After figuring out how to build this file by looking at the history of the openstack tutorial:
I0222 17:21:33.436162 7198 create_cluster.go:1456] Using SSH public key: /home/ubuntu/.ssh/id_rsa.pub
W0222 17:21:34.466253 7198 create_cluster.go:713] Running with masters in the same AZs; redundancy will be reduced
error populating configuration: error getting section of Designate: section 'Designate' does not exist
After adding the appropriate section for Designate:
I0222 19:16:35.615058 7833 create_cluster.go:1456] Using SSH public key: /home/ubuntu/.ssh/id_rsa.pub
W0222 19:16:36.651965 7833 create_cluster.go:713] Running with masters in the same AZs; redundancy will be reduced
I0222 19:16:37.694541 7833 subnets.go:184] Assigned CIDR 192.168.220.32/27 to subnet nova
I0222 19:16:37.694681 7833 subnets.go:198] Assigned CIDR 192.168.220.0/30 to subnet utility-nova
Previewing changes that will be made:
I0222 19:16:43.940036 7833 builder.go:297] error reading hash file "https://kubeupv2.s3.amazonaws.com/kops/1.12.0-alpha.1/linux/amd64/utils.tar.gz.sha1": unexpected response code "403 Forbidden" for "https://kubeupv2.s3.amazonaws.com/kops/1.12.0-alpha.1/linux/amd64/utils.tar.gz.sha1": <?xml version="1.0" encoding="UTF-8"?>
<Error><Code>AccessDenied</Code><Message>Access Denied</Message><RequestId>0688EECF03949ED6</RequestId><HostId>kltff0LzZZrQuots7KsjVGHzGbFi9KBYJD/KbWeZegHikjFlGkpQ2cBxuN/UD5lDPh4pQKR2mjI=</HostId></Error>
cannot determine hash for "https://kubeupv2.s3.amazonaws.com/kops/1.12.0-alpha.1/linux/amd64/utils.tar.gz" (have you specified a valid file location?)
After setting KOPS_BASE_URL to https://kubeupv2.s3.amazonaws.com/kops/1.11.0:
I0222 19:37:23.864263 7906 create_cluster.go:1456] Using SSH public key: /home/ubuntu/.ssh/id_rsa.pub
W0222 19:37:24.948779 7906 create_cluster.go:713] Running with masters in the same AZs; redundancy will be reduced
I0222 19:37:25.923374 7906 subnets.go:184] Assigned CIDR 192.168.220.32/27 to subnet nova
I0222 19:37:25.923516 7906 subnets.go:198] Assigned CIDR 192.168.220.0/30 to subnet utility-nova
Previewing changes that will be made:
W0222 19:37:31.655041 7906 urls.go:71] Using base url from KOPS_BASE_URL env var: "https://kubeupv2.s3.amazonaws.com/kops/1.11.0"
error building tasks: error reading manifest addons/dns-controller.addons.k8s.io/k8s-1.6.yaml: error opening resource: error executing resource template "addons/dns-controller.addons.k8s.io/k8s-1.6.yaml": error executing template "addons/dns-controller.addons.k8s.io/k8s-1.6.yaml": template: addons/dns-controller.addons.k8s.io/k8s-1.6.yaml:38:17: executing "addons/dns-controller.addons.k8s.io/k8s-1.6.yaml" at <DnsControllerArgv>: error calling DnsControllerArgv: unhandled cloudprovider "openstack"
6. What did you expect to happen? Honestly, about this, it's alpha after all.
7. Please provide your cluster manifest. Execute
kops get --name my.example.com -o yaml to display your cluster manifest.
You may want to remove your cluster name and other sensitive information.
[https://gist.github.com/wfhartford/787e015c04f0c6f0b4d82b14097c92a6]
8. Please run the commands with most verbose logging by adding the -v 10 flag.
Paste the logs into this report, or in a gist and provide the gist link here.
[https://gist.github.com/wfhartford/7405cca2e7507520491303fcdd4a1d36]
9. Anything else do we need to know? Thanks so much for working to support openstack. Please let me know if there are any other things I should try, or if I can help at all.
Answer
questions
wfhartford
Scaling down my cluster to 1 master and 1 node with no bastion node, (our openstack cluster is very small) the kops create cluster command succeeded, followed by kops update cluster --name sd-dev.k8s.local --yes with no unexpected warnings.
However, none of the supplied commands (kubectl get nodes, kops validate cluster) succeeded. The IP address in the kubeconfig file is linked to a load balancer that was created. That load balancer seems to be configured correctly (it lists the internal IP of the master node), but connection attempts fail; curl -k https://<ip address> fails with a connection time out.
I added a static route to the router created by kops (that's required on all our routers in openstack, not really sure why) that allowed curl to connect, but it still fails with curl: (35) OpenSSL SSL_connect: SSL_ERROR_SYSCALL in connection to 172.20.57.11:443.
All kubectl commands fail with:
Unable to connect to the server: EOF
kops validate cluster fails with:
unexpected error during validation: error listing nodes: Get https://172.20.57.11/api/v1/nodes: EOF
Related questions