profile
viewpoint

Ask questionsElasticsearch stuck in ApplyingChanges

Bug Report

What did you do? Added a secureSettings section to an existing Elasticsearch resource, as described in the docs.

What did you expect to see? The Elasticsearch pods should be configured and restarted, and the keystore should contain the new settings.

What did you see instead? Under which circumstances?

Elasticsearch resource is stuck in phase ApplyingChanges.

Environment

  • ECK version: 1.0.0-beta1

  • Kubernetes information: EKS 1.14

$ kubectl version
Client Version: version.Info{Major:"1", Minor:"16", GitVersion:"v1.16.1", GitCommit:"d647ddbd755faf07169599a625faf302ffc34458", GitTreeState:"clean", BuildDate:"2019-10-02T17:01:15Z", GoVersion:"go1.12.10", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"14+", GitVersion:"v1.14.6-eks-5047ed", GitCommit:"5047edce664593832e9b889e447ac75ab104f527", GitTreeState:"clean", BuildDate:"2019-08-21T22:32:40Z", GoVersion:"go1.12.9", Compiler:"gc", Platform:"linux/amd64"}
  • Logs: If found no relevant entries in the operator logs.
elastic/cloud-on-k8s

Answer questions a-hat

Apparently the cause of this problem is that the ES cluster had only one node. I suppose the operator always wants to keep one node running to guarantee the availability of the cluster, which is fine. But with version 0.9 of the operator, the behaviour was different if I remember correctly. The operator spawned another node with the new settings, and finally terminated the old node.

I suppose this is due to the implementation with StatefulSets. Will this behaviour change, or is it intended?

useful!

Related questions

Allow ECK to specify a custom readiness check for Elasticsearch and Kibana hot 1
Kibana pods crashing with FATAL Error hot 1
no matches for kind "Elasticsearch" in version "elasticsearch.k8s.elastic.co/v1alpha1" hot 1
Provisioning of custom configuration files hot 1
Elasticsearch InitContainer not being scheduled hot 1
Cluster goes into CrashLoopBackOff after 5-10 minutes of running hot 1
minimum_master_nodes is removed or not updated when upgrading+upscaling hot 1
"Resource was created with older version of operator, will not take action" cause Es cluster pods to stuck hot 1
Can't add data nodes hot 1
error: unable to recognize "kibana.yaml": no matches for kind "Kibana" in version "kibana.k8s.elastic.co/v1beta1" hot 1
Connecting with Elasticsearch running in Elastic on Kubernetes hot 1
no matches for kind "Elasticsearch" in version "elasticsearch.k8s.elastic.co/v1alpha1" hot 1
Cannot delete resources created with ECK 0.8 if 0.9 is running hot 1
Github User Rank List