profile
viewpoint
nodejs Spring Boot React Rust tensorflow

Ask questionsInconsistent results with kubectl and curl for SelfSubjectAccessReview api calls

<!-- Please use this template while reporting a bug and provide as much info as possible. Not doing so may result in your bug not being addressed in a timely manner. Thanks!-->

What happened:

  1. We create a kind cluster 18.8 version.
  2. Setup a service account
apiVersion: v1
kind: ServiceAccount
metadata:
  name: myserviceaccount
  namespace: default
  1. Created an impersonator clusterrole that allows impersonating users and groups
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: impersonate-myserviceaccount
rules:
- apiGroups: [""]
  resources: ["users", "groups"]
  verbs: ["impersonate"]
  1. Assign this role to myserviceaccount
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: impersonate-myserviceaccount
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: impersonate-myserviceaccount
subjects:
- kind: ServiceAccount
  name: myserviceaccount
  namespace: default
  1. Now we wanted to test if this serviceaccount can only impersonate users and groups as assigned by RBAC above and can not impersonate any other service account. Similar to doing
kubectl auth can-i impersonate serviceaccounts --token=<token-created-in-secret-for-myserviceaccount> --v=10
  1. When we execute the command above, we get allowed as true meaning that the serviceaccount can impersonate other serviceaccounts and which would mean our RBAC is not setup correctly.
I0504 18:01:30.495678   13926 request.go:1068] Request Body: {"kind":"SelfSubjectAccessReview","apiVersion":"authorization.k8s.io/v1","metadata":{"creationTimestamp":null},"spec":{"resourceAttributes":{"namespace":"default","verb":"impersonate","resource":"serviceaccounts"}},"status":{"allowed":false}}
I0504 18:01:30.495895   13926 round_trippers.go:423] curl -k -v -XPOST  -H "User-Agent: kubectl/v1.18.2 (darwin/amd64) kubernetes/52c56ce" -H "Accept: application/json, */*" -H "Authorization: Bearer <redacted-token-created-in-secret-for-myserviceaccount>" -H "Content-Type: application/json" 'https://127.0.0.1:58114/apis/authorization.k8s.io/v1/selfsubjectaccessreviews'
I0504 18:01:30.804568   13926 round_trippers.go:443] POST https://127.0.0.1:58114/apis/authorization.k8s.io/v1/selfsubjectaccessreviews 201 Created in 308 milliseconds
I0504 18:01:30.804674   13926 round_trippers.go:449] Response Headers:
I0504 18:01:30.804728   13926 round_trippers.go:452]     Cache-Control: no-cache, private
I0504 18:01:30.804761   13926 round_trippers.go:452]     Content-Type: application/json
I0504 18:01:30.804785   13926 round_trippers.go:452]     Content-Length: 497
I0504 18:01:30.804824   13926 round_trippers.go:452]     Date: Tue, 04 May 2021 12:31:30 GMT
I0504 18:01:30.804942   13926 request.go:1068] Response Body: {"kind":"SelfSubjectAccessReview","apiVersion":"authorization.k8s.io/v1","metadata":{"creationTimestamp":null,"managedFields":[{"manager":"kubectl","operation":"Update","apiVersion":"authorization.k8s.io/v1","time":"2021-05-04T12:31:30Z","fieldsType":"FieldsV1","fieldsV1":{"f:spec":{"f:resourceAttributes":{".":{},"f:namespace":{},"f:resource":{},"f:verb":{}}}}}]},"spec":{"resourceAttributes":{"namespace":"default","verb":"impersonate","resource":"serviceaccounts"}},"status":{"allowed":true}}
  1. However if we execute the curl command directly printed above, it gives the expected result
curl -k -v -XPOST  -H "Accept: application/json, */*" -H "Content-Type: application/json" -H "User-Agent: kubectl/v1.18.2 (darwin/amd64) kubernetes/52c56ce" -H "Authorization: Bearer <redacted-token-created-in-secret-for-myserviceaccount>" 'https://127.0.0.1:58114/apis/authorization.k8s.io/v1/selfsubjectaccessreviews' -d '{"kind":"SelfSubjectAccessReview","apiVersion":"authorization.k8s.io/v1","metadata":{"creationTimestamp":null},"spec":{"resourceAttributes":{"namespace":"default","verb":"impersonate","resource":"serviceaccounts"}},"status":{"allowed":false}}'

* Using HTTP2, server supports multi-use
* Connection state changed (HTTP/2 confirmed)
* Copying HTTP/2 data in stream buffer to connection buffer after upgrade: len=0
* Using Stream ID: 1 (easy handle 0x7ff236808200)
> POST /apis/authorization.k8s.io/v1/selfsubjectaccessreviews HTTP/2
> Host: 127.0.0.1:58114
> Accept: application/json, */*
> Content-Type: application/json
> User-Agent: kubectl/v1.18.2 (darwin/amd64) kubernetes/52c56ce
> Authorization: Bearer <redacted-token-created-in-secret-for-myserviceaccount>
> Content-Length: 242
> 
* We are completely uploaded and fine
* Connection state changed (MAX_CONCURRENT_STREAMS == 250)!
< HTTP/2 201 
< cache-control: no-cache, private
< content-type: application/json
< content-length: 498
< date: Tue, 04 May 2021 12:40:07 GMT
< 
{"kind":"SelfSubjectAccessReview","apiVersion":"authorization.k8s.io/v1","metadata":{"creationTimestamp":null,"managedFields":[{"manager":"kubectl","operation":"Update","apiVersion":"authorization.k8s.io/v1","time":"2021-05-04T12:40:07Z","fieldsType":"FieldsV1","fieldsV1":{"f:spec":{"f:resourceAttributes":{".":{},"f:namespace":{},"f:resource":{},"f:verb":{}}}}}]},"spec":{"resourceAttributes":{"namespace":"default","verb":"impersonate","resource":"serviceaccounts"}},"status":{"allowed":false}}
* Connection #0 to host 127.0.0.1 left intact
* Closing connection 0

allowed is false which is expected result. 8. We tried the same on other hosted clusters on Oracle OCI but could not reproduce this issue. However on kind its reproducible every time, i.e. with kubectl the result is "allowed":true and on a subsequent curl command with same headers and body , its "allowed" : false

What you expected to happen: Since the same api is being called from both kubectl and curl and with same headers/body , it should return same result.

How to reproduce it (as minimally and precisely as possible): Create the serviceaccount, cluster role and cluster role binding and try to execute kubectl followed by curl as mentioned above

Anything else we need to know?: This works consistently on other hosted Kubernetes clusters and issue is only on kind

Environment:

  • kind version: (use kind version): kind v0.10.0 go1.15.7 darwin/amd64 but the cluster was created using v1.18.8
$ kind create cluster --config - <<EOF
kind: Cluster
apiVersion: kind.x-k8s.io/v1alpha4
nodes:
  - role: control-plane
    image: kindest/node:v1.18.8@sha256:f4bcc97a0ad6e7abaf3f643d890add7efe6ee4ab90baeb374b4f41a4c95567eb
    kubeadmConfigPatches:
      - |
        kind: ClusterConfiguration
        apiServer:
          extraArgs:
            "service-account-issuer": "kubernetes.default.svc"
            "service-account-signing-key-file": "/etc/kubernetes/pki/sa.key"
EOF
  • Kubernetes version: (use kubectl version):
Client Version: version.Info{Major:"1", Minor:"18", GitVersion:"v1.18.2", GitCommit:"52c56ce7a8272c798dbc29846288d7cd9fbae032", GitTreeState:"clean", BuildDate:"2020-04-16T11:56:40Z", GoVersion:"go1.13.9", Compiler:"gc", Platform:"darwin/amd64"}
Server Version: version.Info{Major:"1", Minor:"18", GitVersion:"v1.18.8", GitCommit:"9f2892aab98fe339f3bd70e3c470144299398ace", GitTreeState:"clean", BuildDate:"2020-09-14T07:44:34Z", GoVersion:"go1.13.15", Compiler:"gc", Platform:"linux/amd64"}
  • Docker version: (use docker info):
bash-3.2$ docker info
Client:
 Context:    default
 Debug Mode: false
 Plugins:
  app: Docker App (Docker Inc., v0.9.1-beta3)
  buildx: Build with BuildKit (Docker Inc., v0.5.1-docker)
  scan: Docker Scan (Docker Inc., v0.6.0)

Server:
 Containers: 11
  Running: 3
  Paused: 0
  Stopped: 8
 Images: 22
 Server Version: 20.10.5
 Storage Driver: overlay2
  Backing Filesystem: extfs
  Supports d_type: true
  Native Overlay Diff: true
 Logging Driver: json-file
 Cgroup Driver: cgroupfs
 Cgroup Version: 1
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
 Swarm: inactive
 Runtimes: io.containerd.runc.v2 io.containerd.runtime.v1.linux runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: 05f951a3781f4f2c1911b05e61c160e9c30eaa8e
 runc version: 12644e614e25b05da6fd08a38ffa0cfe1903fdec
 init version: de40ad0
 Security Options:
  seccomp
   Profile: default
 Kernel Version: 5.10.25-linuxkit
 Operating System: Docker Desktop
 OSType: linux
 Architecture: x86_64
 CPUs: 4
 Total Memory: 1.941GiB
 Name: docker-desktop
 ID: 33ZG:JI56:DXHW:46UM:VKBQ:HQLJ:B5FL:WFAV:O2NU:RDZI:4GGW:A4HK
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 HTTP Proxy: http.docker.internal:3128
 HTTPS Proxy: http.docker.internal:3128
 Registry: https://index.docker.io/v1/
 Labels:
 Experimental: false
 Insecure Registries:
  127.0.0.0/8
 Live Restore Enabled: false
  • OS (e.g. from /etc/os-release):
macOS Catalina
v10.15.7
kubernetes-sigs/kind

Answer questions lake-of-dreams

@enj I followed the steps you suggested please but I get following error on last line

bash-3.2$ kubectl auth can-i impersonate serviceaccounts
error: You must be logged in to the server (Unauthorized)
useful!

Related questions

zsh completions can't be sourced hot 10
Errors compile: version "go1.12.1" does not match go tool version "go1.10.2" hot 9
Support for Apple silicon M1 hot 6
[liquorix kernel] error when running `kind create cluster`: failed to create cluster: failed to init node with kubeadm - kind hot 1
source：https://uonfu.com/
answerer
Ravi Singhal lake-of-dreams @oracle https://www.linkedin.com/in/ravisinghal11

repos

42

gists

0

followers

1

following

2
Github User Rank List