Ask questionsPushing images to dockerhub stopped working

Actual behavior

Kaniko exits with exit code 1 with the following message and and does not build the image:

error checking push permissions -- make sure you entered the correct tag name,
and that you are authenticated correctly, and try again: checking push permission for
"foo/bar:bionic-99": UNAUTHORIZED: authentication required; [map[Action:pull Class:
Name:mgit/clamav Type:repository] map[Action:push Class: Name:foo/bar Type:repository]]

This worked with the same build pipeline and no changes 3 months ago with the following image:

Using Docker executor with image ...
Pulling docker image ...
Using docker image sha256:2aa254b4837c242c7de87956438eaba70f97a2768ab0870819fd20e09df15cf6 for ...

Expected behavior

Kaniko to upload image to dockerhub like the version 3 months ago was able to. There where no changes, and it works if i go back to an older kaniko version.

To Reproduce Steps to reproduce the behavior:

  1. ... with the following pseudo .gitlab-ci.yml
  entrypoint: [""]

  - foo

  stage: foo
    - echo "{\"auths\":{\"\":{\"auth\":\"${CI_DOCKERHUB_AUTH}\"}}}" > /kaniko/.docker/config.json
    - >
      /kaniko/executor --context "${CI_PROJECT_DIR}/foo" --dockerfile "${CI_PROJECT_DIR}/foo/Dockerfile"
      --destination foo/bar:blub-${CI_PIPELINE_IID}"
      --destination foo/bar:blub"
  1. ... build it

Additional Information

  • Dockerfile Please provide either the Dockerfile you're trying to build or one that can reproduce this error.
  • Build Context Please provide or clearly describe any files needed to build the Dockerfile (ADD/COPY commands)
  • Kaniko Image (fully qualified with digest)
Using Docker executor with image ...
Pulling docker image ...
Using docker image sha256:2ec307dcf7f52dcf700ea0fbc65d448f46365cfac69567e8177bf12b80942f54 for ...

Triage Notes for the Maintainers <!-- 🎉🎉🎉 Thank you for an opening an issue !!! 🎉🎉🎉 We are doing our best to get to this. Please help us by helping us prioritize your issue by filling the section below -->

Description Yes/No
Please check if this a new feature you are proposing <ul><li>- [ ] </li></ul>
Please check if the build works in docker but not in kaniko <ul><li>- [x] </li></ul>
Please check if this error is seen when you use --cache flag <ul><li>- [ ] </li></ul>
Please check if your dockerfile is a multistage dockerfile <ul><li>- [ ] </li></ul>

Answer questions gebi

Ah, sidenote, i confirmed that the credentials still work for pushing images to dockerhub, and they work fine, both manually and as mentioned with the old kaniko version.

For now we have pinned the kaniko version to this one mgit/base:kaniko-executor-debug-stable (which also fixes the problem of kaniko unable to build images on bigger FS because of the included busybox on bigger filesystems due to 64bit inodes and the included busybox not being compiled with large file support #656 )


Related questions

Possibility to pull-tag-push without build hot 2
Support for Azure Container Registry hot 2
HTTP 401 returned when trying to push to Dockerhub with correct credentials hot 2
Can't move files as part of a RUN? hot 1
Kaniko pod - Run as root or priviliged mode? hot 1
Kaniko Logs Leakage: Kaniko pod exiting before errors could be read by Skaffold or any other tool. hot 1
The container base on kaniko/executor image hangs when use knative to build image hot 1
Some log lines prefixed with "ERROR: logging before flag.Parse:" hot 1
Warmer does not work with private images hot 1
error building image: error building stage: lstat /tmp/apt-key-gpghome.VoPBz66R2g/gnupg_spawn_agent_sentinel.lock: no such file or directory hot 1
kaniko cant push to quay ent on prem hot 1
Multistage Dockerfile with copy and glob uses cached layer when it shouldn't hot 1
Build fail : Value too large for defined data type hot 1
Kaniko executor image fails to start (unknown command true for executor) hot 1
does kaniko support docker registry mirrors ? hot 1
Github User Rank List