Nomad version

Nomad v0.11.3 (8918fc804a0c6758b6e3e9960e4eb2e605e38552)

Operating system and Environment details

• Arch Linux
• Nomad running with -dev flag

Issue

Given Docker has been configured to use the pass credential helper (from docker-credential-helpers)

{
	"credsStore": "pass",
	"credHelpers": {
		"quay.io": "pass"
	}
}

And pass has been initialized

And that credential helper has been populated with credentials via docker login (which stores them in the credsStore)

And Nomad has been configured to use that credential helper

plugin "docker" {
  config {
    auth {
      helper = "pass"
    }
  }
}

When I run a Nomad job that refers to an image in the configured repository (in this case, quay.io) Nomad fails to pull the image due to an authentication failure

Reproduction steps

See Issue above.

Additional info

This issue seems to be related to the way Nomad requests credentials from the credential helper. If I ask the credential helper for creds, I give "quay.io" as the input, and it returns the creds. However, Nomad prepends "https://" in its call to the helper.

I was interested in knowing if the ecr credential helper handled this, and it does; the code that handles fetching credentials strips incoming "https://":

• GetCredentials calls ExtractRegistry
• ExtractRegistry strips "https://"

The pass credential helper does not support prepending "https://"; doing so yields no credentials.

Workaround

The workaround for this is to create a new entry in pass that matches the domain with the https scheme prepended:

docker-credential-pass get <<< "quay.io" | jq '.ServerURL = "https://quay.io"' | docker-credential-pass store

Note also that trying to log in to docker with docker login https://quay.io is not a workaround; the scheme is stripped when storing the credentials.

After doing the above, Nomad will be able to pull the image.

Job file (if appropriate)

N/A

Nomad Client logs (if appropriate)

The logs here just show the same 500 errors that the UI shows regarding being unable to pull the image. I didn't find anything else that was helpful.

Nomad Server logs (if appropriate)

N/A