Sure, but we never want to end up researching why the two behave differently when we can easily prevent it before merging the PR.

Actually, how does this fail precisely? Looking at storage commit b8e0174ae6b2dc083d9ada365b9a207371aa62a6 , CompressedSize and CompressedDigest are always updated together, and immediately before CompressedDigest is updated, the map used to look up LayersByCompressedDigest is updated. So, we should never get into the blobinfo.Digest != layers[0].CompressedDigest case AFAICS.

(Overall, before roughly the introduction of TryReusingBlob, the code is not very ready for substituting different blobs, and I’d lean towards dropping the reuse (forcing a re-pull of the layer) if we did have to change something, rather than introducing the new / never-before-exercised situations into a very old very stable branch. But right now I must be missing something.)

Following the idea in the comment above, added a commit to check if, when we found a cached layer by the uncompressed digest, the compressed digest matches too: if it does, we return the cached compressed size as before. If not, we have different compressed archives and we return the compressed size of the source image.

That doesn’t work in general because the input blobInfo.Size value may be -1 (unknown), but must not be unknown when returned from ReapplyBlob.

No, the legacy file always uses $HOME AFAICS, and is not ignored when DOCKER_CONFIG is set.

A quick drive-by comment: Rather than update formatting to conform to some recent version, it would be cleaner to freeze the Go or gofmt version back to the version at the branching time, to minimize the number of changes on the branch (and indirectly in CRI-O.). (OTOH I haven’t checked whether that’s even possible.)

Good point! I completely agree. I checked, the gofmt is taken from the distro.

It’s actually coming from the Go 1.9.2 tarball extracted in .travis.Dockerfile AFAICS.

Also switching back the the "artic" release, the gofmt version will report the unsorted import. You’re right about that — actually the breakage comes from a naive backport of 42fb012c4cb109d8a57a954e2673b6eb0b541834 , it’s not a version discrepancy as I assumed.

So, please just fix the code to conform; I’m sorry about the hassle.

The tests are failing due to

gpg: agent_genkey failed: No pinentry
gpg: key generation failed: No pinentry

And fedora:30 is newer than this code. At least this particular failure can be avoided by going back to :29.

It doesn’t make much sense to me to use Skopeo commit 2b97124e4ac18c5f56650b89b8710bd5dc450fee (which introduces the two-return-value version of copy.Image) without including the immediately following 2734f93e301c441aabadf1902b7771e3c42bec70 .

The immediately preceding commit might be a better fit.

Does anything change about what this line does? If not, wrapping the git checkout in a subshell (cd … && git checkout ) && should allow not changing this line, to make it clearer that nothing is going on, and avoid hard-coding the container layout in this Makefile.

The readme currently says this:

skopeo uses credentials from the --creds (for skopeo inspect|delete) or --src-creds|--dest-creds (for skopeo copy) flags, if set; otherwise it uses configuration set by skopeo login, podman login, buildah login, or docker login.

I have already logged into my private registry via docker login. My ~/.docker/config.json file looks like this:

{
	"auths": {
		"<private-registry>": {}
	},
	"HttpHeaders": {
		"User-Agent": "Docker-Client/19.03.12 (darwin)"
	},
	"credsStore": "desktop",
	"experimental": "disabled",
	"stackOrchestrator": "swarm"
}

However, using skopeo v1.2.0, skopeo inspect docker://<private-registry>/<img>:<tag> always fails with an error:

FATA[0000] Error parsing image name "docker://<private-registry>/<img>:<tag>": unable to retrieve auth token: invalid username/password: unknown: Authentication is required 

Passing credentials manually via --creds does work as expected. The private registry is hosted on JFrog Artifactory, if it matters.

Thanks for your report. As you note, this is a missing functionality in c/image, and will be fixed in there.

@dependabot rebase

… to fix CI.

Thanks to Jesse Rittner [email protected] (@rittneje ) for preparing this in #1056.

@vrothberg PTAL again

Actually:

• Let’s extend the lifetime of certificates to 10 years
• And update the rejected_certs one as well, so that we are testing trust, not just expiration.

Looking at https://github.com/projectatomic/docker/blob/master/cliconfig/config.go as well as more recent versions, the DOCKER_CONFIG path replaces the $HOME/.docker, it does not add one more file to check. So this code should do the same thing.

FWIW the --authfile option allows directing Skopeo to use a specific file in a much more direct and transparent way.

Thanks for the certificate update — it’s unrelated and urgent, so let’s split it into #1057 so that we can discuss the details of this PR here.

… to fix CI.

Thanks to Jesse Rittner [email protected] (@rittneje ) for preparing this in #1056.

From a Podman / c/image point of view, the thing to check is just running the upstream registry server on 5443 . If that works, and the nginx proxy doesn’t, my starting hypothesis would be that the nginx proxy is not implementing the protocol as expected.

The PATCH URL comes directly from a Location header sent the server, so the server is misconfigured and does not know its own port. The -p 5443:443 might have something to do with it.

Reopening to see what the parser is doing — and moving to c/image.

A quick drive-by comment: Rather than update formatting to conform to some recent version, it would be cleaner to freeze the Go or gofmt version back to the version at the branching time, to minimize the number of changes on the branch (and indirectly in CRI-O.). (OTOH I haven’t checked whether that’s even possible.)

It should be possible to wrap the io.Reader passed to PutBlob with something that adds a “happened during read:” context (and that would end up in the middle of the final error string), or possibly even wraps it in a marker type that can be detected/unwound to discard all the PutBlob-related error context (“Error writing:”).

@fdegir The configuration file is mixing up v1 and v2 syntax; drop all lines before the unqualified-search-registries line.

That will make it work, but the way it fails is surprising to me — I can’t see how we can end up using the v1 search value while using the v2 registries list; AFAICS the code should refuse to work on mixed v1/v2 files completely.

This code originates from https://github.com/openshift/machine-config-operator/blob/9e770a71b70d52651eb4c26abb0b9eb8d0b1a681/pkg/controller/container-runtime-config/helpers.go#L331 , and at least openshift-controller-manager needs the same code to correctly interpret ImageContentSourcePolicy: https://github.com/openshift/openshift-controller-manager/pull/19 .

ImageStream import/mirror/update code might need the same code to build correct registries.conf to support mirrors, for https://bugzilla.redhat.com/show_bug.cgi?id=1741391 .

https://github.com/openshift/machine-config-operator/pull/1087 validates that this API is usable for machine-config-operator, and the review in there confirms that the openshift-controller-manager code can use it as well.

ImageContentSourcePolicy is supplied by the caller directly, to be fully interpreted by this function; the caller is typically expected to just list the CRD (or, in the case of MCO bootstrap, read it from disk).

The ignored/blocked registries lists are passed as individual parameters instead of using Image.Spec.RegistrySources because MCO modifies the RegistrySources data to remove the cluster’s release ClusterVersionStatus.Desired.Image.

This allows callers to (optionally supply a starting point) and create a /etc/containers/registries.conf configuration based on the ImageContentSourcePolicy CRD and insecure/blocked registries (typically from Image.Spec.RegistrySources).

Currently known users that need this:

• openshift/machine-config-operator (for nodes’ registries.conf)
• openshift/openshit-controller-manager (for builds’ registries.conf)

The code now lives in https://github.com/openshift/runtime-utils/blob/master/pkg/registries/registries.go .

As a very quick guess without actually checking, skipping the copy also skips the opportunity to add a signature via --sign-by. I guess some earlier copy was optimized out, causing the image to not be signed as expected; this error complains about a different signature that does exist; if there were a valid signature, that complaint would not be reported to the user.

I think we typically do document search order and the like in the file format man pages. (Otherwise it would have to be documented not only in the login page, but also in pull and push and so on, in every single consumer of c/image, with no way to tell which man page is authoritative if they differed.)

.docker/config.json is a bit different in that this project is not authoritative about the format specification — but mentioning it in the containers-auth.json(5) man page could also be useful.

OTOH with the long-planned move to prioritize credential helpers, we admittedly might need a different place for the primary documentation of the mechanism. I’m not sure what that place would be; registries.conf would not be easy enough to find.

/remove-lifecycle stale

#111 has dropped the copy of containers/image/pkg/sysregistriesv2 (although a very old version is vendored), but the registries subpackage still exists.

Thanks for your report.

This does not ring a bell. I’d suggest first isolating whether this is a bug in the writer or in the reader:

• read the raw manifest with skopeo inspect --raw and see the data in the manifest
• skopeo copy docker://… dir:$tmpdir, verify the layer digest, and see its size.
• If the on-registry contents are already unexpected, maybe try using a dir: destination or a different registry implementation to check whether it might be registry-specific.

installpodman?

The way I’m thinking about mirrors (which is may not be the way anyone else thinks, and isn’t documented anywhere), they are not intentionally specifically restricted to pulls (e.g. pushes are not included because it’s much harder / impossible to do across unsynchronized destinations, not because it’s fundamentally not interesting to do).

It might well make sense to implemented mirrors for list-tags. OTOH…

Or at least could lead to unexpected results in some configurations. this suggests the situation where mirrors only have a subset of images available.

For pulls, that’s an expected and valuable benefit — but that’s also a good reason not to use mirrors for listing tags.

Also, I’d argue that a skopeo sync by the mirror admin is one of the cases where the user is clearly aware of the mirroring topology, so the user can easily change the skopeo sync command line to read directly from the intended mirror instead of referencing the primary registry and expecting the mirror configuration to apply.

So, on balance I’m leaning towards not changing the current implementation; but there may well be something I’m missing.

I do agree that the documentation should be clarified.

ACK, I don’t think there’s any urgent work in progress we need to wait for.

Thanks for your report. You’re right, the original compressed representation can’t be regenerated.

#1035 was merged, making this no longer applicable.

PR Response to https://bugzilla.redhat.com/show_bug.cgi?id=1848150

Minor change to error messages to provide more user-friendly experience

Signed-off-by: Parker Van Roy [email protected]

Set default rootless sigstore to ~/.local/share/containers/sigstore if the caller is non-root. Fix rootless podman push --sign-by.

Signed-off-by: Qi Wang [email protected]

Reproducible with podman v1.8.0, If I want to use my local GPG keys to push signed images:

> podman pull docker.io/alpine
> podman tag docker.io/alpine quay.io/saschagrunert/alpine
> podman push --sign-by [email protected] quay.io/saschagrunert/alpine
Error: Error copying image to the remote destination: Error writing signatures: mkdir /var/lib/containers/sigstore: permission denied

I think the issue is that signatureTopLevel returns the wrong path when running rootless: https://github.com/containers/image/blob/143904cdaa199b9f3e0a19e1e2a965df10f1f0f9/docker/lookaside.go#L178-L190

If write == true then ns.SigStoreStaging == "file:///var/lib/containers/sigstore"

This feature https://github.com/containers/image/pull/842 was merged in containers/image to support authentication using a provided Bearer token (example use case: integration with Harbor webhooks).

This PR adds a flag to the skopeo CLI to provide support for the registry Bearer token.

Signed-off-by: Alvaro Iradier [email protected]

Thanks for the other fixes!

In the implementation of utils.go, the flag parsing can be done without or with a prefix ("src" or "dest" currently), so I guess that testing flag parsing with prefix "whatever" and then testing the parameters --whatever-registry-token is enough.

Yes, that’s the idea. “whatever” just happens to be spelled dest in the tests.

Please keep the code in the same order as dockerImageOptions to make the correspondence easier to follow.

Move this just below DockerAuthConfig.

Shouldn’t `skopeo layers’ have the option added as well?

Move this just below DockerAuthConfig.

(Here, similarly, it would be a bit easier to follow to have this option after credsOption, the order is already different, let’s not make it worse.)

The same user can already specify --authfile pointing anywhere else (unless the user somehow able to manipulate the environment of a future su/sudo, but not the command being run… I’m not too worried about that, and setting XDG_RUNTIME_DIR would allow the same attack already).

The suffix is a known constant, we don’t need securejoin to protect against that constant containing ../.

Looking at the other comment — this should probably end up as a “get XDG_CONFIG_HOME path” + “append a path relative to XDG_CONFIG_HOME”.

Please introduce a new xdcConfigHomePath constant, or maybe somehow rename the old one; don’t just silently reuse it for a different purpose.

Using a “nonLinux” path on Linux is also confusing — and on non-Linux that path was already added.

(In this context, it’s not a “non-linux” code path; it’s a “following the XDG base dir spec, XDG_CONFIG_HOME is not set so use the spec-defined default” path; i.e. the if/else should conceptually only get the $HOME/.config)

LGTM after dropping the docker/docker_transport.go part, actually.

(Maybe something like

If no `docker` section can be found for the container image, and no `default-docker` section is configured

to go from the most-specific to the most general, in the “order of evaluation”)

This should probably be dropped.

(This can be return nil, errors.Errorf(…))

Addressing https://github.com/containers/image/issues/999

Signed-off-by: Brandon Lum [email protected]

No, that doesn’t immediately look expected.

(… but it’s close :) The Docker archive format does not have manifests, so it also does not have manifest digests. (c/image manufactures an in-memory manifest for “obscure internal reasons”.) So, it would not be surprising that (podman save + podman load on a different machine with no images) does not preserve the digest.)

The Docker archive format does, though, preserve schema2 (only) configs, so I would expect the image ID to be the the same, and the thus deduplicated on podman load on this reproducer where the original image remains in storage. And it is a rather surprising to me that podman inspect of the same image with a different way to reference it leads to a different Digest value.

I have reproduced this, and verified that the image ID ({{.Id}} for both lines is the same), but from a very quick look at the code, the .Digest value available to image inspect comes straight from c/storage, so both ways to find it should AFAICS be copies of the same data and contain the same values.

Currently It might be possible to use podman push --remove-signatures podman save -o $image@sha256:… docker-archive:test.tar (probably losing some functionality like the ability to add extra tags); I didn’t try in practice.

I agree adding --remove-signatures to podman save makes some sense. (To an extent, arguably it should be the default for Podman because podman pull, in general, breaks signatures of the typical compressed images, so pushing the pre-existing signatures is rarely useful. OTOH I’m not sure it always breaks signatures, it might be possible to construct a workflow with uncompressed images without registries or something like that, and always using --remove-signatures would break that.)

RepoTags is part of the entities return type where we have full control over. We can change the name to better reflect the semantics (and also document the fields) in the future.

AFAICT it is visible in the JSON output of podman images, but I may well have overlooked something; I didn’t read the code end-to-end.

Either way, displaying unexpected digests is better than not displaying anything at all, so this is clearly an improvement.

Thanks for your report.

The image is a multi-architecture image, and skopeo copy by default copies only the image for the current architecture. Recent versions of Skopeo provide skopeo copy --all to copy the full multi-architecture image; that should preserve the digest (as long as the destination registry supports the format).

Signed-off-by: Daniel J Walsh [email protected]

My initial guess is that the fields should be initialized in copier when creating it, and not present in imageCopier at all.

This issue also serves as a more general reminder to check what else I should have paid attention to — and to document how it should work for future contributors.