@cgwalters I think I've seen coreos-ci coming back on other repositories, but you may need to force-push this one to re-trigger it.

Fine for doing both later, to be honest I thought the new 0o755 was exactly a reaction to the TODO.

Just noting here, possibly for future consideration, from an out-of-band chat: in order to avoid spiraling into a scripting trap and VM-to-kola dependencies, we could explore having an hidden bootupd exttests runner and move the testing logic into the codebase.

One small nit/confusion from my side, but otherwise big thanks for also covering this with a regression test!

Nit: I think you also took care of the TODO above with this. If so, it can be dropped. Also, there is a very confusing #[allow(dead_code)] // Used for testing at the top of this function, which judging from this patch I think it isn't really true.

** Host system details **

Any

** Expected vs actual behavior **

Atomic has /home as a separate partition, but /var/log and /var/log/audit should also be separate, so malicious containers cannot fill the partitions with logging and create a denial of service.

Would you like to work on the issue? I'm happy to do the implementation here.

Closing this one as it looks like a stale distro-related report (against Fedora Atomic).

/cc @ashcrow for a quick proof-read and @bh7cw for review as she was recently going through this.

This clarifies a few existing details related to log levels and introduces a section covering journalctl and how to inspect logs on a live system.

This converts all daemon logic to stop directly printing to stdout/stderr, and instead perform logging at the appropriate level.

/cc @cgwalters

Check out how it's set up in coreos-metadata.

Closing as this is already in place and working well, see https://github.com/coreos/rpm-ostree/pulls?q=is%3Apr+is%3Aclosed+label%3Adependencies.

Hey @mattdm, reading the linked material it is my understanding that this RFE has been scraped and it evolved into the count-me system instead, which we are tracking at https://github.com/coreos/rpm-ostree/issues/2142. Can you double-check there isn't anything else missing here and that this can be closed in favor of #2142?

LGTM too, leaving to @jlebon for the final pass/merge.

We're using CentOS atomic host to run on 2 embedded platforms - so on bare metal, I guess. Basically, we have a centos-atomic-host-<custom>.json and we use 'rpm-ostree compose' to build the image. Upgrades involve doing 'rpm-ostree rebase' to a new repo with updated packages. Our latest version is rebased on top of CentOS 7.7 - we basically use 7.7 repos to get the most up to date stuff from CentOS 7. My question is - what would our upgrade path to CentOS 8 be? Any recommendation for that?

I see the recommendation for moving from Fedora atomic to Fedora CoreOS is to re-provision (https://discussion.fedoraproject.org/t/launch-faq-how-do-i-migrate-from-fedora-atomic-host-to-fedora-coreos/51), but obviously that won't work on bare metal. Would simply updating the centos-atomic-host.json file and the repos to point to CentOS 8 work, and then 'rpm-ostree rebase' on that?

Closing as this question has been answered already.

For Atomic -> Fedora CoreOS migration there is a doc-page at https://docs.fedoraproject.org/en-US/fedora-coreos/migrate-ah/, which could be further enhanced with more details as needed.

(I've retriggered the CI as it got stuck somewhere, it then merged on green)

Oh indeed, sorry I missed that aspect.

** Host system details ** [root@localhost local]# rpm-ostree --version rpm-ostree: Version: 2017.4 Git: 9ebfecb859228eadeccd3486b7e908005cf875f8 Features:

• compose

Provide the output of rpm-ostree status. State: idle Deployments: â— xxxxx:xxxxxx/2/x86_64/standard Version: OSversion-20170923_163753 (2017-09-23 08:38:40) Commit: 024369683cbdf493dcdb54cdcbcdde6c147a246a58883974aaf20826d310f52f PendingCommit: 0c1af55607027ce7a8d0d43492585c650fa8b13566bb05948554a1b2514a4dba PendingVersion: OSversion-20170925_162900 (2017-09-25 08:29:48) OSName: xxxxxx

xxxxxx:xxxxx-xxxxxx/2/x86_64/standard Version: OSversion-20170925_162900 (2017-09-25 08:29:48) Commit: 0c1af55607027ce7a8d0d43492585c650fa8b13566bb05948554a1b2514a4dba OSName: xxxxx

** Expected vs actual behavior **

[root@localhost local]# rpm-ostree upgrade No upgrade available. Run "systemctl reboot" to start a reboot

Expected:

[root@localhost local]# rpm-ostree upgrade No upgrade available.

Steps to reproduce it

1. install OS
2. upgrade to a newer version(no reboot)
3. rpm-ostree rollback(no reboot)
4. upgrade to the original installed version

Would you like to work on the issue? No.

I think it maybe caused by "PendingCommit" info? but I don't know why it works like this?

Thanks a lot!

Tentatively closing as the issue should have been fixed in the meanwhile, feel free to open a new ticket with fresh details if not.

Minor: you can move this .to_string_lossy() directly into target definition, that should spare you the .display() a few line below (same applies to name too).

Rebased and followed up on comments.

I think this would need to source libtest.sh in VM, no?

I think @dustymabe wants to hold this for a bit and synchronize with the rest of the work at https://github.com/coreos/fedora-coreos-tracker/issues/277. This LGTM as is (but I haven't tried a full provisioning flow right now, I stopped midway due to lack of ACL/time), I'll let him merge when he prefers.

I'm confused about this report. The actual "Getting Started" page at https://docs.fedoraproject.org/en-US/fedora-coreos/getting-started/ actually mentions the core in several places already.

@dustymabe merged; do you need this quickly in a tagged release too?

This updates to actix-0.10 and adapts the existing code to the new API.

@lfarkas thanks for both. It looks like the install flow is failing on

coreos-installer-service[759]: coreos-installer install /dev/vda --ignition-url ftp://192.168.209.1/download/vidux/tftpboot/bios/pxelinux.cfg/vidux.ign --insecure-ignition --firstboot-args rd.neednet=1 ip=dhcp  --platform qemu
coreos-installer-service[779]: Error: parsing arguments
coreos-installer-service[779]: Caused by: unknown protocol for URL 'ftp://192.168.209.1/download/vidux/tftpboot/bios/pxelinux.cfg/vidux.ign'

which indeed matched the PXE entry:

coreos.inst.ignition_url=ftp://192.168.209.1/download/vidux/tftpboot/bios/pxelinux.cfg/vidux.ign

I think this another incarnation of https://github.com/coreos/coreos-installer/issues/381, see the answer there for more details.

/cc @bgilbert on whether he wants to keep/move/close/document this one.

and not it's not the rootfs kernel parameter since i update it to the recommended way ie: add to file to the initrd parameter.

Thanks for the report and for mentioning this, it would have been my first suggestion too.

To inspect what's going on, can you please share the whole PXE configuration? Assuming you have an input mechanism for this node console, can you please continue into the maintenance console and check the output journalctl to see if there are any hints of what is failing in the initramfs?

This has the side-effect of swapping the order of cp grub.cfg and grub-install, would that have actual implications on the bootloader bits on disk?

This introduces a basic pretty-printer at the top-level entrypoint in order to print critical failures in a way which is simpler to understand. New output on error-exit looks like this:

[INFO ] starting update agent (zincati 0.0.14-alpha.0)
[ERROR] critical error: failed to assemble configuration settings
[ERROR]  -> failed to validate agent identity configuration
[ERROR]  -> failed to build default identity
[ERROR]  -> failed to introspect OS base architecture
[ERROR]  -> failed to run 'rpm-ostree' binary
[ERROR]  -> No such file or directory (os error 2)

Fixes #34.

Not sure if that's the expected behavior, but sending a request to cincinnati with a non-supported basearch leads to a 500 internal error.

For example:

$ curl  --header 'Accept:application/json' 'https://updates.coreos.fedoraproject.org/v1/graph?stream=stable&basearch=mock-amd64'
HTTP status server error (500 Internal Server Error) for url (http://127.0.0.1:8080/v1/graph?basearch=mock-amd64&stream=stable)

The issue seems to be here: https://github.com/coreos/fedora-coreos-cincinnati/blob/master/fcos-graph-builder/src/main.rs#L147

    let arch_graph = policy::pick_basearch(cached_graph, basearch)?;
    let final_graph = policy::filter_deadends(arch_graph);

Instead of returning an HttpResponse, we're just erroring out.

I haven't used rust in ages, but I can attempt a fix. Should we send a 400 bad request?

LGTM, thanks for fixing this!

Feels like systemd really needs to grow some sort of ExecStartArgs=

We can probably cover this RFE somewhere before going to the systemd folks, but I'm not so sure and I think that the final interface would be quite similar (a map of string -> string) with the only difference being leaving envp pristine.

Wouldn't it be better to support configuration via environment directly? Can't they already do that via RUST_LOG?

Yes RUST_LOG would still work, especially for more advanced things (e.g. selectively enabling debug messages from a specific library). It isn't a very discoverable/friendly UX though, and for bootupctl specifically there are few bleeding edges as the crate name (used to control per-module logging) does not match the binary name. This is also narrower in scope, as it only bridges the top-level crate level (versus the global logging level) to a CLI flag, allowing to directly get debug/trace messages from the application without all the transitive dependencies too (i.e. RUST_LOG=bootupd=debug). To me it seems worth at least for bootupctl, and possibly on bootupd to avoid having to each every single user about the module-syntax of RUST_LOG.

This fixes error context on rpm-ostree failed runs in order to avoid duplicating the underlying error cause. Failure chain already includes the root cause as the next element, thus there is no need to also embed it in the parent context.

I do not know actix/futures well enough to understand the new -> pin change.

For reference, the change aligns to the new type constraints introduced with https://github.com/actix/actix/pull/355. In short, the new API requires that the future response chain can be pinned to a stable memory address, which we are upholding here.

As a side-effect, this avoids a few blank lines and also reverses the order in which the error-chain is traversed, pushing outer error context towards the top. That seems more intuitive to me.

Before:

[INFO ] starting update agent (zincati 0.0.14-alpha.0)
Error: Os { code: 2, kind: NotFound, message: "No such file or directory" }

failed to run 'rpm-ostree' binary

failed to introspect OS base architecture

failed to build default identity

failed to validate agent identity configuration

failed to assemble configuration settings

After:

[INFO ] starting update agent (zincati 0.0.14-alpha.0)
[ERROR] critical error: failed to assemble configuration settings
[ERROR]  -> failed to validate agent identity configuration
[ERROR]  -> failed to build default identity
[ERROR]  -> failed to introspect OS base architecture
[ERROR]  -> failed to run 'rpm-ostree' binary
[ERROR]  -> No such file or directory (os error 2)

This introduces a basic pretty-printer at the top-level entrypoint in order to print critical failures in a way which is simpler to understand. New output on error-exit looks like this:

[INFO ] starting update agent (zincati 0.0.14-alpha.0)
[ERROR] critical error: failed to assemble configuration settings
[ERROR]  -> failed to validate agent identity configuration
[ERROR]  -> failed to build default identity
[ERROR]  -> failed to introspect OS base architecture
[ERROR]  -> failed to run 'rpm-ostree' binary
[ERROR]  -> No such file or directory (os error 2)

This fixes error context on rpm-ostree failed runs in order to avoid duplicating the underlying error cause. Failure chain already includes the root cause as the next element, thus there is no need to also embed it in the parent context.

s/etc../etc./ ?

I think something went wrong with this sentence, possibly some copy-paste mixup?

Nit: I think ibmcloud login -r ${REGION} --apikey @my-key.json is equivalent here and also covers the API-login step (but please double-check that it also affects commands further below).

Good catch! I've directly amended the typo in the existing commit.

This looks like a useful generalization of the base.ign + user.ign merge :+1:. Among other things, right now base & user configs are explicitly identified by kind. @bgilbert were you planning to have separate base.d and user.d paths, or to only touch the former, or to get rid of the split, or something else?

@tomleb code looks fine, but this needs a cargo fmt pass to make CI happy.

Zincati is fine with empty errors. For reference, the plaintext error would have been fine too, but it would have been discarded as not matching the expected JSON format, so it was mostly pointless.

@tomleb no problems, happy to merge that if it makes the flow less surprising, I was just missing the whole context about the ticket thus asking for more hints :)

This updates to actix-0.10 and adapts the existing code to the new API.

/cc @dustymabe

This adds triggering condition to SSH-keys service units on OpenStack.

Overall this is ok to land, but it's quite likely that we make uncoordinated changes when merging/ratcheting this, thus ending up with behavioral side-effects in RHCOS. Before merging this, @bh7cw it would be great if you could:

1. already prepare and get merged the override in RHCOS config repo
2. add a README here pointing back to that copy to be kept in sync