Do clean up in cosa for bootc v1.6.0

[PeaceRebel]

As bootc v1.6.0 is expected to fix the reproducibility of ostree commit, these changes mainly contains changes to use ostree commit from meta.json

Closes: #4275

[openshift-ci]

Skipping CI for Draft Pull Request.
If you want CI signal for your change, please convert it to an actual PR.
You can still manually trigger a test run with /test all

[PeaceRebel]

=== RUN   fcos.upgrade.basic
=== RUN   fcos.upgrade.basic/setup
=== RUN   fcos.upgrade.basic/upgrade-from-previous
=== RUN   fcos.upgrade.basic/upgrade-from-current
--- FAIL: fcos.upgrade.basic (50.43s)
    --- PASS: fcos.upgrade.basic/setup (2.71s)
    --- PASS: fcos.upgrade.basic/upgrade-from-previous (23.88s)
    --- FAIL: fcos.upgrade.basic/upgrade-from-current (0.08s)
            cluster.go:151: error: No such metadata object 61de0ca4d69d8454e5ddaaf76ffda4ab3ef131b1209c58d4d9caccf0cd041c11.commit
            cluster.go:173: "sudo ostree commit -b testupdate --tree=ref=61de0ca4d69d8454e5ddaaf76ffda4ab3ef131b1209c58d4d9caccf0cd041c11 --add-metadata-string version=44.20250825.dev.0.kola" failed: output , status Process exited with status 1
FAIL, output in tmp/kola

Looks like the issue still exist in the latest bootc version.

[jlebon]

=== RUN   fcos.upgrade.basic
=== RUN   fcos.upgrade.basic/setup
=== RUN   fcos.upgrade.basic/upgrade-from-previous
=== RUN   fcos.upgrade.basic/upgrade-from-current
--- FAIL: fcos.upgrade.basic (50.43s)
    --- PASS: fcos.upgrade.basic/setup (2.71s)
    --- PASS: fcos.upgrade.basic/upgrade-from-previous (23.88s)
    --- FAIL: fcos.upgrade.basic/upgrade-from-current (0.08s)
            cluster.go:151: error: No such metadata object 61de0ca4d69d8454e5ddaaf76ffda4ab3ef131b1209c58d4d9caccf0cd041c11.commit
            cluster.go:173: "sudo ostree commit -b testupdate --tree=ref=61de0ca4d69d8454e5ddaaf76ffda4ab3ef131b1209c58d4d9caccf0cd041c11 --add-metadata-string version=44.20250825.dev.0.kola" failed: output , status Process exited with status 1
FAIL, output in tmp/kola

Looks like the issue still exist in the latest bootc version.

I think the issue there is that we needed new enough rpm-ostree (v2025.10, which pulls in the newer ostree-ext) in the starting disk image, i.e. the qcow2 that kola run-upgrade downloads. I think that's in all the streams we care about now, so it should work. Can you retry?

[PeaceRebel]

I think the issue there is that we needed new enough rpm-ostree (v2025.10, which pulls in the newer ostree-ext) in the starting disk image, i.e. the qcow2 that kola run-upgrade downloads. I think that's in all the streams we care about now, so it should work. Can you retry?

Using 44.20250908.91.0, the test fails with the same error.

EDIT: rpm-ostree version in the above image is rpm-ostree-2025.10-3.fc44.x86_64

[jlebon]

Hmm, and you're sure you're using new enough cosa that has the newer bootc as well?

[PeaceRebel]

Hmm, and you're sure you're using new enough cosa that has the newer bootc as well?

Yup, using local build from this branch.

[coreos-assembler]$ rpm -qa bootc
bootc-1.6.0-1.fc42.x86_64

[jlebon]

Can you dig into this more? E.g. compare the output of ostree show --raw $merge_commit between the two imports to show the raw commit objects (which will also include the commit metadata) and see where the difference is.

[jlebon]

I think bootc-dev/bootc#1637 could also be a factor here that breaks reproducibility.

[dustymabe]

@PeaceRebel what's the status of this?

[PeaceRebel]

@PeaceRebel what's the status of this?

This is parked for now due to the issue with bootc. I may be able to look at it after ignition subpackage work.

[openshift-ci]

PR needs rebase.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

[openshift-ci]

@PeaceRebel: The following test failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name	Commit	Details	Required	Rerun command
ci/prow/images	ecac0c3	link	true	/test images

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.