fstab cleanup of / does not work

[vrothberg]

It looks like we regressed after commit d46b072 since the fstab cleanup does not work (anymore).

The systemd-remount-fs.service fails with

mount: /: fsconfig system call failed: overlay: No changes allowed in reconfigure.
dmesg(1) may have more information after failed mount system call.
/usr/bin/mount for / exited with exit status 32.

/etc/fstab includes

UUID=d457c96a-3adb-48aa-9bb3-3c34ba1c338b / btrfs subvol=root,compress=zstd:1,ro 0 0

[cgwalters]

What generated that /etc/fstab? How did you install this system? Was it with Anaconda or not?

The reason we don't change that fstab line today is it already contains the ro option which is not expected here.

Hmm, I'm guessing this didn't come from Anaconda, but I did find osbuild/bootc-image-builder@c609003 and while that wasn't wrong I really should have taken the next step and fixed it to not emit a fstab entry at all.

[vrothberg]

Was it with Anaconda or not?

Yes, I installed the system with an anaconda-iso produced by BIB.

[cgwalters]

Umm so much technical debt in this area. AFAICS bib still uses this code for that case which is really suboptimal in so many ways because it uses the wrong filesystem type by default across different distros...not even honoring the bootc default filesystem.

More generally default partitioning can differ by OS/distro or desired base image too...again ideally we support embedding this in the container image and in this embedded ISO case Anaconda could read that (ref rhinstaller/anaconda#5197 more generally).

There is a big sub-thread here though that on the bootc side here we really do need to give guidance for people on how to use /etc/fstab correctly...composefs basically breaks the historical concept of /. The baseline guidance would probably just be to recommend root flags as a kernel argument and don't write an /etc/fstab entry for / at all.

[cgwalters]

➡ #972

[cgwalters]

• anaconda: https://bugzilla.redhat.com/show_bug.cgi?id=2332319
• bib: switch from /etc/fstab to root flags karg osbuild/bootc-image-builder#756

[cgwalters]

For the record #1113 is attempting to fix one special case of this from our generator.

However we also have outstanding issues with fedora's selinux policy breaking our generator (ref https://bugzilla.redhat.com/show_bug.cgi?id=2344767 )

Again, the right fix overall is for things to stop using /etc/fstab for the rootfs as noted in our updated docs from #972

But for Anaconda...well, fixing this would be quite messy there I think so we probably do either need to do full fix in our generator or add bootc install finalize-filesystem phase where we could do arbitrary mutations/checks/cleanups on whatever the install tool did after intall to-filesystem (or in Anaconda's case via the legacy ostree container deploy path).

[cgwalters]

add bootc install finalize-filesystem phase where we could do arbitrary mutations/checks/cleanups

Prepared in #1157

[travier]

Also tracked in https://bugzilla.redhat.com/show_bug.cgi?id=2348934

[bdegeeter]

Does using transient [etc] and [root] in the ostree prepare-root.conf further complicate this issue? I'm running into issues creating an anaconda installer iso via BIB with the read-only /etc with overlayfs. The anaconda installer cannot write an /etc/fstab that remains after reboot. No /boot or /boot/efi partitions are created. You can't do a bootc switch to upgrade the OS via an image because it complains about the lack of a /boot/loader structure.