Demonstrate a debian or arch base image

[cgwalters]

Let's do the work necessary to help push forward a Debian, Arch or other OS base image. The issues in order:

bootloader

systemd-boot: #806
grub/bootupd: coreos/bootupd#468 (also xref coreos/bootupd#432 soon to close the iteration loop.)

ostree dependency

Some distributions only have a quite old libostree. We should probably support a build mode where it's actually vendored.

[LukeShortCloud]

If you need help testing anything, please let me know. I'm very interested in using bootc with my Arch Linux-based winesapOS project! I've put a lot work into it recently to support building OCI images.

[marcoh00]

I had some time on my hands this weekend and decided to have a look at building an Archlinux bootc container.
Good news is that it wasn't too hard to get a system that boots :-)
In case it helps, I'll do a writeup on what I've done and the roadblocks I faced (if only interested in roadblocks just scroll down!).

Base Image

I started with building an OSTree repository for a arch-based rootfs and used M1cha's archlinux-ostree tooling for this.

./arch-ostree build_builder_container
./arch-ostree build_rootfs_container --rootfs-containerfile examples/Containerfile.minimal --rootfs-tag localhost/archlinux:rootfs
ostree init --repo=$(pwd)/ostree
./arch-ostree --ostree-repo $(pwd)/ostree commit_rootfs_container --rootfs-tag localhost/archlinux:rootfs -- -b minimal

I did modify the Containerfile.minimal to include btrfs-progs, dosfstools, exclude grub and in a subsequent COPY/RUN installed additional packages for bootupd, bootc and some unholy package that is a compilation of various fedora boot files extracted from their rpms (more on that below).
It also copies a directory structure into /usr/lib/bootupd that I obtained from a fedora-bootc:41 image.

I could use the OSTree repo to generate a base image using bootc internals ostree-ext container encapsulate -l containers.bootc=1 -l ostree.bootable=1 --repo $(pwd)/ostree minimal containers-storage:localhost/archlinux:ostree.
For testing, I also created a derived image that just sets some simple root password so I could login after booting.

Custom packages

Maybe I find time to upload the PKGBUILDs if anyone's interested.
But they're certainly not ready for production in any way and just had the purpose to get something up and running ;-)

bootc: This is just a simple PKGBUILD looking like any other "rustapp-git" package.

bootupd: Same as bootc, but additionally creates a symlink from bootupd to bootupctl.

fedora-boot: This one cobbles together everything from Fedora's grub2-{common,pc-modules,tools-minimal,tools,efi-x64,pc} and shim-x64 RPMs.
Apart from that, you only need to make sure that the stuff that's going into /usr/sbin is moved to /usr/bin as /usr/sbin is a symlink in arch and otherwise this would conflict with filesystem.
I did not try this yet, but I think bundling everything together like that could make bootupd generate-install-metadata work in combination with a few hacks on the bootupd source code that I did (i.e. swapping the hardcoded rpm calls with calls to pacman).

Roadblocks

The following is pretty much arch-specific, however I'm sure a lot of aspects also apply to other distributions.
Also, I had pretty much zero trouble with bootc, all the hurdles I faced had something to do with either bootupd or boot configuration (I didn't look into who creates these yet, it didn't seem to be bootc, is it OSTree itself?).

• bootupd uses hardcoded RHEL/Fedora-specific parts everywhere.
  • /usr/sbin/grub2-install is /usr/bin/grub-install
  • generate-install-metadata expects everything to live in either /boot or the corresponding ostree directory when trying to query the package manager for version information. However, neither arch has packages putting files directly into boot, nor did I notice that any other distribution apart from Fedora does this.
  • Expecting to find a shim binary. I think most distributions do call it shimx64.efi so the hardcoded filename shouldn't be problematic. However in the case of Archlinux, setting up shim is (1) completely optional and (2) no signed shim packages are offered, so the wiki only explains how to use Ubuntu's shim and adding own keys via mok. As such, it would be handy if an existing shim file wasn't a strict requirement for running bootupd.
  • Currently bootupd has a strict requirement on rpm to generate metadata. When I first read this issue, I had some hope that implementing a pacman backend could be enough to make all of this work so I tried to implement that. If you're interested, I could contribute this, however it seems like there aren't any abstractions for different package management systems, so currently I just replaced the rpm code in my local branch. Unfortunately, I don't know how stable pacman's CLI output is. It certainly hasn't got RPM's capabilities that are currently used in bootupd, i.e. multiple calls are needed to get the package's build date. Also, the output seems to be locale-dependent (however setting LC_ALL did help). Moreover, I don't think all package management systems (looking at you, apt/dpkg!) have information on the build date? Maybe this could turn out to be problematic for other distributions?
• Whatever generates the bootloader configuration writes it into /boot/grub2 while Archlinux's grub searches inside /boot/grub.
• Archlinux GRUB doesn't support BLS so even if it could find the configuration it wouldn't work anyway. Maybe I should open an issue with arch and ask to add BLS support as it seems sensible to expect this.

Of course all of this is mitigated by using Fedora's boot-related packages, but that certainly is only a workaround to get it running at all.
Until now, I only tried to MBR-boot via qemu. I'm not sure if shim/grub check any signatures when not in secure boot mode, if they don't, then EFI could work, too. It wouldn't work with secure boot however (but nothing apart from self-signing and manually adding keys to shim would).

[cgwalters]

bootupd uses hardcoded RHEL/Fedora-specific parts everywhere.

Yes...sorry. There's some active discussion in coreos/bootupd#468 again about generalizing it.

Archlinux GRUB doesn't support BLS so even if it could find the configuration it wouldn't work anyway.

That's...unfortunate indeed, though it's probably worth noting that there's been some long lingering conflict and the BLS that grub implements is not the same as the BLS type 1 as documented.

Thanks for working on this! Bigger picture I was thinking about trying to make bootupd a soft dependency here. One thing that hurts is that bootloader bits are spread across both ostree and bootupd.

This all said I am sure the thing that would help the most is for us to support systemd-boot. That is a long, longstanding TODO on the ostree side but brings up its own complexities.

[frap129]

@marcoh00 How did you install the bootc container? bootc install?

I've been using ostree w/ Arch for a while now and would like to work towards migrating to bootc in the future. I've been trying to deploy an arch container from an exisiting bootc os (bluefin/ublue, deploying using bootc switch) to try avoid dealing with the bootupd issue for now. I've setup a minimal containerfile (https://github.com/frap129/arch-bootc) for working on this. bootc is able to deploy it but it fails to boot at the switch root stage. Falling back to the emergency shell also fails with "Failed to connect to system scope bus via local transport". This is in qemu setup w/ bios, not uefi, so secure boot shouldnt be an issue here.

At this point, I am wondering if I have misunderstood the purpose of your fedora-boot package. I assumed it was just to get bootupd to play nice during installation, and that using a bootc system w/ grub already installed would avoid it.

I should note that I am not encapsulating the container in an ostree commit first, since #887 is merged, which may also be problematic.

Edit: added that boot fails during switch root

Edit 2: The boot issue was because I copied the dracut config to enable the ostree hook to the wrong dir (my bad for copy+pasting and not double checking). The arch-bootc repo I linked above now houses a pretty minimal build setup for an arch-bootc base image. I have not attempted to deploy it via bootc install, but using bootc switch from an existing bootc OS works flawlessly.

[marcoh00]

@frap129 Hi, sorry for answering so late. I see you could figure out most of your issues by yourself?

Yes, I did install via bootc install, specifically the bootc install to-disk --via-loopback method. However, it needs bootupd to work - including all the trouble this brings along (which can be fixed by using a Fedora-like directory structure and packages).

[frap129]

Thanks for clarifying that. I was able to resolve all my issues, and I'm currently using bootc on one of my personal archlinux systems that was already set up using ostree.

As for the bootupd issue, the main problem with supporting archlinux seems to be that bootup expects that the grub and shim packages own their respective ".efi" files, which is only true of shim by nature of it being presigned. Given that bootc works and bootupd is really the only problem here, I'll be posting some thoughts on this in coreos/bootupd#468

[yjftsjthsd-g]

By way of update, I tried to do this with Arch using frap129's repo today, and wrote the whole process up as https://brianecole.com/blog/2025-03-27_bootc-movingarch.html ; the summary is that there are some papercuts that are easy enough to hack around (ex. by putting a fake VERSION_ID in /etc/os-release), but eventually bootc-image-builder just expects to run dnf and that's the end of it as things currently stand. I'll ask after that in the linked issue, but I wanted to post some update in case people wanted to see where things currently appear to stand.

[fnzr]

Just to put a step-by-step setup in writing, I was able to create a bootable arch-based baremetal installation with the following steps:

Start from a real installation of any arch-based system. Trying to install from scratch from a live iso always fails because there isn't enough storage space. There seems to be a way to increase the partition on the live iso(ArchLinux wiki ) but I couldnt get it to work.

Use https://github.com/GrabbenD/ostree-utility from base. Customize the script as desired, like modifying the partitions and such, but modify the Containerfile with the following:

• include sudo, git, base-devel and vim (or whichever text editor) in the image

RUN pacman --noconfirm --sync \
    ...
    git \
    sudo \
    base-devel \
    vim

• create a user with sudo rights, ie:

RUN useradd archlinux --create-home --user-group
RUN echo "archlinux:ostree" | chpasswd
RUN echo "archlinux ALL=(ALL) ALL" >> /etc/sudoers.d/10-archlinux

Run the script. This should make a ostree-ready, bootable drive. Boot into it, login with the non-root user.

Clone https://github.com/frap129/arch-bootc

Unlock /bin by running sudo ostree admin unlock

Now this part is a bit tricky, because makepkg inside arch-bootc/build.sh will try to build/install bootc and bootupd. You might have to edit the makepkg script to allow running as root. Also swap the order of bootc and bootupd on arch-bootc/build.sh , bootupd is a dependency of bootc.

This part is very nebulous to me, and I ended up just building the packages in a working installation, mounting it in the ostree drive and linking the binaries.

Regardless, you must end this first step with bootc in your $PATH and the *tar packages under arch-bootc/pkgbuilds/{bootc,bootupd}

Edit arch-bootc/Containerfile to install whatever you need. Remember to create a non-root user for you.

Run arch-bootc/build.sh (or just the podman command inside the file) this will generate the image localhost/arch-bootc

Run bootc switch --transport containers-storage localhost/arch-bootc --apply

This should install the image and reboot, booting into the newly created image!

To be clear, I have very little understanding of ostree and bootc and I mostly just stumbled my way into this seemingly working state, I don't know know the pitfalls I may or may not have gotten myself into.

[leonheldattoradex]

@cgwalters any chance we might get a bootc crate up on crates.io like we have for https://crates.io/crates/bootupd?

It's a pre-condition* to locally create and upload a package to the feeds.

Thanks!

*edit: not a pre-condition to have it locally, it's just much easier ;-)

[omenos]

@cgwalters Do you have any ideas or pointers for creating bootable container images of Ubuntu/Debian? I'm interested in potentially converting a fleet of thin clients to a bootc-like paradigm, but a specific application they're used for is only supported on Ubuntu.

[wommy]

I have something similar to @omenos
trying to netboot some diskless workstations

[marcoh00]

@omenos @wommy I have played quite a bit with bootc and not-RH-like systems by now and I'm afraid there's no quick way to achieve this. This doesn't mean it's impossible (it's certainly not!) or particularly difficult, but it means you probably have to do a bit of maintenance on a regular basis (as you probably will need to maintain a fork of bootupd while coreos/bootupd#468 is still unsolved).

What you'll need to do:

• Start with a minimal base image. Fortunately, the image doesn't need to be OSTree-based anymore, so you can probably just use docker.io/ubuntu:latest or something similar.
• Install kernel, shim, ostree and bootc packages and generate a suitable initrd containing ostree-prepare-root. composefs-related packages are probably also a good idea.
• Make sure you put kernel and initrd to the appropriate places as described in the bootc documentation
• The hardest part: Patch bootupd according to your distribution's needs and install it inside your container. This means you will need to adjust the hardcoded paths (especially shim, grub2-install) and make some adjustments to the logic that looks for files updated by the package manager. Two weeks ago, I tried to make openSUSE work for the fun of it, you can have a look at the patch. Fortunately, openSUSE collects the EFI files in a common directory and uses RPM, so it was pretty easy - but I don't know about Debian/Ubuntu. For Debian/Ubuntu, you'll probably also have to patch the code that asks the package manager for the current version of boot-related files, as RPM is currently hardcoded.
• Of course, you'll have to install your patched bootupd inside your container and let it update/generate the metadata.
• Pay attention to the handling of /etc vs. /usr/etc. This was a major painpoint for my openSUSE tests, as aaa_base both creates files in /etc and /usr/etc. Because the /usr/etc/* files oftentimes include the /etc ones, I got some SIGSEVs I couldn't explain at first ;)

Good luck!