bootc enablement for ppc64le

[sacsant]

I am trying to figure out how bootc (bootable container) works and started experimenting with it on Power(ppc64le arch).

I came across https://developers.redhat.com/articles/2024/05/07/image-mode-rhel-bootable-containers# and attempted to create a bootable container image on Power using the recipe mentioned in bootc image builder project .

I used following command (Fedora40 running on PowerVM Logical Partition [LPAR])

podman run --pull=newer --rm --privileged --pid=host quay.io/sacsant/fedora-bootc:40 bootc install to-disk --target-no-signature-verification --wipe --block-setup direct --filesystem xfs /dev/sdc
WARN[0000] Failed to decode the keys ["storage.options.thinpool"] from "/usr/share/containers/storage.conf"
time="2024-05-25T06:03:33-04:00" level=warning msg="Failed to decode the keys ["storage.options.thinpool"] from "/usr/share/containers/storage.conf""
time="2024-05-25T06:03:34-04:00" level=warning msg="Failed to decode the keys ["storage.options.thinpool"] from "/usr/share/containers/storage.conf""
Installing image: docker://quay.io/sacsant/fedora-bootc:40
Digest: sha256:6c88359c24a8c351bf2f742c0f5cfd9a5b32b1e6e475499fc9612c017e2cf860
Wiping /dev/sdc1
Wiping device /dev/sdc1
Wiping /dev/sdc2
Wiping device /dev/sdc2
Wiping /dev/sdc3
Wiping device /dev/sdc3
/dev/sdc3: 4 bytes were erased at offset 0x00000000 (xfs): 58 46 53 42
Wiping /dev/sdc
Wiping device /dev/sdc
/dev/sdc: 2 bytes were erased at offset 0x000001fe (dos): 55 aa
/dev/sdc: calling ioctl to re-read partition table: Success
ERROR Installing to disk: Creating rootfs: Unsupported architecture: powerpc64

Can someone help me debug this problem? Is code missing for ppc64le arch?

[prb112]

It's x86_64 and aarch64 only L251
anyhow::bail!("Unsupported architecture: {}", std::env::consts::ARCH);

It'd be great to get ppc64le support.

[sacsant]

Following the s390x enablement tracker, I was able to make some progress on this.

• Updated the image.go to include ppc64le arch
• Updated partition_tables.go with ppc64le specific data
• To get around the (panic: required mounts for bootupd stage [/boot/efi] missing) issue for now commented the /boot/efi line from images repo
• AnacondaContainerInstaller specific customisation code in image.go also possibly needs some update to avoid following errors. For now I commented these statements to move forward

cmd/bootc-image-builder/image.go:237:7: img.Users undefined (type *"github.com/osbuild/images/pkg/image".AnacondaContainerInstaller has no field or method Users)
cmd/bootc-image-builder/image.go:238:7: img.Groups undefined (type *"github.com/osbuild/images/pkg/image".AnacondaContainerInstaller has no field or method Groups)
cmd/bootc-image-builder/image.go:240:50: img.KickstartKernelOptionsAppend undefined (type *"github.com/osbuild/images/pkg/image".AnacondaContainerInstaller has no field or method KickstartKernelOptionsAppend)
cmd/bootc-image-builder/image.go:242:7: img.KickstartNetworkOnBoot undefined (type *"github.com/osbuild/images/pkg/image".AnacondaContainerInstaller has no field or method KickstartNetworkOnBoot)
cmd/bootc-image-builder/image.go:246:14: img.Users undefined (type *"github.com/osbuild/images/pkg/image".AnacondaContainerInstaller has no field or method Users)
cmd/bootc-image-builder/image.go:246:36: img.Groups undefined (type *"github.com/osbuild/images/pkg/image".AnacondaContainerInstaller has no field or method Groups)

I have run into following boot loader related error:

Installing image: docker://quay.io/sacsant/fedora-bootc:latest
Initializing ostree layout
Initializing sysroot
ostree/deploy/default initialized as OSTree stateroot
Deploying container image
Deployment complete
Running bootupctl to install bootloader
Installing for powerpc-ieee1275 platform.
/usr/sbin/grub2-install: error: the chosen partition is not a PReP partition.
error: boot data installation failed: installing component BIOS: Failed to run "/usr/sbin/grub2-install" "--target" "powerpc-ieee1275" "--boot-directory" "/run/osbuild/mounts/boot" "--no-nvram" "/dev/loop0"
ERROR Installing to filesystem: Installing bootloader: Task Running bootupctl to install bootloader failed: ExitStatus(unix_wait_status(256))
Traceback (most recent call last):
File "/run/osbuild/bin/org.osbuild.bootc.install-to-filesystem", line 53, in
r = main(args["options"], args["inputs"], args["paths"])
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/run/osbuild/bin/org.osbuild.bootc.install-to-filesystem", line 48, in main
subprocess.run(pargs, env=env, check=True)
File "/usr/lib64/python3.12/subprocess.py", line 571, in run
raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['bootc', 'install', 'to-filesystem', '--source-imgref', 'containers-storage:[overlay@/run/osbuild/containers/storage+/run/containers/storage]b3a026083aa7f61628b5fdc3ab90b5b692139668ec583485d0f2e48fe34e46d9', '--skip-fetch-check', '--generic-image', '--karg', 'rw', '--karg', 'console=tty0', '--karg', 'console=ttyS0', '--target-imgref', 'quay.io/sacsant/fedora-bootc:latest', '/run/osbuild/mounts']' returned non-zero exit status 1.

Any suggestions on how to solve this error? I am unable to connect the dots and having trouble identifying code that needs a change to fix this.

[sacsant]

To be specific I am currently focussed on creating a qcow2 based bootable image.

podman run --rm -it --privileged --pull=newer --security-opt label=type:unconfined_t -v $(pwd)/config.toml:/config.toml -v $(pwd)/output:/output -v /var/lib/containers/storage:/var/lib/containers/storage quay.io/sacsant/bootc-image-builder:latest --type qcow2 --rootfs xfs --local quay.io/sacsant/fedora-bootc:latest

[cgwalters]

/usr/sbin/grub2-install: error: the chosen partition is not a PReP partition.
error: boot data installation failed: installing component BIOS: Failed to run "/usr/sbin/grub2-install" "--target" "powerpc-ieee1275" "--boot-directory" "/run/osbuild/mounts/boot" "--no-nvram" "/dev/loop0"
ERROR Installing to filesystem: Installing bootloader: Task Running bootupctl to install bootloader failed: ExitStatus(unix_wait_status(256))

That's in https://github.com/coreos/bootupd/

[sacsant]

Thanks Colin for the pointer. Since I am completely new to this project/code, I am still unable to connect the dots.

Should I use the coreos/bootupd code as reference and make similar changes to other repositories (bootc-image-builder or images or perhaps bootc??) or should I change coreos/bootupd? Can you guide me on this?

I noticed that coreos/bootupd uses powerpc64 as a value to identify IBM Power architecture while other projects like bootc-image-builder and likes use ppc64le as a value. Is there a theory behind this difference ?

[mkumatag]

I noticed that coreos/bootupd uses powerpc64 as a value to identify IBM Power architecture while other projects like bootc-image-builder and likes use ppc64le as a value. Is there a theory behind this difference ?

This is based on how your programming language detects the platform

e.g:
golang

# go env GOARCH
ppc64le      <=============================

rust:

# rustc --target=powerpc64le-unknown-linux-gnu --print=cfg
debug_assertions
panic="unwind"
target_abi=""
target_arch="powerpc64"             <=============================
target_endian="little"
target_env="gnu"
target_family="unix"
target_has_atomic="16"
target_has_atomic="32"
target_has_atomic="64"
target_has_atomic="8"
target_has_atomic="ptr"
target_os="linux"
target_pointer_width="64"
target_vendor="unknown"
unix

[cgwalters]

What's extra fun here is that it's actually only for ppc64le on Fedora derivatives that the "Rust vs RPM" architecture naming is different. Rust=powerpc64 RPM=ppc64le whereas they match for s390x, x86_64 and aarch64. Note that the Go and RPM names are otherwise different for amd64 vs x86_64 etc.

[sacsant]

Going back to the original error of "Unsupported architecture: powerpc64" , I added following code in bootc
} else if cfg!(target_arch = "powerpc64") { // BIOS-BOOT sgdisk_partition( &mut sgdisk.cmd, 1, "0:+4M", "BIOS-BOOT", Some("9E1A2D38-C612-4316-AA26-8B49521E5A8B"), );

Is the above code correct?

With this code using install to-disk command the operation proceeds further but fails eventually with

DEBUG exec: "sgdisk" "-Z" "/dev/sdb" "-U" "R" "-n" "1:0:+4M" "-c" "1:BIOS-BOOT" "-t" "1:9E1A2D38-C612-4316-AA26-8B49521E5A8B" "-n" "3:0:0" "-c" "3:root" "-t" "3:0FC63DAF-8483-4772-8E79-3D69D8477DE4"
[14381.355893] sdb: sdb1 sdb3
DEBUG Created partition table
[14381.356751] sdb: sdb1 sdb3
DEBUG argv0=Some("exe")
ERROR Installing to disk: Creating rootfs: Missing partition for index 3

Following partition layout was written to the disk
Device Start End Sectors Size Type
/dev/sdb1 256 1279 1024 4M PowerPC PReP boot
/dev/sdb3 1280 26214394 26213115 100G Linux filesystem

What partition is missing and is referenced by index 3? I guess there should be /dev/sdb2 pointing to /boot . Can someone help decode this error?

[sacsant]

What partition is missing and is referenced by index 3? I guess there should be /dev/sdb2 pointing to /boot . Can someone help decode this error?

I was able to modify the code to move past this issue.

Using block setup: direct
Initializing partitions
Creating boot filesystem (xfs) on device /dev/sdb2 (size=510M)

mkfs.xfs -m uuid=e14c7dd8-4297-4cb4-b41b-eb027167d941 -L boot /dev/sdb2
Creating root filesystem (xfs) on device /dev/sdb3 (size=99.5G)
mkfs.xfs -m uuid=ad5cdc91-0ca9-4cf5-afda-5aeaf53b1370 -L root /dev/sdb3
Mounting /run/bootc/mounts/rootfs
Mounting /run/bootc/mounts/rootfs/boot
Initializing ostree layout
Initializing sysroot
ostree/deploy/default initialized as OSTree stateroot
Deploying container image...Freed objects: 118 bytes
done
Running bootupctl to install bootloader

Now both the methods qcow2 and install to-disk fail during boot loader install

Running bootupctl to install bootloader

bootupctl backend install --write-uuid --update-firmware --auto --device /dev/sdb /run/bootc/mounts/rootfs
Installing for powerpc-ieee1275 platform.
/usr/sbin/grub2-install: error: the chosen partition is not a PReP partition.
error: boot data installation failed: installing component BIOS: Failed to run "/usr/sbin/grub2-install" "--target" "powerpc-ieee1275" "--boot-directory" "/run/bootc/mounts/rootfs/boot" "--no-nvram" "/dev/sdb"
ERROR Installing to disk: Installing bootloader: Task Running bootupctl to install bootloader failed: ExitStatus(unix_wait_status(256))

[sacsant]

With some additional changes (read hard coded values) I was able to successfully use bootc install to-disk and to-filesystem commands on ppc64le architecture. I was also able to successfully verify boot of these images (disk device as well as qcow2) on IBM Power.

[cgwalters]

Cool! Feel free to send a PR, it's OK if it's not polished, we can iterate on it!

[sacsant]

I have submitted following pull requests for initial set of changes
PR 497 against bootc-image-builder
PR 636 against bootc

Working on following changes:

• images : Remove dependency on /boot/efi for ppc64le arch
• bootc-image-builder compile time failure

cmd/bootc-image-builder/image.go:237:7: img.Users undefined (type *"github.com/osbuild/images/pkg/image".AnacondaContainerInstaller has no field or method Users)
cmd/bootc-image-builder/image.go:238:7: img.Groups undefined (type *"github.com/osbuild/images/pkg/image".AnacondaContainerInstaller has no field or method Groups)
cmd/bootc-image-builder/image.go:240:50: img.KickstartKernelOptionsAppend undefined (type *"github.com/osbuild/images/pkg/image".AnacondaContainerInstaller has no field or method KickstartKernelOptionsAppend)
cmd/bootc-image-builder/image.go:242:7: img.KickstartNetworkOnBoot undefined (type *"github.com/osbuild/images/pkg/image".AnacondaContainerInstaller has no field or method KickstartNetworkOnBoot)
cmd/bootc-image-builder/image.go:246:14: img.Users undefined (type *"github.com/osbuild/images/pkg/image".AnacondaContainerInstaller has no field or method Users)
cmd/bootc-image-builder/image.go:246:36: img.Groups undefined (type *"github.com/osbuild/images/pkg/image".AnacondaContainerInstaller has no field or method Groups)

• bootupd changes

[yoheiueda]

Hi @sacsant

I am working on s390x support, and I just raised this PR osbuild/images#758

images : Remove dependency on /boot/efi for ppc64le arch

I think my patch may be relevant to this issue.