6 ms·

bootc-image-builder: Build your entire OS from a Containerfile

twelvenmonkeys 1y ago
We also have a GUI for trying this out! https://github.com/podman-desktop/extension-bootc https://github.com/podman-desktop/extension-bootc We’re also starting to see other projects adopt a “OS as a Container image” such as Bazzite: https://bazzite.gg/ https://bazzite.gg/ using bootc :) Feel free to ask any questions!
- ChocolateGod 1y ago
  Why swap from the OSTree storage to OCI? Doesn't that negate the space saving offered by OSTree having a content addressable store.
  - jeffro_rh 1y ago
    By using zstd:chunked, we get those atomic diffs at each layer using an enabled container registry. So diffs are still over the wire.
tt726259 1y ago
You can also achieve this with your current system > nix-build '<nixpkgs/nixos>' -A vm -I nixpkgs=channel:nixos-25.05 -I nixos-config=./configuration.nix I use nixos btw
- indigodaddy 1y ago
  Can this do vmdk format?
  - jchw 1y ago
    I don't know the answer using the built-in VM attributes (I mean I'd guess probably, but I don't know how if so) but there's always nixos-generators for making VM images. Definitely used this for deploying VMs to cloud providers, haven't tried the VMWare one yet though. https://github.com/nix-community/nixos-generators https://github.com/nix-community/nixos-generators
  - zimbatm 1y ago
    Yes, you can target VMDK, AMIs, Azure, ... `nixos-rebuild build-image --image-variant vmware` See https://nixos.org/manual/nixos/stable/#sec-image-nixos-rebuild-build-image https://nixos.org/manual/nixos/stable/#sec-image-nixos-rebui...
yjftsjthsd-h 1y ago
> A container for deploying bootable container images. ...as long as the images are in the Red Hat family (Fedora, CentOS Stream, RHEL).
- tayo42 1y ago
  Is there something about this makes it red hat specific. An OS is just a specific collection of files in the end. Whether things are installed with rpm or Deb shouldn't matter?
  - yjftsjthsd-h 1y ago
    You'd think so:) Unfortunately the current implementation hardcodes calls to dnf: https://github.com/osbuild/bootc-image-builder/issues/869 https://github.com/osbuild/bootc-image-builder/issues/869
    - deleted 1y ago
      [deleted]
- jeffro_rh 1y ago
  Ublue also builds Ubuntu versions of a lot of this.
- ethan_smith 1y ago
  The project roadmap actually includes plans to expand beyond Red Hat family distributions - there's active work to add support for Debian/Ubuntu and potentially other distros.
- whs 1y ago
  I was going to try this to perhaps use it in production. Turns out the RHEL clones like Alma or Rocky doesn't have this thing in production-ready grade. All options you have now are owned by Red Hat themselves.
  - jeffro_rh 1y ago
westurner 1y ago
Does bootc-image-builder build Native Containers? Do Native Containers work as VM images that can be stored in an OCI Image/Artifact/Package Registry? I've been mentioning Native Containers since I realized that was how bazzite works now. Is vagrant necessary anymore if host, vm, and container images can all be signed and stored in an OCI Image store? From https://news.ycombinator.com/item?id=44137501 https://news.ycombinator.com/item?id=44137501 re: Firecracker and Microsandbox VMs : > ostree native containers are bootable host images that can also be built and signed with a SLSA provenance attestation; https://coreos.github.io/rpm-ostree/container/ https://coreos.github.io/rpm-ostree/container/ ublue-os/image-template: https://github.com/ublue-os/image-template https://github.com/ublue-os/image-template : > Build your own custom Universal Blue Image ublue-os/akmods has nvidia GPU drivers, nvidia-open, zfs: https://github.com/ublue-os/akmods https://github.com/ublue-os/akmods : > A caching layer for pre-built Fedora akmod RPMs > OCI images providing a set of cached kernel RPMs and extra kernel modules to Universal Blue images. Used for better hardware support and consistent build process. nvidia-container-toolkit (CDI) is necessary for --gpus=all to do CUDA and libEGL 3D with podman. Is this also already installed in bazzite? ublue-os/toolboxes: "quadlets and systemd service units for management", boxkit : https://github.com/ublue-os/toolboxes#images https://github.com/ublue-os/toolboxes#images ublue-os/devcontainer .devcontainer/devcontainer.json: https://github.com/ublue-os/devcontainer/blob/main/src/base/.devcontainer/devcontainer.json https://github.com/ublue-os/devcontainer/blob/main/src/base/... It looks like the Just Justfile 40-nvidia.just has moved due to image topology simplification? https://news.ycombinator.com/item?id=39364975 https://news.ycombinator.com/item?id=39364975 : > ublue-os/config//build/ublue-os-just/40-nvidia.just defines the `ujust configure-nvidia` and `ujust toggle-nvk` commands
- lotharcable 1y ago
  What does "native containers" mean in this context?
  - westurner 1y ago
    > ostree native containers are bootable host images that can also be built and signed with a SLSA provenance attestation From https://coreos.github.io/rpm-ostree/container/#ostree-native-containers https://coreos.github.io/rpm-ostree/container/#ostree-native... : > rpm-ostree inherits work in ostree-rs-ext to create “container native ostree” functionality. This elevates OCI/docker containers to be natively supported as a transport mechanism for bootable operating systems. I think it means simplification of complexity and unnecessary re-duplication.
indigodaddy 1y ago
Huh, this is kinda wild. So for esxi images, this would seem to beat/potentially be simpler than the traditional Packer + interacting with an ISO on esxi infra, yes?
- eraser215 1y ago
  Arguably yes. I think the big improvement is that an upgrade is really just switching from image A to image B, rather than dozens to hundreds of individual package transactions. Furthermore parts of the system are fully mutable (e.g. /etc) allowing you to run automation against a system post install for more customisation.
nullify88 1y ago
I've been very excited on progress on bootc. I've tried to make my own coreos distro and its quite complicated in comparison. I've used this to start from a minimal base and added what I've needed on top. Best of all, updates are delivered via a container registry.
tmaier 1y ago
Universal Blue (Bluefin etc.) has a reusable GitHub template. https://github.com/ublue-os/image-template https://github.com/ublue-os/image-template
- eraser215 1y ago
  ... and it works fabulously. I have been running Bluefin (same folks as Bazzite) from one of these templates for about 6 months and it has been a near on flawless experience. I have moved from Fedora 40->41->42 without having to touch a traditional "upgrade". https://projectbluefin.io/ https://projectbluefin.io/
hardwaresofton 1y ago
I wonder which gets more actual usage, this project or linuxkit. Does anyone have experience worth sharing with both?
- Wuzado 1y ago
  If I had to wager a guess, bootc might get more actual use now that it's supported in RHEL 9.6 and 10 as "image mode". It's an exciting piece of technology, especially from the perspective of a platform engineer. Also, bootc is a basis for the Universal Blue family of distros, especially Bazzite, which is very popular with gamers.
  - hardwaresofton 1y ago
    yeah you're probably right -- going forward the usage is likely going to be a lot higher, at the very least. I thought of the underlying tech for those other distros being ostree more than anything but this is the better interpretation.
Kudos 1y ago
I've used this to bootstrap bootc-based Fedora on my workstations. I've got a CI job that builds updated container images every night, a simple `rpm-ostree upgrade` pulls in the new image and `systemctl reboot` activates it. What I like about this is always having a known working image I can quickly swap to, particularly for the machine with an nvidia card.
franga2000 1y ago
I'd love to have something like this for embedded system images, like for Raspberry Pi deployments.
- eraser215 1y ago
  Totally. Appliances are perfect candidates for this tech.
rgovostes 1y ago
Roman Shtylman has an example of using a Dockerfile to produce a rootfs for the Jetson Nano: https://github.com/defunctzombie/jetson-nano-image-maker https://github.com/defunctzombie/jetson-nano-image-maker (2022) I've always been hesitant to use this method over debootstrap: the Ubuntu container images ("FROM ubuntu:20.04") are created from a tarball that Ubuntu's convoluted CI system spits out and I'm not confident I understand if it's somehow suitable only for a container and not for real hardware.
- ValdikSS 1y ago
  The alternative is mkosi from systemd developers https://github.com/systemd/mkosi https://github.com/systemd/mkosi However beware that they break backwards compatibility almost every 6 months. This is probably the most backwards-incompable project I know, you can't rely that the minor version update won't break your projects.

We also have a GUI for trying this out! https://github.com/podman-desktop/extension-bootc https://github.com/podman-desktop/extension-bootc We’re also starting to see other projects adopt a “OS as a Container image” such as Bazzite: https://bazzite.gg/ https://bazzite.gg/ using bootc :) Feel free to ask any questions!

ChocolateGod 1y ago
Why swap from the OSTree storage to OCI? Doesn't that negate the space saving offered by OSTree having a content addressable store.
- jeffro_rh 1y ago
  By using zstd:chunked, we get those atomic diffs at each layer using an enabled container registry. So diffs are still over the wire.

tt726259 1y ago

You can also achieve this with your current system > nix-build '<nixpkgs/nixos>' -A vm -I nixpkgs=channel:nixos-25.05 -I nixos-config=./configuration.nix I use nixos btw

indigodaddy 1y ago
Can this do vmdk format?
- jchw 1y ago
  I don't know the answer using the built-in VM attributes (I mean I'd guess probably, but I don't know how if so) but there's always nixos-generators for making VM images. Definitely used this for deploying VMs to cloud providers, haven't tried the VMWare one yet though. https://github.com/nix-community/nixos-generators https://github.com/nix-community/nixos-generators
- zimbatm 1y ago
  Yes, you can target VMDK, AMIs, Azure, ... `nixos-rebuild build-image --image-variant vmware` See https://nixos.org/manual/nixos/stable/#sec-image-nixos-rebuild-build-image https://nixos.org/manual/nixos/stable/#sec-image-nixos-rebui...

yjftsjthsd-h 1y ago

> A container for deploying bootable container images. ...as long as the images are in the Red Hat family (Fedora, CentOS Stream, RHEL).

tayo42 1y ago
Is there something about this makes it red hat specific. An OS is just a specific collection of files in the end. Whether things are installed with rpm or Deb shouldn't matter?
- yjftsjthsd-h 1y ago
  You'd think so:) Unfortunately the current implementation hardcodes calls to dnf: https://github.com/osbuild/bootc-image-builder/issues/869 https://github.com/osbuild/bootc-image-builder/issues/869
  - deleted 1y ago
    [deleted]
jeffro_rh 1y ago
Ublue also builds Ubuntu versions of a lot of this.
ethan_smith 1y ago
The project roadmap actually includes plans to expand beyond Red Hat family distributions - there's active work to add support for Debian/Ubuntu and potentially other distros.
whs 1y ago
I was going to try this to perhaps use it in production. Turns out the RHEL clones like Alma or Rocky doesn't have this thing in production-ready grade. All options you have now are owned by Red Hat themselves.
- jeffro_rh 1y ago

westurner 1y ago

Does bootc-image-builder build Native Containers? Do Native Containers work as VM images that can be stored in an OCI Image/Artifact/Package Registry? I've been mentioning Native Containers since I realized that was how bazzite works now. Is vagrant necessary anymore if host, vm, and container images can all be signed and stored in an OCI Image store? From https://news.ycombinator.com/item?id=44137501 https://news.ycombinator.com/item?id=44137501 re: Firecracker and Microsandbox VMs : > ostree native containers are bootable host images that can also be built and signed with a SLSA provenance attestation; https://coreos.github.io/rpm-ostree/container/ https://coreos.github.io/rpm-ostree/container/ ublue-os/image-template: https://github.com/ublue-os/image-template https://github.com/ublue-os/image-template : > Build your own custom Universal Blue Image ublue-os/akmods has nvidia GPU drivers, nvidia-open, zfs: https://github.com/ublue-os/akmods https://github.com/ublue-os/akmods : > A caching layer for pre-built Fedora akmod RPMs > OCI images providing a set of cached kernel RPMs and extra kernel modules to Universal Blue images. Used for better hardware support and consistent build process. nvidia-container-toolkit (CDI) is necessary for --gpus=all to do CUDA and libEGL 3D with podman. Is this also already installed in bazzite? ublue-os/toolboxes: "quadlets and systemd service units for management", boxkit : https://github.com/ublue-os/toolboxes#images https://github.com/ublue-os/toolboxes#images ublue-os/devcontainer .devcontainer/devcontainer.json: https://github.com/ublue-os/devcontainer/blob/main/src/base/.devcontainer/devcontainer.json https://github.com/ublue-os/devcontainer/blob/main/src/base/... It looks like the Just Justfile 40-nvidia.just has moved due to image topology simplification? https://news.ycombinator.com/item?id=39364975 https://news.ycombinator.com/item?id=39364975 : > ublue-os/config//build/ublue-os-just/40-nvidia.just defines the `ujust configure-nvidia` and `ujust toggle-nvk` commands

lotharcable 1y ago
What does "native containers" mean in this context?
- westurner 1y ago
  > ostree native containers are bootable host images that can also be built and signed with a SLSA provenance attestation From https://coreos.github.io/rpm-ostree/container/#ostree-native-containers https://coreos.github.io/rpm-ostree/container/#ostree-native... : > rpm-ostree inherits work in ostree-rs-ext to create “container native ostree” functionality. This elevates OCI/docker containers to be natively supported as a transport mechanism for bootable operating systems. I think it means simplification of complexity and unnecessary re-duplication.

indigodaddy 1y ago

Huh, this is kinda wild. So for esxi images, this would seem to beat/potentially be simpler than the traditional Packer + interacting with an ISO on esxi infra, yes?

eraser215 1y ago
Arguably yes. I think the big improvement is that an upgrade is really just switching from image A to image B, rather than dozens to hundreds of individual package transactions. Furthermore parts of the system are fully mutable (e.g. /etc) allowing you to run automation against a system post install for more customisation.

nullify88 1y ago

I've been very excited on progress on bootc. I've tried to make my own coreos distro and its quite complicated in comparison. I've used this to start from a minimal base and added what I've needed on top. Best of all, updates are delivered via a container registry.

tmaier 1y ago

Universal Blue (Bluefin etc.) has a reusable GitHub template. https://github.com/ublue-os/image-template https://github.com/ublue-os/image-template

eraser215 1y ago
... and it works fabulously. I have been running Bluefin (same folks as Bazzite) from one of these templates for about 6 months and it has been a near on flawless experience. I have moved from Fedora 40->41->42 without having to touch a traditional "upgrade". https://projectbluefin.io/ https://projectbluefin.io/

hardwaresofton 1y ago

I wonder which gets more actual usage, this project or linuxkit. Does anyone have experience worth sharing with both?

Wuzado 1y ago
If I had to wager a guess, bootc might get more actual use now that it's supported in RHEL 9.6 and 10 as "image mode". It's an exciting piece of technology, especially from the perspective of a platform engineer. Also, bootc is a basis for the Universal Blue family of distros, especially Bazzite, which is very popular with gamers.
- hardwaresofton 1y ago
  yeah you're probably right -- going forward the usage is likely going to be a lot higher, at the very least. I thought of the underlying tech for those other distros being ostree more than anything but this is the better interpretation.

Kudos 1y ago

I've used this to bootstrap bootc-based Fedora on my workstations. I've got a CI job that builds updated container images every night, a simple `rpm-ostree upgrade` pulls in the new image and `systemctl reboot` activates it. What I like about this is always having a known working image I can quickly swap to, particularly for the machine with an nvidia card.

franga2000 1y ago

I'd love to have something like this for embedded system images, like for Raspberry Pi deployments.

eraser215 1y ago
Totally. Appliances are perfect candidates for this tech.

rgovostes 1y ago

Roman Shtylman has an example of using a Dockerfile to produce a rootfs for the Jetson Nano: https://github.com/defunctzombie/jetson-nano-image-maker https://github.com/defunctzombie/jetson-nano-image-maker (2022) I've always been hesitant to use this method over debootstrap: the Ubuntu container images ("FROM ubuntu:20.04") are created from a tarball that Ubuntu's convoluted CI system spits out and I'm not confident I understand if it's somehow suitable only for a container and not for real hardware.

ValdikSS 1y ago
The alternative is mkosi from systemd developers https://github.com/systemd/mkosi https://github.com/systemd/mkosi However beware that they break backwards compatibility almost every 6 months. This is probably the most backwards-incompable project I know, you can't rely that the minor version update won't break your projects.