7 ms·
The fun thing about learning to boot from PXE, is that you have to learn it every time you onboard a new type of hardware... or a new VM hypervisor... or new NI
by zorlack 10mo ago
The fun thing about learning to boot from PXE, is that you have to learn it every time you onboard a new type of hardware... or a new VM hypervisor... or new NIC firmware... or new BIOS firmware.
God help you if you actually want to install an operating system.
PXE is such a vital capability for working with on-prem servers. But it's ten different things which all have to play nicely together. Every time I build a PXE system I feel like I'm reinventing the universe in my tiny subnet.
- webdevver 10mo agowe need to go /stalinmode/ on the whole bootup and initialization industry subsector. it should be required by law for that stuff to be open source and documented. "but muh competitive advantage??" its literally a for loop that reads sectors from disk/network into memory and jumps to the start address. if a local build of the (vendor provided source code) firmware doesn't match the checksum of the build thats flashed on the actual mobo, you get sent to a cobalt mine.
- pjc50 10mo agoYou're getting downvotes for being hyperbolic about it, but boot integrity is really both a consumer safety and a national security issue.
- toast0 10mo agoBoot by committee (UEFI) doesn't seem much better than boot by fiat (BIOS). For everything nice it gives you, you lose something nice that BIOS gave you ... or you have something nice that you lose when you exit boot services. Or there's an extension for something nice that isn't usable on mainstream hardware. UEFI gives you nicer video modes, but not a text mode after boot services. UEFI has an extension for booting images from the network, but afaik, it's impossible to use, and there's no reasonable way to boot from a disk image; working UEFI network boot has to pull pieces out of the filesytem and present them seperately; as opposed to MEMDISK which makes the image available as a BIOS disk and the image is labeled so that one the OS is loaded, the image can be used without BIOS hooks. If this is possible on UEFI generally, it isn't widely distributed knowledge. Something that will work on any UEFI system that makes it to iPXE, subject to changes to the OS in the image (which is reasonable... MEMDISK needs changes too, unless the OS runs all disk I/O through BIOS APIs)
- dfe 10mo agoIt's been a long while since I did anything with UEFI, but my recollection is that the standard data structures are reasonably well documented, especially when they are meant to be part of booting an OS I imagine that whatever UEFI extension implements loading a disk image over the network probably also implements some way of knowing where the sectors are in RAM so that the OS bootloader can choose to hand off access to the memory disk to the OS. Is such support implemented in any bootloaders? I have no idea. My guess is probably not because people would rather just have the bootloader use the available services to download the disk image itself.
- legooolas 10mo agoI've not found this at all -- PXE "just works" on legacy boot or UEFI for me. I've used it for years to install hosts via Foreman (https://theforeman.org/ https://theforeman.org/), as well as for personal stuff on my home network, and it's so much better than getting people to use USB sticks or whatever else!
- happyPersonR 10mo agoYeah in order to automate, you’ve gotta know something about what you’re automating. PXE is not different.
- generalizations 10mo agoI’m confused, are you talking about getting PXE enabled in the hardware, or customizing something about your PXE software for the new hardware?
- kasabali 10mo agoAll of 'em.
- zorlack 10mo agoThere's a lot of nonsense at every level. Especially when dealing with heterogenous infrastructure. Some NICs support http. Some NICs support tftp. Some NICs have enough memory for a big iPXE, other NICs don't. Some BMC systems make next-boot-to-lan easy, but not all. We almost always use iPXE in order to normalize our pxe environment before OS kickstart. There's a lot to it and quite a lot of little things that can go wrong. Oh, and every bit of it becomes critical infra.
- generalizations 10mo agoOk, that makes more sense. I'm used to iPXE, and I guess that quick bootstrap from PXE->iPXE bypasses a lot of the nonstandard weirdness.
- hardwaresofton 10mo agoAgreed, PXE seems ideal for provisioning things, but it's just too hard to use, especially when you're not on a network you fully control. I just want to start the computer, and have it download an immutable OS image from somewhere I decide (and supply a checksum for, etc). I don't want to set up TFTP or any of this other stuff. It feels like I should be able to just specify an IP (let's say) a checksum (maybe supply that information to the NIC directly somehow), and be off to the races after a reboot.
- convolvatron 10mo agoreplace the PXE stack with an OS installer written in UEFI. This bootload can be installed through a guest running on the host in the EFI partition, or possibly through PXE or direct UEFI http load. this allows you intermediate the boot process without coordinating with the administrative owner of the DHCP server, and is actually less janky than PXE
- hardwaresofton 10mo agoNot to sound incredibly lazy (because I am), but is there anything off the shelf that does this? Anyone doing something similar would also be great -- I saw some UEFI-in-Rust projects recently so maybe it's not too hard to hack through myself. That said, my original need for this was on Hetzner, and what I did instead was actually completely automate their reset setup (thankfully they have an API to that), so I have a solution, but I would much rather if I could prebake and load images much easier. I think these days Hetzner has much more UEFI support across it's dedicated server fleet. Side note: Every time I see tinkerbell and other relatively new PXE boot projecs and it's set of tools I feel relief, then dig in, then feel dread again around iPXE. Side note 2: I really want to do two things: 1. Load OSes from the network easily 2. Run the OS in RAM (ECC) I feel like it would take my server management experience to the next level -- I've spent an inordinate amount of time messing with Hetzner USB add-ons and Alpine to try to get things to work, but it wasn't reliable (it worked, but wasn't reliable).
- ahepp 10mo ago