How does Linux handle testing? They don't seem to have a CI system as far as I know. Presumably there's no big lab with automated testing on real hardware. Does anyone know how releases are tested?
Several companies (including mine, Toradex) have setup board farms to deal with Kernel regressions and bugs during the `-rc` window, ie, the kernel that is going to be released is heavily tested.
Take a machine, set it up to network boot, plug it in to a ZigBee-controlled plug. Every time you have a new version, a script bumps the netboot kernel and power cycles the machine, then runs tests. Those tests include "does it boot and respond to ssh", but also "can it run render jobs on the GPU", "does KDE start and if we connect over VNC did it render correctly", "do NFS performance tests show expected numbers", etc. Ideally, this box has a lot of hardware plugged in and those tests exercise all the drivers you can think of. For bonus points, scale horizontally; each kernel version can get booted on a Dell server and a Raspberry Pi and a Macbook and a Raptor POWER workstation and a Steam Deck (probably with battery removed) and [...], and all of these can netboot and run tests in parallel.
Honestly, I'd love to set up this kind of thing, but I've not seen a job for it that doesn't need C expertise:(
