
Grub2 security update renders system unbootable - beefhash
https://bugzilla.redhat.com/show_bug.cgi?id=1861977
======
peteri
I assume this is related to this from yesterday
[https://news.ycombinator.com/item?id=23990075](https://news.ycombinator.com/item?id=23990075)
Which is about revoking secure boot keys

------
Wowfunhappy
There was a story on HN last night from Debian where they laid out this issue,
and basically stated "Yes, this security update is going to render some
systems unbootable, here is why we're doing it anyway."

[https://www.debian.org/security/2020-GRUB-UEFI-
SecureBoot/](https://www.debian.org/security/2020-GRUB-UEFI-SecureBoot/)

Stability is important, especially when it comes to unbootable machines—but I
don't quite know what anyone was supposed to do here. If a user has secure
boot enabled, the OS has to assume that the user wants/needs security at that
level of the chain—and it is therefor responsible for ensuring the chain's
integrity. In this case, there was no way to do that without some machines
(temporarily) failing to boot.

What would have been a better way to handle this?

~~~
RedShift1
Is there proof that secure boot actually prevented an attack?

~~~
g_p
Secure boot has, for the most part, prevented the underlying system from being
compromised by malware that can run before boot (and therefore hide itself
from the OS). I don't think we can really prove a negative here, but it means
no more "boot sector" viruses, either on the HDD itself, or on external
bootable media. That certainly is a good first step to preventing foot canon
incidents, and giving the OS a decent chance at preventing malware running
before the OS has control of the system.

Couple in, as another post said, a tpm which only releases the decryption key
if the system is in a valid state with secure boot running, it helps a lot
with basic cold boot protection. TPM is far from perfect, but it is arguably
better than a user not encrypting their disk at all, and this prevents attacks
like replacing the utilman.exe with cmd.exe etc.

------
kd913
I will state the same comment as last time.

Can distros maybe consider moving to systemd-boot at some point? Systemd is
already built in and can handle things like mounting pretty easily and simply.

It is a hell of a lot leaner than grub, doesn't use a billion superfluous
modules. That and it is a lot easier to prevent tampering compared with the
cumbersome nonsense that is grub passwords.

Oh and it enables distros to gather accurate boot times and enables booting
into UEFI direct from the desktop.

It works with secureboot/shim/Hashtool. Also each distro has it's bootloader
entries in separate folders to avoid accidental conflicts.

~~~
dijit
systemd-boot (AKA, gummiboot) is leaner, for sure- much in the same way a
bicycle is leaner than a truck.

It might surprise you to know this but systemd-boot does not support, among
other things: "BIOS"/MBR boot, EXT4, XFS, mdraid, LUKS (hidden boot), OPAL
(aka hardware FDE, self-encrypting drives) or even btrfs!

mdraid being pretty critical to a lot of server work.

Then there's the topic of altering bootloaders for distro's which value
stability, it wouldn't happen in a point release.

~~~
Spivak
I feel like you're making this sound worse than it actually is

* systemd-boot doesn't support those filesystems as the /boot partition, all of those things for your / are totally fine. For most people having your /boot partition be XFS vs FAT32 is firmly in the "who cares?" realm.

* Nobody really supports OPAL, even grub. The only reliable solution is using [https://github.com/sedutil/sedutil](https://github.com/sedutil/sedutil) to unlock the disk and then soft reboot into any normal bootloader.

* You can do mdraid, you just have to stick the metadata at the end of the drive rather than the beginning. The utilities that set up your raid even warn you about this because this setup is so standard.

~~~
dijit
> systemd-boot doesn't support those filesystems as the /boot partition, all
> of those things for your / are totally fine. For most people having your
> /boot partition be XFS vs FAT32 is firmly in the "who cares?" realm.

XFS is also not supported. But yes, your /boot can be fat32, but saying
"nobody cares" is just needlessly dismissive, I can think of many reasons to
care at least a little. Primary of which is lack of encryption and ability to
mirror easily, another can be the possible corruption of /boot (FAT is not
typically used for it's data consistency properties)

> Nobody really supports OPAL, even grub. The only reliable solution is using
> [https://github.com/sedutil/sedutil](https://github.com/sedutil/sedutil) to
> unlock the disk and then soft reboot into any normal bootloader.

I guess this is a fair statement, if you ignore that the "PBA" is actually
using grub itself, but yes, the PBA is non-encrypted and usually lives as a
separate partition, making it functionally equivalent between systemd-boot and
grub2.

> You can do mdraid, you just have to stick the metadata at the end of the
> drive rather than the beginning. The utilities that set up your raid even
> warn you about this because this setup is so standard.

You can't boot to an mdraid device with systemd-boot, you can only have a
separate (non mirrored or raid'd) partition on a drive, and point to that
drive to boot.

~~~
Spivak
> You can't boot to an mdraid device with systemd-boot, you can only have a
> separate (non mirrored or raid'd) partition on a drive, and point to that
> drive to boot.

Sorta kinda true in theory but not in practice. Yes systemd-boot isn't raid
aware but if you configure mdraid to store its metadata at the end of the
drive then you wind up with a, say, mirrored disk that to the firmware doesn't
look like it's mirrored at all. And since the firmware just reads the
partition it's safe and works.

So yes, you have to configure your firmware to look at both drives in a
mirrored setup as a fallback but you can boot.

> but saying "nobody cares" is just needlessly dismissive

Fair, I'm more saying that having firmware that only supports a small set of
simple filesystems is the usually the more normal situation, the fact that
GRUB comes with all the drivers makes it the weird one.

------
jacquesm
If you are wondering why many people _hate_ updating working systems, no
matter what the security implications, look no further than this.

Time and again it is an innocent security update that would end up in a
reinstall, finding a bunch of bloat ware on a system, losing critical
functionality, data loss and time lost.

Updates should be restricted to the absolute minimum and tested to the point
that deploying them does not put customer data at risk.

~~~
tanzann
What can you recommend as a balanced linux distro between the constant updates
of arch on one hand and mostly very outdated package versions of debian/centos
on the other?

~~~
_ph_
There is of course Fedora. The latest is quite cutting edge, but you can
choose when to upgrade to the next release. And there are security updates for
older releases, so you don't have to immediately upgrade. Still, keeps you in
the Red Hat universe.

~~~
freedomben
This is exactly why I settled on Fedora after a few years on Arch, a few on
Ubuntu/Debian/etc. Fedora is the best blend I've found. New releases about
twice a year but you can skip a release if you want and upgrade every other
release (or always stay on release n-1 which some people I know do).

Arch and Fedora follow the same conventions so everything on Fedora is where
you expect it from Arch (this also makes much of the great Arch wiki
applicable directly).

There will always be a soft spot in my heart for Arch, but Fedora is the
perfect balance for me. I can't have my work machine getting broken and
forcing me to read forums and mailing lists, etc.

------
znpy
The stablest systems I had the pleasure to manage were two identical rhel6
clusters into different geographical locations for higher availability and
fault tolerance. Such systems were installed, turned on and never touched
again. Kernel 2.6.32 that managed about six-seven years of uptime up to
mid-2019. We operated a lot onto such systems, mounting and unmounting iscsi
devices, starting and stopping stuff, turning on and off network interfaces
and clustered filesystem (thanks to Veritas cluster manager).

The key move was never updating.

Such systems were literally mission critical, without that cluster the whole
company was unable to produce its main products.

Considering how much stuff they ran and how many simultaneous users were
connected, I was humbled by their stability (and by rhel's stability).

If you're getting angry at this post: the customer was not in the it field and
was completely okay with buying new hardware and doing a full reinstall every
X years.

~~~
MaxBarraclough
Usually security is one of the top reasons to update software systems. Would a
security flaw in these clusters have been an issue? Were they aggressively
firewalled?

~~~
znpy
Yes, there were multiple firewall layers. Also, services were not accessed
from outside the corporate network.

------
psanford
The link is to a redhat bug report, but the issue is also affecting other
distros:
[https://bugs.launchpad.net/ubuntu/+source/grub2/+bug/1889509](https://bugs.launchpad.net/ubuntu/+source/grub2/+bug/1889509)

------
awill
This is why I dislike grub. It's really, really bloated. A bootloader just
needs to pick the partition to boot, and little else. I switched to gummiboot
ages ago, and it's so simple. There's far less to go wrong (gummiboot got
absorbed by systemd, so it's now called systemd-boot)

------
protomyth
Am I reading this correctly that "yum update" is all I have to do to screw up
an 8.2 minimal install?

~~~
fctorial
sudo yum update

------
jkingsbery
"This update enhances security by making the system unbootable, which is the
most secure a computer can be."

------
bjornedstrom
After reading this I decided to downgrade my Ubuntu machine for now until it's
figured out. There are instructions here:
[https://wiki.ubuntu.com/SecurityTeam/KnowledgeBase/GRUB2Secu...](https://wiki.ubuntu.com/SecurityTeam/KnowledgeBase/GRUB2SecureBootBypass)
under the heading "DOWNGRADE `GRUB2`/`GRUB2-SIGNED` TO THE PREVIOUS VERSION
FOR RECOVERY"

Under the heading is a small shell script that will download the old debs for
you. Note that for it to work and not have wget spam 404:s, you have to update
the entire GRUB2_LP_URL and GRUB2_SIGNED_LP_URL to the links in the little
table. At first glance it looks like you only have to change GRUB2_VERSION and
GRUB2_SIGNED_VERSION.

------
RedShift1
Grub2 is not a bootloader. I'm not even really sure what it is. In Grub 1 you
had a configuration file with the operating systems you wanted to boot.
Simple, effective. With grub2 all I'm seeing is a bunch of sh scripts that are
impossible to write by hand.

------
tmikaeld
Again? I remember when zfs didn't init in time so grub found no root. It was
fun panic when it's on a virtualization server..

------
sgt
On this subject, a few months ago I had Ubuntu servers suddenly failing due to
a bizarre automatic snap update that basically took most of our Docker
containers down. Did anyone experience this with production systems?

------
wtracy
Any word on whether the equivalent update on Debian has similar issues?

~~~
psanford
Yes, there are reports of failures on Debian:
[https://bugs.launchpad.net/ubuntu/+source/grub2/+bug/1889509...](https://bugs.launchpad.net/ubuntu/+source/grub2/+bug/1889509/comments/6)

------
gojomo
Well, a system that can't be booted is totally secure against many kinds of
remote & local attacks. So, "mission accomplished", I guess?

