Secondly, I think a lot of people have forgotten what things were like in the early 90's. Back then encryption software was still export controlled, so we couldn't put DES (AES wasn't released until 2001) into the kernel without triggering all sorts of very onerous US government restrictions (and Europe was guilty of this too, thanks to the Wassenaar Agreement); you couldn't just put things up on an FTP. Also, back then, there was much less public understanding of
cryptography; the NSA definitely knew a lot more about cryptoanalysis that in the public world, and while it was known by 1996 that MD5 had Problems, there wasn't huge trust that the NSA hadn't put a back door into SHA-1 which was designed by them with zero explanation about its design principles.
This is why the original PGP implementation, as well as the Linux Kernel random number generator, was very much focused on entropy estimation. We knew that it was potentially problematic, but then again, so was relying on cryptographic algorithms that were poitentially suspect. There was a good reason why in the 90's, it was generally considered a very good idea to be algorithm agile; there simply wasn't a lot of trust in crypto design, and people wnated to be able to swap out cryptographic algorithms if it was found that some algorithm (e.g., like MD4, and later MD5) was found to be insecure. So the snide comments about people not trusting algorithms seems to miss the point that even amongst the experts in the field --- for example, at the Security Area Directorate at the IETF, of which I was a member during that time --- there was a lot of thinking about how we could deploy upgrades if it were found that some crypto algorithm had a fatal weakness, and we would need to swap out crypto suites with minimal interoperability issues.
Unfortunately, being able to negotiate crypto suites leads to downgrade attacks, such as we've seen with TLS --- but what people forget is that when the original SSL/TLS algorithm suites were designed, people thought they were good! It was only later that some crypto suites were found to be insecure, leading to the downgrade attack issues. But it also shows that people were right to be skeptical about crpyto algorithms in that era.
Since then, we've learned a lot more about cryptographic algorithm design, and so people are a lot more confident that algorithms can be relied upon to be secure --- or, at least, other issues are much more likely to be weak link. That's why Wireguard is designed without any ability to negotiate algorithms, and as a result, it makes it much simpler than IPSEC. And it's probably the right choice for 2019. (At least, until Quantuum Computing wipes out most of our existing crypto algorithms; but that's a rant for another day.)
As far as monitoring entropy levels in Linux, in general, the primary reason why we need it is because even if we are willing to invest a lot of faith into the ChaCha20 CRNG, we still need to provide a secure random number seed from somewhere. And that can be tricky. If you fully trust a hardware random number generator, then sure, no worries. Or if you are using a cloud provider, so you have to trust the hypervisor anyway, then using virtio-rng to get randomness from the cloud provider is fine. (If you cloud provider wants to screw you, they can just reach into the guest memory or intercept network or disk traffic at boot time, so if you don't trust not to backdoor virtio-rng, you shouldn't be using the cloud provider at all.)
As far as whether or not to trust RDRAND, the blog post seems to assume that it's absurd to trust that NSA couldn't possibly have backdoored the CPU instruction. On the author hand, there are those who remember DUAL-EC-DRBG, where most people do now believe the NSA did put in a backdoor. And Snowden revelations did show that NSA teams were putting backdoors into Cisco routers by intercepting them between when they are shipped and when they were delivered. So given that you can't audit the Intel CPU's RDRAND, and Intel is a US company, it's not that insane to perhaps have some qualms about RDRAND. After all, if you were using a chip provided from a Chinese company (where the owner of said company miight also have been a high ranking general in the PLA), or a CPU provided by a Russian company controlled by a Russian Oligarch who is good friends with Putin and
who also had a background from the KGB --- is it really insane to be worried about those CPU's? Let's not even talk about concerns over China and 5G telephony equipment. Why is it then completely absurd for some people to be considered about the complete inauditability of RDRAND, and the fact that no functional or statistical test can determine whether or not there is a backdoor or not?
Of course, if you really don't trust a CPU, you should simply not use it. But creating a CPU from scratch, using only 74XX TTL chips really isn't a practical solution. (When I was an undergraduate at MIT, we did it as part of an intro CS class; but MIT doesn't make its CS students do that any more.) So the best we can try to do is to try to spread out the entropy sources; that way, even if source 1 might be compromised, if it is being mixed with source 2 and source 3, hopefully at least one of them is secure. (Or maybe source 1 is backdoored by the NSA, and the source 2 is backdoored by the Chinese MSS, but if we hash it all together, hopefully the result will only be vulnerability if the NSA and MSS work together, which hopefully is highly improbable.)
The bottom line is that it's complicated. Of course I agree that we should use a CRNG for most purposes. But we still have to figure out good ways of seeding a CRNG. And in case the kernel memory gets compromised and read by an attacker, or if there is a theoretical vulnerability in the CRNG, it's good practice to periodically reseed the CRNG. And so that means you still need to have an entropy pool and some way of measuring how much entropy you think you have accumulated, and how much has been possibly revealed ("used") for reseeding purposes. In a perfect world, of course, assuming that we had perfectly trustworthy hardware to get an initial seed, and in a world where we are 100% sure that algorithms are bug free(tm), then a lot of this isn't necessary. But we in real world engineering, we have safety margins because sometimes theory breaks down in the face of reality....
For the avoidance of doubt: I can confirm that I didn't add chacha20 to the kernel or have anything to do with that. I did move get_random_int() and get_random_long(), which are used for things like aslr and wireguard handshake ids, from some weird md5 horror to using Ted's chacha20-based rng (when rdrand is unavailable) a few years ago, but that's different from /dev/urandom. https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/lin... Ted is the one who added chacha20 for /dev/urandom.
Linux does have support for it, but you have to drag in the crypto system, which is optional, and it's a super-heavyweight and complex interface. Jason tried to simplify it for Wireguard, but ran into a lot of resistance, and he's now adding Wiregard with an interface layer to the crypto subsystem. He's still going to work on trying to add a simpler crypto interface, but a core principle of Linux's RNG is that it must always be present; I didn't want to make it an optional component that could was enbled or disabled at compile time. That means I couldn't rely on the crypto subsystem, even if I was willing to put up with its rather horrific interface. (There are some reasons for its complexity, but it adds no value to the random driver or Wireguard.)
In any case, if you really need more speed than the current ChaCha20 CRNG, you're doing something wrong. It's almost certainly not for cryptographic purposes. So if you do want that kind of speed, number one, you probably really don't need the security, and so a PRNG will always be faster. Or if you do need the security, grab a seed value using getrandom(2), and then implement a userspace CSRNG. (But I bet you really don't.)
> Let’s take this comment at face value and say Linux rng throughput is 180MB/sec: https://www.reddit.com/r/crypto/comments/ednj0x/comment/fbjr...
> Let’s take one of the more moderate types of AWS instances and suppose our max network speed is 10gbps = 1250MB/sec.
> Supposing we have an application that does not need to do any slow operations like reading from a database and is only performing very simple operations in memory (like, let’s say, generating a symmetric key), we see that the Linux rng is roughly an order of magnitude slower than our network throughout.
It's not, so far as I can tell, Pornin's claim that RNG design is mooted by RDRAND. I would be surprised to see Pornin argue for a system design that replaced a standard RNG design, with some kind of secret-unpredictable-event secure seed, with calls to RDRAND.
But your argument that it's sensible to be cautious about RDRAND because of Dual EC might itself be misleading. Dual EC is a complete design for a CSPRNG. RDRAND is a component of a CSPRNG. In every proposed system I've seen that uses RDRAND, RDRAND is one of several secret inputs to the RNG. I've never seen a design that chains Dual EC; it was surprising to see Dual EC even used, anywhere, because it is so comically expensive. Most of the BULLRUN revelations (and the subsequent Juniper fiasco) weren't new discoveries about Dual EC, but rather the discovery of systems not thought to be relying on Dual EC that were.
Even Bernstein doesn't really argue that RDRAND hurts security in the design context you're talking about; for the same reason that you say cloud users should trust virtio, a "snooping" RDRAND is simply out of the threat model.
But of course, RDRAND isn't really the point. Clearly Pornin doesn't think that haveged makes sense in systems without RDRAND. So what are you really saying here? That he's not charitable enough about the historical context you were working in?
Would you mind explaining https://blog.cr.yp.to/20140205-entropy.html then?
> Linux 5.3 will turn back getrandom() into /dev/urandom with its never-blocking behavior, because, quite frankly, Linus’s opinions on his own mastery of RNG theory exceed his actual abilities. The reasoning seems to be that if there is “not enough entropy”, then the application should make an interpretative dance of some kind to promote the appearance of new hardware events from which the entropy can be gathered. How the application does that, or why the kernel should not do it despite being much closer to the hardware, is not said
In this particular case, what triggered this was an optimization in ext4 in how we did directory readahead, which reduced the number of I/O's done during the boot sequence. Some user space program in early boot tried to call getrandom(2), and it blocked because with a smaller number of I/O's, for some hardware platforms, it would stop the boot sequence in its tracks, and this then resulted interrupt events, leading to no further activity, leading to an indefinite hang.
So what do we do? We could revert the ext4 optimization, but what it did was draw attention to the fact that in the absence of a hardware random number generator which everyone trusted, what we had in terms of CRNG initialization was fragile.
Now the blog posting is inaccurate here as well, it is not the application which needs to make an interpretative dance of some kind. We actually are doing it in the kernel, and there is at least some hope that on an x86, making some assumptions about how the caches and clocks work, it probably is secure. I am actually worried that it won't be secure enough on simpler RISC cores, such as ARM, RISC-V, MIPS, etc.
The real right answer is that we should be pulling from as many hardware random number generators (that are designed for crypto purposes) that are available, such as from the UEFI, or the TPM, and mix that in as well. We'll probably continue to have config options so that the person building the kernel can decide whether or not those will be trusted. That hasn't happened yet, but I've been trying to recruit some volunteers to implement this in UEFI boot, or using NERF or Coreboot, etc. Util we do, or for hardware that doesn't have trusted hwrng's, starting in 5.3, we now have an in-kernel interpretative dance which is the fallback, for better or worse.
I'm not super-fond of that, but it was better than the alternative, which was no interpretive dance, and simply having getrandom(2) return "randomness" regardless of whether or not we thought it was random or not. On modern x86 processors, we will be mixing in RDRAND, so if you trust RDRAND, you'll probably be OK, interpretative dance or not. But the big worry is going to be on simpler CPU's such as RISC-V.
Ultimately, there are no easy solutions here. Arguably, just gathering timing events during the boot was also an "interpretive dance", since how much uncertainty there really is from SSD operations, and whether the SSD is using a oscillator different from the one used by the CPU, etc., involves a certain amount of hand-waving. So the only real solution is real, carefully designed, hardware RNG's. But then the question is how an you be sure they are trustworthy? This conundrum has always been there.
For myself, I use a ChaosKey and make sure it is contributing to the entropy pool before I generate long-term public keys. Of course, can I be sure that the NSA hasn't intercepted my ChaosKey shipment and trojaned it, the way they did with Cisco Routers? Nope. I can only hope that I'm not important enough so that they wouldn't have bothered. :-)
Yeah, I was thinking the article looked inaccurate there. Thanks for confirming.
My understanding is that XORing a trusted seed source with any number of untrusted seed sources results in a trusted seed source. What would be the point of such kernel options?
For example, if you are an NSA employee, you might be utterly confident that the NSA didn't twist Intel's arms to put in a backdoor into RDRAND --- e.g., that it isn't AES(NSA_KEY, SEQ++) --- and even if it were, you would be sure that as a US citizen, you would be safe from intrusive attacks to spy on your communications without a FISA warrent, which of course would only be done with a super scrupulous attention to legal process, the recent IG report of the Carter Page FISA warrant to the contrary.
In 2019, if you are a Republican member of the House of Representatives, such as Devin Nunes, you might be sure that the FBI is playing fast and louse with all FISA warrants, and so if so no one is safe from politically motivated investigations, especially if you are working for the Trump campaign, such as Carter Page.
See? Two different people might have very different opinions about whether a particular source should be trusted or not.
So having options to remove some sources is pointless and can only make things worse, never better.
My apologies if I didn't understand the point you were making.
And yes, I would love to see us adding more entropy sources in the future. The problem is I don't have a lot of time, and a lot of other projects to work on, but I've been trying to recruit people to add support to pass entropy from UEFI into the kernel (which requires changes to Grub, NERF, Coreboot, etc.), being able to pass entropy from one kernel to the next when using kexec, etc. I can't do it all myself; this is something that requires a lot of people to make things better.
RDRAND is used, but not trusted by default (and there was a fault in AMD RDRAND with old un-patched bios). systemd applies a seed file generated during last boot, but again disables the option to credit the entropy pool, because it does not trust that this wasn't an improperly prepared and distributed VM image with the same seed file used over and over. There are lots of things that very likely contribute usable entropy, but the kernel can't know with absolute certainty that any particular one does not have some flaw. The final straw was ext4 optimizations and modern SSDs resulting in very few storage controller interrupts, and some programs using getrandom() during early boot, and boot locking up indefinitely for some users running the latest linux kernel, ultimately due to paranoia about the RNG entropy sources.
So if you have to choose between failing to boot up, or just giving the best random numbers you can despite lack of certainty/guarantee of sources of entropy, Linus prefers to not fail to boot up. I know that OpenBSD trusts that seed file, which Linux+systemd uses but does not trust, and I'm not sure what macOS and Windows do, but they probably trust RDRAND, which Linux also uses but does not trust. (My personal fix is to trust rdrand, there's a kernel cmdline option for it.)
Could systemd choose to read a seed file after it mounts the root file system? Sure, but then it's on systemd to believe that the seed file is always secure, even on IOT devices where the seed file might be imaged onto millions of devices with an identical value out of the box. Using a seed file means you also have to trust how the OS is installed; security is a holistic property involving the entire system's design and implementation.
Ultimately, it's not going to be up to kernel or systemd to tell users and system administrators what they should or shouldn't trust. If you trust RDRAND, and you're on x86, you can enable the config option or provide the boot command line flag, and you're all set. But I'm not going to tell you one way or another whether or not you should trust RDRAND. And even if I did, you could just reject my advice, either way. As I said in another reply, "trust" is as much a social construct as it is a technical one.
> it's good practice to periodically reseed the CRNG. And so that means you still need to have an entropy pool and some way of measuring how much entropy you think you have accumulated, and how much has been possibly revealed ("used") for reseeding purposes
Periodically re-seeding the PRNG can be argued to make sense, but doesn't require entropy estimation: consider https://en.wikipedia.org/wiki/Fortuna_(PRNG) (and the paper mentioned under "Analysis".) Not needing to estimate entropy does come at the cost of a longer time-to-reseed, but that still seems a good trade-off to me.
(Just in case you weren't already aware of that design.)
Thank you for posting this.
“Those who do not know history's mistakes are doomed to repeat them.”
add eax r8d
rol r8 32
add edx r8d
register uint64_t ra,rd,...
ra += 0xFFFFFFFF&r8
rd += r8 >> 32