Cryptographers are certainly not responsible for this superstitious nonsense.
Think about this for a moment: whoever wrote the /dev/random manual page seems to
simultaneously believe that
(1) we can't figure out how to deterministically expand one 256-bit /dev/random
output into an endless stream of unpredictable keys (this is what we need from
(2) we _can_ figure out how to use a single key to safely encrypt many messages
(this is what we need from SSL, PGP, etc.).
For a cryptographer this doesn't even pass the laugh test.
There are people that assume that both:
1 - 256 bits aren't enough entropy to create a long chain of random numbers in a CSPRNG;
2 - 256 bits (after adjusting for known attacks) are enough entropy to encrypt a long chain of data in a symmetric cryptography algorithm.
Since encryption and creating random numbers are normally the same operation, 256 can't be both enough and not enough.
I don't know if SSL even supports symmetric encryption with more than 256 bits of entropy. I've certainly never seen it.
His basic point is that we can use the 256-bit random output as the key for a stream cipher, and voilà you have yourself an endless stream of unpredictable keys. And if you don't believe that, then in what sense do you trust the stream cipher?
There can be benefits from depending on non-portable implementation details but also significant drawbacks.
There are many standards and cross-platform interfaces defined outside of POSIX. Some explicit and top-down, some organically evolved.
The spec for /dev/*random is here, as was published in the mid-90s: http://git.kernel.org/cgit/linux/kernel/git/stable/linux-sta...
Applications, yes. Appliances built on it - now that's more open to interpretation.
Back in 2003/2004 I was building a centrally managed security appliance system. At the time I made the hard choice that first-boot operations (such as generating long-term device keys) MUST use /dev/random. It made the initial installs take longer, but I refused to take the chance that an attacker could install and instrument a few hundred nodes and find out possible problems with entropy sources.
Once the first-boot sequence was over, applications used /dev/urandom for everything. This included the ipsec daemons. Forcing everything to /dev/random during first boot made sure that on subsequent boots there would be (for all practical purposes) enough entropy available for urandom to work securely.
The first-boot problems were amplified by the fact that we were running our nodes inside virtualization. (At the time: UML, and we built our own on top of it. Xen wasn't nearly ready enough back then.)
It's fascinating to see that the problems we had to deal with 10 years ago are now becoming an issue again. To this day I choose to use /dev/random if I need to generate key material shortly after boot (which could be install), or for my own long-term use. Good thing personal GPG keys have a shelf-life of several years...
I'm sympathetic to people's concerns about generating long-term keys. But my problem is, /dev/random isn't addressing the major risks there either. You should generate long-term keys on entirely separate hardware.
The author keeps coming back to this fact, but I don't understand how it negates any of the arguments in favour of using /dev/random. What does it matter if the numbers are ultimately coming out of the same PRNG? I don't see how this negates the "self-healing" property of /dev/random or any of the other advantages. It seems like the author is really just saying, "/dev/urandom is probably good enough".
The idea that a lot of people get from that is that urandom is not much better than Mersenne twister, while /dev/random is the real thing.
And yes, you're right, the argument is precisely "computational security is good enough".
By the way, /dev/urandom is also self-healing.
Which other advantages of /dev/random might be there, apart from the cold boot case?
> And yes, you're right, the argument is precisely "computational security is good enough".
Point of order. MT is not, and is not meant to be, a cryptographically secure random number generator.
Generally speaking, good RNGs may be divided into two categories. First, there are RNGs designed for simulation. Goals: high periods, strong statistical randomness, and speed. Then there are RNGs designed for crypto. These have one primary goal: that even if you knew how the RNG worked, and had a long stream of previous output, you could not [easily] predict the next output.
MT is in the first category. It is trivially predictable given about 650 past integers, since they collectively defined its internal seed.
Sure, but only when possible and not all the time. I do realize we are only talking about seconds of vulnerability at a time, though.
> Which other advantages of /dev/random might be there, apart from the cold boot case?
I don't know enough about linux's PRNG to say, but as mentioned in some of the other posts, the point is that both APIs are there because they represent two different sets of guarantees. If you need the guarantees that /dev/random provides, then it's there and there's nothing wrong with using it. There are some additional considerations to be aware of, like the fact that it blocks, but I don't think that makes it something to be avoided.
Yes, I understand. It's the same one that /dev/urandom reads out of, if I am not mistaken.
> /dev/random has bolted onto it an "entropy estimator" that tries to match the draw from the generator to the inputs to the generator
Right, and it blocks if it estimates that there is little or no entropy left. So doesn't that mean that approximately (i.e. subject to the quality of the estimation) 100% of the time there is no entropy left, it will block?
Now, I understand that entropy doesn't "run out" on a PRNG in the sense that it can "run out" with an entropy pool. However my understanding is that the purpose of /dev/random is to behave "like an entropy pool" in that the continual reseeding reduces its vulnerability to RNG attacks. So when I say that that the PRNG has "run out" of entropy, I mean that it is behaving more like a PRNG than an entropy pool due to a lack of reseeding.
Is this a valid concern or not? I have seen mixed answers in this thread.
* Continual reseeding happens to both /dev/random and urandom
* Continual reseeding does not improve resistance to cryptanalytic attacks on the RNG itself
* Continual reseeding does provide a degree of forward secrecy; if your box gets owned up and you lose the RNG state to an adversary, you eventually recover from it
* Blocking based on "entropy estimators" is an incoherent response to any risk that the kernel RNG faces; it has no relationship to any actual policy about forward secrecy
* The blocking /dev/random does is based on a misunderstanding its developer has about how CSPRNGs provide security.
You are seeing mixed answers because 'saurik doesn't like the tone of the "just use urandom" articles. He thinks it glosses over the issue of first-ever-boot cold-start-entropy. I disagree strongly that you need to care about that issue, ever. But he and I probably do agree about every bullet in this comment.
Maybe it's just me, but it seems to me that the programming field is chock full of myths and misinformational tales that people actually make decisions with!
In the end, it boils down to:
Why not use /dev/urandom and get good enough security, instead of insecurity.
I think this is a key point that the article is missing. Silent security failures are very, very bad. Blocking will at least provide assertion that either things are as expected, or the issue will be detected early.
Linux's /dev/urandom happily gives you not-so-random numbers before the kernel even had the chance to gather entropy. When is that? At system start, booting the computer.
I invite you to read the article, because there's more.
But my key point is that urandom can fail silently. /dev/random create an assertion that there is enough entropy, which is a useful assertion to do until there are guaranties from urandom of the same.
Also note that most POSIX systems will cat a random seed file into /dev/random at startup and save entropy from /dev/urandom to the seed file at shutdown (and also hopefully right after adding to the entropy pool at startup). As long as an attacker doesn't have access to the disk or root permissions on the host, insufficient entropy should only be a problem the first time a host boots after install, presuming it gathers enough entropy after first boot.
FreeBSD and OSX use Yarrow for /dev/random and /dev/urandom, with absolutely no difference in behavior between /dev/random and /dev/urandom. I'd like to see Linux adopt Fortuna and make /dev/random non-blocking, though I can see the case being made for making /dev/random block after each boot until the estimated pooled entropy first goes above some threshold.
I would like to see a smarter /dev/urandom which asserts that there is enough fresh entropy. It would increase the trust of using it, and would put /dev/random usage in the special category of one-time pads. Until then, each case which the saved seed can cause issues with will have to be addressed if one choose to use /dev/urandom.
Note also that the only time urandom fails is on first boot. Linux distributions save seeds between boots, so your second and subsequent boots don't have this problem.
This really only comes up in VM settings and some small embedded systems, I think.
I have made a mental note to see if I can find a way to make this section a bit more prominent.
The operation is explained starting on line 52. The 2 paragraphs starting on line 78 explain the relationship behind the entropy estimator and the CSPRNG (It relies on SHA-1).
> As more and more random bytes are requested [from /dev/urandom] without giving time for the entropy pool to recharge, this will result in random numbers that are merely cryptographically strong
The one thing I still don't like about /dev/urandom is this: urandom's proponents say that it only fails on first boot, because modern linux distros capture entropy to seed urandom on following boots.
This means that in order for /dev/urandom to be ok, I have to depend on specific functionality of both the kernel and the linux distribution. In contrast, /dev/random is (arguably) harder for the distro to mess up, as it's primarily a function of the kernel.
Given the number of times distros have screwed up crypto, what are the odds that this will someday matter in the real world?
There is work on fixing this problem (or, at least, improving the situation) on Ubuntu: http://blog.dustinkirkland.com/2014/02/random-seeds-in-ubunt...
But rewinding and restoring to a previous save point? What's the harm in keeping the seed?
The other one is that after rewinding you've got the identical internal state of the CSPRNG.
So you're possibly reusing the same random numbers (or closely related ones) as before.
Of course, after a short while the rewinded VM diverges. It's another flavor of the cold boot case, really.
Or if the change in semantics is too onerous, a new /dev/drandom (delayed-random) could do this.
This is like saying that using consistent databases for things that require consistency is also "wrong", because the developer is going to add some ludicrous poorly-implemented caching layer in front on production as the performance dips, leading you to have a poorly-controlled inconsistent system anyway. It's an argument I appreciate, but one that indicates a deeper problem of developers thinking they can fix things they can't.
...and that's the only argument in the entire document for why /dev/random is bad: everything else is a justification for why /dev/urandom should be considered a reasonable alternative to /dev/random, most of which is difficult to confirm or deny unless you are a cryptographer. I mean, from a naive standpoint, it would sound like "putting all your eggs in one basket" could lead you to a problem, but how would I know, right?
Helpfully, the article claims to quote cryptographers on the matter. However, and this pretty much damns this article for me: he takes these quotes out of context. If you read the original sources, Daniel Bernstein makes it clear that /dev/urandom is broken on Linux (more on this later), and Thomas Pornin was responding to a user that was simply trying to get unique random values (where /dev/urandom is obvious).
As for Thomas Ptacek's emphatic "use /dev/urandom" repetition, his article was about people attempting to use user-land replacements for the kernel implementations, not people who were using /dev/random. In essence, he's trying to make the argument that people shouldn't be so afraid of /dev/urandom so as to use something clearly worse. He does state that /dev/random is overrated, but correctly points out its flaws.
Back to this article: after a ton of argumentation that the people suggesting that there is a key difference between /dev/random and /dev/urandom are inherently being misleading, the author gets around to quoting a section of the man page that sounds perfectly reasonable, and even is willing to cede that there is no issue using /dev/random for those use cases. I think the author is just mad because some service is slow. :/
Now, here's what I'll leave you with: an argument why this author's article is not just wrong but promoting beliefs that are actively harmful, an argument that can I hope can be believed even if you aren't a cryptographer and does not rely on me being a cryptographer: its an argument based on practical concerns only, with a demonstration of the problem not just in theory, but that happened to real people/developers.
[edit: Apparently, I misremembered this Android issue, so when I pulled sources for it I ended up with something that is not actually an issue with /dev/urandom. I have added details on a different situation where /dev/urandom actually was the culprit, but the argument isn't quite as strong. I was hoping for an argument that anyone would believe without invoking even a single hypothetical, but sadly I screwed up: I'm sorry for the confusion :/. I think the new example still makes a strong, though not impervious, point.]
So, if the author got his way, then all source code everywhere would be referencing /dev/urandom instead of /dev/random for all use cases (including "long-lived keys"). However, the author also admits that /dev/urandom "isn't perfect", and that on Linux it never blocks, even when it really should. It is then claimed that the correct solution to this problem is that you should seed virtual machines with entropy before using them.
Look: that just isn't always possible. I say this not from theory, but from the trenches of information security. On Android, some poor developer made java.security.SecureRandom use /dev/urandom. This meant that all Java libraries that rely on random numbers were getting ones from urandom, even when they claimed that they wanted "cryptographically strong" ones. Put short: the author's dream come true, codified into the library.
Sadly, users expect to be able to use Android devices soon after they boot, the user is not in control of the entropy sources, and the people who build the kernel drivers for these devices often didn't include enough entropy sources to fill the pool faster. This meant that all of this code was now broken: instead of blocking until "seriously-random numbers" could be generated, it would get "reasonably random-looking ones".
This is not a theoretical issue: people built bitcoin wallet applications sitting on top of these APIs, and some of the people using those wallets ended up with bitcoin addresses secured by keys that were painfully predictable across usages on different devices (the specific concern people bring up using /dev/urandom!). This was looked into, traced back to this design flaw, and Google even commented on the situation.
[edit: For clarity, this is the part that is wrong; as pointed out in the author's rebuttle to my comment, SecureRandom was actually doing something ludicrous here--the kind of thing that Thomas Ptacek was angry about ;P--and thereby Google's own article, which I had pulled up as a source for something I had apparently misremembered, claims that developers could consider using /dev/random or /dev/urandom to replace it.]
[edit: Thankfully, there is another situation I can cite: the attack on routers from a year or two ago that involved low-entropy keys that were generated at boot. It is the same issue, but what makes this a weaker argument is that the user is now the developer, and the developer is actually in control of the device and could have fixed the entropy pool as stated by both this author and by Daniel Berstein's full comment.]
So, in this new example, the key issue is that a bunch of embedded systems--routers and firewalls from companies such as Cisco--were generating keys to secure their configuration portals and administrator consoles using /dev/urandom. This was done, on first boot (when there would be no entropy available), and the result was that many RSA keys were generating the exact same input prime numbers. If you are trying to factor a large semi-prime, and you have another large semi-prime that shares a factor, you can break it apart very easily with simple math.
Some researchers went through to determine why this happened, and their conclusion was that reliance by library developers on /dev/urandom, being used on devices and at moments where /dev/urandom didn't end up working very well, was the "root cause" of the issue, and made a recommendation to library developers that usage of less secure mechanisms should be a non-default option and that any tradeoffs in its selection should be clearly documented, lest security be contingent on factors that do not immediately seem related to security.
> In the final component of our study, we experimentally explore the root causes of these vulnerabilities by investigating several of the most common open-source software components from the population of vulnerable devices (Section 5). Based on the devices we identified, it is clear that no one implementation is solely responsible, but we are able to reproduce the vulnerabilities in plausible software configurations. Every software package we examined relies on /dev/urandom to generate cryptographic keys; however, we find that Linux’s random number generator (RNG) can exhibit a boot-time entropy hole that causes urandom to produce deterministic output under conditions likely to occur in headless and embedded devices. In experiments with OpenSSL and Dropbear SSH, we show how repeated output from the system RNG can lead not only to repeated long-term keys but also to factorable RSA keys and repeated DSA ephemeral keys due to the behavior of application-specific entropy pools.
> When we disabled entropy sources that might be unavailable on a headless or embedded device, the Linux RNG produced the same predictable stream on every boot. The only variation we observed over 1,000 boots was the position in the stream where sshd read from urandom.
> On stock Ubuntu systems, these risks are somewhat mitigated: TLS keys must be generated manually, and OpenSSH host keys are generated during package installation, which is likely to be late in the install process, giving the system time to collect sufficient entropy. However, on the Fedora, Red Hat Enterprise Linux (RHEL), and CentOS Linux distributions, OpenSSH is installed by default, and host keys are generated on first boot. We experimented further with RHEL 5 and 6 to determine whether host keys on these systems might be compromised, and observed that sufficient entropy had been collected at the time of key generation (due to greater disk activity than with Ubuntu Server) by a slim margin. We believe that most server systems running these distributions are safe, particularly since they likely have multiple cores and gather additional entropy from physical concurrency. However, it is possible that other distributions and customized installations do not collect sufficient entropy on startup and generate weak keys on first boot.
> For library developers: Default to the most secure configuration. Both OpenSSL and Dropbear default to using /dev/urandom instead of /dev/random, and Dropbear defaults to using a less secure DSA signature randomness technique even though a more secure technique is available as an option. In general, cryptographic libraries should default to using the most secure mechanisms available. If the library provides fallback options, the documentation should make the trade-offs clear.
Of course, this is covered by the idea that these devices were not properly configured; but that's kind of the point, right? What everyone is saying about /dev/urandom is that it isn't guaranteed to produce anything "seriously" random; it happens to, most of the time, on most operating systems, produce something incredibly random, but some times, on some operating systems, it might produce something that is barely random at all.
This means that if everyone switches to using /dev/urandom in their code, a ton of libraries are going to work on some systems, and not on others; they might work on a properly configured server, but not on an Android device, or might work on one cloud provider, but not another... this is insane... if I download "secure off-the-record messaging implementation library for Linux", am I going to expect it to fail like this?
Certainly, the developers this article seems to believe are both common and acting reasonably enough to be worked around by people writing code--the ones who are supposedly going around patching random() out their libraries ;P--are not going to realize their code isn't secure, or that they they need to configure their kernel or operating system distribution to make it secure; but this is even going to burn the people reasonable enough to download code and not try to hack it :/.
[edit: And, to close the earlier edit: clearly the people who were building these embedded routers didn't realize these tradeoffs. The author of this article is thereby asking developers to be cognizant of how these different pseudo-random number generators operate and design their entire systems in ways that will make specific sub-components, components they probably don't understand well and "just want to be secure", continue to work correctly under potentially-modified assumptions. While they technically can do this--unlike the sadly-flawed Android argument I wanted to make, where users and developers had no control--this seems like a much higher bar than keeping people from taking random() and patching it out entirely due to a performance benefit, and comes down to a simple heuristic.]
Let me rebut two points that kind of irk me:
You claim that the Android problem that bit several Bitcoin wallets was due to java.security.SecureRandom using /dev/urandom.
That is mistaken, as the blog posting you gave clearly shows:
"Developers who use JCA for key generation, signing or random number generation should update their applications to explicitly initialize the PRNG with entropy from /dev/urandom or /dev/random."
So, in their opinion, /dev/urandom is fine.
Furthermore you claim that I've taken quotes out of context.
I was aware that there is the danger of misunderstanding these people's point of view, so I emailed all three of them, right when I put this web page up some days ago, specifically asking them if they felt that I might be misrepresenting them or if they are otherwise unhappy about me using their names.
Of those three two replied (both very quickly).
Daniel Bernstein replied very shortly with "Seems reasonable.", but noted that he disagrees with the "more entropy cannot hurt" point. That's why I added the little sidebar with a link to his blog posting.
Thomas Pornin seems to have taken the time to really read the article, he wrote "That's a fine page. I like it.", noted a mistake in the boot scripts section (that I have fixed) and suggested some additional points to discuss (which I did not put on the page).
All in all I was really fascinated how easy it is to get in touch with those highly-respected people, and how respectful they treat "us normals". :-)
> Furthermore you claim that I've taken quotes out of context.
As for this, I don't just claim it: I demonstrate it; the fact that these people also separately said your article sounded reasonable doesn't change the original reasons they made their comments. It hurts my position that they approved of your article, but it still irks me that you are using these quotes as evidence, when they aren't actually good arguments for your position. "I e-mailed these people and they agreed with my article" would have been a much stronger statement to me than "they said these things, out of context and in ways that sort of undermine my position, that you should believe".
I could probably live with /dev/random and /dev/urandom as they are, except that /dev/urandom also depletes the entropy counter. So if something (in my case nmbd from samba) is always reading from /dev/urandom for some reason, there's never anything for anyone to read from /dev/random on those rare occasions when it's "needed".
So on that system I use the hack of using udev rules making /dev/random just another name for urandom ( http://www.ploxiln.net/wiki/linux_urandom_for_random ). But with the boot-time no-entropy-at-all problem, I find myself wishing /dev/random on linux worked like you describe freebsd works. In fact, I might replace my udev rules with a startup script that waits to read 128 bytes from /dev/random, then switches it to actually be urandom :)
Once urandom is seeded, there is absolutely no security benefit to using /dev/random, for any cryptographic application, anywhere. urandom is a stream cipher engine generating a keystream whose keys are entropy events. Keystreams don't (in practical terms) "run out of key", and urandom outputs don't "run out of entropy".
I would also love it if you could actually respond to the relative merits of the author's "developers will hack randomness out of their projects" belief (which I claim only happens to idiots) vs. my alternative "developers will not realize the security of the libraries they are using is reliant on an unrelated set of decisions made by the system integrators" (which apparently happens to engineers who don't sound like idiots).
(Also, I appreciate it adds rhetorical strength to your comment to point out that key streams don't "run out of key" and urandom doesn't "run out of entropy", but I never stated that, never believed that, and would not now claim it: those statements are entirely orthogonal to any of the arguments I made, and seem to have only been included to make your comment sound like it responded to mine.)
I guess put shorter: care to respond to anything I said? Anything at all? :( All you've done is restated something from the article... (I mean, you certainly don't have to respond to me, but then I question the decision to leave a comment in the first place: my comment's are long because I had to respond to all the substantive points in the article, not just state my position :/).
I don't understand how people who understand the relationship between CSPRNGs and (say) CTR mode can believe that there's merit to using /dev/random after urandom has been seeded.
And again: I have not, am not, and will not, state that /dev/urandom runs out of entropy. I further did not state that your old post claimed these things: I only stated that your old post pointed out the seeding limitation of /dev/urandom. I continue to not see why you keep stating that. Nothing I have said, either explicitly or implicitly, claims that when /dev/urandom "runs out of entropy" it causes a problem: I would never claim that, as even if I thought I believed that (and I do not) I do not consider myself qualified to say that (hence why I rely on arguments from practicality only).
I think it's a rubber chicken security measure, but if it makes you feel better, that's fine.
What's not fine is relying on /dev/random as a source of entropy for your application, as opposed to seeding urandom.
(However even reading a limited number of bits from /dev/random on the first call of your code can freeze the router with no mouse and keyboard (or anything else if this is done often enough). So I still consider the above practice a bad strategy for a library writer. The person who designs the initial OS boot however should think about it. The reason that even Linux kernel doesn't do this automatically even once at the start of the whole system is because the kernel developers know that they don't have access to enough entropy unless the hardware somehow provides it. I believe having RDRAND type (not in the sense of alleged backdoor but in the sense of a known hardware source) of entropy in every CPU or chipset is actually a good thing solving an actually existing problem. We are far from that state still, unfortunately.)
There may of course be a specification stating that this relationship between the two has to be this way but then many things have been carve in stone and changed later. So unless there is a compelling argument to prefer implicit seeding of /dev/urandom over doing it explicitly, I would definitely avoid it.
Neither today's article nor any of your comments today (before this one) nor what you wrote a while back makes the statement "the correct solution seems to therefore be that when your program starts you should seed /dev/urandom from /dev/random and then continue to use /dev/urandom".
So, is this the correct option? All everything has been saying so far is "do not use /dev/random, use /dev/urandom", which I maintain is actively-harmful advice and should probably be retracted by everyone saying it: making it into a black/white issue is not just silly but will lead people into a situation where they've made confusing mistakes.
This statement so far seems to be well-argued (if nothing else, no one, including yourself, has provided any argument against it; in fact, you seem to presuppose it in all of your statements, assuming that someone smart enough to realize that this is a problem has gone to the trouble of making certain /dev/urandom is seeded sufficiently, despite all evidence to the contrary):
"What's not fine is relying on /dev/urandom as a source of entropy for your application, without first seeding urandom."
Your version, however, seems to only have one argument going for it: that under load a system using /dev/random might block. If the choice is down to using /dev/urandom or /dev/random, I have argued in various ways that this should not be considered as serious of a problem. So far no one (including yourself) has provided any arguments that I'm wrong except to emphatically restate the premise. Like, seriously: I've now spent hours arguing with you, and you haven't actually made any arguments :(.
I also, however, have stated (such as in the other part of this thread I linked to while responding to your other comment) that I think that there are opportunities for a more nuanced discussion of this. Humorously, the solution I came up with was different than yours (I suggested blocking on /dev/random once to make certain /dev/urandom is "ready") and int thinking about it might not work (it relies on an implementation detail of /dev/urandom that I realize I don't know), which frankly just goes to underscore my argument: you and this author telling people "do not use /dev/random, use /dev/urandom" is harmful advice that should apparently be replaced by "use /dev/random to see /dev/urandom, and then use /dev/urandom for subsequent random usage", lest people either 1) use /dev/urandom in a way where it might be used before it is seeded or 2) hear about this issue and then enact some incorrect workaround.
The most interesting thing I've learned from this discussion is a comment of Dan Bernstein's, quoted by the OP:
> Cryptographers are certainly not responsible for this superstitious nonsense. Think about this for a moment: whoever wrote the /dev/random manual page seems to simultaneously believe that
> (1) we can't figure out how to deterministically expand one 256-bit /dev/random output into an endless stream of unpredictable keys (this is what we need from urandom), but
> (2) we _can_ figure out how to use a single key to safely encrypt many messages (this is what we need from SSL, PGP, etc.).
> For a cryptographer this doesn't even pass the laugh test.
If I understand this correctly, what DJB is saying is that the idea that reading random numbers from /dev/[u]random decreases the amount of entropy in the pool is ludicrous. So the real bug here is that: once the pool reaches a threshold level of entropy, even /dev/random should never block again, because the entropy in the pool is not decreasing.
() There should be only one kernel PRNG device
() It should be seeded from keyboard presses, mouse movements, etc. (i.e. real entropy sources) at OS installation time (if there's a persistence mechanism) or at boot time (if there isn't)
() If called before being seeded, it should hang, but in practice this should never happen
() It should never hang thereafter
() If we accept DJB's argument that adding entropy later can be harmful (and since we've already agreed it's unnecessary), we shouldn't do it
() There should be a standard protocol by which a freshly cloned VM can be told it needs to reseed
So, how do we convince the kernel devs of this, and put an end to this whole argument?
People are working on a better Linux random driver.
Do you know if those folks are working on making /dev/random faster*? I'd really like to see even traditionally "non-cryptographic" RNG's (e.g. rand()) just use the kernel's CSPRNG.
I don't understand the rest of this comment. You build an argument about I didn't write "use random them urandom". But I did; I just didn't word it the way you seem to want me to have. Specifically, I wrote: "But [the problem is] also easily fixed in userland: at boot, seed urandom explicitly".
Developers should use urandom to the exclusion of all other CSPRNGs.
That is not "use random then urandom": that is "configure your computer differently"; the person who has the job of following that advice is no longer the person working on the secure software, but the person building the computer; it might even be the end user, whom we have absolutely no reason to believe has any clue how to secure anything (including his lunchbox). Now more people are involved, we have reason to believe that these new people do not understand this issue well, and there is a possibility of failure mode.
This failure mode seems to only be being defended due to some minor performance loss on startup of a select subset of processes. Can I hope to assume you also encourage people to not use stack protector, because the 1% global performance loss is unacceptable, as clearly we should simply trust developers everywhere to write programs that don't have stack overflows? It is exactly the same argument: relying on someone to do something correctly, someone we have evidence to show doesn't do that thing correctly, because we are afraid of some minor performance cost associated with guaranteeing that the problem simply cannot ever happen.
> No developer is making a mistake by using urandom. Every Nacl user, for instance, is relying on urandom. Every Golang user is relying on urandom. Those are two crypto libraries with impeccable pedigrees.
And apparently OpenSSL, as stated in the paper that analyzed the wide-spread cryptographic fail that happened a year or two ago.
In most settings, a dos attack is an inconvenience. A key compromise a disaster, always.
e: dos, not ddos.
You have argued that people should always use /dev/urandom, as opposed to the common understanding that /dev/random is safer to use if you need high quality randomness, due to the cold-start issues with urandom.
You said that "Not using urandom has caused way, way, way more real-world security problems than any bizarro embedded systems cold start entropy problem ever has.".
I asked you for an example of security problems caused by using /dev/random instead of /dev/urandom. You provided a link with no such examples.
I'm still not reading anything about "long lived keys" in your clarification here.
If you are concerned about cold-start entropy on first-ever boot, then seed urandom from random explicitly at boot. There is no reason ever to use /dev/random in crypto code, and you will find that the best crypto code doesn't.
I think I'll end my part in this discussion right here.
If I were more cynical, I'd say that the NSA is spending money on giving such people a loud voice, so that they can filibuster attempts to actually fix things across the board. ;)
Additionally, what the article recommends (Section 6.2) is that OS generators behave like FreeBSD: block only until enough entropy has been gathered, and then block no more. That is achievable on Linux using tptacek's suggestion, but of course a native solution would be better.
- Using /dev/random occasionally blocks applications, but that's acceptable because it's only a small high-throughtput subset of them, and those users can deal with it themselves.
- Using /dev/urandom never blocks, but a small headless subset of devices will in some situations generate predictable streams. This is unacceptable.
I fear the assessment of the damage done by /dev/random blocking is understated. When it comes to generating long-term keys, yeah, sure, use /dev/random if you want; it doesn't matter how long that takes. But in online applications, these blocks can make an application unusable. Also remember than CSPRNGs are not only used for short key generation: generating CBC IVs is a nice example of something relatively high-throughput that really should not block.
You claim that only idiots beyond help will resort to working around /dev/random to prevent blocking. I disagree. I claim that any sufficiently frustrated developer will eventually go down that route if no other obvious choice is available.
As an aside, these arguments also put some amount of faith in the entropy estimator's quality, i.e., that /dev/random blocks only when it should. This may not be the case, as shown by  in (admittedly) somewhat contrived conditions.
The good workaround for /dev/random is trivial: you use /dev/urandom instead; if you are sufficiently frustrated that you hardcode /dev/random to a file of static data, you probably have made numerous other horrible mistakes. Yes: I maintain that this developer is an idiot who is "beyond help"; it is harder to make a static file for this than to use /dev/urandom.
This can also be a configuration file option: "potentially blocks, but always works" vs. "will never block, but you'd better have setup your system correctly". Even better, the developer could go to steps to verify /dev/urandom is ready (such as reading from /dev/random and writing it to /dev/urandom) before using it receptively in ways that could block.
> I fear the assessment of the damage done by /dev/random blocking is understated.
When /dev/random blocks you know it happened, and a FAQ entry can guide you to a reasonable solution. (And again: given that you can apparently seed /dev/urandom from /dev/random, a solution that the advocates of /dev/urandom are still unhappy with for some reason, it seems like there's no excuse to end up with /dev/random blocking; instead, what everyone should be encouraging is to seed /dev/urandom from /dev/random and then use /dev/urandom.)
Alternatively, the people who end up building systems that do not have enough entropy at boot have no way of knowing this happened, and are sufficiently divorced from what is being executed that they are unlikely to even realize that someone is relying on something they shouldn't. This is where "sane defaults" come in: on Linux, it seems like software should not rely on /dev/urandom until the software proves somehow to itself that it is "ready".
I have no problem with this whatsoever; it's probably overkill for most users, but it's failure-proof. What I object to is using /dev/random all the time under the misconception that it is somehow more secure.
Developers use bogus userland libraries because the man pages tell them (wrongly) that urandom is unsafe for cryptography, and when they use /dev/random, their application randomly hangs. Exactly why would you ever recommend that a developer attempt to use /dev/random?
This is just asking for system integration errors, such as the wide-spread issue with headless routers generating keys at boot. Hell: even the desktop systems tested by the researchers were only just barely managing to run the key generation late enough that the keys weren't running into the "deterministic data returned from /dev/urandom" problem.
Yes: this is fixable by making certain the system has enough entropy sources and doesn't attempt to do anything with /dev/urandom until enough entropy has hit the system to make the data returned actually-random. However, in practice, that isn't happening, it isn't happening often, and it is trivially defendable against by using /dev/random.
As it stands, the only reason anyone has come up with for using /dev/urandom instead of /dev/random is "occasionally /dev/random blocks". The question is then whether the "sometimes the system blocks under load" is more or less dangerous of a default than "sometimes keys are generated that are actually the same as every other device".
Now, maybe you believe that the former issue is more important than the latter: that people building systems under load are more likely to break them by replacing /dev/random with a file full of constant "random" data (rather than replacing it with, for example, /dev/urandom) than people generating keys at boot without realizing their device is nothing more than a CPU with 16kB of flash and 32MB of ROM attached to a dead network.
If you do feel this way, (and it isn't at all clear to me, as you still seem to be beating the "/dev/random doesn't generate better data than /dev/urandom iff the random entropy pool has been correctly seeded" horse, which I maintain was dead long before I arrived with my argument) I encourage you to read the other comment I've written elsewhere in this thread showing why this seems like a poor choice of default behavior.
On these machines, reading /dev/random at boot time can hang indefinitely. Your "1 line in an rc script" will make the OS appear to be broken.
It's possible the problem can be ameliorated by a combination of approaches. Educating sysadmins is clearly important. I can imagine a userspace process monitoring /dev/random at boot time and screaming if someone tries to read from it and it blocks. (Though on a headless VM, "screaming" in such a way as to reliably be heard may be nontrivial.)
The point is, work needs to be done here, by a variety of people, including the distro publishers. Getting that to happen is going to take some advocacy.
[voltagex@arch ~]$ cat /dev/random | dd of=/dev/null
^C0+0 records in
0+0 records out
0 bytes (0 B) copied, 8.91021 s, 0.0 kB/s
[voltagex@fedora ~]$ cat /dev/random | dd of=/dev/null
^C0+1 records in
0+0 records out
0 bytes (0 B) copied, 8.09669 s, 0.0 kB/s
voltagex@saucy:~$ cat /dev/random | dd of=/dev/null
^C0+1 records in
0+0 records out
0 bytes (0 B) copied, 8.31898 s, 0.0 kB/s
voltagex@saucy:~$ sudo service haveged start
voltagex@saucy:~$ cat /dev/random | dd of=/dev/null
^C0+81630 records in
20406+0 records out
10447872 bytes (10 MB) copied, 2.92772 s, 3.6 MB/s
Of course if I'm completely misunderstanding what you're saying, please correct me but yes this seems to be pretty bad.
It would make designing a one-time pad generator chip very cheap. The factory could print a chip, seed it, and the product would be done. Would you put your name to such product, making the argument that it would be fully cryptographic safe to use?
> lack forward secrecy
>> I don't understand how people who understand the relationship between CSPRNGs and (say) CTR mode can believe that there's merit to using /dev/random after urandom has been seeded.
I would say that forward security is a good counter to the argument that urandom only, ever, need a initial seed. urandom need to be constantly updated with more entropy to maintain forward security.
This mean that after reboot, any cryptographic application that use urandom will run the risk of loosing forward secrecy if the saved seed is someone recovered (say, from backup).
That makes me also wonder, are the stored seed securely erased on Linux after boot?
As for forward secrecy, as I understand it, /dev/urandom is constantly updated with more entropy to maintain forward secrecy. In fact it is updated in exactly the same way as /dev/random. The only difference between /dev/urandom and /dev/random is that /dev/random blocks when it deems you have "run out of entropy" (which is really nonsense).
So even if the saved seed is compromised, as soon as enough new entropy comes in, /dev/urandom will be secure again. Also note that if the seed is compromised, /dev/random is not much better because it doesn't know that the seed has been compromised. So /dev/random will freely produce for you up to N bits of tainted output based on the broken seed, where N is the current entropy estimate.
No. One-time pads require true randomness. Any algorithm involving a pseudorandom-generator, no matter how computationally secure, is actually simply a stream cipher.
Is this a parody? Because I think it's a strong argument for continuing to disable the clusterfuck that is SELinux.
I didn't; just luck.
Time and time again, ignorant and lazy human beings are the enablers of security breaches. You can blame humans all you want, but if systems aren't designed with such human factors in mind, they simply won't work in the real world.
Look: that just isn't always possible. I say this not from theory, but from the trenches of information security.
I'd have to say that you're confusing "not possible" with "hasn't been done yet." Manufacturers have been flashing unique identifiers and even completely individually customized firmware as part of the assembly line process for decades now. (I know of one Fortune 500 manufacturer that can automatically customize virtually every engine controller's firmware.) Considering how foundational good CSPRNG are to cryptosystems and how cheap such resources are now, I'd say that this is one situation where there should be government regulations. How about a regulation mandating generous initial entropy pools in all computing devices that will be encrypting end user data? That doesn't cover all situations where insufficient entropy can cause harm, and such regulations would ultimately be much more complicated than my 3 paragraph comment might seem to suggest, but it would be a good start. (Yes, embedded devices are going to complicate this a lot, I know.)
The security field needs to start using some of the same pragmatic strategies the NTSB uses to prevent plane crashes. We can't just stand around wishing programmers know more about crypto and security. We need to put measures in place that actually work.
This same statement, however, underlies my argument in the other direction: that ignorant and lazy human beings build systems that don't have entropy pools that fill fast enough at boot to make /dev/urandom secure for the things they are running there. If the software were using /dev/random instead, it defends against the systems integrator making this mistake by blocking until the system has enough entropy.
Now, it also will potentially block later as well, which sucks: however, this is going to happen to a subset of users building systems under heavy load. Rather than letting the user run rampant, profiling the code, finding /dev/random to be the bottleneck, and replacing it with a file of hardcoded "random numbers", how about giving the user a setting "if you are reasonably confident you are not using this during boot before the entropy pool is ready for /dev/urandom, turn on this non-default setting and the software will never block"?
Under the assumption, however, that this somehow has to be a black/white "choose one, only one, no options, no workarounds" issue, defending against the ill-informed system integrators seems like a much more reasonable position than defending against the downright-idiotic system integrators. The downright-idiotic system integrators probably also left a bunch of other holes in their systems: let's help out the ill-informed ones a little, and use defaults that are more difficult to break.
> > Look: that just isn't always possible. I say this not from theory, but from the trenches of information security.
> I'd have to say that you're confusing "not possible" with "hasn't been done yet."
The argument in that paragraph involved an Android device. If my Android device doesn't sufficient entropy when I go to run the software I downloaded from the Android Market at the moment that software attempts to generate an encryption key I need, it is "not possible" for any of 1) me (most devices have locked bootloaders), 2) the developer of the app, 3) the developer of any cryptography libraries the developer used, to fix the problem.
> The security field needs to start using some of the same pragmatic strategies the NTSB uses to prevent plane crashes. We can't just stand around wishing programmers know more about crypto and security. We need to put measures in place that actually work.
I agree: how about someone makes it so that Linux itself has the FreeBSD behavior of blocking /dev/urandom until the entropy pool has reached the critical mass required to make its randomness generation secure? ;P Alternatively, how about someone gets it into all the common init daemons to read enough data from /dev/random at boot to guarantee work around this kernel limitation? It sounds like everyone is just asking for mistakes by encouraging people to write software that uses /dev/urandom to run on systems that have a known-limitation in the usage of /dev/urandom.
I realize that I'm preaching to the choir by writing this, but it's for 3rd party readers: What if the airline industry just encouraged people to try and do the right thing, then stood around shaking their heads when it turned out lots of pilots, mechanics, and companies didn't? Well, there would be a lot more accidents.
That's where we are as an industry and a society, with regards to security.
There were problems with routers with too little entropy, but unless you are the person implementing the code which initializes the router's keys after the first boot, you still should not use blocking.
I guess the simplest rule is: 1) If you are programming something which is to be used as a normal program, on the normal computer, use /dev/urandom. 2) If you are responsible for developing the device or virtual device which doesn't have valid entropy sources after the first boot, do consult somebody in the know how you can provide the needed initial entropy. After it's there, use /dev/urandom.
In short, I really don't know where using /dev/random regularly is a good practice.
The parent article is targeted to the user-space developers. Or people who will use /dev/random in their shell scripts to "clean the disk" and then not understand why it's slow.
Where would you, if you develop user-space code, have advantage from /dev/random?
Now, as a developer, you write a library or daemon, or some other piece of userland code. You do not know on what operating systems or at what point in the bootup process it will be used. You know it works fine if and only if /dev/urandom has been seeded, and so use /dev/urandom.
However, apparently, people install your daemon at boot and generate the keys at boot, on VMs that are created often from scratch. There's nothing inherently wrong with these computers: they have entropy sources, they just don't have any entropy at boot, the referenced "boot-time entropy hole".
Your code now "doesn't work". It isn't that it crashes, it doesn't block: in fact there's no way at all to tell that anything wrong happened, except that some point a long time later someone finds out that every encryption key you generated from it is from a limited, guessable, "weak" subset.
The argument here seems to be that this is the fault of the person who built the computer. Honestly, I somewhat agree. However, let's imagine the real-world conversation that would happen if the developer of this library/daemon actually stated this limitation up-front in their README somewhere.
README: "Note: this program only works correctly if the entropy pool is adequately seeded before running it."
User: "Wow, ok; can you tell me if Ubuntu is up to the challenge?"
Developer: "No. It isn't that simple: the system doesn't block boot for the entropy pool to fill, so whether your system is setup correctly is up to whether entropy fills fast enough; your chosen distribution is one aspect of this, but we will also need to look at what kinds of hardware you have attached and external factors such as whether the network you are using gets much traffic."
User: "Seriously? Is there some simple test I can run on my computer that would let me determine if my system is broken?"
Developer: "No. These are complex factors that are difficult to verify statically. Your attempts to analyze the data yourself will also almost certainly come long after the entropy pool has already received the critical mass required for /dev/urandom to become secure. Your attempting to run the test would also affect a lot of the entropy sources directly, including network and input."
User: "But it is possible if I spent more time on it?"
Developer: "Well, sure: there was a paper published where some researchers instrumented the kernel attempting to analyze these issues. They were able to determine that OpenSSH on RedHat was generating its host keys only just barely after the entropy cutoff on standard desktop computers. You could make similar patches to your kernel and attempt to see whether your usage seems safe."
User: "That sounds ludicrous... was there really no way for you to code around this?"
Developer: "Oh, sure, we could use /dev/random; but then sometimes the program would block under heavy load when it really didn't need to; rather than just changing to /dev/urandom, some of our users were responding by hardcoding the randomness of our program entirely, defeating all of the security in our system. To protect these people we decided to switch to /dev/urandom ourselves."
User: "Couldn't you have made that an option, choosing the most generally secure option by default, but providing /dev/urandom as a sane alternative 'when using under heavy load', thereby guiding people who have this issue to not do something insane? Maybe you could read /dev/random once when your program starts, to get the blocking behavior, and then switch to /dev/urandom for later repeated use?"
Developer: "No. This is apparently a black and white issue, and we have to decide emphatically whether /dev/random should always be used or should never be used. The idea that this might be a complex issue that requires nuance in thought and might have to be decided on a case by case basis, or for which there might be some solution that involves working around the issues with one by using the other momentarily, should not occur to us."
[edit:] (So, it sounds like maybe the correct solution at this point is that the user seeds /dev/urandom themselves using /dev/random. Of course, this solution only works if the user is smart enough to do this and has access to do it, neither of which are likely true on many Android devices. The developer could do this for the user, but that's back into the "nuance" discussion. Instead of being so emphatic that "/dev/random should never be used" people should be advocating developers use actual solutions, potentially "workarounds", to the problems that come up in real-world deployment :/.)
It can be a tough call. In a previous job I worked on a product that used /dev/random. We never had a problem with it until one customer installed our product on a newly created VM, and it hung during startup.
We switched to urandom, but we definitely had misgivings about it, as this was exactly the case where it could matter. It's also a situation where reading from /dev/random only the first time wouldn't fix the perceived bug in our product.
Perhaps it would be useful to have a userland process monitor /dev/random and, in a case like this where there has never been sufficient entropy in the pool, send a notification to the console (though there's no guarantee it will be noticed). On Android it could pop up a window.
If this was the case, that the block was happening on the very first usage of random data, then you actually managed to build a system that wasn't secure: that first time /dev/random was blocking, when you switched to /dev/urandom, on a newly-created VM, meant that the customer was getting deterministic data from /dev/urandom, and no one ever realized because there is no error condition for this. (Maybe the system clock was accurate on these new VMs and not set later by ntpd; if so, there might be hope that at least there were a few bits of entropy involved ;P.)
I think we only had "misgivings" because the key we were generating had only relatively minor use in the product. Alas, I don't recall exactly what that was, and it's certainly possible, in retrospect, that we didn't care as much about the implications of using /dev/urandom as we should have. But that was pre-Snowden.
My question was about those that write user programs, not those that configure Ubuntu to boot right the first time. I claim that I as the writer of user code am not supposed to have the responsibility of the initial seeding, unless I'm writing the interactive code, where I can show you a nice looking UI which says "now type on the keyboard to provide more entropy" if I'm waiting for background thread to stop blocking on /dev/random.
The initial entropy of the routers etc which don't have keyboard are something that must be taken care of by the designers of the system, not by an application or library writer, especially if the later doesn't have access to any hardware/external entropy source.
Put differently: why are we encouraging people to "run with scissors" and allow this mistake to even be possible? Either demand that Linux itself fixes the boot-time entropy hole, encourage developers to seed the entropy pool before relying on /dev/urandom, or encourage developers to only use /dev/random; but it seems "actively harmful" to encourage developers to only use /dev/urandom and then just pray that the system integrator or user understood entropy enough to make the system have a working /dev/urandom by the time it is needed. If I were a pen-tester this would be one of the first things I check for on every setup, as it is a really esoteric issue that almost no one understands well and that apparently developers of even popular software like OpenSSL/Dropbear completely ignores in the name of not accidentally blocking under load.
And it has to be done at the distro level. Distros have to be persuaded to make random seeding a standard part of their installation process. They should also give the user a command to run in newly cloned VMs.
The new x86 RDRAND instruction would be very useful here, of course, if we ever decide we can trust it. One could argue that calling RDRAND at boot would still be better than having no entropy at all; the NSA may have a backdoor, but probably no one else does.
So even if once Intel gets to be "less evil in the eyes of worried people" for a "sin" of providing RDRAND although the same people otherwise trust the company to let its CPU's execute all their programs, what should the poor kernel writers do on the systems without RDRAND?
But you are completely right, having the VM's and distributions agreeing on protocols for providing initial randomness to the VM is certainly a step in the right direction and the proper solution to one real problem.
Whereas promoting belief in regular user-space use of the /dev/random as the magic solution is, in my humble opinion, something like a cargo cult. I'm not commenting your statements here, just attempting to make my position in this whole discussion clear.
This isn't fair. Most instructions compute deterministic functions of their operands. It's easy for people to tell if the CPU is computing the correct result. (Remember the Pentium FDIV bug?) RDRAND, by definition and design, is nondeterministic. Only by examining a long sequence of its results could one gain any confidence that it isn't backdoored ... and then one might still be overlooking something.
Actually, you can't. A properly backdoored RNG is indistinguishable from a non-backdoored RNG. Imagine a backdoored RNG that is simply a counter encrypted with AES. The output looks completely random, but it can be completely broken by the holder of the AES key.
But it supports the point I was making: trusting RDRAND is a very different thing from trusting the rest of the instruction set.
Once again, kernel can't manage to do it only once after the boot, in general. Library/server application writers could not be expected to do more or better. UI applications can at least collect mouse movements, if they actually know they have an access to a mouse! Enough initial entropy is a hardware problem not the question of "simply calling the right calls, and 'random' is righter."
If you seriously doubt that an encryption library like OpenSSL has to always generate keys then this discussion is between the two people who shouldn't have even started exchanging opinions.
Ditto for your claim that blocking for entropy is something that doesn't happen or is trivial. It's not, the kernel writers know.
If you still believe you're right, try to explain to the kernel writers how they can just initialize the urandom pool just once after the boot, and you solved the problem for everybody at once! You may even enter some hall of fame.
I use OpenSSL in a ton of programs for the purposes of computing SHA1 hashes. So yeah: I doubt most programs linked against OpenSSL are generating keys. Only processes that generate a key matter for the purpose of seeding entropy.
What, exactly, do you think your computer is generating so many new keys for during a routine boot process? Can you tell me what these new keys are for?
> Ditto for your claim that blocking for entropy is something that doesn't happen or is trivial. It's not, the kernel writers know.
Clearly it does happen: the question is how much does it happen; a desktop system generates enough entropy that it is easily capable of keeping up with generating enough random data from /dev/random to seed /dev/urandom every now and then because a new SSL-enabled daemon spawned.
> If you still believe you're right, try to explain to the kernel writers how they can just initialize the urandom pool just once after the boot, and you solved the problem for everybody at once! You may even enter some hall of fame.
1) Apparently FreeBSD does this. Can you tell me how FreeBSD apparently does the impossible? 2) The kernel can clearly store a boolean; I mean, the kernel does all kinds of things only once... are you seriously thinking that the reason the kernel hasn't implemented this is because they can't?
> a desktop system
How is OpenSSL supposed to know if it's used on a desktop system?
> Can you tell me how FreeBSD apparently does the impossible?
I guess by pretending that problem doesn't exist: e.g. postulating a desktop system, like you do. But do ask them and pass the advice to the Linux kernel writers! They just waited for you to get that idea to ask the FreeBSD guys (and I'd be grateful if you post the results of that here, please, we all want to learn).
> are you seriously thinking that the reason the kernel hasn't implemented this is because they can't?
Yes, in general, they can't! Entropy on most of the systems still doesn't grow on the CPU trees. But do please prove me wrong.
Also, both should be a lot faster. With some buffering you should be able to use them to replace libc's rand() with little to no performance penalty.
Every application developer who wants "random" numbers should simply get cryptographically-strong random numbers, period. We should not expect them to choose, since it's just another opportunity to make a mistake.
I however prefer having both and expecting from the programmers to actually understand what they need. I see it's not easy, in my discussions here you can see that people still believe that "random" is "righter" than urandom for their actual urandom needs and that they have to "do something and that something is waiting for the entropy" even when only the system designer or owner should be responsible for the proper urandom seeding. It's hard.
I suggest changing the man file of these two streams to explain the basic idea better. Then those wanting to "do something" would at least lose that support. I know it's against the tradition of the man pages to be clear though.
Not exactly. /dev/urandom also sucks, because it's slow, and it never blocks, even when the system knows that it can't deliver cryptographically-strong entropy.
There should only be one interface, and it should deliver cryptographically-strong random bytes at high speed and with high reliability. There is currently no character device on Linux that does this.
> I however prefer having both and expecting from the programmers to actually understand what they need.
I'd prefer it too, but it's unrealistic. Keeping up with the state-of-the-art in crypto is a job for specialists, and even halfway-decent crypto implementers make RNG mistakes on a regular basis.
Look at the sketches in the article.
Sorry, you're wrong. /dev/random in Linux is made to deliver the new entropy to you as the user, even if these "entropy" bits do pass through the block you marked as CSPRNG. That doesn't contradict with my claim. The fact they pass through the CSPRNG block doesn't change the fact that the system makes its own best attempt to collect the actual entropy and pass it to you as the user. That's also the reson why it blocks until it has collected as much new bits as you requested -- it does its best to not lie to you. These bits are not the "continuous CSPRNG stream" they are just the nicely polished entropy bits.
Think about this this way: some system stream can be made that would deliver the unprocessed "source" entropy bits. But such bits would have to be additionally reprocessed (in effect, passed through something like CSPRNG or whatever, there are enough papers written about that) to be useful in any sensible way. Even very random natural processes (e.g. radioactive decay) can't be used in their pure form for cryptographic purposes. So the kernel does that additional processing for you. That's all. They are still entropy bits, only additionally shuffled with the "encryption" code to actually improve their characteristics for the cryptographic purposes.
Now you may claim that the claimed entropy doesn't exist in the case that estimator estimates wrong, but it's still the best attempt there is. Accepting imperfections of the real world, /dev/random in Linux is there to give you the quality entropy bits, based not only on some known states, but the very real "entropy" bits not discussing here how these can be collected, estimated and their characteristics improved.
The meaning of "entropy" here is loosely "the non-calculable randomness which can be observed by the functioning and the interaction of the system, not the direct calculable product of the system itself." The system running in the VM and without special inputs has much less chance to observe much of such randomness.
I am aware that, according to the claims I've read here, by not using blocking, entropy estimations or whatever, FreeBSD is not delivering the entropy once it stops blocking. But it's just the FreeBSD's difference in the contract.
In Linux, it is actually the other way around to your claim: not even /dev/random on Linux actually strives to return the new entropy, the /dev/urandom won't just return CSPRNG stream but will occasionally get the added benefit of the real random bits collected via the entropy collectors. It's just guaranteed that it won't block while again, doing its best.
The source of kernel's random.c already linked in this discussion is actually very nice and clean:
Do compare random_read vs. urandom_read.
We as a technical community and we as a society need to start using the same stances and strategies that governments and agencies like the NTSB use to prevent accidents. When planes crashed in the wooly pioneering days of aviation, people probably stood around sadly tsk-tsking someone's failure to recognize a stall. Today, the NTSB conducts root cause analyses and the industry takes concrete steps to prevent future mishaps. (Like equipment that detects stall and warns the pilot.)
In this case, why don't we simply have classes of devices that are expected to implement crypto for end-users, and require them to be built with generous entropy pools? (To head-off the expected smart-asses: Each one gets their own unique one, of course.)
(EDIT: An analogy: What if you knew that pilots all had conflicting opinions on how to fly and maintain a plane, with constant acrimonious debates, and where some large swathe of all pilots simply were operating on misinformation? Would you fly? Would you expect lots more crashes? Now ask yourself what the situation is like with computer security, where knowledgeable people expect security to always be broken.)
Intel added a generous entropy source to their processors. The result is that nobody trusts it as a single entropy source, and /dev/random blocks anyway. Also, we are learning that using it only as an added source may also be harmful, and maybe we should ignore it completely.
I don't think this'll have any simple solution, although the idea of a CSPRNG that may block only at boot is interesting.
For example, on Solaris from random(7D):
The /dev/random and /dev/urandom files are suitable
for applications requiring high quality random numbers
for cryptographic purposes.
While bytes produced by the /dev/urandom interface are
of lower quality than bytes produced by /dev/random, they
are nonetheless suitable for less demanding and shorter
term cryptographic uses such as short term session keys,
paddings, and challenge strings.
Also, one of the Solaris Security Engineers wrote about random number generation on Solaris extensively just last year:
Some blogs from other Solaris Security engineers:
...and last, but not least, Enrico Perla (doesn't have a blog at the moment as far as I know) did author this book and is a Solaris Engineer and someone that works on Solaris Security-related things:
I'm glad I could provide useful info.
Of course that probably only only counts if an external or local unprivileged entity manages to become informed of the random source's internal state. If the attacker has direct access to the kernel's state then they probably have access to influence (or at least monitor) the incoming entropy such that they can stay informed of the full internal state.
There is a point in some attacks beyond which your only half way guaranteed solution is the metaphorical orbital nuke platform.
You don't need access to the machine.
Of course if the attacker is not that far in and has derived the PRNG state by other means (and you can stop them just relearning the state by the same means) then reseeding would work.
This would make sure that distro vendors don't even get a chance to mess initial seeding at boot time. It will also force vendors of embedded or "cloud" distributions (eg. Ubuntu's AWS images) to find a way to pre-seed the images to reduce the initial boot times.
Unfortunately there is such a huge amount of software that depends on this particular difference between /dev/random and /dev/urandom that I don't see the change happening soon.
/dev/urandom is a pseudo random number generator, a PRNG,
while /dev/random is a “true” random number generator.
Fact: Both /dev/urandom and /dev/random are using the exact same CSPRNG (a cryptographically secure pseudorandom number generator). They only differ in very few ways that have nothing to do with “true” randomness.
This a commonly believed myth
Fact: This is the real fact
Not that different than this format:
"He said that he is a great coder. In fact, he can't even write a Hello World program".
The assumption here is that the initial state can't be guessed, i guess you would rather use /dev/random for that, even if it blocks. Now be careful with the initialization step of your favorite crypto library, most inconsistencies /fishy tricks do happen right here !
Also another issue is that that such a random number generator introduces locks; I have a small program that encrypts everything with RSA, and uses multiprocessing to speed up the process; RSA encryption is supposed to use padding schema that needs to generate random numbers, so it locks, however you don't have this problem with decryption - that's where it counts because RSA decryption is much slower than encryption.
here is my project:
> If so, then why is the CSRNG re-seeded every 60 seconds? It would seem that there are reasons for doing so.
From what I understand about it (IANACryptographer), mostly to safeguard against situations where an attacker can learn the internal state of the CSPRNG at one moment. But I don't think this is too relevant for the problem at hand.
The only difference in Linux between /dev/random and /dev/urandom is that /dev/random always plays a guessing game about the seed entropy. The main argument here is, I guess, that this is a silly game. How can you really estimate entropy? It doesn't only depend on what physical sources in the machine give you, but also on what the attacker knows or can know about these sources and the state of the machine. How do you estimate that?
This was the main motivation behind the Fortuna CSPRNG by Schneier, Ferguson and Kelsey. There is no guessing going on, rather the algo is designed to not need to guess. Interestingly, the way Fortuna works seems to me would also somewhat mitigate the kind of attack DJB talks about in the article referenced by OP.