
Myths about /dev/urandom (2014) - mindcrime
https://www.2uo.de/myths-about-urandom
======
Taniwha
(disclaimer I sell a HW random number generator)

I think both the diagrams here are simplistic - in reality pre 4.8 there were
3 pools (an input pool and 2 output pools, one of which blocked) and post 4.8
where there are 2 pools, an input pool and a blocking pool (urandom now pulls
from the input pool thru a CSPRNG).

One big downside of the new (post 4.8) architecture is that
urandom_min_reseed_secs is ignored - pre-4.8 you could ask the kernel to
periodically pull a little bit of real entropy from the input to refresh the
urandom output (non-blocking) pool, by default that was once every 60 seconds,
but in our case with lots of entropy available we'd set it to happen once a
second - now post 4.8 it looks like urandom_min_reseed_secs is ignored and the
input pool (where /dev/urandom gets its data from) is not regularly updated
unless data is also pulled from /dev/random - which may make sense if entropy
is rare, but is not so useful when back pressure from /dev/random is left to
control the rate at which external entropy flows into the system - I suspect
it's time to present a patch that makes urandom_min_reseed_secs work again

Nevertheless the conclusion "use /dev/urandom" is still valid

~~~
Taniwha
BTW - a tiny bit of python to keep /dev/urandom fed post 4.8 - useful if you
have a hardware RNG (either an external one or the one in your CPU):

    
    
      #! /usr/bin/python
      #
      #       OneRNG /dev/urandom tickler
      #       (C) Copyright Paul Campbell Moonbase Otago 2018
      #       Released under a GPL 3.0 license
      #
      import time
    
      with open("/dev/random", 'rb') as r:
              while True:
                      with 
      open("/proc/sys/kernel/random/entropy_avail", 'r') as e:  
                              iv = e.read()
                              print(iv)
                              if iv > 1500:
                                      x= r.read(1000)
                              e.close()
                      time.sleep(1)

~~~
acqq
GPL 3.0 for a few obvious lines seems to be really an unfitting license.

Non-licensed English language version, containing the same information:

Once per second, read /proc/sys/kernel/random/entropy_avail and if the
resulting number is more than 1500 perform a read of 1000 bytes from
/dev/random.

~~~
nothrabannosir
(this is more of a tangential licensing PSA, since we're on the subject)

Not licensing something is, ironically, restrictive. If you want to release to
public domain (or equivalent), you have to explicitly mention that. E.g. by
licensing under CC0.

Otherwise, default copyright and "all rights reserved" applies.

[https://creativecommons.org/share-your-work/public-
domain/cc...](https://creativecommons.org/share-your-work/public-domain/cc0/)

(But, IMO, nothing wrong with publicizing use of GPL 3! :) thanks either way
to both)

~~~
semi-extrinsic
I think GP is getting at something else. As the length/complexity of the code
goes to zero, the copyright and "all rights reserved" vanishes; this is well
established, or else we'd all have to license "Hello world" examples from
Kernighan and Ritchie's book.

If someone writes the same one-sentence pseudocode as GP in a book or
blogpost, nobody could reasonably claim that an implementation would fall
under the copyright of that book. And I think you'd be very hard pressed to
argue it's copyrightable just because someone typed it out into real code.

------
matbess
This article naturally leads to a question that I didn’t see addressed,
perhaps you can answer it here:

If it’s unnecessary to ever prefer /dev/random to urandom, then why does
/dev/random’s current blocking implementation still exist? Surely between all
of the kernel developers, between all of the Unix-ish OS’s, over the last
couple of decades would have realized that /dev/random should simply act like
/dev/urandom?

~~~
Scaevolus
Because Theodore Ts'o, a major Linux contributor and maintainer, created and
maintains /dev/random and blocks attempts to make it nonblocking like every
other cryptographically secure random source. It's a political problem, not a
technical one.

See
[https://news.ycombinator.com/item?id=6550256](https://news.ycombinator.com/item?id=6550256)
for a much longer argument about it.

~~~
John_KZ
Why should it be made non-blocking? If you want a large string with little
entropy just use /dev/urandom. The political/dogmatic argument is that is
should be made non-blocking. The process of acquiring entropy in typical
computers is _definitely_ blocking.

~~~
tptacek
What does "acquiring entropy" mean here? There is no reason a properly seeded
CSPRNG should ever need to block.

~~~
John_KZ
I don't think you understand how CSPRNGs work.

~~~
tptacek
Do go on.

------
dang
2014:
[https://news.ycombinator.com/item?id=7359992](https://news.ycombinator.com/item?id=7359992)

2015:
[https://news.ycombinator.com/item?id=10149019](https://news.ycombinator.com/item?id=10149019)

2017:
[https://news.ycombinator.com/item?id=13332741](https://news.ycombinator.com/item?id=13332741)

~~~
matbess
Nice! Thanks

------
jMyles
This document is very informative for sure, but I don't really think it's
dispositive for many of today's use cases for randomness. In fact, I think
this document is pretty dangerous.

I referenced this document here in this stackexchange question, which I think
has a good and lively discussion about the real stakes of /dev/urandom today:

[https://security.stackexchange.com/questions/186086/is-
alway...](https://security.stackexchange.com/questions/186086/is-always-use-
dev-urandom-still-good-advice-in-an-age-of-containers-and-isola)

~~~
tptacek
I don't think you're going to find too many cryptography engineers
recommending that you use /dev/random, or, for that matter, the blocking
variant of getrandom(2).

This might be a live debate on message boards, or on the wrong stack overflow
boards --- though Pornin's answer[1] on this question is one of the all-time
good Stack Overflow answers --- but among practitioners, this is long settled.

[1]: [https://security.stackexchange.com/questions/3936/is-a-
rand-...](https://security.stackexchange.com/questions/3936/is-a-rand-from-
dev-urandom-secure-for-a-login-key/3939)

~~~
jMyles
Well, perhaps you have a better answer to my question than forest's answer, to
which I awarded the bounty. For my money, I think that's one of the finest and
most balanced pieces of writing on this topic available on the internet today.

> I don't think you're going to find too many cryptography engineers
> recommending that you use...the blocking variant of getrandom(2).

It depends on what you mean by "the blocking variant". To quote Cory Benfield:

> Can we please try to be clear about what kind of blocking we mean?
> getrandom(flags=0) absolutely _can_ block: that's what the original issue
> was all about. To ensure it _never_ blocks you need to call
> getrandom(GRND_NONBLOCK): that's why the flag exists.

It may (or may not) surprise you to learn that PEP 524, which uses this
(sometimes blocking) variant of getrandom, has been accepted.

Here's a discussion of the matter featuring several reputable, kickass
security-minded python people discussing the matter and indeed endorsing the
sometimes-blocking (flags=0) variant of getrandom:

[https://bugs.python.org/issue27266](https://bugs.python.org/issue27266)

For full transparency: I read your post on this topic before I posted my
question and found that, as far as I was able to read, it added nothing new to
the public discourse on this topic. It does not seem to overcome the very real
and well-sourced concerns raised by forest in his answer to my question.

Anyway, I'm not sure what you mean by "the wrong stack overflow boards" \- can
you tell me which are right and which are wrong?

The link which you've provided, which of course I had already read thoroughly,
as I pointed out in my question, is not from Stack Overflow, but from Security
StackExchange, just like my question.

I haven't yet marked an answer correct, so if you actually have something
substantive to add that you think is better than forest's answer, it's
possible that you can grab the check.

~~~
tptacek
Regarding that Stack Exchange answer: avoiding the Linux KRNG in favor of
alternate RNG's like OpenSSL's is literally the opposite of what cryptographic
software security engineers will tell you. For the past decade and a half,
virtually all notable cryptographic failures owing to bad randomness came from
systems that weren't using the kernel RNG.

~~~
jMyles
Yeah, as you can see, I pushed back on that.

But, as far as I can tell - and I do indeed write with this in mind - the
movement to create applications for very fresh, naive installations such as
containers or IoT devices, does indeed present a problem for purely-blocking
random.

My sense is that you and others in the community encouraging blind trust of
/dev/urandom need to adjust your language when consulting on this matter in
the future.

Since I mostly write python, I'm happy that this has been worked out and that
python now uses blocking random (via getrandom(flags=0)) in these very rare
situations.

~~~
tptacek
Once again: there is exactly one time in the lifecycle of an entire system
when you want a blocking read from the Linux KRNG: when the system is first
booting, to ensure it's seeded (FreeBSD is smarter about this and handles that
automatically). Apart from that one time, every other read on your system
should be from urandom.

No, this doesn't need to updated for IoT or "containers" (Linux containers
share the KRNG).

~~~
jMyles
> there is exactly one time in the lifecycle of an entire system when you want
> a blocking read from the Linux KRNG

Right. I'm glad to hear you phrasing it this way.

And listen: I don't mean to be flamy here, and it's possible that my sense of
these things is tainted by our many... colorful discussions here on HN, but
the unradom orthodoxy, to which getrandom and python's os.urandom are in part
a response, is something for which you are arguably a tiny bit responsible,
having actually written "Use urandom. Use urandom. Use urandom. Use urandom.
Use urandom. Use urandom" on your blog. That kind of punditry has no place in
security engineering.

There is a need for a degree of sobriety with these sorts of security
considerations. Sometimes edge cases move toward the mainstream, and that's
exactly what has happened with this odd little phenomenon of programs being
executed much, much earlier in the OS lifecycle in the case of
containerization and IoT devices.

Yes, some containers are deployed in settings where the host KRNG can be and
is used. But not all. And many IoT devices suffer from a sufficiently weak set
of entropy sources as to need to block.

These are real concerns, as identified by the study I linked from the
University of Michigan. The researchers there were able to replicate over 1%
of the SSH keys that they found in the wild because the devices that created
them had insufficient entropy at the time of the key generation.

All I'm saying is: don't treat these sorts of security issues as subject to
orthodoxy. Clear thinking and solid engineering are the orders of the day.

~~~
tptacek
The blog post you're referring to contains exactly the advice I just gave in
the preceding comment.

I don't see a link to a Michigan study in this thread. If you post it again,
I'll be happy to explain why it doesn't say that you should use /dev/random,
or a userspace RNG.

~~~
jMyles
Well, man, that's not the debate. As so often in these conversations, you're
running away with the goalposts. If I may advise: these are exactly the antics
that are costing you credibility with those of us out here in the field
actually building cryptographic systems.

Good day.

~~~
tptacek
I don't understand your objections, like, at all. Every "concession" you think
you're extracting is right there, in a blog post I wrote 4 years ago. Those
are the goalposts. They haven't moved since 2014.

Everything else you have to say just reads like word salad. I should be
concerned about container RNGs, but not Linux containers, unless they're the
Linux containers that don't share the KRNG? Or the IOT devices where blocking
somehow solves the cold-start entropy problem, and at the same time can't be
done in the init scripts? Even the Python issue thread you posted --- yes, I
read it --- disagrees with you. I have no idea what you're talking about and,
candidly, I don't know that you do either.

I've written (until now) perhaps 1 word on this thread for every 30 of yours,
and I think my comments are straightforward and make obvious points. You
respond to each of them with a wall of text. Can I suggest that maybe instead
of a grand disagreement, you've instead just done a poor job communicating
whatever your concern might be?

Regarding that last sentence of yours (er, the one before your "good day"): I
am _extremely_ comfortable with who does and doesn't find me credible. Thanks
anyways.

~~~
jMyles
Well look, the problem is solved, at least for people using python. So there's
really no need on my end to discuss it further.

As for community communication and understanding: I really don't think there's
any good reason that we all have to lock horns all the time. I do think that
you'll find more joy in these sorts of discussions if you can be less caustic
and more deliberative. It's no more difficult to be nice than it is to be
mean; give it a try. And if you get to the edge of your knowledge (and, it
seems to me that you do, fairly frequently, which is great - you are obviously
interested in a wide variety of societal topics just like I am), don't just
keep marching. Stop and acknowledge your short-comings and recognize that
there are others who might have had some original / compelling / correct
thoughts on these matters also.

Again, this is no skin off my nose. My current project is python, and python
has wisely decided to stop using urandom in favor of getrandom(flags=0), so
this is a non-issue.

But won't it be nice if the next non-issue can be handled with greater ease
and less toxicity?

~~~
pvg
_and less toxicity?_

Wait a minute, you have made nasty insinuation after nasty insinuation in just
about every single comment you've written on this thread and then sprinkled
them liberally with finely ground condescension. That's where the toxicity is
coming from.

~~~
jMyles
I'm sorry if I've been unkind. That wasn't my intention.

But insinuation? I think I've been very clear and explicit about my criticisms
rather than insinuating anything. Is there something about which I can be more
transparent that will increase the quality of this and similar discussions?

I also don't mean to condescend, and I can understand from my writings how my
tone might be interpreted that way. But what is the right thing to do in these
situations? Discussions on HN of security and the political ramifications are
dominated by the volume and aggression of this person and s73v3r_ and others
of similar style, and it is taking a toll - people who are quiter but more
knowledgeable (and to be clear: I'm not saying I'm either of those things) are
being shut out. Can't you see it?

It reminds me of far-right AM radio. I don't want to find myself just tuning
out of HN entirely, so I'm trying to be a little more proactive.

On the other hand, I certainly don't want to respond to vitriol with vitriol,
but I do want to try to find common ground so we can all learn from each other
instead of bark about who's right and who's wrong on these minuscule, mundane
points.

Again: Although I think I've been quite explicit rather than lean on
insinuation, I apologize for portions of my tone. I'm not a mean person; I'm
just trying something new in an effort to change the direction of these
things.

------
adrianmonk
> _I just claim that it 's bad to block when the entropy estimate is low._

As always when it comes to engineering, good and bad can only be defined when
you are specific enough about what it is you're trying to accomplish.

Is it a single-user client system with an intelligent user who won't try to
subvert security (maybe because you've defaulted to /dev/urandom but given
them a choice to override and use /dev/random), and you're trying to do
something of crucial importance like generate a long-lived crypto key? Then
blocking might be fine or even the better way.

Is it a multi-user and/or server system where randomness is a scarce resource
whose exhaustion could bring down the system like any other scarce resource
(disk, network, RAM, CPU, etc.)? Then blocking is probably a terrible idea and
could even open you up to denial of service attacks, thus causing you to
achieve _less_ security.

~~~
tptacek
First problem with this response: programmers have a terrible understanding of
what "crucial importance" is. In fact, most things in cryptography that want
random values _need_ unpredictable values. The old Linux man page creates the
impression that you want "good" random for keys, and "mediocre" random for
everything else, which is false.

Second problem is, there is no such thing as "good" and "poor" random. There
are two states: unpredictable (seeded) and predictable (not yet seeded). Once
you read seeded state, you're done; _blocking serves literally no purpose_.

~~~
adrianmonk
> _programmers have a terrible understanding of what "crucial importance" is._

If you cannot understand requirements, you cannot produce satisfactory
software.

Anyway, it was intentional that I gave an example where an educated end user
is able to make the choice rather than a programmer having to guess.

> _The old Linux man page creates the impression that you want "good" random
> for keys, and "mediocre" random for everything else, which is false._

It certainly is false, and it is a myth that many people believe, but it isn't
what I was saying.

> _Once you read seeded state, you 're done; blocking serves literally no
> purpose._

As the article explains under the last section ("Not everything is perfect"),
/dev/urandom does not guarantee that you are reading seeded state.

Also, those numbers' unpredictability rests on the crypto being really good.
Which I'm sure it is, but crypto weaknesses have been discovered in the past.
It would be defensible to seek additional protection against that possibility
for peace of mind if peace of mind is more important to you than running time
and/or availability.

------
matbess
Well written post, I enjoyed it, it makes sense.

The part tearing into the man page was especially excellent!

Following your point a little further.. the thought of updating the man pages
with warnings like the following made me laugh:

“AES, SHA-3, RSA, Diffie-Hellman, Elliptic curves, and all those crypto
packages you're using, OpenSSL, GnuTLS, Keyczar, your operating system's
crypto API, are all theoretically vulnerable to a cryptographic attack.
Knowledge of how to do this is not available in the current unclassified
literature, but it is theoretically possible that such an attack may exist. If
this is a concern in your application, do not use them.”

------
loeg
This seems to just muddy the waters.

[https://sockpuppet.org/blog/2014/02/25/safely-generate-
rando...](https://sockpuppet.org/blog/2014/02/25/safely-generate-random-
numbers/) is shorter, more succinct, and to the point.

------
Johnny555
On a VM that someone else runs (i.e. Google or AWS), is either /dev/random or
/dev/urandom cryptographically secure?

If no, are there any "safe" choices? (other than an external source like an
HSM or hardware random number generator)

~~~
TheDong
Most vms / cloud providers can provide a virtualized source of entropy to be
mixed into the pool (google "virtio-rng"). Good OSs (like linux with the right
virtio modules) will use that to have a high amount of entropy very early into
boot. If that's what you're asking, then the answer is "yes, clouds have good
urandom available, usually earlier than on real machines".

If instead you're asking "but can the cloud compromise me?" then the answer is
"you're wrecked".

If you do not trust the hardware your vm is running on (be it real or virtual
hardware), then you have no hope.

Any source of random numbers you read from will read those numbers into
memory. The hypervisor or sufficiently-malicious-hardware will then be able to
read your memory and get the data, even if it came from an hsm.

The "safe" choice is to trust that your hardware/hypervisor is not stealing
such data.

If you can't trust that, you may have to start fabricating your own processors
and running your own data-centres.

Be sure to hire better physical security than AWS has or else it's all for
naught when someone comes in and yanks your memory to read it.

