

On cryptography and dogmas - jcsalterego
http://antirez.com/post/crypto-dogmas.html

======
jgrahamc
"It is very important to force users to add non alphanumerical characters and
a few capital letters in the password IF security is very important for your
application. "

Not it's not. What's important is that users pick a password randomly from a
large pool. For example, there's nothing wrong with a long password all in
lowercase if the characters are picked randomly (see, for example, how Google
2-factor authentication handles application specific passwords).

"But guess what? This morning I discovered that actually the algorithm PBKDF1
described into RFC2898 does exactly what I proposed."

Actually PBKDF1 has been deprecated since 2000 and replaced by PBKDF2 which
doesn't use SHA1 (it uses HMAC-SHA1 instead). And PBKDF1 is a key derivation
function, it's not designed to be slow (as is, for example, bcrypt).

Nevertheless, you could iterate SHA1 if you wish.

~~~
viraptor
What's wrong with long password with all lowercase characters which are not
picked randomly? Apart from someone looking at you typing, why would
"fuwaiunviohugihyeurpqwjiosnxjcewiorhewuioahfdsfeaw" be worse than "i like
unicorns in the morning and hedgehogs in the evening"?

~~~
jgrahamc
There's nothing really wrong with that as long as it's not predictable in some
way. For example, if you knew that all my passwords were song lyrics in
lowercase then you could attack my passwords using list of all the world's
song lyrics (which would be much smaller than all random lowercase strings of
length X).

If you come up with something you can remember that's long and unlikely
someone else can guess the search space for then you'll be ok.

~~~
pork
If I knew your passwords were all song lyrics, that's already a failure of
security.

~~~
eru
In a sense. But it's a principle of cryptography that you assume the attacker
knows all your schemes / algorithms, and only the random bits are secret.

~~~
pork
Your scheme can be a few more random bits, selecting from a finite number of
"schemes".

~~~
eru
Yes.

------
aaronsw
This is moronic. Cryptography is very hard and one tiny mistake can ruin
everything. In that kind of situation, do you want to do something clever and
new that you just thought up or do you want to go with what's been tried and
tested by many?

Anyone can invent a cryptosystem that they themselves can't break. That's why
you need a community, over a long period of time, searching for flaws. Going
with the herd is exactly the right thing to do here.

~~~
tptacek
The whole situation is moronic.

It's hard to look at the whole discussion here and not wish that Cody had just
asked Salvatore in private if he wanted a better hash function, rather than
calling him out for it on the thread announcing Salvatore's new program. I've
been exactly where Cody is and have learned that there's little productive
conversation to be had when someone is excitedly announcing a new project.

At the same time, Salvatore was too prickly about this. His response was
dictated by emotion and not his head, and it's painted him into a corner of
referring to sound crypto as "dogma" that can be navigated by programmer
common sense. He's wrong about that and I suspect he knows it. He could still
have been snippy about being told to add bcrypt to his sample application,
without trying to make a principled stand about the merits of different KDFs.
This isn't the first time Salvatore has been stridently wrong about crypto on
HN.

Coda, like the fabled honey badgers of yore, does not give a fuck. If you
understand that going in, it's hard to be pissed at him.

One of the charming things about Salvatore's code is that it's build largely
without deps. It is probably my favorite thing about Redis, that you can
download it and simply type "make"; it doesn't have an autoconf script and
implements its own event library. It takes craftsmanship to do that on
something as significant as Redis.

It is indeed a downside of bcrypt that it pulls in a dep. If you are avoiding
deps as a matter of principle, use a different KDF (this applies _only to
KDFs_ ; if you need encryption and you DIY, you're boned). But as soon as you
write a Gemfile, I reserve the right to make fun of you for hand-rolling your
KDF.

~~~
antirez
Just to say that I apologize with the HN community and with my twitter
followers about my behavior in the past hours. I and coda both made mistakes
in our interaction.

What is good is that the result is that, at least, I'm understanding more on
the topic. But I guess all this was not needed to reach this goal.

Edit: oh and another thing is, I'm very entusiast about lamer news, but not
because of the code, that many programers can implement without troubles, but
because of the project.

Slashdot, programming reddit, HN, gave me _a lot_ as a programmer. It is an
honor for me to partecipate to a public discussion with so much skilled people
here. So I think creating a consortium, like a non profit org, to make this
even better, could be awesome.

~~~
tptacek
I'm going to be using Lamer News as the basis for a (non-programmer) project.
Thank you for releasing it.

------
Fluxx
I think it's important to not confuse "dogma" with "development best
practices," a lack of which is one of the problems that is hurting software
development. People go off and do their own versions of password hashing
because they don't know what's the best practice or think they'r e being safe.

Is it important that you know _why_ brcypyt is a best practice? Yes. I
understand that it (tunably) is slow and uses a random salt, which I also know
is a best practice for hashing passwords. Is doing repetitive sha1s going to
work well enough? Probably, but I don't know enough about cryptography to know
fo sure. So when smart people say, "use bcrypt," I do just that.

Programming is too big of a topic for everyone to understand everything about
what they're using. We have to trust others.

~~~
lusis
IMHO, the thing that bothered me about the whole exchange was that it could
have been avoided with a little bit of common courtesy. Instead it turned into
a "oh let's go roll that pull request".

I picked up pretty early on that antirez wanted to know the "why" first since
one of his stated goals was keeping deps to a minimum. What could have been a
nice productive discussion about crypto standards was fucked up by ego,
asshattery and language barriers.

Oh and github pull requests are TOTALLY the appropriate place to have those
kinds of discussions =/

~~~
thaumaturgy
Absolutely. Coda Hale's "fuck tact" statement does a neat job of ignoring that
"fuck tact" leads to stupid arguments like this.

------
timf
This comment from Coda Hale is worth a look:
[https://github.com/antirez/lamernews/pull/8#issuecomment-248...](https://github.com/antirez/lamernews/pull/8#issuecomment-2481391)

~~~
llambda
Looks like an unfortunate interaction all the way around: I have to admit
reading the blog post I felt that the "dogma" lay on the other end of the
table...now this link seems to confirm that suspicion.

There is a reason people said, "use bcrypt," and certainly Coda is well
qualified to elucidate the details.

Side note: It's a little disappointing to see this kind of negativity shoot up
on the front page. But by the same token I'm glad that HN readers are so
observant and timf linked to a more complete history of what went down.

~~~
boundlessdreamz
antirez was very civilized initially. That pull request has a wall of "use
bcrypt" comments. It is just a despicable instance of mob mentality.

~~~
tptacek
"Despicable"? Are you sure that's the word you want to use? I'm not fully on
Coda's side on this (it's a silly news site, and I've told more than one HN'er
to just stretch SHA1 with iterations), but a great way to guarantee that I end
up there is to polarize the discussion with comments like this.

It's actually people like you who create stupid flame wars like this. Do you
actually have an opinion about bcrypt versus PBKDF1? Or are you just sitting
on the sidelines chanting "FIGHT FIGHT FIGHT"?

Incidentally: in the fully polarized discussion, Salvatore loses.

~~~
boundlessdreamz
Despicable to just add "use bcrypt" comment without understanding why antirez
was against it and not responding to his comments. There were several people
on that thread who corrected antirez but did it in a civilized manner. I was
specifically talking about people who just added "use bcrypt" and not coda

What is "people like me"?

I don't have an opinion about bcrypt v/s PBKDF1. I have an opinion about how
people should provide feedback. I don't know where you get the idea that I'm
chanting "fight fight fight". Exactly the opposite actually. Provide
constructive feedback and don't just mindlessly litter a discussion with "use
bcrypt"

~~~
tptacek
Since you don't have an opinion about the actual subject we're discussing,
you're really only here to talk about people. People I assume you don't
actually know. You think you're contributing, but you're just like the kids on
the playground taunting the two kids who look like they're about to fight.

~~~
boundlessdreamz
You are just trolling. Calm down! I don't think you are even reading my
comments fully.

Again I don't know where you are getting the idea that I'm taunting anyone.
Also I'm not commenting on people but their collective behaviour. It is an
important difference. You are just making baseless allegations.

"two kids" ? If you are referring to coda & antirez, I specifically said I was
not referring to coda.

------
khafra
Be careful. Tptacek may show up soon to explain in more detail why this is a
dangerous direction to go, but the fact that PBKDF2 is a reasonably well-
studied key stretching algorithm is the only reason I'd be comfortable
iterating SHA1.

To put it simply, there's just so many incredibly subtle ways to introduce
weaknesses into crypto that even if you really do understand everything that's
going on, you'll still break things by building it yourself.

~~~
antirez
the whole point is in the "to explain". As far it is a matter of explaining
everything is fine and I and others can improve. The problem is when you see
8000 tweets just saying "use bcrypt" without having a clue about what is wrong
about another approach.

Note that I also trust that we should try to follow well established and
proven standard, simply not as drones.

~~~
xtacy
antirez, while I really appreciate the sentiment that one should not simply
follow dogmas but be curious/question/understand them, there is a fine
balance. In your case, if it were a learning exercise, it's perfectly fine to
invest time and understand why other approaches are bad.

But if you're writing a software that other people depend on, perhaps it's
best if you follow widely accepted principles, but do the questioning and
curiosity _later_. On a meta level, depending on a person's level of
curiosity, it's very difficult to answer why certain things are the way they
are. Sometimes, you just have to take the word for it. :)

I am not pointing fingers at you specifically, but it's just a general
observation. I do not know if Lamer News was just an experiment, or a serious
effort that others can use.

Cheers,

~~~
antirez
I agree with you. The problem comes when we are so focused on proven stuff
that we end allowing only one possibility at all, since the implementation I
suggested was well established practice with an RFC.

~~~
thaumaturgy
> _The problem comes when we are so focused on proven stuff..._

I sympathize with your overall point, but there's a really good reason why the
focus _should be_ on proven stuff: because it's been proven.

You are building an application which will be responsible for safeguarding
some of its users' information. You should typically want to do that using a
proven approach. If nothing else, it's a cover-your-ass situation: if it turns
out that the proven approach has a flaw, you can at least say that you
followed standard practices. If your invention has a flaw, you hold all the
responsibility.

I hate to fall back to argument-by-analogy (but I have to, since I'm not a
cryptographer), but you're somewhat in the situation of installing a fire
suppression system for someone, and you're saying, "Well, I don't want to use
halon for this, even though it's what all the experts say I should use,
because I don't want to deal with the company selling the halon equipment. So
let's just use lots and lots of kitchen fire extinguishers instead. They do
the same job, and if I use lots of them, it will work just as well."

Maybe. Maybe it will. But, if it doesn't, you will be responsible for having
made a decision that flies in the face of lots of advice from lots of very
smart people, and the consequences of that decision will affect not just you
but also anyone that uses your system.

edit: On the more technical side of things, maybe nobody has yet pointed you
to Bruce Schneier's 2005 article on SHA1 weaknesses
([http://www.schneier.com/blog/archives/2005/02/cryptanalysis_...](http://www.schneier.com/blog/archives/2005/02/cryptanalysis_o.html)).
There are several very good points in there about the dangers of using broken
cryptographic algorithms. By stretching a broken hash, you could be -- and
probably are -- compounding its weaknesses. i.e., you might be making it
_easier_ to find a collision, not harder. And, that article was written almost
7 years ago; since then, newer low-cost hardware has been made available which
can compute 33 billion MD5 hashes per second
(<http://blog.zorinaq.com/?e=42>), and that was almost a year ago. I swear I
read this year about some Russian kids doing a lot better than that, and that
ignores things like EC2 which make it economical to do heavy-duty distributed
hash breaking.

~~~
xtacy
Interesting points. Another way to convince oneself, is that even experienced
professionals are prone to make mistakes.

This is an interesting set of slides that captures the same thoughts:

    
    
        http://www.bsdcan.org/2010/schedule/attachments/135_crypto1hr.pdf

------
falava
When I saw SHA1(password|salt) in the readme of
<https://github.com/antirez/lamernews> I knew a flame was coming, and that
"use bcrypt" and "cryptography is hard" will popup, but not really exactly
why?

Thanks for this post antirez, because after diving a little, now I know better
the reason, the risks, that tarsnap creator uses scrypt
(<http://www.tarsnap.com/scrypt.html>), that really "cryptography is hard", he
also makes bugs ([http://www.daemonology.net/blog/2011-01-18-tarsnap-
critical-...](http://www.daemonology.net/blog/2011-01-18-tarsnap-critical-
security-bug.html))

And this funny little gem: <http://xkcd.com/538/>

------
yogsototh
It seems it started a flamewar about sha1 and bcrypt.

First: scrypt (<http://www.tarsnap.com/scrypt/>) is far better than bcrypt.

Second here is the general idea behind the scrypt algorithm (from the scrypt
slides):

Algorithm ROMix:

    
    
        Given 
        a hash function H, 
        an input B (the user password), 
        and an integer parameter N (computational hardness), 
        compute:

￼￼￼￼￼￼￼￼￼￼￼￼￼￼￼￼ V_i =H^i(B) (0 <= i < N) and X = H^N(B), then iterate

    
    
            j ← Integerify(X ) mod N 
            X ← H (X ⊕ Vj )
    
        N times; and output X.
    

The function Integerify can be any bijection from {0, 1}^k to {0...2^k −1}.

Theorem:

Under the random oracle model, the class of functions ROMix are sequential
memory-hard.

More intuitively,

    
    
        V_0 = H(B)
        V_1 = H(H(B))
        ...
        X = V_N = H(H( ... H(B) ...) )
    

V_i are filled with pseudo random values.

Then the algorithm access them in pseudo random order.

Which means that is is not only very long to compute but also need a lot of
memory.

Now zealot should say _use scrypt_! not bcrypt. And also, even if cryptography
is difficult, the idea behind is not so difficult.

~~~
rnicholson
> First: scrypt (<http://www.tarsnap.com/scrypt/>) is far better than bcrypt.

Is there any independent analysis on scrypt? Only paper on scrypt I've seen
comes from Colin. That's not meant as an attack; I'm interested in learning
more, but can't find anything else.

------
gcp
I couldn't be bothered to read the article entirely, but what he proposes at
the end (iterating SHA) is likely broken:

[https://groups.google.com/group/sci.crypt/msg/92fe3e4e1edf0d...](https://groups.google.com/group/sci.crypt/msg/92fe3e4e1edf0d0f?hl=nl)

~~~
antirez
I think this is a different context where it does not make the attack harder.
Our context is different AFAIK and the approach works.

~~~
gcp
I guess the difference is that you're not actually interested in slowing down
the generation of a collision, just that of a dictionary attack.

~~~
antirez
This is how it works. Here the attack they want to mount is the following:
find another string, ANY string, that will hash to the same output, but only
32 bits of the output.

Since it is any string, it can also be a SHA1 itself. So what you do is to
start with an "X" that can be ANY ANY value, even "foo". And you start doing:

    
    
        x = SHA1(x)
        x = SHA1(x)
        ... again and again ...
    

right? Well in the average case after 2^31 iterations you find a collision,
right?

But the output 65536 iterations ago was it! The string that will output that
specific 32 bit output after SHA1() nested 65536 times. So you want to go
backward but it is not possible, SHA1 can't be inverted.

So what you do? You start again from "X" and stop exactly 65536 iterations
before you found the wanted value.

Obviously doing 65536 more SHA1s of that string you get the previous output.
So you found your string.

Why the original poster says that the attack takes 2x time but can even
optimized? Since you can store the value of SHA1 at 10000 iterations, at 20000
and so forth. Then instead of re-running the iteration again you start from
the nearest cached value.

Is it clear now? Otherwise please ask me and I'll be willing to help.

Edit: do you see that chat? This is the dogmatic approach we don't need. gcp
found a random message on sci.crypt, and it used it as an universal proof that
nested SHA1 is wrong (well he said "likely" actually). It may be wrong for
other reasons perhaps, but not for this. First of all: try to understand what
you read.

~~~
gcp
I think there's a fairly huge difference between pointing out an exact attack
on the primitive you propose versus dogmatically rejecting it because it's not
commonly used. For one, it's possible to have this discussion and arrive at an
obvious conclusion (attack does not apply).

 _First of all: try to understand what you read_

I didn't read your blog post, as I clearly stated at the beginning :-)

------
shin_lao
You need to salt each round of SHA-1 otherwise you don't add much security.

You may also want to use SHA-256 instead of SHA-1 and that doesn't add any new
dependency AFAIK. Ideally you should use SHA-3 as soon as it's available.
SHA-1 is becoming weaker each day...

You may want to offer the possibility to easily replace the password hash
algorithm via a snapin (so people who like bcrypt can use it).

~~~
antirez
Hello, please care to share why without an iteration dependent salt it is less
secure? I guess what you mean is:

    
    
        x = SHA1(password|salt)
        for i from 0 to N do {
            x = SHA1(x|i)
        }
    

But in our specific context how this helps? In other words, how the attacker
is able to compute N-times-nested-SHA1 faster than performing all the
iterations?

Note: this is not a pre-image attack. Thanks.

~~~
skeptical
I tend to be of the opinion that strength of hashes s overrated, other
security measures should jump in and this whole discussion should be cooled
down.

That said, but nesting sha1 calls you are reducing the size of the output set
at each step. You will end up with a reasonably smaller possible hashes. I
don't know why everybody seems to ignore this.

Does password hashing qualifies as chryptography? I don't think so.

~~~
antirez
"That said, but nesting sha1 calls you are reducing the size of the output set
at each step"

I don't think so otherwise finding collisions would be trivial. What you do
when using nested SHA1 is just to run a cryptographically secure PRNG with a
160 bit internal state.

~~~
shin_lao
The in-depth security means you design your system so that it doesn't fall
apart if only one of the pieces fall.

As for your comment on the PRNG, it's actually not a very good design. You
would rather use a cipher in stream mode which is indeed fed from a hash,
because the cipher will have a better and more predictable behaviour over
time.

I however don't understand why people were repeating bcrypt as a mantra on
your git pull.

------
jasonwatkinspdx
_"It is very important to force users to add non alphanumerical characters and
a few capital letters in the password IF security is very important for your
application."_

This is a mistake. Virtually all users respond to these requirements using one
of a small number of tactics. The requirements add little to no entropy, which
makes them very dangerous, because that leads to the mistaken belief that "all
our passwords are strong".

The only secure password is a randomly generated one. You can use
words/phrases a la diceware or xkcd if it's easier to remember.

------
nathanwdavis
In case you haven't seen the long thread of comments on the pull request that
lead to this blog post: <https://github.com/antirez/lamernews/pull/8>

------
mef
Regardless of who is right or wrong on this issue, I have to say I'm pretty
impressed with antirez for keeping his cool and being up for genuine dialogue
in the face of some pretty brutal and abusive comments from Coda Hale and
others.

~~~
jcdavis
Security stuff like this is exactly what you _shouldn't_ be having this type
of "genuine dialog" about unless you are an expert. Thats not a slight against
antirez-I sure as hell am not a security expert either. The point is, this is
an area where people shouldn't be trying to get clever. Even in this same
thread, jgc has pointed out that PBKDF1 is deprecated, and that it's not
designed to be slow (and thus not a good candidate for a password hashing
function). Is he right? Maybe. I don't know. Do you? Does antirez? Yes, the
folks commenting on that pull request were being dicks, but that doesn't mean
this is the appropriate response.

~~~
mef
This is a well reasoned response, but I want to point out that there is
nothing wrong with acknowledging that one is a non-expert and wanting to
engage in discussion regardless.

Some participants in this thread and in the conversations with antirez seem to
think that all developers who are not experts should always blindly obey the
cryptography best practices handed down from on high without ever being
interested in the details or questioning the whys or wherefores of those
practices.

------
yogsototh
I am the creator of YPassword, and each time I asked if my algorithm was
secure I stumbled upon a lot of people not understanding anything about
cryptography that claimed to me:

"Hey! sha1 is sooo absolutely insecure! Use bcrypt!" :-/

"Hey! bcrypt is sooo incredibly insecure! Use scrypt!" (only implemented in
C).

You know what? I finished to read the scrypt paper. In fact _theoretically_
scrypt is far more secure than bcrypt, himself (AFAIK) better than PBKDF1. And
the general algorithm behind scrypt is _simple_. Not as simple as sha1^n but
almost as simple.

I tried to answer my theoretical question two times now. Each time I never
meet any real security expert, only a bunch of zealot. And I know they are all
zealot because I discussed with a lot of security expert (searchers) during my
Ph. D. And I know far more about security than people reading my question
might think. Even if it feels like, my question is _not_ a newbie question.
But I am not an expert either.

I try my chance with you dear HNers. Here is the question:

Let

    
    
      sha1(salt|pass)=S
    

Knowing "salt" and "S", is there a known attack better than brute force to
discover "pass" or sha1(salt2|pass) where salt2 is known and dependent only of
salt (for example salt2 = salt + 1)?

~~~
tptacek
This is exactly the kind of batshit comment I'm talking about when I say
Salvatore shouldn't be calling bcrypt "dogma", even when not using bcrypt is a
reasonable choice for him.

In the nineteen seventies, Unix password files had _random_ per user salts ---
not salts derived from one another, but random ones --- and those password
hashes were better than this one.

~~~
yogsototh
Hi! As you are an expert in the domain, do you know if my method to generate
personal password seems secure?

Typically, instead of memorizing a lot of random password, I memorize only one
password and for each website I use the password:

sha1(password|domainname)

Until here nobody give me a clear answer, and therefore I assume it is secure
while not any flaw is discovered.

And more precisely, I use

sha1(password|number|domainname)

because, if I fear my password was discovered, I change it by incrementing the
number.

I would be very grateful if you mind answer my question. Thanks!

~~~
JoachimSchipper
That's vulnerable to a length extension attack: sha1(passwd|am) can be used to
derive sha1(passwd|amazon). (See Wikipedia for a discussion.)

Don't try to get clever, just use random passwords.

~~~
yogsototh
Thank you so much! It is exactly what I was looking for.

------
epenn
Can someone explain to me how salts help a hashing algorithm secure itself
against a rainbow table? For example, let's say my password is "justinbieber",
which then has a salt prefixed to it which turns it into "iheartjustinbieber".
If an attacker is unfortunately able to gain access to the hash of my password
and the rainbow table he uses has an entry for hash("iheartjustinbieber"), how
is that any more secure than the original?

Edit: changed "iheartjustinbieber" to hash("iheartjustinbieber") in the last
sentence.

~~~
Robin_Message
You have to use the salt in a way that is not trivially breakable. Look up
HMAC for the only approved way I know of to do it (whoops, dogma; you can
probably invent many ways to do it, but HMAC is simple and considered secure,
and I don't know anything else that is). Anyone suggesting just making the
salt longer is missing the whole damn point of this debate: Cryptography is
really hard to get right and smarter people than all of us have screwed it up.
Thus, relying on only the best decisions checked by many others is not
"dogma"; its the only sensible decision.

~~~
gizzlon
I completely agree with your "don't roll your own crypto" argument. But
there's a difference between using a hash and inventing your own cryptography.

What kind of use did you have in mind with _"trivially breakable"_?
hash(salt+pass) ?

My point is just that _some_ (not all) things considered unsafe in crypto is
still good enough for this kind of use.

~~~
politician
The wikipedia article on HMAC has a good discussion of why hash(salt+pass)
isn't the best way to good.

"The design of the HMAC specification was motivated by the existence of
attacks on more trivial mechanisms for combining a key with a hash function.
For example, one might assume the same security that HMAC provides could be
achieved with MAC = H(key ∥ message). However, this method suffers from a
serious flaw: with most hash functions, it is easy to append data to the
message without knowing the key and obtain another valid MAC. The alternative,
appending the key using MAC = H(message ∥ key), suffers from the problem that
an attacker who can find a collision in the (unkeyed) hash function has a
collision in the MAC. Using MAC = H(key ∥ message ∥ key) is better, however
various security papers have suggested vulnerabilities with this approach,
even when two different keys are used.[1][3][4]

No known extensions attacks have been found against the current HMAC
specification which is defined as H(key1 ∥ H(key2 ∥ message)) because the
outer application of the hash function masks the intermediate result of the
internal hash. The values of ipad and opad are not critical to the security of
the algorithm, but were defined in such a way to have a large Hamming distance
from each other and so the inner and outer keys will have fewer bits in
common."

The why crypto works is that cryptographers -- some of the most OCD pedants
you will ever know -- find a nano-scale fracture in one small relatively
unimportant part of an algorithm, and then wrench it open into a gaping lava-
spewing chasm of exploitation and credit card theft.

Please just use best practices, but realize that they will be periodically be
updated.

------
Zash
Obligatory: <http://xkcd.com/936/>

------
JonnieCache
Twitter: the worlds premier platform for sanctimonious whining from people who
don't know you.

