
NIST Selects Winner of Secure Hash Algorithm (SHA-3) Competition - dchest
http://www.nist.gov/itl/csd/sha-100212.cfm
======
tptacek
Keccak's (pronounced: ketchup) team includes Joan Daemen, who was also on the
Rijndael team, giving him a hand in both AES and SHA-3.

Keccak's structure is simple and markedly different from that of MD4, MD5,
SHA1, and the SHA-2 family, all of which share a common design pattern called
Merkle-Damgard (MD). MD-style hashes chunk their inputs into blocks and tag
the last block with a length; they then run in a manner similar to a block
cipher, where each block is combined with the previous block after the hash
function core is applied. The resulting output of these MD-style hashes is
literally the internal state of the hash at the last block; you can take most
SHA-2 hashes, for instance, and "feed them back" to the SHA function with more
data to generate a continued hash. This gives rise to a common crypto protocol
flaw called a "length extension attack".

Unlike MD, Keccak uses a design called a "sponge function". It's called a
"Sponge" because it splits hash into two distinct stages: one in which the
hash function "absorbs" data, applying the hash core function to permute an
internal state, and another in which the hash is "squeezed" to produce output
(which further permutes the state). Somewhat like a cryptographic PRNG, the
security of the hash function is delegated to the confidentiality of the
internal state, which isn't disclosed by producing the hash.

Furthermore, Keccak's Sponge design derives security by only allowing inputs
to directly influence a subset of the internal state bits. Like MD hashes, the
inputs are chunked into blocks and fed to the hash algorithm. But each block
fed to the hash affects only that block's worth of internal state bits; the
remainder of the hash's state (called the "capacity bits") are mixed in with
the "outer" bits during the application of the hash core. Here's a picture
that may tell the story better:

<http://tinyurl.com/cryptosponge>

Notice how the "M" bits of the input hit only the "r" bits of "outer" state in
the hash, while the "c" bits of "capacity" are used only during the "f"
function.

~~~
cdavidcash
If all it does is prevent length extension attacks, then there are much
simpler and less risky ways to do that (i.e., MD variants).

Also, your explanation of the sponge structure omits the real difference
between it and MD: It is a transform that turns a non-compressing function (f
in that diagram) into a compressing function. MD, on the other hand, starts
with a fixed-input-size compressing function and extends its domain.

By the way, what do you mean by "Furthermore, Keccak's Sponge design derives
security by only allowing inputs to directly influence a subset of the
internal state bits."? That's as true for an MD-type construction as it is for
a sponge construction. In fact, it's a crucial fact that allows us to build a
reduction from, say, the collision-resistance of MD[f] to the collision-
resistance of f.

~~~
tptacek
Regarding the bits exposed to inputs in Keccak, I read the claim in the same
manner as the claim that CTR is more side-channel resistant because attacker
ciphertext bits never hit the AES core; here further margin is given by the
additional capacity bits. That's my attempt at exposition from the Sponge
paper. You would know far better than I would, though; I'm a tester, not a
cryptographer.

Regarding length extension, strong disagree; we see the SHA functions
routinely abused this way.

------
apawloski
At some points throughout the competition, the determination criteria used by
NIST were somewhat unusual, with the decision committee stating that

"We preferred to be conservative about security, and in some cases did not
select algorithms with exceptional performance, largely because something
about them made us “nervous,” even though we knew of no clear attack against
the full algorithm."

[http://csrc.nist.gov/groups/ST/hash/sha-3/Round3/documents/E...](http://csrc.nist.gov/groups/ST/hash/sha-3/Round3/documents/Email_Announcing_Finalists.pdf)
[PDF]

~~~
zokier
That is actually bit more interesting in the light of Schneier complaining
about SHA3 not being significantly better than SHA2.

------
ChuckMcM
I found this overview <http://keccak.noekeon.org/> quite approachable.
Especially the pdf linked here : <http://keccak.noekeon.org/Keccak-
reference-3.0.pdf>

------
tveita
My impression is that people, in the absence of obvious (to them) differences
in security, will choose a cryptographic function based on speed, which is
simple to understand and quantify.

Going by the eBASH benchmarks, software implementations of Keccak are mostly
the same speed as SHA-2, and for some CPUs it's significantly slower.

It does have better throughput/area ratio in hardware, but chip manufacturers
are not early adapters.

With that in mind I can't see any major push for general SHA-3 adoption unless
the SHA-2 family is broken.

Still, I'm looking forward to the reading the report.

~~~
Zenst
"Keccak, created by Guido Bertoni, Joan Daemen and Gilles Van Assche of
STMicroelectronics and Michaël Peeters of NXP Semiconductors"

I don't think there will be any issue with regards to some early adopters.
Maybe even a concideration with regards to common hardware usage like
smartcards at some level.

But whilst your right about SHA-2 not being broken and making the rush to
SHA-3 moot, it does not stop marketing types getting sales over SHA-2 only
complient. In that it will be marketing and drive for sales and standing out
that will drive it into market with all things being as they stand. Besides,
everybody likes a plan B to fall back upon.

------
lifeisstillgood
Surely, as tptacek's professional life (you know the one here on HN) is
dedicated to proving that the security of the encryption algorithm is mostly
incidental to the security of the system as a whole, then I would like the
next competition to be about producing reference implementations or similar.
Sort of like keyczar but down to the point where all browsers simply
incorporate the same approved crypto libraries and Apis, securely use areas of
memory without file handles anywhere in the background and all the other good
stuff that takes us into a 80% done by default world.

I am not saying this isn't good stuff, but as someone struggling to implement
a user login properly it would be much nicer to have a template to follow.
(and he waits for someone to point to the template ...)

~~~
pjscott
A user login, you say? First, find bcrypt or scrypt bindings for your
programming language of choice. Here's one for Python, with some very concise
example code:

<http://www.mindrot.org/projects/py-bcrypt/>

Store the hashed password that bcrypt gives you in a database somewhere. Make
sure your login goes over HTTPS. I think that should cover the basics; do you
have any specific questions?

~~~
lifeisstillgood
What about handling single sign on for multiple applications - what do I put
in the cookie if anything? If I am encrypting a cookie what scheme? What key
rotation? How many failures of a cookie to decrypt is suspicious

What about password reclamation - do I keep the answers to their grandmothers
maiden name in the same database? What about adding a new email address to the
list of ones I will mail a password reset token to - do I force them to prove
ownership of the first email address before adding the second ? Or will a SMS
to their phone do?

I am sure I am missing a hundred items and none of them are to do with bcrypt
vs scrypt. Thus is hard stuff and I am avoiding most of it with the magic
words "Mozilla persona" but I still don't know how to do sso well. I can do
not badly but not well.

Edit: I am being a bit snipy and I apologise but I know I am in deep waters
and would be happier if NIST were producing reference implementations. Perhaps
actually they have - there is kerberos. If you want a key visit room 303 with
your passport

~~~
pjscott
Sounds like I misinterpreted what you were asking. Yes, those are some tricky
problems. It looks like our best choice for a longer-term answer is Persona,
but it'll be a while before it's as slick as it should be. :-(

~~~
saurik
Unless Persona drastically changed in the last couple months, it seems to
fundamentally misunderstand the problem of user authentication by assuming
email addresses are both canonical and authoritative.

<http://news.ycombinator.com/item?id=4233391>

~~~
lifeisstillgood
Well what is canonical and authoritative?

Most people keep an email address for years and that's a lot more than own
their own domain.

I simply don't know what would work better than an email address as a virtual
identifier - if u have a suggestion please (seriously I want to know) say

edit: wow am I in a bad mood. sorry! I would still like to know if there is a
better answer than email addresses - but more politely. Cheers

~~~
saurik
When people decide to change their email address, they don't want to lose
their accounts, and when someone else gets their old email address, they
shouldn't have their accounts compromised. Seriously: even just "username" is
better than email address. All of the advantages of using an email address as
the primary key for an account are either not true or _actively dangerous_
once you realize how little they mean to the vast vast majority of "normal
people".

Facebook is a reasonable example: they allow you to use email addresses to log
in, but your account is not tied to your email address. Ever since nearly
forever, they have let you add new email addresses to your account, and you
can remove old ones. They also seem to have some schemes in place for
mitigating "I lost access to my University email address, and someone else got
it" (which one would imagine to be nigh unto endemic for their use case).

~~~
tveita
This is getting a bit off topic, but:

Changing addresses can be dealt with just as it is today: Let logged in users
add additional email addresses to their accounts.

Other people getting your old email address is a bit worse. The easy solution
is for providers to not reuse names. You could extend the protocol with a
version token, so a provider could say that new bob is different from old bob,
and shouldn't be able to log in to old bob's account.

------
sp332
Here's a competing author's take on the final round of this competition:
<http://news.ycombinator.com/item?id=4564190>

~~~
tptacek
I don't think I understand Schneier's point here. This is only the second time
(AFAIK) that NIST has held a contest like this to standardize a crypto
primitive. In the first contest, there was a clear and urgent need to retire
DES. In the case of SHA-2, it was clear going in that there wasn't a "speeds
and feeds" problem with SHA so much as there was a gathering unease about the
basic design of the family of hashes from which SHA-2 is derived: MD4 and MD5
are broken, and SHA-1 isn't looking long for this world. Meanwhile, hash
functions are not as well studied as block ciphers and number theoretic
crypto, and the pace of development in hash research is increasing.

There is value simply in diversifying the gene pool of hash functions.

~~~
apawloski
I think it was just a case of Schneier being a bit of his usual contrarian
self. But is he correct in his assertion that the security of SHA-3 is not
going to be much greater than that of SHA-2? (I don't mean that as a leading
question.)

~~~
tptacek
What does "greater" mean? I buy into the idea that the attack that breaks
SHA-2 is unlikely to break SHA-3, and vice versa, and that's apparently a key
design goal for the standard.

~~~
B-Con
"greater" here means that we would think that SHA-2 is easier to break than
SHA-3. Technically speaking, it's a guess, but I think that the general
consensus is that it is more likely that SHA-2 will be broken before SHA-3.

------
B-Con
A little more detail from NIST here:
<http://www.nist.gov/itl/csd/sha-100212.cfm>

~~~
dchest
It's the same link.

~~~
B-Con
Yep. Wrong tab. A different submission did not link to this page and I meant
to post there.

