

SHA-3 to Be Announced - stalled
http://www.schneier.com/blog/archives/2012/09/sha-3_will_be_a.html

======
exDM69
I implemented the Skein hash in a crypto class at my uni. What is remarkable
about that hash is that it has a "hash tree" mode which provides an
interesting opportunity for parallelization and doing hashing of partial data.
In contrast, many traditional hash algorithms are inherently sequential by
nature.

On the other hand, as Mr. Schneier points out in the article, the Skein hash
utilizes a modified Threefish block cipher, while many of the SHA-3
contestants were AES-based (edit: seems like none of the finalists are). Now
we have a hardware AES implementation shipping in mainstream processors, so it
gives an edge to the AES-based hash functions out there.

edit: I went through the list of finalists and it seems none of them actually
use the whole AES block cipher, although several of them use AES S-boxes or
other parts of AES.

~~~
Zenst
I believe they worked towards speed mostly and whilst AES is in alot of
hardware the basis of there design was to be hardware independant and with
that in mind you can see why no design bias was placed upon what is or not
currently enabled in some hardware. In a world were we still have no standard
in ragards of BIG or LITTLE ENDIAN then you can see they took the right
approach.

Only aspect that Mr Schneier has not highlighted and which you touched upon is
that some of the hashing finalists are faster and in that they will in brute
force comparisions be not as good. That is another consideration, albeit one
of limited time as you already said AES is now in some hardware as
instructions and how long since AES being a standard has that taken.

But the whole ability to work on partial blocks and that is what you need to
do to gain from parallelisation does open up a while new ballgame in
modifiying hashed code and returning the same hash. Now if you can pick a
partial block size and then do a brute force test modifying from 0-FF each
byte of the block incrementaly for the every permutation and only get a match
on the original then that will by design be the one I'd be picking, i'm sure
they will be testing each parallel finalist in that way. But personaly if they
all pass that test then why pick one winner and have them all as part of the
standard SH-3.[1-5]. After all if they all work well then having the choice
can only add to the entropy we call security these days.

~~~
tptacek
Again: the "brute force" thing you're mentioning here is a non sequitur. It's
akin to suggesting that Intel should manufacture slower chips to make
passwords hard to crack. Hash functions should be as fast as possible; it's
the job of a KDF construction to slow the process down to resist brute
forcing.

~~~
Zenst
It's the aspect of being able to change part of the data and only have to
brute force that part of the data so if you have a block of data that you can
modify and rehash check until you get the same hash you have a whole new issue
with hashing by design of being able to work on partial block and paralise
things. No I'm not suggetsing intel should make slower chips, I am suggesting
that any design of hashing that allows you to modify a small part of the
entire lump of data/code and you only have to recheck the hash of that small
block instead of the entire lump of data/code then even if they hashing of X
bytes took the same or even slower than SH-2 you will still by design induce a
weaker system that is open to a easier attacks than any system were you have
to rehash the entire block of data sequentualy. Nothing to do with KDF!

~~~
ryan-c
It is trivial to create constructions where changing a single bit of the data
requires recomputing the hash over all of the data. A really simple option to
that end is to feed the hash back into itself multiple times. Please go read a
paper on PBKDF2 and/or scrypt - they are designed to work with an arbitrary
hash function be impossible to parallelize.

~~~
Zenst
ok read about PBKDF2 and what your desfribing/on about is key stretching or
hash feedback cycles and is a great appraoch to take for what it does. Though
for what I was on about it would still be a case of working out the hash for a
smaller part than the sum of the entire data, but as I have thought though. If
you stripe the data blocks across the sum of the data then the ability to
modify a small part of the data would still result in you taking just as long
to work out the hash for the entire sum of data. It is when blocking is done
in contiguous blocks (100 byte data block 1 is bytes 1-10, block 2 is 11-20
etc) then is were you have the issue I describe and yes what you say about
hash feedback is correct in that for any block it will add a extra time factor
but it will still be less than the entire lot of data and if you can erhash 10
bytes instead of the entire 100 bytes it can only be quicker. But if those
blocks are stippped then you would get the best of both worlds and have the
ability to parallise your hashing without fear of being any weaker by design,
but thats only if you stripe the data blocks you paralise and not if you use
contiguise blocks. I'll have a good read of these finalists and more over the
weekend for this, though far better brains than I wrote them so I expect this
is me just learning what they already know.

~~~
ghshephard
Zenst - it sounds like you have a lot of interest in cryptography, and your
lack of familiarity with PBKDF2 and friends suggests that you have just
entered this space.

I highly recommend <https://www.coursera.org/course/crypto> as a great
introduction into some of the more interesting elements you are working
through.

~~~
Zenst
Thank you and signed up. I have interests in too many things that cross over,
but this is one area I do need to step back a bit and learn the lingo a bit
more. Looking forward to this course now.

------
helper
I'm a bit surprised that Schneier is advocating for "no award". Even if the
SHA-3 candidates are not fundamentally better than SHA-512, we really do need
a standardized algorithm that has built in protection from length extension
attacks.

~~~
kbolino
Is there something about HMAC, which can augment any hash function, that makes
it insufficient for the task?

~~~
tptacek
Yes. HMAC is very slow. Also, length extension is a pernicious and scary
little property to have lurking around your crypto designs; you want it gone.

------
swordswinger12
I think NIST should have a big Apple-esque unveiling event for new crypto. I
for one am that excited about SHA-3.

~~~
DanBC
That's a good idea.

There's a risk of creating hype among people who don't know enough to be able
to make a decision, but who do have power to make that decision.

(<http://dilbert.com/strips/comic/1995-11-17/>)

~~~
thrill
I can envision a great Samsung-ish commercial about waiting in line to get the
new hash - one of you creative types could have fun with that spoof (of a
spoof)

------
Zenst
Interesting that the reason for SHA-3 has been missed in that the finalists
offer no better way to hash with the main difference being some are faster and
some slower than the best SH2 variations.

What does this mean, well in effect no extra value is being directly offered,
sure some have extra abilities by design like being more able to liberate
parallel processing by sbeing able to split the data to be hashed into chunks
and work on partial blocks of the final data and use the results to get the
final hash result. That is nice.

But when it comes to brute forcing then being faster works against you, also
the ability to work on partial chunks of the data allows you to modify the
code and rechecking the partial hash for the part your changing until you get
the same result, this alows you to do nasty things to code and get the
official hash answear alot easier than having to rehash the end result every
time and getting the same result or modifying the code to get the same result
(usualy have area you jump over all nop and modify that to influence the hash,
but more sane ways to do this but offtopic).

So in essence any hash that can be run faster in any way will make it weaker
in terms of brut forcing (yes I know people assume there passwords will be the
last one on the list to be checked bia brute forcing and assume if it takes 10
years to test all variations then there password is 10 years strong, you see
the flaw in mentality there).

Now NIST still have an opertunity here and it is a simple, tried and tested
approach and that would be to have all finalists winners and have them all in
the standard as variations. This then allows end users/admins to pick there
variation of choice or even perish the thought allow mixed usage so say your
/etc/password file could have some users using one variation, others using
another, etc. Whilst it add's no obvious extra benifit, it will allow more
variations and in that fallbacks/choice and that is what n BIT
encryption/hashing is all about, each bit being a choice in a way.

So in summary I believe NIST should let them all win and have SH3.n with n
being the variation of finalist, let them all win, choice is good and that is
what n bit encryption is after all, extra choices.

~~~
tptacek
Ugh. Being faster does not work against secure hash functions! Holding all
else equal, faster is invariably better.

What you're thinking of are _password hashes_ , which are a
variant/application of key derivation functions (KDFs). KDFs often use secure
hash functions, which is where the confusion comes from.

You want your core crypto to be as fast as it conceivably can be, because you
want to be making progress towards a state where all communications are
encrypted by default.

~~~
Zenst
In some instances yes faster is always betetr and take AES, whilst initialy
was slower, now that it is enabled in hardware it is now alot faster.

The point I was making is that it is a concideration and the general mentality
is that the larger the version number then the better it is and a point the
original article was making in that none of them are any better than what is
on offer with regards to security. Its is tha aspect of being able to get a
large hashed file and modify part of that file and recheck just that partial
HASH without having to rehash the whole thing. This for comminucations starts
to open up a faster way to modify encrypted communications as by changing a
small part you only have to rehash that part and know the final block is still
ok. This is a area which makes by design any hash function can work with
partial blocks, less secure.

So fast is good but it often comes as a compromise against security and any
new standard should at least be better than what it is designed to replace and
not open up whole new avenues of attack.

~~~
mjb
> was making in that none of them are any better than what is on offer with
> regards to security

In this case, possibly. It is quite clear by now that SHA-1 and MD5 are
flawed, so the 'higher version' SHA-2 variants (especially the bigger ones)
should be preferred.

> So fast is good but it often comes as a compromise against security and any
> new standard should at least be better than what it is designed to replace
> and not open up whole new avenues of attack.

Brute force attacks against 512 bit hashes are not practical today, and won't
be practical for a long time. The concern with password storage is seldom pure
brute force attacks, but rather attacks against a dictionary. This is because,
for password storage, the input entropy is typically much less than 512 bits
(or even 128 bits). It's a completely different use case.

> Its is tha aspect of being able to get a large hashed file and modify part
> of that file and recheck just that partial HASH without having to rehash the
> whole thing.

Is this an argument against hash trees? Can you explain more about the this
potential attack? It seems to be to be equivalently hard to finding preimages.

~~~
Zenst
>Is this an argument against hash trees? Can you explain more about the this
potential attack? It seems to be to be equivalently hard to finding preimages.

If you only have to rehash a branch as apoosed to the entire tree and match
hash's then you have a easier time as it is alot faster by design.

Now if the way the hashing work is that only say 1234567 will get the hash
value 11 and no other variation then i will have no issues and welcome this as
a great achievement and pure brilliance. But I dont feel this is the case and
nor could it be by reducing any large amount of entropy into a shorter
definition and that is what a hash function does after all and one hash value
will match more than the original data.

~~~
Zenst
Actualy if you pick the block to paralise striped (ie 100 bytes and block 1 is
every 10th byte so byte 1,11,21,31... and block 2 is 2,12,22,32....) then the
ability to modify the code/data in any nefarious means would be as hard (if
not harder ) than having to rehash the entire lot.

The only issue with this approach of blocking is that it works on all the data
and as such would for example be no use for streaming which is a terrable
exmaple but you get the idea.

------
dochtman
djb thought in March it was going to be Keccak:

<https://twitter.com/hashbreaker/status/183552364953878528>

------
JeremyBanks
Schneier picked a very misleading headline here. I was wary when I saw that
the NIST page he links regarding the timeline still hasn't been updated since
June, and then I saw him reply in the comments:

 _" > When will SHA3 be announced? Were you given special information the rest
of us don't have access to?_

 _I have no inside information on when SHA-3 will be announced. My guess is
that they've made the decision, and are going over the final rationale again
and again. My guess is that it won't be Skein."_

Even though this is the original title, I'd prefer the HN title be edited to
something about Schneier hoping NIST will pick no SHA-3.

------
Zenst
Out of interest these hash functions can be implemented in very few bytes with
100 being mooted for this skien hash. With that in mind when it comes to brute
forcing I do wonder if it would be possible to just brute force a better
solution easier than brute forcing a hash, I say that in jest.

But it does make you reliase how much empressive stuff you can do in just a
few bytes and what else is out there.

