Keccak's (pronounced: ketchup) team includes Joan Daemen, who was also on the Rijndael team, giving him a hand in both AES and SHA-3.
Keccak's structure is simple and markedly different from that of MD4, MD5, SHA1, and the SHA-2 family, all of which share a common design pattern called Merkle-Damgard (MD). MD-style hashes chunk their inputs into blocks and tag the last block with a length; they then run in a manner similar to a block cipher, where each block is combined with the previous block after the hash function core is applied. The resulting output of these MD-style hashes is literally the internal state of the hash at the last block; you can take most SHA-2 hashes, for instance, and "feed them back" to the SHA function with more data to generate a continued hash. This gives rise to a common crypto protocol flaw called a "length extension attack".
Unlike MD, Keccak uses a design called a "sponge function". It's called a "Sponge" because it splits hash into two distinct stages: one in which the hash function "absorbs" data, applying the hash core function to permute an internal state, and another in which the hash is "squeezed" to produce output (which further permutes the state). Somewhat like a cryptographic PRNG, the security of the hash function is delegated to the confidentiality of the internal state, which isn't disclosed by producing the hash.
Furthermore, Keccak's Sponge design derives security by only allowing inputs to directly influence a subset of the internal state bits. Like MD hashes, the inputs are chunked into blocks and fed to the hash algorithm. But each block fed to the hash affects only that block's worth of internal state bits; the remainder of the hash's state (called the "capacity bits") are mixed in with the "outer" bits during the application of the hash core. Here's a picture that may tell the story better:
Notice how the "M" bits of the input hit only the "r" bits of "outer" state in the hash, while the "c" bits of "capacity" are used only during the "f" function.
If all it does is prevent length extension attacks, then there are much simpler and less risky ways to do that (i.e., MD variants).
Also, your explanation of the sponge structure omits the real difference between it and MD: It is a transform that turns a non-compressing function (f in that diagram) into a compressing function. MD, on the other hand, starts with a fixed-input-size compressing function and extends its domain.
By the way, what do you mean by "Furthermore, Keccak's Sponge design derives security by only allowing inputs to directly influence a subset of the internal state bits."? That's as true for an MD-type construction as it is for a sponge construction. In fact, it's a crucial fact that allows us to build a reduction from, say, the collision-resistance of MD[f] to the collision-resistance of f.
Regarding the bits exposed to inputs in Keccak, I read the claim in the same manner as the claim that CTR is more side-channel resistant because attacker ciphertext bits never hit the AES core; here further margin is given by the additional capacity bits. That's my attempt at exposition from the Sponge paper. You would know far better than I would, though; I'm a tester, not a cryptographer.
Regarding length extension, strong disagree; we see the SHA functions routinely abused this way.
BLAKE and Skein inject a message block by first running it through the block cipher (as key in BLAKE, as plaintext in Skein), and then XORing the result with the message block to get a new state. JH XORs message block into one half of the state before permutation, and into the other half after it. Keccak XORs message block into a part of state before permutation, and that's it.
Oh great, for the next year every tech meeting I go to is going to sound like five people with terminal bronchitis coughing themselves to a standstill as everyne name drops the encryption scheme they want and cannot agree how to pronounce it. I wish Schneier had won .
If Keccak isn't vulnerable to a length extension attack, is using it to hash a secret key concatenated with a message a good MAC algorithm? If not, why not?
Keccak's structure is simple and markedly different from that of MD4, MD5, SHA1, and the SHA-2 family, all of which share a common design pattern called Merkle-Damgard (MD). MD-style hashes chunk their inputs into blocks and tag the last block with a length; they then run in a manner similar to a block cipher, where each block is combined with the previous block after the hash function core is applied. The resulting output of these MD-style hashes is literally the internal state of the hash at the last block; you can take most SHA-2 hashes, for instance, and "feed them back" to the SHA function with more data to generate a continued hash. This gives rise to a common crypto protocol flaw called a "length extension attack".
Unlike MD, Keccak uses a design called a "sponge function". It's called a "Sponge" because it splits hash into two distinct stages: one in which the hash function "absorbs" data, applying the hash core function to permute an internal state, and another in which the hash is "squeezed" to produce output (which further permutes the state). Somewhat like a cryptographic PRNG, the security of the hash function is delegated to the confidentiality of the internal state, which isn't disclosed by producing the hash.
Furthermore, Keccak's Sponge design derives security by only allowing inputs to directly influence a subset of the internal state bits. Like MD hashes, the inputs are chunked into blocks and fed to the hash algorithm. But each block fed to the hash affects only that block's worth of internal state bits; the remainder of the hash's state (called the "capacity bits") are mixed in with the "outer" bits during the application of the hash core. Here's a picture that may tell the story better:
http://tinyurl.com/cryptosponge
Notice how the "M" bits of the input hit only the "r" bits of "outer" state in the hash, while the "c" bits of "capacity" are used only during the "f" function.