Good work from Bruce Leidl's team at Netifera (Bruce is brl, a regular).
Long story short: with all the mainstream hash algorithms, if you know the length of a hashed message, you can take its hash and from it synthesize the hashes of messages with arbitrary suffixes. It's called a message extension attack.
A really common pattern in hand-rolled authentication schemes is to hash a message needing authentication with a secret key, and then send the message and the hash. If you know the secret key, you can verify the message. The secret key doesn't need to be sent at all.
If your scheme is vulnerable to message extension attacks, attackers can mint arbitrary messages with valid authentication by taking an already-authenticated message and extending it to cover a suffix that changes the meaning of the legitimate message. It's a game-over flaw.
Colin Percival noticed a related (but less serious) flaw in Amazon's AWS authentication scheme. At the time, people asked if he'd looked at Flickr's scheme. He had, but not very closely. Colin's flaw was in canonicalization of messages, which is a more obscure attack (by a little bit) than message extension, which is is described in the Wikipedia under "Cryptographic Hash Function". Colin didn't flag this flaw.
Which just shows to go you: crypto primitives are radioactive. You are not going to get them right. Use high-level interfaces like GPG or SSL or cryptlib, or design your app not to need crypto.
No, but apparently it takes better than Google can muster to implement that MAC in such a way that Nate Lawson can't extract secrets from it remotely. You will never win this argument.
I was surprised that both flaws they exploited were mentioned in the talk I gave at Yahoo back in June. One of the researchers that found this attack informed me that they independently found this in Flickr, not as a result of my talk. Interesting coincidence!
Isn't "game-over flaw" a bit overstated? It's a serious blunder, but if the data you're authenticating is highly structured then in many cases it won't be exploitable because the final padding from the original message becomes part of the body of the extended message, as unparseable gibberish.
I'm talking about exploitation of this vulnerability in general, not the specific case of Flickr. Not everything is a sequence of key-value pairs formatted in UTF-8.
Most of the crypto you're going to run across as a pentester will be in apps written in Java and C#, and in almost every one of those cases, garbage characters won't break a parse. ".split()" works just fine even if you have 16 characters of random high ASCII. It's one of those things C programmers definitely have to unlearn.
I trust a generated parser more than a hand-written one. Any parsing algorithm that involves the use of split() is almost certain to be weakly thought-out and have ill-defined behavior for unexpected input. A well-written parser will never read past the first unexpected character, or at least token.
This reminds me a lot of the problem the amazon s3api v1 suffered from. (It's referenced in this paper.) Colin Percival did a wonderful job alerting amazon and getting it fixed. Reading the paper it looks like they tried to alert people, but they didn't go into detail.
Yahoo has dedicated internal people who, if you ask them nicely, will pick your app apart to find these kinds of design and implementation errors. Reviewing your code with them is very educational but not pleasant to the ego. Either they completely fell down on the job, or the Flickr devs decided they were smart enough to do security by themselves.
Every major tech firm has an in-house app security or "SDLC" team. But outside of Microsoft --- or, even in some cases at Microsoft --- most people sub crypto design and verification out to people like Paul Kocher's CRI.
It's probably not fair to trace a cryptosystem flaw back to a web app security team.
Long story short: with all the mainstream hash algorithms, if you know the length of a hashed message, you can take its hash and from it synthesize the hashes of messages with arbitrary suffixes. It's called a message extension attack.
A really common pattern in hand-rolled authentication schemes is to hash a message needing authentication with a secret key, and then send the message and the hash. If you know the secret key, you can verify the message. The secret key doesn't need to be sent at all.
If your scheme is vulnerable to message extension attacks, attackers can mint arbitrary messages with valid authentication by taking an already-authenticated message and extending it to cover a suffix that changes the meaning of the legitimate message. It's a game-over flaw.
Colin Percival noticed a related (but less serious) flaw in Amazon's AWS authentication scheme. At the time, people asked if he'd looked at Flickr's scheme. He had, but not very closely. Colin's flaw was in canonicalization of messages, which is a more obscure attack (by a little bit) than message extension, which is is described in the Wikipedia under "Cryptographic Hash Function". Colin didn't flag this flaw.
Which just shows to go you: crypto primitives are radioactive. You are not going to get them right. Use high-level interfaces like GPG or SSL or cryptlib, or design your app not to need crypto.