Example #489643876834 of why you, a very smart, very capable developer, should not be writing crypto code. Rule #1: if you have to type the letters A-E-S or S-H-A into your code, you're doing it wrong.
There are pretty big mistakes to be made on the other side of this philosophy as well. If you need to solve some cryptographic problem which is not exactly the same as something which has already been designed, implemented, and reviewed by competent cryptographers, then you essentially need to create a new protocol and accept all the pitfalls which that entails.
PGP seems to be a popular 'primitive' to try to roll your own broken crypto protocol out of.
Even SSL/TLS is not easy to use correctly (while understanding all the risks) for purposes other than what it was originally designed for: Preventing credit card numbers from being sniffed between a browser and a web merchant.
Design a transport security protocol that is different in some way from TLS, and a good chunk of those differences are going to be vulnerabilities that the TLS designers figured out that you missed.
SSLv1 may have been originally designed for securing credit cards, but I'm not sure that argument survives a close read of TLS; why waste time with mutual authentication, with forward secrecy, with a custom record layer, with arbitrary length certificate signing chains, etc etc etc. You might be surprised at the number of systems that leverage TLS successfully on ports other than 443.
We probably agree; no matter how you choose to implement crypto security in a system, you're probably worse off for trying.
Forward secrecy was one of the important mistakes I had in mind actually. If you use TLS naively, you have none. You probably didn't even realize you needed it.
Verifying certificates correctly is not so easy to get right either, if you can even figure out what PKI means for your particular application in the first place.
Certificates are a lot like git branching. They're intimidating if you consider them in their entirety, but simple if you stick to your old use cases.
Enterprise apps make good use of certificates. A common use case is to set up a private CA (a three-line shell script) and issue internal certs to all endpoints in a system; you can then do crypto-strong access control just by checking signatures. This is approximately to PKI what "using git just like you used Subversion" is to git.
Another place where TLS has been a huge win is in wireless security; EAP-TLS with mutual certificate auth is the de facto corporate wireless security standard now.
You seem to be knowledgable on these topics, so I have a question... If I take the query string parameters and then a ridiculously long pre-determined salt (30 chars) to create a SHA-256 signature for POSTing, what's the chance that this will actually result in collisions / cracking?
Even if null data is submitted and someone sniffs the salt, it's still 30-chars long and last I checked rainbow tables don't work too well on SHA-256 hashes of that length.
I ask because I use a system similar to this to just do a basic check to ensure that a form POST is coming from a user clicking an action and not just some random spammers' form just posting the needed vars. It's nothing particularly secure (I think), just a nuisance to any spammer.
First, you're talking about a nonce, not a salt. You'd be a happier person if you took the word "salt" out of your CS vocabulary.
Second, the vulnerability here isn't password cracking; it's that AMZN verifies the signature on canonicalized data, but operates on non-canonicalized data.
Why are you "signing" your data at all? You're doing it wrong. Use SSL to secure the whole transport; use form or HTTP auth to make sure you know who you're talking to.
HTTPS is irrelevant to what I said. End-to-end authentication is better than transport-level authentication. Using an HMAC to sign requests and then sending those signed requests over HTTPS is better than using HTTP auth over HTTPS.
The developer who made this mistake probably didn't think s/he was writing crypto code. (Switching to "he" for brevity) He probably thought he was merely making use of someone else's crypto code. The mistake was in choosing what string to sign, not in the function applied to that string after it was chosen. I think the real lesson here is that if your app needs cryptography, then you need at least a black-box understanding of what crypto does and doesn't do.
In my experience, the people who do the worst at building secure systems are the ones who know "just enough" about how crypto works. These are the people who implement Diffie-Hellman and don't check parameters, the people who implement AES in ECB mode, the people who store passwords hashed with ultra-fast digest schemes, the ones that leak timing information or cipher state in distinguishable error messages, and the people who implement mirrorable challenge-response protocols.
The "dumb" developers just use OpenSSL and PGP, and they usually come out ahead in the end.
Also: I doubt the Amazon implementers here were dumb. I bet they were pretty smart. They just forgot about canonicalization; this is a classic problem in signature systems. Building crypto features is a good way to get screwed.
(PS: I feel qualified to take a contract breaking systems like this, but probably not qualified enough to engineer my own crypto protocol.)
