To be honest, I don't see why anyone would use bcrypt over PBKDF2, if the security of the primitives is of any serious concern. I am new here on HN, so I'm not aware of your arguments on why bcrypt is better than PBKDF2.
As for the "branding" question, would you recommend XTEA or MARS or Threefish over AES for someone in need of a block cipher? Of course not. Standards are not always perfect (and many a flaw has been found in standards), but they are generally beneficial.
PBKDF2 also has the advantage of being modular. It takes in an arbitrary PRF (although HMAC-SHA1 is the usual); maybe HMAC-SHA1 turns out to be poor for the job, just plugin a better PRF (hell, you can plugin a provable PRF that reduces to integer factorization or the elliptic curve discrete log). bcrypt is just bcrypt --- a seemingly not too peer-reviewed modification of an ancient cipher, that is not even recommended anymore.
scrypt is better than both, of course. Provable time-memory hardness is great, and should be made standard.
It takes an arbitrary PRF that is in practice virtually always SHA2, and practically always a well-known cryptographic hash function. If you're going to bank on a cryptographic primitive and you have a choice between "cipher" and "hash function", you pick "cipher".
Also, for lay developers, choosing a crypto construct for its modularity is like choosing a smoke detector because it allows you to use different radiological bits in it. You're not supposed to be messing with those bits. The whole point of the package is not to have random developers changing them.
Also, I want you to note something:
You pick AES not because it's a standard but because it's the product of a contest in which many of the world's best cryptographers competed to design the replacement to DES. That's not what PKCS standards are. A PKCS standard is simply something that survived a standards group discussion.
I see you've adopted Moxie's argument, block ciphers against hashes. Very well. The hard part of designing a good hash function is achieving collision-resistance; one-wayness is easy. In this context, we don't really care about collision-resistance, since HMAC can be a PRF without collision resistance of the underlying hash (there was a recent proof by Mihir Bellare) --- this is why it's still "OK" to use HMAC-MD5, despite MD5 being completely broken otherwise. The adage that we know much more about ciphers than hashes, although still mostly true, is an exaggeration at this point in time, where we have the HAIFA mode, good block ciphers, and soon SHA-3.
I didn't mean to say that Joe the webdev would be choosing the PRF, that's insane. But whoever providing the (library) implementation would have their life facilitated by having modular primitives, instead of having to code another construct from the ground up.
Edit: True, the AES competition was a much higher-profile event. There is an important difference (this is not a rebuttal, but a remark): AES is a cipher, PBKDF is a construction. AES has no proof of security, nor hope of one: it's a purely heuristic security argument. Through models like the random oracle, however, we could show that the PBKDF construction is secure, if H is secure. In such a case, there is not as much need for a competition, unless you're competing for performance or the like. That said, I would love proofs of security (or show the lack thereof) for PBKDF2.
It's not "Moxie's argument". It's also Schneier and Ferguson's argument from Cryptography Engineering:
Event hough hash functions are used in almost every system, we know far less about hash functions than we do about block ciphers. This is one of the failures of the cryptographic community. Compared to block ciphers, very little research has been done on hash functions, and there are not many practical proposals to choose from.†
It is also the reason why we are sponsoring contests to replace SHA2, because the research horizon for the current generation of cryptographic hash functions is... ominous.
You're making a noncontroversial statement (ciphers are better studied than hashes) sound like a controversy. It's not really a controversy. And please note: I didn't bring "conservatism" into this discussion; the blog post we're responding to. If you put it to me directly, I'll say bcrypt is more conservative than PBKDF2/SHA2 (which is what every current PBKDF2 system is going to end up using). But I didn't write a blog post that says "don't use PBKDF2".
† I say this only to make the point that it's not an argument pulled from thin air or from Moxie comments; something that Schneier commits to writing is, I mean to say, very likely to represent conventional wisdom.
You're right, I do recall that passage. Apologies if I sounded patronizing.
I tried to be careful with the wording, precisely to avoid making a common-sense statement into a controversy. Yes, block ciphers are more well-studied (2012-70s > 2012-80s). My point was that the gap is much smaller now than (say) when AES was standardized (I wonder, was that passage also in the "Practical Cryptography" 2003 book?).
As a personal note, I doubt 5-10 years from now SHA-2 will have been compromised for password hashing (or HMAC), though. It has been remarked several times during the process that SHA-2 would have made a great SHA-3 candidate, its only major flaw being length extension attacks. The main fear was that SHA-2 would succumb to the same techniques as SHA-1 and MD5; so far, that has not been the case (perhaps because everyone is fighting it off with the SHA-3 candidates).
Of course, I didn't write a blog post either with anything. I don't blog. Given the choice, I'd give priority to PBKDF2; that's about it. Of course, I'm not a customer-facing developer, so my worries are different.
No problem. I think you probably know this stuff better than I do. In practice, I think PBKDF2 really means "PBKDF2/SHA2", and that in 5 years attacker tools will be most efficient for PBKDF2/SHA2, less efficient for bcrypt, least efficient for scrypt, and won't address PBKDF2/AES-256-CBCMAC at all because nobody will know how to code it.
Moreover, what we (= provable security wonks) really don't understand is collision resistance, or, more generally, cryptographic security notions that are stronger than one-wayness (really UOWHF) and that do involve a secret key.
Trusting blockciphers over hashes based on this high-level argument is suspect, at least in principle. (To be clear, I'm not arguing with your practical recommendations as they seem pragmatically justified.) It could well be the case that the ways we build password hashes from blockciphers appear to be more secure simply because we haven't sufficiently cryptanalyzed blockciphers for the necessary collision-resistance-type properties. Perhaps collision resistance is just too much to hope from any highly-efficient function, and it is only a matter of investing the effort to find collisions in the functions derived from blockciphers. Adding to the suspicion is the fact that blockciphers give us good hashes by coincidence, not design (And formal evidence doesn't help explain the situation - although I see hand-wavy claims about related-key attack resistance being sufficient, in my work I've only found reverse connections).
So are you aware of any deeper reasoning behind the blockciphers-over-hashes argument? Could trusting blockciphers for collision-resistance just be a good usage of security-through-obscurity, because a tremendous amount of effort has been invested in finding collisions in the hash functions but not in the blockciphers?
I think that book is going to need an update soon as the SHA-3 competition has directed a significant amount of research behind hash function security.
The good news is that SHA-2 and HMAC remain in pretty decent shape (known limitations notwithstanding).
As for the "branding" question, would you recommend XTEA or MARS or Threefish over AES for someone in need of a block cipher? Of course not. Standards are not always perfect (and many a flaw has been found in standards), but they are generally beneficial.
PBKDF2 also has the advantage of being modular. It takes in an arbitrary PRF (although HMAC-SHA1 is the usual); maybe HMAC-SHA1 turns out to be poor for the job, just plugin a better PRF (hell, you can plugin a provable PRF that reduces to integer factorization or the elliptic curve discrete log). bcrypt is just bcrypt --- a seemingly not too peer-reviewed modification of an ancient cipher, that is not even recommended anymore.
scrypt is better than both, of course. Provable time-memory hardness is great, and should be made standard.