
How and why writing crypto is hard (2012) - ColinWright
http://www.acooke.org/cute/WhyandHowW0.html
======
lvh
Excellent points!

As for the library problem, I'm teaming up with Alex Gaynor, Hynek Schlawack,
Donald Stufft and Christian Heimes to write a better cryptography library for
Python:
[https://github.com/alex/cryptography](https://github.com/alex/cryptography)

New contributors very welcome!

On the other side of the coin, I agree that cryptography education is very
lacking. That's why I presented Crypto 101 last year at PyCon, which was an
attempt at teaching just enough crypto in 45 minutes. Obviously, that's pretty
near trying to square the circle: it's a best-effort attempt, but that's about
it. (If you're still interested, the recording is available for free:
[http://pyvideo.org/video/1778/crypto-101](http://pyvideo.org/video/1778/crypto-101))

That's why I'm also writing the book version of Crypto 101. Early draft stages
right now, but getting there. After listening to tptacek say it so many times,
I've come to agree: if you want to teach crypto well, you probably want a
bunch of exercises that teach you how to break it. As a result, both the
exercises and the text-adventure game I'm adding to the book rely on breaking
real-world crypto: everything from bad password storage to MITMing SSL
handshakes to stream compression :)

Additionally, a significant portion of the book will be devoted to _when_
(i.e. rarely ;)) and _how_ to write crypto. I would hope that your journey
would've been a lot less painful if you had had access to the book. Perhaps
for future programmers? :)

If you'd like to review, there's a reviewers mailing list, feel free to shoot
me an e-mail explaining why you think you'd be a good reviewer, and I'll
probably sign you up :)

~~~
peterhunt
I'm so glad about this.

I think it's pretty ridiculous that whenever a "non crypto" person tries to
use crypto the security folks go lambast them, yet there is simply not a truly
usable crypto api out there! Please go out and make this the right way; I'm
looking forward to it :)

~~~
tptacek
There isn't one kind of "security" person. I work in security. But I don't
install antivirus, I don't write hardened runtimes, I don't configure
firewalls, I don't analyze malware samples, I don't design ciphers, I don't
write policies, and I don't do forensic investigations.

All I do is break into software; I do vulnerability research, and that's
pretty much it right now.

As a vulnerability researcher, it's not on me to provide you with a good
crypto library. People like us write software in order to find
vulnerabilities. We're specialized so that we can be very good at doing that
by dedicating all our time and attention to that one problem.

But much more importantly: like vuln researchers, don't like vuln researchers,
it doesn't matter. When vuln researchers complain about crypto libraries, they
are informing you of facts. You can ignore those facts and probably be
vulnerable, or accept them and act on them and not be vulnerable.

Having said all that, there _are_ in fact good crypto APIs. You can use
Keyczar, which was a Google project. You can use NaCL, which is Daniel
Bernstein's project; it even has a Ruby port now. You can use Peter Guttman's
cryptlib.

~~~
peterhunt
Ok, so maybe I came off a bit harsh there. I've worked with a number of
excellent security professionals who have found weaknesses and provided great
prioritizations and solutions to the problems they've found.

However that's not the case whenever some project on HN claims to use
cryptography. The answer usually is: "you don't know what you're doing, so
don't use cryptography." While that is "safe", for many it's not actionable:
sometimes we actually do need to use it and not everyone can afford a security
consultant. I wish instead there was a recommendation for some library out of
the box that would handle everything from key management to supporting my
specific use cases.

Said a different way: I want an API that is simply giveMeANewKey() and
encryptSecretMessage() and decryptSecretMessage() and the library takes care
of both secrecy and integrity with strong defaults.

You can't expect it to make the whole system secure of course, but I think a
better API that makes it harder to shoot yourself in the foot (both by having
sane defaults and being very opinionated to the point where it affects your
system design) would go a long way, no?

EDIT: looks like the stuff you linked to is more what I am looking for. I wish
I had heard of them sooner!

------
raverbashing
"This was because the "iv" parameter in the pycrypto Cipher API is ignored in
CTR mode. Instead, you need to provide the data to the Counter object.

I don't know if I am being muddle-headed in thinking of the initial counter
value as an IV, but I was a little annoyed with pycrypto. Couldn't it throw an
error if it's given an IV in CTR mode,"

THIS

API creators have to _stop this bullshit_ of putting death traps in their
code.

This is not about go RTFM, or "you're incompetent".

If you allow it to be used in a wrong manner, it's _your liability_ as well.

If your IV sometimes goes into one field, but in another occasion goes in
another field, THROW AN ERROR IF IT'S NOT USED CORRECTLY

~~~
lvh
See also: if someone (hi, Salt) passes in 1 as the public exponent for RSA,
you API should probably not just go ahead and use that value.

(Of course, Salt had no business being anywhere near an API that had the
letters R-S-A in it, but I digress...)

~~~
gizmo686
That seems like a case of determining what the API should check for. An
explicit check for e=1 seems stupid, the reasonable check would be to make
sure that e was a valid exponent, which I believe means confirming that it is
primes, which can have a non-trivial overhead (although in the case of RSA,
the exponents are gennerally small enough that it doesn't matter.) The real
question in this case is why the API let you select the exponent in the first
place.

~~~
lvh
Hence, like I said: they should not have been near an API that had the letters
R-S-A in it. They should've been near a very nice API with no configurable
knobs to get wrong. The entire thing was part of an authenticated encryption
scheme: they could've just use NaCl's `box` API and the entire thing would be
two lines of code, give or take. Oh, and not utterly pointless ;-)

------
kingkilr
Crypto library authors seem to have taken the philosophy that "it was hard for
me to learn all this stuff, so it should be hard to use". Since I started
working on the cryptography library that lvh mentioned I've seen some
absolutely amazing APIs from crypto libraries:

* Undocumented error conditions (actually in general the documentation is atrocious for crypto libraries, "crypto is easy, why would you need docs"?) * ECB as the default mode for block ciphers * Ignoring invalid parameters * Allowing you to provide parameters which basically your cryptography a no-op (e=1) * Modes whose purpose is undocumented, merely "This mode should not be used"

And finally, absolutely NO python crypto libraries expose AES GCM. (This is
all to say nothing of the myriad of other issues I have)

~~~
illumen
Crypto is hard - we should teach it better.

This book helps with teaching crypto and python at the same time...
[http://inventwithpython.com/hacking/](http://inventwithpython.com/hacking/)

------
tsotha
He doesn't even get to the _real_ problem with writing crypto - unless you can
get some attention from experts, there's no way you can know you didn't leave
a big gaping hole in your security. It's something you literally can't test
yourself.

------
ww520
Kudos to OP for trying a hard problem and be frank about the hurdles. That's
the way to learn. Try it, make mistakes, so what. We learned best from
mistakes and failures. Mistakes burn into our memory. We'll be better the next
time.

------
zb
Incidentally, the PyCrypto API is considered so dangerous that I believe it's
actually _banned_ from Fedora. M2Crypto is a better alternative.

~~~
donaldstufft
I'm not sure I'd called M2Crypto a much better API. It more or less exposes
the OpenSSL API which is also notoriously hard to get right. Further more it
hasn't seen an update since 2011, the SSL certificate for the bug tracker
expired in 2011, and it does not compile without patches against any modern
OpenSSL (1.0+).

The state of crypto in Python is horrendous, hence the new library.

------
DanielRibeiro
The point is: coding mistakes in crypto code are often fatal. And there are
many implementation details to make mistakes on.

------
ronaldx
Another problem is: I know I shouldn't roll my own, but I can't or don't know
how to identify implementations that meet my needs and are written by someone
sufficiently competent.

------
bitwisebro
I noticed this
[https://news.ycombinator.com/item?id=4964720](https://news.ycombinator.com/item?id=4964720)
advice is wrong. >Possible timing attack. Consider doing another HMAC of each
HMAC and comparing those instead.

The time-insensitive method is to XOR every bit of HMAC_0 with HMAC_1 (keep
going even if they are not of equal lengths use 0s) and then (not before)
check if the lengths are equal.

~~~
andrewcooke
either approach avoids leaking useful information, doesn't it? one does so by
guaranteeing identical times. the other by passing the information through a
one-way function.

the link given in the advice you point to includes a fairly detailed
discussion / comparison -
[https://www.isecpartners.com/blog/2011/february/double-
hmac-...](https://www.isecpartners.com/blog/2011/february/double-hmac-
verification.aspx)

------
illumen
Another problem with simplecrypt is the password is exposed in plain text to
any other process.

[http://www.gnupg.org/faq/GnuPG-FAQ.html#why-do-i-get-
gpg_war...](http://www.gnupg.org/faq/GnuPG-FAQ.html#why-do-i-get-
gpg_warning_using_insecure_memory)

Also the source code looks like encrypted text. I can't figure out what much
of it does, or why it does it by reading it.

Also, I think the API is not the best, because it asks for a password. Where
are people storing the password? This promotes easy mistakes... like for
example, saving it to a file, or reading it from a web input form.

~~~
andrewcooke
i don't know a good way to avoid the in-memory password from within python -
if that is possible, please show me how and i'll fix the code. the larger
point about passwords is, i think, part of the reason for keyczar's approach.
but that is not yet available in python 3 (certainly not when i wrote the
code; i did check since and last time i looked there was a patch being
considered).

however, i was quite proud of the source -
[https://github.com/andrewcooke/simple-
crypt/blob/master/src/...](https://github.com/andrewcooke/simple-
crypt/blob/master/src/simplecrypt/__init__.py)

edit: i've added some warnings to the project page -
[https://github.com/andrewcooke/simple-
crypt#warnings](https://github.com/andrewcooke/simple-crypt#warnings)

