
How I implemented my own crypto - loup-vaillant
http://loup-vaillant.fr/articles/implemented-my-own-crypto
======
tptacek
I would like to clarify:

The thing that I said was "table stakes" for implementing cryptography was
_passing the algorithm test vectors_ , which this author's previous post
claimed as a security feature.

If you're unfamiliar with the concept, a test vector is a series of strings
and intermediate values used to ensure that your (say) OCB3 is the same as
everyone else's OCB3.

Had he asked, I'd further claim that not having dumb C bugs is also table
stakes for cryptography, since serious crypto vulnerabilities happen at a
higher level of abstraction. The author grazes past (somewhat disconcertingly)
one such class of bugs when he discusses carry propagation and limb scheduling
in the context of Poly1305. Improper carry propagation is an example of the
kind of security vulnerability for which there are no test vectors and no
memory safety validation tools.

You just have to know what a carry propagation bug is, where to look for them
(not Poly1305), and what the impact of one is.

Hopefully, the author of this library does.

~~~
lisper
> You just have to know what a carry propagation bug is

This is one of the things that drives me nuts about the crypto community. The
natural response upon reading, "You just have to know X" if you don't already
know is to go search for X. Well, if you go search for "carry propagation bug"
you will find lots of examples of carry propagation bugs being found and
fixed, but no explanation of what one is. The thing that "you just have to
know" is not so easy to find. The courteous thing to do when mentioning
something that "you just have to know" but which is not easy to find is to
provide a pointer or a short in-line explanation. When you don't do that you
leave your reader with no choice but to publicly expose their ignorance of
this purportedly crucial knowledge by asking, which I will now proceed to do
as a public service: What the hell is a carry propagation bug? Where do you
look for them? And what is their impact?

And while we're at it, what are "table stakes"?

~~~
tptacek
Give me a fucking break.

Sean, Marcin, and I spent the better part of a year _individually replying to
tens of thousands of emails_ from people doing the Cryptopals challenges that
we wrote and published for free. Do you know how much work that was? Here's a
hint: it was a lot of work.

There are a lot of charges you can level at me to try to win a dumb message
board argument, but the one where I want cryptography to be a secret
priesthood known only to the anointed select is not one of them --- nor is the
argument that I haven't taken the time to help make these kinds of
cryptographic attacks easier for people to learn about.

Please immediately find one of those other arguments to substitute for this
one, and, if you have it in you, take a moment to retract.

Thanks in advance.

~~~
lisper
I do sincerely apologize for taking my frustrations out on you, because you
have indeed gone above and beyond the call to help make crypto accessible.

Nonetheless, I still stand by the substance of my comment: when you (not you
specifically, but anyone) say, "You have to know X" you should check that
doing a Google search for X yields some reasonable results, and if it doesn't,
provide a hint on how to proceed.

~~~
peterwwillis
> Nonetheless, I still stand by the substance of my comment: when you (not you
> specifically, but anyone) say, "You have to know X" you should check that
> doing a Google search for X yields some reasonable results, and if it
> doesn't, provide a hint on how to proceed.

Do you actually believe you are personally entitled to demand special research
tips from domain experts?

~~~
sn9
I don't think it's too much to ask for links to resources.

If it's a common enough remedy that needs linking, you could just write a
dedicated blog post describing the recommended resources or whatever and just
link to that.

Sometimes the hard part of researching things is knowing what to look for or
where to look for it.

~~~
peterwwillis
You can _ask_ for anything you want. But I wouldn't expect anyone to do work
for you just because you demand it.

Give me $100 and I'll send you a link to some good crypto books.

------
ZoFreX
I am terrified that I do not consider myself competent enough to write a
crypto library, and yet there isn't a single mention - in this article, nor at
the time of writing the comments here on Hacker News - of many of the pitfalls
I know to avoid when undertaking such an endeavour. There is even a list of
"you have to do A, B, C, and that's about it" that is missing some major -
well known, even! - items.

I know "don't roll your own crypto" comes across as dismissive or patronising.
I'm not a huge fan of the phrasing. But as a first-order approximation, it is
correct. If you want to roll your own crypto, that needs to be your _thing_.
It's very unlikely you're going to be a fantastic full-stack web developer
_and_ be able to do that. If it's really what you want to do then awesome! Go
study it, learn it, practice it. But if you're doing it as a side-hobby, never
put it into production.

~~~
JohnStrange
I apologize if that sounds dismissive (it's not intended to) but your post is
not more than the usual vague FUD. Take a look at so-called 'professional'
cryptographers and their libraries, and you'll find that practically all of
their code had severe bugs. Professionals make errors like everyone else,
'professional' just means you're being paid for it. You will have a hard time
finding a professional and widely used crypto library that wasn't essentially
compromised in one of its functions at one time or another. Not only that,
professionals from the _closed-source_ department have a long history of
coming up with compromised or bogus, sometimes even ridiculous cryptographic
algorithms and implementations. Two typical examples: CMEA cell phone
encryption, and the Crypto AG backdoor debacle.

Surely people who invent and implement cryptographic algorithms for a living
can be expected to be better than your run-off-the-mill programmer, but as
I've said elsewhere implementing crypto is also not a black art. It requires
about the same level of skill as e.g. implementing a production-ready low-
level network protocol or a file system. That rules out many programmers but
not all. That concerns implementations. As for algorithms, their development
requires extensive experience in _cryptanalysis_ (not just applied
cryptography!). Some pros have that, others don't.

Last but not least, many popular and widely used crypto libraries started as a
side hobby. For example, libtomcrypt started that way. In fact, many widely
respected cryptographers started as hobbyists. For example, Bruce Schneier. Of
course, the ones who are widely accepted as peers are also those who are in
the 'hire me as a consultant, not the other guy' business, and they will
naturally advise everyone to not roll their own crypto but to rely on _their_
expertise...

~~~
dsacco
So I have two points for you:

 _> Take a look at so-called 'professional' cryptographers and their
libraries, and you'll find that practically all of their code had severe bugs.
Professionals make errors like everyone else, 'professional' just means you're
being paid for it. You will have a hard time finding a professional and widely
used crypto library that wasn't essentially compromised in one of its
functions at one time or another. Not only that, professionals from the
closed-source department have a long history of coming up with compromised or
bogus, sometimes even ridiculous cryptographic algorithms and implementations.
Two typical examples: CMEA cell phone encryption, and the Crypto AG backdoor
debacle._

This doesn't diminish the claim that the modal individual should not attempt
to "roll their own crypto." You're pointing out that professionals make
mistakes, but of course they do - that indicates nothing about the propensity
for amateurs to make mistakes.

 _> It requires about the same level of skill as e.g. implementing a
production-ready low-level network protocol or a file system._

I have never implemented a low level network protocol or a file system
(although at least the former sounds like a fun project) - so I cannot claim
you're wrong here. _However_ , I will point out that a network protocol is not
designed to be error-proof in an intentionally antagonistic environment, which
is a difficulty unique to cryptographic software. There are incentives
involved in breaking crypto code that do not present themselves in other areas
of software development, even if the programming difficulty itself is not
entirely dissimilar.

 _> Last but not least, many popular and widely used crypto libraries started
as a side hobby. For example, libtomcrypt started that way. In fact, many
widely respected cryptographers started as hobbyists. For example, Bruce
Schneier. Of course, the ones who are widely accepted as peers are also those
who are in the 'hire me as a consultant, not the other guy' business, and they
will naturally advise everyone to not roll their own crypto but to rely on
their expertise..._

They can _start_ as side hobbies, but they are not going to remain that way
for any meaningful amount of time if they are to be widely deployed and safe.
You can only choose two out of those three.

More importantly, there is no cabal of cryptographers spreading FUD to line
their own pockets with consulting fees. The cryptography consulting industry
is _very_ small compared to the actual security consulting industry. In the
application security consulting industry I do believe there are firms that try
to secure business this way, but that industry is much larger (and has firms
which try to misrepresent cryptographic competency). I have had significant
interactions with engineers at Riscure and NCC Crypto, which is a sizable
portion of all the "real" cryptanalytic consulting work that occurs (at least
in the United States) - they never struck me as being the sort to suggest what
you're saying. That leaves Rambus/CR and a select few other firms doing real
crypto consulting, which leads me to believe the industry is, on the whole,
very legitimate.

~~~
loup-vaillant
> _a network protocol is not designed to be error-proof in an intentionally
> antagonistic environment_

Err, really? That may be the case sometimes. Also these days many things can
be antagonistic: anything online is exposed to adversaries. So is anything
that routinely reads untrusted input on everyone's computer (like a video
player, or a pdf reader).

Thinking of the sheer amount of potentially vulnerable code out there makes me
a little scared.

~~~
dsacco
But is the code you're speaking of the security-specific code, or the network-
specific code? If you attack a networking protocol, what are you attacking if
not the associated cryptography? A networking protocol without any
cryptography doesn't really need to be attacked because it's already wide
open.

You introduce the confidentiality, authentication and integrity (which is
really just going to be part of the authentication) through cryptography and
security-specific code.

I'm not saying people don't attack networking protocols, or that there is no
incentive for doing so. I'm saying that the portion that plays the _defensive
role_ is in the security software, not the overlying networking protocol.

~~~
nickpsecurity
re networking vs security-critical code

Anything that accepts data from potentially-malicious sources is security-
critical if it's itself privileged or produces data/events that go to
something else privileged. Such analysis is usually used to determine the
"Trusted Computing Base" (TCB) of a system that encompasses everything that
might be used to break the security policy. All people who do real, security
engineering keep it in mind, minimize it, strengthen it, and document what's
left.

Here's an example from high-assurance field of what that looks like where they
prototype an architecture for VPN's that minimizes TCB of key components:

[https://os.inf.tu-dresden.de/mikrosina/dach2005.pdf](https://os.inf.tu-
dresden.de/mikrosina/dach2005.pdf)

Far as qualifying the TCB, the seL4 team's What is Proved and What is Assumed
is a great illustration of keeping it real about the security claims:

[https://sel4.systems/Info/FAQ/proof.pml](https://sel4.systems/Info/FAQ/proof.pml)

Also shows their TCB is a lot bigger than their tiny kernel. Fortunately,
there's been high-assurance developments to solve most of _those_ things. I've
seen robust, networking stacks but don't think they're high-assurance yet. If
they're complex and done in C, here be dragons.

------
reikonomusha
It was a fun read, except when all the C bugs were listed. It's getting
tiresome to hear about folks writing nominally Important software in a
language where it is extraordinarily difficult to get things absolutely
correct, with paltry excuses such as "it needs to be fast" or "it needs to be
portable [0] to a VAX-11/750." It doesn't give me confidence that these
completely usual bugs popped up early on, and it will not surprise me when
more show themselves.

The author's lack of practical familiarity with the memory hierarchy, caching,
and pipelining—evidenced by his surprise by the speed gained by fetching more
than a byte at a time—also makes me suspicious of his understanding of the
semantics of his program. (I understand that this is a form of _ad hominem_ as
it relates to arguing the correctness of software, but it's a consideration
worth noting in a broader context of trust.)

It would be great if this was written in another language.

[0] Most "portable" code written in C is not portable. Look at any mature C
project and see the layers and layers of macros and hacks to make things
portable.

~~~
thinkMOAR
Curious about the other language. What other language that doesn't add another
layer between the code and system instructions were you thinking about?

~~~
pjmlp
Most programming languages with compilers to native code (JIT/AOT), the myth
of high level Assembler for C only applies if your computer is a PDP-11 or a
basic 8-bit CPU like a 6502 or Z80.

The ANSI C and C++ standards define the concept of abstract machine for the
language semantics, just like in most languages.

Additionally you have the concepts of sequence points, the new memory model
semantics for multi-threaded code and the beloved UB.

UB which doesn't exist in most languages, because their rather leave it
implementation defined or lose the opportunity to target some strange CPU not
able to support the language semantics.

Also Assembly doesn't has UB, making it ironic that it is safer to write
straight Assembly at the expense of portability than C or C derived languages.

~~~
bandook_raja
> Also Assembly doesn't has UB, making it ironic that it is safer to write
> straight Assembly at the expense of portability than C or C derived
> languages.

You just need to know your compiler flags/configuration to produce exact
assembly output (if you want that). And about assembly not having UB, BSF and
BSR are pretty good examples for that.

~~~
pjmlp
That _just_ is quite ironic, because it is even version dependent across
releases of the same compiler, which means anything fine tuned for version X
can break on version X + 1, due to changes on the optimizer.

And I doubt anyone is regularly reviewing the produced Assembly every single
time they change compilers.

Assembly doesn't have more than 200 documented cases that hardly any human is
capable of remembering.

------
technion

        found an error in the Argon2 reference implementation.
    

This is exactly why I like as many people writing crypto as possible[0]. Bugs
like the one the writer found show up in reference implementations, and are
found by absolutely no one unless someone writes an alternate implementation
that behaves differently.

Edit: A further example, a lot of people write crypto as part of various CTFs.
This is a case of "writing crypto" that poses no threat to the community.

[0] That doesn't mean they should use it in production.

------
ktta
I've always seen 'rolling your own crypto' as not being recommendation against
writing your own library, but creating your own primitive.

Sure, writing your own library is _very_ difficult, but you have a simpler set
of problems, which proper testing, another set of eyes and enough tools will
take care of the big problems.

Now, implementing your own primitive and recommending to use it is _bad_. For
a primitive to be deemed secure, it has to undergo years of review and testing
by seasoned cryptographers.

This isn't a recommendation to write your own library, but my opinion that it
is okay to do so if you want something that none of the major libraries offer.

~~~
viraptor
Even just using crypto primitives is dangerous unless you've got experience.
There's an old post from matasano about it
[https://www.nccgroup.trust/us/about-us/newsroom-and-
events/b...](https://www.nccgroup.trust/us/about-us/newsroom-and-
events/blog/2009/july/if-youre-typing-the-letters-a-e-s-into-your-code-youre-
doing-it-wrong/)

~~~
ktta
Every line of code written can be a severe vulnerability, not just code
dealing with crypto. In case of crypto, being well-informed is good enough to
not mess up those kind of implementations (on a theoretical level). The
problems mentioned can be pointed out and verified on paper. But you can't say
the same thing for code.

~~~
Ar-Curunir
No, you can write crypto code, even when knowing stuff about the crypto, and
still mess up. Crypto is problematic because you often cannot easily check
whether you're wrong (it could be a tiny edge case that isn't tested because
your input space is huge).

~~~
ktta
I didn't deny messing up crypto code is a thing. I'm saying messing up code is
a separate problem from messing up how one architects projects using crypto.

What you say as messing up with edge cases, etc. is a programming problem. The
errors talked about in the link he posted is about using faulty crypto. Today
it is common knowledge to never use ECB.

What cipher mode you use is a problem you think about before you even start
writing a single line of code. But you are talking about programming errors,
which one can make in all areas, not only ones involving crypto. Those errors
can break security similar to messing up crypto implementation.

------
convivialdingo
Hey, fantastic work! Really happy to see some amateur crypto development out
there. I started as an amateur also, now I work in crypto product dev.

Definitely impressed if this is your first crypto work. I really like the
simplicity, reminds me of OpenSSH.

I gotta agree with some of the naysayers though - this needs more time to
brew. Even if it's perfect, there still more proof to show.

My suggestions:

* A long-running interoperability test between your lib and another. Test vectors don't catch everything. Run it for a week, check results and publish the results.

* Fuzzing. You need to to catch false results, random crashes, stack overflows, etc. Fuzz every input, do it for random lengths (both underflow and overflow).

* Maybe port it and test on Win, Mac, ARM, MIPS. I find a lot of bugs shake out during ports.

* Timing. You can't prevent all side-channel attacks, but bits going out are easy attack vectors. Use a high-precision timing mechanism to catch any unusual spikes. Capture a few hours of points for each functional path, and publish the results. (crypto_aead_unlock has two return paths, it might be better to maintain timing).

* Personal preferences: Rather than using int returns, use bool. Zero your ctx vars, e.g. a_ctx ctx = {0,};

I spend more time in testing products than development, honestly. Publish as
much relevant data as you can.

~~~
loup-vaillant
> _Definitely impressed if this is your first crypto work._

It is. :-)

> _A long-running interoperability test between your lib and another._

Already have one for all primitives. Just tweak the parameters to make it run
for a long time. The test code is in test/sodium.c and test/donna.c

> _Fuzzing. You need to to catch false results, random crashes, stack
> overflows, etc. Fuzz every input, do it for random lengths (both underflow
> and overflow)._

I'm not sure how to go about it. I already check every input size, from zero
to several times the size of the internal block, in my comparison tests.

Of course, all inputs are correct. Incorrect inputs result in undefined
behaviour, checking for that is out of scope. (That's why most functions
return void.)

> _Maybe port it and test on Win, Mac, ARM, MIPS. I find a lot of bugs shake
> out during ports._

Yeah, I only tried GCC and Clang on my Intel Ubuntu machine. But I did use
Valgrind, ASan, MSan, UBSan, and the TIS-Interpreter. I also compiled under
various standards: C99, C11, C++98, C++11, C++14, C++17.

> _Timing. […] Use a high-precision timing mechanism to catch any unusual
> spikes_

Isn't that a bit overkill? I'd rather manually review the source code for the
absence of secret dependant branches or secret dependent array indices. I'm
not implementing AES or RSA, the primitive I have chosen were specifically
designed to easily avoid timing attacks.

> _crypto_aead_unlock has two return paths, it might be better to maintain
> timing_.

No. There are 2 cases to consider: failure, and success. Each path runs in
constant time, so timing can only distinguish success from failure… which is
revealed anyway by the return code.

> _Rather than using int returns, use bool._

I'm afraid bool is a tiny bit less portable than int. I also went with the
flow about error codes, which are traditionally int.

> _Zero your ctx vars, e.g. a_ctx ctx = {0,};_

I'm not sure how to do it portably, with zero dependency.

------
rijoja
Whenever I feel the need for a tin foil hat I start to wonder if there is a
FUD campaign powered by the "establishment" to encourage people not to
investigate this area of computer science so that security holes will remain
unnoticed.

But, yes I wouldn't start out on writing a crypto library, then again I
wouldn't attempt to build an OS or a 3D stack or even an web server either.
All cases where a security breach could have devastating effects as well.

~~~
baby
Writing your own crypto is the only way to become good at it, or to understand
more about crypto.

* DJB wrote NaCl

* Frank wrote libsodium / libHydrogen

* Brian wrote Ring

* Thai Duong and Bleichenbacher wrote Tink

* Eric Young wrote OpenSSL

* Jason Donenfeld wrote Wireguard

* Shoup wrote NTL

* Emily Stark, Mike Hamburg and Dan Boneh wrote SJCL

* Thomas Pornin wrote 6 SSL libraries and then BearSSL

* Adam Langley wrote everything else

* ...

~~~
tptacek
This is a deeply misleading list.

Daniel Bernstein, Daniel Bleichenbacher, Dan Boneh and Thomas Pornin are
professional cryptographers and world-renowned experts. Even I would feel
comfortable writing crypto if Bleichenbacher was watching behind my back.

Frank "wrote" libsodium, but libsodium is effectively a port of NaCl. Frank
had Daniel Bernstein watching behind his back.

Eric Young wrote OpenSSL. Look how that turned out! Has there been a class of
cryptographic vulnerability that OpenSSL hasn't had?

And yet consider: for all the effort that people like Bernstein and Boneh put
into writing strong, safe crypto, the entire world has blundered into
vulnerability after vulnerability because the amateurish crypto in OpenSSL is
the one every else ended up using.

It's as if you set out to prove my point.

Finally: you obviously know that "writing your own crypto" isn't the only way
to become good at it. In fact, it's a terrible way to get good at it. The
right way to get good at crypto is to learn how to break it. But that takes
effort, and you can have a blog post crowing about your new crypto library
just a few weeks after deciding to write one.

~~~
baby
> are professional cryptographers and world-renowned experts

They were not in the beginning. And implementing crypto helped them get there.
That's my point.

> Finally: you obviously know that "writing your own crypto" isn't the only
> way to become good at it. In fact, it's a terrible way to get good at it.
> The right way to get good at crypto is to learn how to break it.

I strongly disagree with you on this, but I'll add a twist: I'm mostly saying
to become good in "applied" crypto you must roll your own. There are too many
problems you can't understand until you actually write code and try to make it
secure. If you want to become good in theoretical crypto, then don't roll your
own obviously.

This works in any field btw. The best web pentesters are the people who have
implemented websites on their own as well.

~~~
tptacek
I would suggest that for each of them, a 10+ year career in academic
cryptography, in each case accompanied by significant new research results, is
what helped them get there. Implementing a library, not so much.

As for your second point: literally everyone can implement cryptography that
appears secure to them. It's tautological.

~~~
baby
> I would suggest that for each of them, a 10+ year career in academic
> cryptography, in each case accompanied by significant new research results,
> is what helped them get there. Implementing a library, not so much.

I believe that what made them exceptional (for most of them) is the
combination of theoretical learning and hands-on programming. That's my entire
point. I think we can agree to disagree. But that's an opinion I'll probably
hold for the next years as this is the direction I want to take as well.

> As for your second point: literally everyone can implement cryptography that
> appears secure to them. It's tautological.

And the process of putting it out there and looking for ways to make it more
secure is how you learn. Proof: he already learned a bunch about Frama-C and
other C oddities and documented these via his blogpost. Plus he found a bug in
the Argon2 implementation. That's a win.

~~~
tptacek
Why don't you ask them? Most of these people aren't hard to get ahold of. One
of them you even share a Slack with. I would be surprised to learn that any of
them believed doing a library implementation of pre-existing crypto
constructions was an important part of their education, but I like to be
surprised.

~~~
baby
I did ;) surprise

~~~
tptacek
What did he say? Share with the class!

~~~
baby
I can't share this kind of personnal information Thomas. I'm sure you
understand! I guess you'll have to take my word for it.

~~~
tptacek
Ok, I'll ask Thomas Pornin myself.

~~~
pornin
I indeed learned a lot, and still learn a lot, by doing implementations. Doing
a proper implementation forces me to consider all aspects; when the code runs
properly, I know that I have, by definition, been exposed to all the parts.
You cannot get that kind of exhaustiveness from simply reading an article.

However, doing implementations is not at all the same thing as publishing
implementations! The first one or two attempts are always flawed in some way;
only the third one can hope to be reasonably good. I took care to properly
kill and dispose of the corpses of all my learning code.

The trick (and it's a difficult one) is to decide in advance that the code you
write to learn will have to be deleted -- and stick to it. Developers have
trouble letting go of their creations, in general. If you can maintain that
discipline, then there is no problem in "writing your own crypto". But that is
a big "if".

~~~
tptacek
I've found that a good motivation for writing learning-only "throwaway" crypto
code is as models for writing attack code; you don't even have to throw the
code away, just publish it with the exploit.

But then, I'm a believer that everyone should learn crypto by breaking it, and
clearly not everyone agrees with me.

------
baby
First let me say that I've followed your work a bit and I'm really impressed
with the library. I'm currently planning on rolling my own crypto as well and
your library is on my list of the things to look at.

One question:

> I was shifting a uint8_t, 24 bits to the left. I failed to realise that
> integer promotion means this unsigned byte would be converted to a signed
> integer, and overflow if the byte exceeded 127.

This sounds weird to me, how is a shift (which only apply itself on the bit
value, not the number value) triggering a sign extension? I don't think you're
talking about integer promotion here unless you had a uint16_t = uint8_t << 24
or something like this.

~~~
kbeckmann
I believe this[1] is the patch that fixes this bug. I tried to reproduce the
behavior but couldn't succeed. Maybe I was doing something wrong. Would really
appreciate it if someone here could show a test case where this matters.

[1]
[https://github.com/LoupVaillant/Monocypher/commit/347189c50c...](https://github.com/LoupVaillant/Monocypher/commit/347189c50c053cf13ce1310818c2913f4904c1eb)

~~~
loup-vaillant
Your failure to reproduce the behaviour is normal: on the compilers I have
tested, this doesn't affect the generated binary.

To see the difference, you need to first modify the makefile to use UBSan
instead of just GCC (just comment the active CC line, and uncomment the right
one). You may want to tone down the optimisations for faster compilation as
well. Then you can run `make clean` and `./test.sh`. You should have a warning
on commits prior to this patch.

------
ktta
I'm amazed that the reference implementation of Argon2 had a bug. So that
means anyone who deployed Argon2 today didn't really use Argon2 (I'm being
pedantic), but something else?

libsodium also got it wrong, so that brought down my opinion of it being a
trusted and well-reviewed library.

Now the question is, will they continue to use the same implementation or move
to the fixed code?

~~~
loup-vaillant
Maybe the authors will fix the code in subsequent versions of Argon2. Right
now, backward compatibility is deemed more important. The effects of this bug
are practically negligible anyway. I bet a single bit of additional entropy in
a password would compensate that a hundred fold.

I'm a bit disappointed however at their not updating the specs. I signalled
the bug in January, and the latest version of the appear to have been
published in March. I guess they had other priorities.

Don't be too hard on Libsodium: they only got it wrong because they reused the
reference implementation. (Also, I don't see them breaking backward
compatibility either.)

~~~
ktta
I'm also referring to the orgs who used Argon2 in production. If they move to
the fixed version of Argon2 in libsodium, they'll have to make a plan to
securely move to the correct implementation by hashing the passwords again. It
can be done, but I'm wondering if they'll even bother.

~~~
user5994461
There is hardly anyone using Argon in production.

The few hipsters who jumped on the new hashing algorithm bandwagon can't have
much users.

------
fefe23
I really liked that read, good writing style and the author was coming from
the same place that I'm usually coming from (can we redo this ourselves and
shave off some bloat in the process).

However, as it stands now, it looks more like a cautionary tale on why he
should have listened when people told him not to do it :-)

I suspect a worthier goal would have been to look for ways to improve the
linking footprint of libsodium instead of rolling your own library. Maybe turn
the "I want the smaller version without the huge lookup table please" in a
compile time option or so. That way more people could have used it without
having to commit to "I'll use this little known experimental library instead
of libsodium". Commit in the sense of "people will ask why and I will have to
defend my decision", not in the sense of "oh no we are locked in!!"

~~~
loup-vaillant
Reducing the linking footprint wasn't my only goal. Reducing the source code
is also important, if only to facilitate tests and audits —thus increasing
confidence. That huge table for instance would need to be tested. Every single
value would need to be checked for correctness, or I would have to demonstrate
they came from a sound generator.

\---

I've been careful not to lock people in. If people want to upgrade to faster
primitives, they can. I don't think they will ever need to, though.

------
conradk
The v1.0.0 was published the day a bug was fixed and yet the first sentence of
this post is "Monocypher is ready for production". Sure, OpenSSL has bugfixes
on regular basis as well, but I believe that further fragmenting the crypto
ecosystem with a new library and little commercial support to properly back it
and pay for security audits is risky.

~~~
baby
It's risky but sometimes it's the only way. To be fair the code size is
relatively small so it should be easier to audit than other products. My guess
is that his blog post is also some motivation to get new pair of eyes on the
library. If there's any fund for that, I'd be happy to look at it ;)

------
HurrdurrHodor
Write a crypto library and blog about how great it is.

Apply the tools that other people used to find bugs in it and repeat blogging
about how great it is.

...

In the end you get a library that is slower and smaller (because you left out
the optimized code, duh). It's still not as small as the smallest.

Is this easier to audit? Dubious but it doesn't matter because if people
hadn't wanted the extra speed they wouldn't have added the optimized code
anyway.

Plus, that code is already audited and rolled out and works! Who cares about
auditing another library that provides nothing new?

The only non obviously-garbage argument here is usability which I am too lazy
to look at because it is too fuzzy to refute anyway.

By all means, write your own crypto but DON'T USE IT! And of course don't tell
other people to use it either.

~~~
loup-vaillant
Let's list the pros and cons of each libraries (let's assume we trust all 3
libraries):

Monocypher vs TweetNaCl:

    
    
      Good:
        Monocypher uses more modern primitives (Blake2b, Chacha20)
        Monocypher provides password derivation
        Monocypher is a bit easier to use
        Monocypher is much faster
      Bad:
        Monocypher is twice as big
      Conclusion:
        Monocypher utterly outclasses TweetNaCl.
    

Monocypher vs Libsodium:

    
    
      Good:
        Monocypher is much smaller
        Monocypher is easier to deploy
        Monocypher is a bit easier to use
      Bad:
        Monocypher is a bit slower (except Argon2i, which is faster)
      Conclusion:
        If performance matters, use NaCl.  Otherwise, use Monocypher.
    

Nothing new, you say?

> _By all means, write your own crypto but DON 'T USE IT! And of course don't
> tell other people to use it either._

Get back to me in a couple years, we'll count the CEVs since version 1.0.

------
AstralStorm
Main lesson not learned: instead of testing, prove correctness in high
assurance code like this. Rigorously and formally. Preferably even refine the
proof to executable code.

(Yes, it would take somewhere on the order of 10k LOC to prove correctness of
this 1k.)

~~~
SAI_Peregrinus
Proving correctness isn't enough. You have to test it too. It's perfectly
possible for the proof to have errors (bugs). It's also possible for the proof
to be incomplete: you can prove the algorithm is correct and implemented
correctly to spec and still have vulnerabilities.

~~~
loup-vaillant
While 100% confidence is impossible (because I'm Bayesian), machine checked
proofs do help. It's then a matter of cost/benefit.

------
mathieubordere
I still believe it's a bad idea to roll your own crypto, interesting as an
exercise, but I would never use it in a production environment.

~~~
timclark
How will cryptography libraries ever develop if someone doesn't roll their
own?

~~~
rootlocus
Through sweat and blood. Don't expect crypto libraries to be secure the first
few years after their release. That's why people favor the mature, battle-
tested libraries.

~~~
jwl
But then again, we still got Heartbleed.

~~~
bananaboy
Yeah and imagine the kind of exciting bugs that exist in new and immature
libraries! :)

~~~
jwl
Yes, but are hackers going after common used libraries to get more vulnerable
systems to attack or are they going to spend time on some unknown homebuilt
crypto? In some cases, security through obscurity works well in practice.

~~~
ZoFreX
> In some cases, security through obscurity works well in practice.

This is not one of those cases. Absolutely not. I'm moderately competent at
finding security bugs in things, but I doubt I could find any in OpenSSL. I am
confident I could find some in your average hand-rolled code.

The thing is people make the same mistakes. There's a set of well-known
mistakes that are very easy to make, especially if you're not versed with the
entire history of implementing crypto - which is the case for the majority of
people rolling their own. This makes it very, very easy to guess what mistakes
they will make, and if you know what you're looking for it's easy to find it.

My "personal best" for finding a crypto bug in a project is 50 seconds.

I doubt anyone has found a bug in OpenSSL (or any established crypto project)
anywhere near that fast.

~~~
jwl
You got to consider your most likely risk for attack. Targeted or at random by
a botnet? For example, you are most likely more secure in practice by writing
your own website than using Wordpress, simply because you are more likely to
get hit by a botnet targeting every Wordpress site than someone going directly
for you.

~~~
ZoFreX
This is off-topic. We are talking about security vs obscurity in the context
of cryptography libraries.

~~~
palunon
Replace WordPress with OpenSSL and website with crypto.

------
sigjuice
Are there other fields where the slogan "don't roll your own XXX unless you
are an infallible expert" is applicable?

~~~
dlgeek
Parachute?

~~~
kabouseng
Actually you do roll your own parachute once you have done enough jumps. But
your reserve chute is always rolled by a certified professional specifically
certified to roll reserve chutes, and not by yourself. (if I remember
correctly, my memory is a bit fuzzy from when I jumped a couple of years back)

~~~
Piskvorrr
Well, you are _using_ the product, but not _manufacturing_ it.

------
chmike
Have the library functions been tested (or proved) to be constant time ? This
is a critical property of libsodium.

Another question is what make the difference in number of line of codes with
libsodium ? I mean what do these lines of of libsodium that your lib doesn't
have contain ?

Also, is this a fair metric ? You know we can have 1000 char long lines in C.
;)

~~~
loup-vaillant
> Have the library functions been tested (or proved) to be constant time ?

Just check the absence of secret dependent branches, and secret dependent
array indices, it's pretty simple.

> _what make the difference in number of line of codes with libsodium ?_

Less lines means less need for eyeballs, tests, and audits. Libraries the size
of TweetNaCl (and, to a lesser extent, Monocypher), can be subjected to very
rigorous audits, and increase confidence accordingly.

> _what do these lines of of libsodium that your lib doesn 't have contain ?_

I can see 3 sources: redundant primitives (Monocypher is not redundant),
alternate implementation (Monocypher sticks to portable C), and verbose
implementations (Monocypher keeps it simple).

> _Also, is this a fair metric ?_

Not quite: I did not count the size of the header in Monocypher and TweetNaCl.
Still, I don't think Libsodium has less than 22K lines of actual code. (Note:
I counted everything with sloccount)

On the other hand, my lines never exceed 80 characters.

------
etatoby
I have read the article and I still don't understand what was the purpose of
such a project.

What's wrong with ZIP or 7-Zip + AES encryption? Or GPG with a GUI? Or any
other existing tool that's already stable and multi-platform?

~~~
snakeanus
This is a library, unlike the ones that you mentioned.

------
vadiml
Hi,

I've browsed through the code and i don't see why say it is not good in MT
setting... an first view the code is completely reentrant...

Another thought: i strongly suspect that inlining these store_le... and
load_le... function will show pretty significant performance boost

------
flanfly
I only had a brief look at it but it seems that crypto_check doesn't verify
whenever the signers public key is on the curve.

~~~
loup-vaillant
I'm not sure. If there is such a test, you should find it on
`ge_frombytes_neg`. I bet it returns -1 in line 1315 if that happens. (It
triggers a rejection when that happens.)

If I'm right about this, it would explain why this code path is never hit: it
would only trigger with invalid public keys. I'll test that as soon as I can.

------
jesse_m
It's interesting that Frama-C was mentioned. I have been playing with it the
last few months and would love to hear exactly how it was used. I've read
through a lot of the documentation but haven't found many examples of it being
used

~~~
dhekir
Part of it is described in the Frama-C blog
([http://blog.frama-c.com/index.php?post/2017/03/07/A-simple-E...](http://blog.frama-c.com/index.php?post/2017/03/07/A-simple-
EVA-tutorial)). It actually presents a tutorial on the EVA plug-in (former
Value Analysis) with Monocypher as a case study.

Note that this is the initial stage of using Frama-C. The next step would
probably include writing some specifications for functional behavior (in
ACSL), and then using the WP plug-in (based on weakest precondition calculus)
to prove those specifications, providing much stronger guarantees.

A few Frama-C case studies are on Github ([https://github.com/Frama-C/open-
source-case-studies](https://github.com/Frama-C/open-source-case-studies)),
including an old release of Monocypher, but they constitute mostly a initial
parametrization of the EVA plug-in. Most advanced users of Frama-C apply it on
industrial code that is not available, but academic users regularly publish
papers with case studies (searching "frama-c" on Google Scholar gives good
results).

------
gizmo385
Heads up the the author: The Reddit and HN links in the timeline are switched
:)

------
pier25
I see crypto as black magic. Can anyone recommend some introductory resource
to demystify it a bit?

Why is it so hard?

~~~
loup-vaillant
Here's a good introduction to cryptography:
[https://www.crypto101.io/](https://www.crypto101.io/)

------
xmichael99
So where is the Mono / C# wrapper, or C# implementation of this? (:

~~~
loup-vaillant
The only bindings I'm aware of right now are for Crystal (it may or may not be
up to date):

[https://github.com/konovod/monocypher.cr](https://github.com/konovod/monocypher.cr)

I think I will personally do C++, OCaml, and perhaps Lua. For the rest, either
someone contributes, or you'll have to look at the FFI yourself…

------
dexterdog
I thought the first rule of crypto was never role your own crypto.

~~~
loup-vaillant
I'm the exception. :-)

Seriously though, Monocypher is getting getting past the "own crypto" stage at
this point.

------
gwerks
Inb4 "don't roll your own crypto"

------
snakeanus
I really liked your previous articles/tutorials about Poly1305 and Chacha20.
Is there any plain to make similar articles about the other constructions that
you implemented on this library?

Is there also any plain to implement scrypt (based on the salsa), SPHINCS
(uses BLAKE(1) and Chacha), Ed448-Goldilocks, Keccak or any of the CAESAR
candidates? (The ones that seemed the most interesting to me were NORX, Keyak
(based on keccak) and HS1-SIV (based on chacha)).

~~~
loup-vaillant
I don't think I'll write for the other primitives, I don't know them well
enough to comment on them properly.

I won't add other primitives to Monocypher, that would defeat the purpose of a
"one true cipher suite". One stream cipher, one authenticator, one hash, one
password key derivation, one key exchange, one signature. That is enough.

When some of those primitives becomes obsolete, I'll consider writing a new,
incompatible version of Monocypher. For instance, we can switch to curve448 if
curve25519 isn't secure enough. I think at least a decade will pass before we
come to that, though.

------
HurrdurrHodor
TLDR: Crypto library with two claimed advantages: Better usability and smaller
code than libsodium. However, it is also slower.

~~~
Piskvorrr
Having all of that would be great, but most of the time you need to weigh one
against another. Smaller code means smaller attack surface and supposedly
fewer ways to shoot yourself in the foot ("I know, I'll use CBC"); of course,
checking whether the author doesn't shoot you in the foot by choice of
algorithms, _or their implementation_ , is advised.

