
Introducing Sodium, a new cryptographic library - kzrdude
http://labs.umbrella.com/2013/03/06/announcing-sodium-a-new-cryptographic-library/#
======
tptacek
This is based on NaCL. NaCL is an extremely great library that we virtually
never recommend because it is very difficult to integrate into most people's
dev environments.

Libraries like NaCL, Keyczar, and Cryptlib work by removing all the design
choices from cryptography. You don't pick the key sizes, you don't pick the
algorithms, you don't even pick what kind of keys you exchange. They implement
a whole cryptosystem, as if for some new unreleased version of PGP, subtract
the file format, and present it as an API. They're _great_. They are literally
the only way you should be deploying cryptography in your applications.

I _think_ this is a great development, but I am not qualified to say. This
sits in a weird place in between a language binding for NaCL and a fork of
NaCL. As a bindings package it's an overachiever; as a fork, it seems to have
made extraordinarily conservative changes.

You'd really like a real crypto dev --- Matthew Green, Colin Percival, DJB
(NaCL's primary author; @hashbreaker on Twitter), Steve Weis, Trevor Perrin,
&c --- to say "this is all totally sane, and if you buy the premise of NaCL,
go ahead and use this instead.

You'd also really like Sodium to say "this is as much as we're changing from
NaCL and nothing else", because what makes NaCL worth building on is the
expertise that went into designing it in the first place, which, like me,
Frank Dennis probably doesn't have.

~~~
moxie
This looks like new development that's just API-compatible with NaCL though,
right? Which feels a little strange.

The reason I'm drawn to the work of DJB is because of his fanatical attention
to security-related detail. He wrote and released his own high performance
mail server, DNS server, web server, and logging replacement during a period
of time when server-side software was plagued with exploitation, and people
have only ever found, what, _two_ somewhat minor bugs? Ever?

So what he creates is almost always amazingly solid, but for all his attention
to security perfection, his software is also amazingly unusable. Want to run
publicfile? First, forget everything you know about standard unix directory
structures, because DBJ doesn't like them.

I feel somewhat the same way about NaCL. We can bet the lives of our first
born children that the implementation won't suffer any of the bugs that
OpenSSL has over the years. Side channel attacks, memory corruption bugs, and
timing problems will almost certainly be nonexistent, and someone could
probably do their PhD thesis on documenting why.

But like his other software, NaCL has weird usability quirks. This project
points out that some of those quirks were certainly manifest in the
implementation, but some of them shine through the API itself as well. Like
his other software, my sense has been that interacting with NaCL is something
that we would almost have to put up with in order to avail ourselves of DJB's
greatness, rather than simple joy.

So a rewrite that's API compatible isn't super immediately appealing to me, in
the same way that I don't think I'd be particularly drawn to an interface-
compatible version of qmail or publicfile that was written by someone else,
when the value was in DJB having written them to begin with rather than the
interface (the horror!) they provided.

Although to be fair, it's certainly not as if anything else is that much more
usable in this world right now.

~~~
jedisct1
As discussed on the mailing-list, there will be a high(er) level API, similar
to the C++ API, that doesn't require dealing with input/output being placed at
different offsets in a shared buffer.

This is not a rewrite. It uses the reference implementations from NaCl and the
same constructions. What it brings over NaCl is a standard build system.

~~~
tptacek
whew.

------
SeoxyS
I just completed a major cryptographic project overhauling the security at my
company. (Re-doing our password hashing and authentication, and the protocol
for all our networking and communications.)

The password hashing was easy: a portable implementation built as a static /
shared C library + language bindings linking to OpenSSL's `libcrpyto` for
PBKDF2 (with an alternate implementation using CommonCrypto on iOS / OSX).

The encryption and authentication layer for the communications was much
tricker. The first draft was an implementation based on industry standards:
RSA2048 + AES256. It needed to be portable to iOS in addition the various
other platforms supported, and Apple has deprecated OpenSSL on iOS and OSX.
Annoyingly, OpenSSL does not ship with darwin-arm support out of the box, so a
custom compile was not an easy option either.

In the end, I ended up picking NaCl, and specifically `libsodium` as a
portable implementation. The library, unlike OpenSSL, is beautifully designed
and very easy to use, and implements asymmetric crypto functions (based on
elliptic curves) which are actually much superior to RSA, providing much
greater security for much shorter key length.

Libsodium is highly recommended.

~~~
jedisct1
s/elliptical/elliptic/ Thanks for using libsodium!

~~~
SeoxyS
Thanks, fixed.

Btw: the CocoaPod for libsodium is broken. I had two problems: 1) the headers
include path for the pod was not set correctly, and 2) one of the algorithm's
alternate implementations would not compile with clang. Removing it and using
the ref implementation instead did the trick though.

I might submit a patch / pull request to fix (1).

------
apaprocki
It is unfortunate so many projects have overlapping names, even playing off
the same kind of puns.

This: NaCL + Sodium

Google: NaCl (Native Client) + Pepper

~~~
tptacek
DJB's NaCL is an academic project that dates at least back to 2008.

~~~
apaprocki
I'm just lamenting to no one in particular that in the universe of available
words/acronyms/abbreviations to describe projects we have as many collisions
as we do. If I had to choose (ugh), DJB's NaCL acronym is at least more
specific to its purpose than NaCl meaning "Native Client" just to make a salt
+ pepper pun. I guess all I'm saying is that if I ever write an open-source
slab allocator, I won't call it OpenSSl.

~~~
tptacek
Of course not. You'd call it "6 Minute Slabs".

------
alexjeffrey
If it works as suggested, this library binding will be a godsend to frontline
developers. Cryptography is exceedingly hard to get right and unfortunately
existing libraries often tend to be written for developers who are already
familiar with cryptography. Hopefully this will see the adoption it deserves
and we'll see less incidents like the Mega encryption scandal.

~~~
missinginaction
Unfortunately, like NaCl, it's missing 2 common and critical functions:

    
    
        1. Password-based key derivation
        2. Persistent key storage
    

So developers are still expected to work out those 2 problems and will
reliably screw them up.

~~~
mpyne
I thought the first one was solved at least 3 different ways over now, and
that we should just pick one of [bcrypt, PBKDF2, scrypt]?

~~~
tptacek
That still really is the answer for password hashes.

The right ordering is scrypt, bcrypt, PBKDF2, but even if you choose PBKDF2
you're still worlds better than salted hashes.

It makes sense for NaCL/Sodium to just pick one, though, and it makes sense
for the choice to come from the hash contest.

------
andrewcooke
how is this better than keyczar?

keyczar is about to support python 3 (there's a patch) and i was planning to
make simple-crypt delegate to keyczar (or just delete the project entirely,
since its only reason for existence was nothing better existed on python 3,
but people seem to be using it). should i delegate to this instead of keyczar?
what is the difference?

<http://www.keyczar.org/> <https://pypi.python.org/pypi/simple-crypt>

~~~
bascule
Keyczar is a library with similar goals to NaCl: relative ease-of-use for
authenticated encryption.

NaCl is using more modern cryptographic algorithms (e.g. XSalsa20, Poly1305,
and Curve25519). These provide faster encryption with smaller keys.

The elliptic curve cryptography, most notably, provides keys an order of
magnitude (or two) smaller than what's available today with Keyczar/NaCl.

Also, the NSA recommends you switch from RSA to ECC:

<http://www.nsa.gov/business/programs/elliptic_curve.shtml>

------
DeepDuh
If it keeps its promises this will be huge. To be honest it's almost too good
to be true (and to have such an open license). Looking forward very much to
using this in future projects.

------
anonyfoo
Excuse my ignorance -- I find myself fairly knowledgeable in crypto/security,
but I'm confused by "Sodium also provides a secure, chroot()-resistant drop-in
replacement for the arc4random() function family, including the ability to
generate random numbers within a given interval with a nearly random
distribution."

What does chroot have to do with arc4random()?

~~~
jedisct1
Take a look at arc4random() implementations for a hint. The generator has to
switch to a new key after (in the OpenBSD implementation, too lazy to check
other implementations now) 1600000 bytes of output. How do you switch to a new
key? OpenBSD implemented a sysctl to retrieve the output of the kernel prg.
This is great, as it perfectly works in a chroot()ed environment.

Other operating systems don't necessarily provide the same facility. They will
try accessing /dev/urandom, and if it doesn't exist, revert to something
pretty bad, like read whatever is on the stack and use that as a key.

Sodium provides randombytes_random(), randombytes_uniform() and
randombytes_buf() that you can use as drop-in replacements for arc4random(),
arc4random_uniform() and arc4random_buf(). On Unix, the file descriptor to
/dev/urandom will be kept open, and accessible after a chroot() call. On
Windows, the crypto services provider will be transparently used instead.

~~~
nwmcsween
Hmm linux provides AT_RANDOM from the kernel at program startup usually for
randomization of stack canaries (but it can also be used for other things).

------
peripetylabs
This is great, especially the Python bindings. I've been looking for something
similar for a while. I was hoping to package just a reference (portable)
implementation of 'crypto_secretbox', but the code seemed to be split up in
multiple files and I couldn't understand their build system...

------
newman314
Next step, openssh ported to use Sodium as an alternative?

~~~
tptacek
SSH is a good example of the kind of thing that can't use high-level crypto
libraries, because the constructions required to implement SSH are part of the
specification.

We're all waiting for Daniel Bernstein to replace SSH, but it hasn't happened
yet.

~~~
jedisct1
Do we need a replacement for SSH? The OpenSSH team is doing an amazing work.
Besides making the tool as secure as possible, they are also improving the
protocol. The recently added support for encrypt-then-MAC modes is a great
step forward.

~~~
tptacek
I have a lot of respect for the OpenSSH team but think that they are saddled
with a protocol that was designed (a) before anyone had a good idea how to
design resilient cryptographic transports and (b) without a clear
understanding of how people would want remote control channels to work in
2013.

------
rosser
NaCl isn't new; it's been around since at least 2009.

~~~
tptacek
Nobody uses it, because it lacks bindings to common languages, which is the
problem this is solving.

~~~
count
Is that why, or is it because nobody can find it? At first I thought you were
talking about google's Native Client, and it's the Google results for 'nacl'
or 'python nacl', etc. have both the crypto lib. and the browser vm
interspersed.

~~~
bascule
I'm the author of RbNaCl, the Ruby binding to Sodium. RbNaCl wouldn't be
possible without Sodium, because RbNaCl is written as a Ruby FFI binding and
thus requires a shared library. Sodium builds a shared library. NaCl does not.
Let me explain why...

There are many problems that hinder building language bindings to NaCl. See,
for example, node-nacl:

<https://github.com/thejh/node-nacl>

See the giant warning in bold at the top:

"WARNING: This library DOES NOT WORK on 64-bit systems, and there's nothing I
can do about it before the next version of NaCl is available."

Much of the assembly in NaCl is presently NOT position independent code which
causes many of these sorts of problems. djb has admitted as much and sought to
fix these sorts of problems.

Here is what djb had to say in email recently:

"More language support. The real work here is making everything PIC. Of
course, if what matters is the API rather than speed, then achieving PIC is
easy: just remove the asm."

At the very least, djb recognizes that removing the ASM is a simple and valid
option for achieving PIC now without additional work on the assembly code. In
the meantime he is doing a lot of that work inside SUPERCOP.

There's also the elephant in the room: Windows. NaCl doesn't support Windows.
NaCl will likely never support Windows. Sodium supports Windows.

~~~
erichocean
> WARNING: This library DOES NOT WORK on 64-bit systems, and there's nothing I
> can do about it before the next version of NaCl is available.

FWIW, that warning is...incorrect. Anyone who actually wants to use NaCl with
Node.js (and we do) can trivially get a 64-bit build (it's even done through
Python). I don't disagree that the original author of node-nacl wasn't skilled
enough to do it though.

> There's also the elephant in the room: Windows. NaCl doesn't support
> Windows.

Say what? NaCl supports Windows just fine – it's just djb's 'do' build script
that doesn't, along with how to get a random number. There's absolutely
nothing to prevent anyone from using NaCl on Windows, if they are willing to
integrate it into a normal Windows build system (which is...trivial) and make
a few, trivial, API updates.

\----

Having addressed those mis-understandings, Sodium is a welcome development and
will certainly make it _easier_ for people to develop bindings to NaCl.

~~~
bascule
> Say what? NaCl supports Windows just fine – it's just djb's 'do' build
> script that doesn't, along with how to get a random number

You realize you're describing exactly what Sodium fixes (portable build system
+ randombytes for Windows), right? That was my original point.

------
jsdalton
Yeah, cryptography libraries in general are a pain to use and typically
require a lot of seemingly arcane configuration and confusing (to the novice)
setup.

I wrote this library recently (primarily to scratch an itch on another
project), which really does nothing more than pass sane defaults to PyCrypto
and eliminate crypto jargon:

<https://github.com/jsdalton/secrets.py>

Honestly I think it took more time to wrap my head around the simple use cases
than it did to implement this wrapper once I did.

~~~
tptacek
Stop using this and start using Sodium, which is more secure than your wrapper
library in significant ways that illustrate why people shouldn't think of this
stuff as "jargon" that they can just write wrappers for; for instance:

* It generates encryption keys insecurely instead of using a cryptographically secure KDF

* It leaks timing information on the MAC comparison

* It makes verification of messages optional; verification should never be optional (in your case, when verification is disabled, I think you have the CBC padding oracle vulnerability)

~~~
jsdalton
Thanks for your comments and recommendation. Your points (and my own failings
here) illustrate why libraries like Sodium (which was not on my radar a few
weeks ago) are a good thing.

As an ordinary developer, I just want to _use_ a library like PyCrypto in a
safe manner. When I described the language as "jargon" I did not mean to be
dismissive of it, but rather to say that libraries like PyCrypto force you to
make decisions about terms that are nothing more than jargon to a non-expert
-- when really what a non-expert needs is an extremely simple API which makes
those decisions on your behalf in a safe manner.

Thanks again and these are humbling lessons to learn.

~~~
tptacek
You seem smart and like someone who enjoys this stuff, so if you want a much
more fun lesson (or, uh, 40+ of them), drop a line to sean AT matasano DOT com
and he'll send you a bunch of exercises that'll teach you how to exploit this
stuff.

~~~
X-Istence
Are any of those lessons public, or semi-public? I am very interested in stuff
like that as I am working on crypto systems for a gov't project.

~~~
tptacek
Open invite! Send mail to sean, he'll get you started. We're not publishing
them and we'll ask you not to circulate them, but instead to let anyone you
know who wants to see them to just mail sean.

~~~
X-Istence
Alright, email sent. Would love to work through them with the rest of the team
at work though, think it would be good for all of us to have a good
understanding of how to break crypto systems.

