
Open-source crypto is no better than closed-source crypto - throwawaymath
https://research.kudelskisecurity.com/2018/10/02/open-source-crypto-is-no-better-than-closed-source-crypto/
======
throwawaymath
The titular thesis of this article is that Linus' Law doesn't really apply to
cryptographic security vulnerabilities.

To lend some authority to that (probably controversial) claim: the author of
this article is JP Aumasson, a well known cryptography researcher and
engineer. There aren't many cryptographers who work in both theory and
application the way Aumasson does. He recently wrote _Serious Cryptography_
and has done professional cryptanalysis since 2012. He is also a co- or
primary author for several cryptographic algorithms; including BLAKE/BLAKE2,
NORX, SipHash, and Gravity SPHINCS.

In the course of establishing his argument he groups cryptographic
vulnerabilities into four main categories. The basic idea is that
vulnerabilities in cryptography require a different set of skills to find than
"regular" security vulnerabilities do. They also have different requirements
for exploitation. Aumasson's overall point is that the higher difficulty of
finding bugs in cryptography for each category means there won't be a
meaningful difference in security between open and closed source code.

And for what it's worth he's only talking about open versus closed source
_implementation_ here. He's not talking about Schneier's Law or closed source
designs.

~~~
snaky
> The basic idea is that vulnerabilities in cryptography require a different
> set of skills to find than "regular" security vulnerabilities do

That may be true for traditional low-level hand-optimized implementations. But
if we consider the trend is clearly toward the high-level proof-based code[1],
which is way easier to audit, then open source is obviously better.

[1] "Simple High-Level Code For Cryptographic Arithmetic - With Proofs,
Without Compromises"
[https://deepspec.org/entry/Paper/Simple+High.2DLevel+Code+Fo...](https://deepspec.org/entry/Paper/Simple+High.2DLevel+Code+For+Cryptographic+Arithmetic+.2D.2D+With+Proofs.2C+Without+Compromises)

~~~
blattimwind
Most issues (apart from legacy asymmetric primitives) are actually either in
the implementation (common in widely deployed software) or the design (even
more common in widely deployed software) of higher level protocols. Many
implementations issues in crypto code are just regular code bugs. Meanwhile,
design issues are not so readily apparent.

There is no linter that will tell you that using the same AES key for years
and keeping a block counter in an untrusted location is a bad idea. And if you
make your code obscure enough ("abstract" in all the wrong ways), then people
will have a really hard time figuring stuff like this out.

And no one does proofs on protocol implementations in practice. Yes, there are
some. Yes, there has been some progress to apply it to e.g. OpenSSL — after
everyone has been using it for critical infrastructure _for decades_ — so
there's like, literally, one practical implementation that has seen a sliver
of formal verification.

~~~
nickpsecurity
There's tooling like Cryptol and SPARK to make this a lot easier than it was
in the past. Cryptol can generate software or hardware that implements the
high-level spec of the algorithm. Far as practical use, certainly not just one
even though still extremely rare. Here's some more:

Altran/Praxis Correct-by-Construction for a cert authority

[http://www.sis.pitt.edu/jjoshi/Devsec/CorrectnessByConstruct...](http://www.sis.pitt.edu/jjoshi/Devsec/CorrectnessByConstruction.pdf)

Same company ports Skein from C to SPARK, finds trouble

[http://www.skein-
hash.info/sites/default/files/SPARKSkein.pd...](http://www.skein-
hash.info/sites/default/files/SPARKSkein.pdf)

SPARK Ada is available under GPL and commercial licenses. The book below is
quite easy to read compared to most stuff with formal methods. The companies
building stuff like this usually also have tools to generate tests (property-
based testing) from contracts/specs and can automatically turn anything you
can't prove into runtime checks. Shocked it's not used more often in areas
like smart contracts or cryptocurrency protocols.

[https://www.amazon.com/Building-High-Integrity-
Applications-...](https://www.amazon.com/Building-High-Integrity-Applications-
SPARK/dp/1107656842/ref=sr_1_1?ie=UTF8&qid=1538687861&sr=8-1&keywords=high+integrity+applications)

Galois built Cryptol for NSA but open sourced it.

[https://www.cryptol.net/](https://www.cryptol.net/)

Rockwell Collins built SHADE and some other tools for high-assurance crypto.
They build stuff for the defense sector mainly funded by NSA. They even have a
separation-preserving, verified CPU.

[http://www.csl.sri.com/users/shankar/VGC05/Hardin.pdf](http://www.csl.sri.com/users/shankar/VGC05/Hardin.pdf)

------
webkike
Let’s invert the thesis: closed source crypto is no better than open source
crypto. So why on earth would you ever choose closed source components?

~~~
Illniyar
Security audits, good implementation and proper reserach require the work of
rare (as in there aren't many) and talented individuals. The funding model for
open-source security is bug hunting and having big corporation who use your
open-source product to give you from their money or their talent.

A funding model where you pay the company to hire auditors, implementors and
researchers directly could be argued to be the better funding model.

I'm not saying it is better, since open-source has proven itself many times,
but it has also failed miserably many times because of lack of funding or
interest.

Open source, especially open-source that is produced and maintained by
companies with different agendas (for instance google) as opposed to a
foundation is prone to having that agenda color the way it works. Commercial
product's agenda usually align better with it's customers then some random
company. Some would say that in a security product it is vitally important
that the creators' motivations are aligned with the users'.

~~~
cestith
If I can hire a code auditor to audit closed source, I get one audit. If I
hire an audit and someone else hires an audit on the same open source code, I
get two audits.

~~~
red75prime
> If I hire an audit and someone else hires an audit on the same open source
> code, I get two audits.

So there's incentive to save money and wait for someone else to do audits.

~~~
cestith
That incentive only weighs so heavily against the incentive to run audited
code.

------
jerguismi
I can easily write closed source crypto that will be absolute garbage. I can
also write open source crypto that will be total shit. With enough seller
skills I could convince some people to use the closed source crypto, I guess
with open source crypto I would be called out quicker.

~~~
watwut
Real world open crypto libraries have sometimes really atrocious code. Openssl
being notorious.

Open source is way to go imo, but the automatic assumption that it is good or
someone reviewed it is wrong.

~~~
umvi
> or someone reviewed it

Sometimes I wonder if everyone assumes everyone else has reviewed it,
therefore nobody has reviewed it

~~~
watwut
It is awful lot of hard work and requires a lot of special knowledge not that
many have. Plus it is guaranteed political fight afterwards.

Average programmer don't know what to look for and can't do it over wekend.

------
gmuslera
It can be said for most software categories. But for specially for crypto,
trust matters a lot. Specially today with governments pushing for forcing
backdoors in crypto.

------
hyperman1
This is a strange way of thinking:

    
    
      For this kind of bug, OSS doesn’t necessarily win, when 82% of projects on Github have fewer
      than 45 stars (a figure I just made up, but you get the idea). 
    

It is true, but also irrelevant. Most OSS are orphaned hobby projects, and
that's what he sees here. But when comparing to CSS, the relevant OSS is the
small parts of it that has users, builds a community, etc.. For these
projects, Linus' law is a serious win for OSS. This won't help the last
category of hard bugs, but is a great booster for OSS that' missing from CSS.

He's probably dead right about the hard bugs, BTW

Additionally, OSS has another less visible bonus: Everybody tests their
tooling on OSS simply because they can. If you write something like AFL,
coverity, ... , you load tons of code from github and see how your tool
behaves on it. Then you write a blog post with your great results. So you
either have to file bugs against the OSS projects you scanned, or the
community will think yo're an asshole.

On the other hand, CSS has a less visible malus: The horrible quality of in-
house tooling at governments, banks, etc...

I've seen a case where a government Architect decided to encrypt every
person's personal data with AES, using the birth date + a few extra digits as
password, and write this in a spec posted on their site. An auditor saw AES
and declared the protocol secure enough for transport over the public
Internet. It took most of a year before our team got through to him that this
amounts to about 20 bits of key space, i.e. brute force-able in a few minutes
on a phone with its some interpreted language, live coded before his eyes in a
meeting. (He then decided to pad the key with zeros until he got it to 256
bits. For some reason you're not allowed to whack government officials with
their own spec, even when their IQ is below that of an average doorknob. But
after long meetings we eventually got to a minimally sane encryption). And we
were the only organization that 'caused so much trouble', the others either
didn't understand the issue or simply didn't care.

I am really really really scared about the security of anything governemental
now.

~~~
dejaime
I would say he misses a big point on the hard bugs though. Saying something is
proprietary does not mean it makes enough money to hire such high caliber
professionals, on the contrary.

I have worked with a lot of extremely competent developers on open source, but
my experience with proprietary software was quite the opposite: professionals
are untrained (to the point where they can't even use versioning properly) and
would be incapable of finding such bugs anyway.

Now assuming that a proprietary cryptocurrency has the resources to hire a
respectable audit firm... it is more far fetched than assuming that such star
developers will just work on an open source one for fun.

------
alexandernst
"Open source" doesnt mean it doesnt have bugs or backdoors. It just means that
you can take the code, read it, and maybe find the bug/backdoor.

------
amelius
> Open-source crypto is no better than closed-source crypto

If the weakest link is the hardware (e.g. Intel ME, or Chinese Trojan chip),
then this proposition doesn't even matter much.

~~~
noobermin
I think the author's point is that all things held equal, OSS crypto isn't
better than CSS crypto in their experience.

------
fipple
The advantage of using open source crypto is that when you get hacked so do a
lot of other sites, so you become only one part of a big story.

------
msla
Open-source crypto wins because it's never just crypto. It's a whole system
with crypto as one element.

The crypto may be the only hard part, but it's still just one part. Like in a
password manager: Even if the crypto is somehow known-good, if the whole
thing's closed-source, I don't know if the non-crypto parts aren't
exfiltrating my plaintext passwords so my accounts can be sold to botters. In
an open-source project, there's a good chance something like that will be
caught without some bored person deciding to run Wireshark at the same time
they're running a password manager.

------
rurban
I don't think it's a matter of open vs closed source, it's more a matter how
many maintainers and contributors the projects has.

As most bigger projects (>3 maintainers) show, the more contributors the worse
is the quality of the code, and logically the rate of bugs is higher.

Closed source projects typically have the best number of maintainers: 1-2.
With open source it could easily degrade into a minefield, like with openssl,
apache, linux, gnome, python, perl, ruby, node, ...

------
tinus_hn
The title is a bit vague and the article doesn’t really clarify. The point of
the article is that open-source implementations of designs involving
cryptography are no better than closed source implementations. The purported
reason is that cryptography is difficult and therefore reviewers are unlikely
to spot issues.

I presume that much open-source software is written by hobbyists while a
closed source software development company may hire a crypto expert for the
sensitive code.

------
m-p-3
At least the open-source crypto can be proven to be good and audited by
anyone.

I'd rather use an open-source crypto than take a security russian roulette on
a closed-source one.

------
jvehent
Open Source crypto is no worse than closed source crypto. This is fantastic
news and a strong validation of the open source development model for security
systems.

;)

------
dejaime
Ignoring the biggest problems of proprietary software (e.g. the fact that such
software can be deactivated at any time for no reason, backdoors) and citing
pseudo problems on free software.

"You can also argue that OSS tends to be reused by other OSS, which amplifies
the impact of a given bug."

This guy clearly as a horse on that race.

------
commandlinefan
Well I guess the only logical solution is to roll your own, then.

------
theyinwhy
If the company devloping the crypto dedicates the same amount of resources,
would open or closed source crypto be more secure?

For me the answer clearly is open source crypto.

------
yitchelle
Open source != good quality code

------
darawk
Doesn't the very fact that he was so readily able to review these blockchain
projects negate his thesis?

~~~
TheDong
No. He's comparing it to closed source software he has also audited.

CSS doesn't mean no one can see the code or that there aren't third-party
audits.

~~~
Nullabillity
> CSS doesn't mean no one can see the code or that there aren't third-party
> audits.

It does mean that there are no meaningful third-party audits. An audit (or
benchmark, for that matter) that aims to be approved by the vendor is as
useful as a rubber stamp.

~~~
TheDong
This is sometimes true, but not always.

There are many auditors who will do a good and fair job.

This is especially true if the audit is not intended to be publicly shared,
but rather for the internal consumption of the company in question.

I'd still say an audit where the reviewers are skilled professionals, but
biased by the source of income, is better than the average open source project
where few people will do such a thorough audit ever; likely none after the
original author, excepting special cases like Linux.

