This is the wrong funding model. It keeps money in OpenSSL developer's pockets, but there is no financial incentive for any OpenSSL developer to work on foundational improvements to OpenSSL. He said himself: there is over $100,000 in open contracts for competent developers to work on non-foundational improvements to the project. If you are an enterprising developer with good C skills and a knack for crypto projects and you apply to work for the OpenSSL foundation, are you going to start servicing that $100,000 pool of contracts or are you going to pretend that money doesn't exist and live on ramen?
If nearly all of OpenSSL's revenue comes from clients that want OpenSSL to meet their particular needs, then none of that money is going to developers to strengthen OpenSSL's foundation. This is why OpenSSL looks like a hodgepodge of hacks upon hacks in order to accomplish narrow goals with limited impact testing. It should be no surprise to anyone else: clients are literally paying OpenSSL developers for this, and nothing else.
Who is paying OpenSSL for developers to clean up the code base and remove ancient #IFDEFs? Who is paying OpenSSL for developers to analyze code paths and do case analysis? Who is paying OpenSSL for developers to write unit tests or even have a test harness at all?
No one will pay an hourly rate to accomplish these tasks. Google is not going to pay by the hour for a developer to stare at a function until they grok it; they want a feature. Joe Company will not pay for developers to write unit tests, they want OpenSSL to handle $QUIRK from a vendor's system, or to know how to make their code handle it.
This model needs to go away. Competent OpenSSL developers time is too valuable to waste on client asks. Their project is too important, and as long as the money is flowing only for novel features and not structural improvement, then that money will dictate that only new features are developed.
"This is why OpenSSL looks like a hodgepodge of hacks upon hacks in order to accomplish narrow goals with limited impact testing."
It doesn't just look like a hodgepodege of accumulated hacks, it is a hodgepodge of accumulated hacks. :)
"It should be no surprise to anyone else: clients are literally paying OpenSSL developers for this, and nothing else."
One could say this with respect to many popular open source projects, including ones with corporate sponsorship. The complexity just keeps building over time and there is no such thing as "finished, accepting bug fixes only".
"Who is paying OpenSSL for developers to clean up the code base and remove ancient #IFDEFs? Who is paying OpenSSL for developers to analyze code paths and do case analysis? Who is paying OpenSSL for developers to write unit tests or even have a test harness at all?"
Those are rhetorical questions. We know the answers. Alas, when the people who pay for (open source) software and consulting pay to have "features" removed instead of added, "pigs will fly".
Doug McIllroy is quoted as saying, "The hero is the negative coder".
(Just in case this need explanation:
Prof. McIllroy is the mind behind UNIX pipes and one of computer science's most prominant contributors.
"Negative coder" means someone who removes code instead of constantly adding, or "committing", new code.)
We could really use some more heros. And as we switch away from OpenSSL there will be a lot of links to libssl to remove.
Meanwhile some people have been writing and testing small, auditable and usable open source crypto, more or less for "free".
My guess (and hope) is that pathological requests for "features" to be added would be met with heavy scrutiny. The authors already have day jobs in academia.
Can you point out specific examples that you view as hacks upon hacks?
Maybe I've spent too many years in the code base, but I've also seen worse.
OpenSSL does a lot. Maybe smaller modules would be better and more testing certainly. Organizations using it should also be contributing back more.
Hacks upon hacks seems like a stretch to me.
I like Dan's work and have used in in projects, I just think your comparison and analysis are quite off base.
As I am to mine.
From the tweetnacl.cr.yp.to paper:
"OpenSSL is the space shuttle of crypto libraries. It will get you to space, provided you have a team of people to push the ten thousand buttons required to do so. NaCL is more like an elevator -- you just press a button and it takes you there. No frills or options.
I like elevators." - Matthew D. Green, 2012
Yes, it is improper to compare a space shuttle to an elevator.
It's also absurd to use a space shuttle when all you need is an elevator.
Use whatever you want. Not everyone's needs are the same.
I like small components that are independent. The OpenSSL binary is feature for feature one fo the most complex I have ever used.
I prefer simplicity. That's just me.
Not for everybody. But some might desire it.
You have my sincere apologies for daring to mention an OpenSSL alternative.
The fact that this NaCl is so small and limited is the whole point.
I guess that point was missed.
Comparing it to a library that is mostly crypto primitives is not a fair comparison.
Also - I'm still curious of examples of "hacks upon hacks" for my own curiosity. I've been using OpenSSL in a number of projects for 15+ years, so maybe I am used to certain things.
Maybe throw in hardware token/PKCS11 support?
With all due respect that is complete bullshit. I do not care that you put quotes around free. Writing "free" will never be considered to include sums in the hundreds of thousands of dollars. More importantly blatant lies like this muddy the debate and set outrageous expectations. The Nacl project gives the following description of funding:
NaCl was initiated by the CACE (Computer Aided Cryptography Engineering)
project funded by the European Commission\'s Seventh Framework Programme
(FP7), contract number ICT- 2008-216499. CACE activities were organized
into several Work Packages (WPs). NaCl was the main task of CACE WP2,
\"Accelerating Secure Networking,\" led by Tanja Lange (at Technische
Universiteit Eindhoven) and Daniel J. Bernstein (at the University of
Illinois at Chicago, currently visiting Eindhoven). CACE nished at the
end of 2010 but NaCl is a continuing project.
...Many of the algorithms used in NaCl were developed as part of
Daniel J. Bernstein\'s High-Speed Cryptography project funded by
the U.S. National Science Foundation, grant number ITR-0716498.
NSF ITR-0716498 funding: (USD) 400,000.00
EU 2008-216499 funding: (EUR) 4,733,078.00 ***NEED WP2 line item***
NSF 1018836 funding: (USD) $436,203.00[^3]
NWO grant 639.073.005 funding: ???????????
NB: This is the nwo funding site: http://www.nwo.nl/en/funding I think the english version may have a reduced set of features. I can not find the this grant information on the site.
The point is that using something like NaCl costs you, the developer/user, nothing more than if you are using OpenSSL.
Do you agree?
I have to say I am confused about your reply in the first sentence you seem to acknowledge that the wordingwas related to the cost of "writing and testing" crypto software. However in the second sentence you seem to indicate that your thesis was about the switching costs users face. Which is it? You did not say I get to use nacl "more or less for free" you said that "people have been writing and testing small, auditable and usable open source crypto, more or less for 'free'." That quote seems to be about the cost of creation not the switching costs.
Do you think djb et al produced nacl "more or less for free?"
I mentioned "free" only to point out that there is no financial cost to switching to it. I guess I did not type the sentence with enough care; words are missing. My apologies.
I imagine people would be willing (and are accustomed) to paying for software of similar quality.
But I'm also wondering why this bothered you so much.
Does it make a difference that grants were received?
Is the funding not transparent enough?
The blog article on OpenSSL mentions payments for consulting and "features" to be added to OpenSSL.
Should I be concerned about what those features are, and who is paying for them? Are you concerned?
I'm just nterested in cleaner code than OpenSSL's. NaCl looks cleaner to me.
Maybe I'm wrong. But I'd rather be compiling programs that use libnacl or some other simpler alternative than ones that use libssl.
We all have to make decisions about what software we choose to use, even if we are not cryptographers.
I see nothing wrong with discussing alternatives to OpenSSL. This bug has been a real PITA.
> I mentioned "free" only to point out that there is no financial cost
> to switching to it. I guess I did not type the sentence with enough
> care; words are missing. My apologies.
> But I'm also wondering why this bothered you so much.
- "Why should I donate to GnuPG/OpenSSL/Tor/Mozilla(NSS)? Those NaCl
devs wrote NaCl for free."
- "How hard could it be to implement a crypto library? Nacl was a side project. The Nacl devs 'have day jobs in
academia' and created nacl in their spare time. They did it for free, so they obviously didn't need to spend money on
testing environment, research material or hire/consult experts. On the other hand look at SelfiesMadeEa.sy they
raised serious cash and had to quit their jobs because they tackle hard problems."
- "Obama and the rest of gubmint are taxing me to death. Government should be pay for the military and maybe some
roads; not waste money on liberal academics in ivory towers, maplethorpe and those pinkos from NEA or some stupid
robot/telescope that cant do metric conversions."
- "OMG NSA is evil. USA does nothing but invade countries and privacy."
> Does it make a difference that grants were received?
Somewhat tangential: Knowledge of the grants also seeks to eliminate the
idiocy in the latter two examples above. People need to be reminded that
big government is not always an evil force, governments can be a force
for good. I do not know if you saw my other comment about tor funding
but tor had revenue of \$2+ million in 2012 and 60% came from US
government. I don't know where you are from but I bet you have met a
simple minded moron wearing a tea party costume or trendy European
threads that will not stop complaining about the evil Obama surveillance
administration. Blow their minds and ask them to wrap their heads around
- $800k from DoD for "Basic and Applied Research and Development in
Areas Relating to the Navy Command, Control, Communications,
Computers, Intelligence, Surveillance, and Reconnaissance"
- $350k from State for "Programs to Support Democracy, Human Rights
and Labor" and "New America Foundation: International Programs to
Support Democracy, Human Rights"
> Is the funding not transparent enough?
> The blog article on OpenSSL mentions payments for consulting and
> "features" to be added to OpenSSL.
> Should I be concerned about what those features are, and who is paying
> for them? Are you concerned?
It's only the wrong funding model if there is a superior alternative. It's a non-ideal funding model, but as is the main point of the article, they are and have been actively looking for alternative sources of funding without success. I hope your gripe is not directed at OpenSSL but at those who could be supporting it financially.
If research was market-driven in 1859 Riemann might not had time to play with zeta functions because at the time no one knew what to do with them. Same for Bayes and so many others. It's not that those man had a better environment around them than current researchers, it's mostly that we're getting results slower than we might as a human kind.
The reason it's not working is that their rates are far too low. This is, for whatever reason, a classic mistake of technical people. If you're turning down work for lack of time, you need to increase your rates until this stops happening.
* Who owns which subsystems?
* Is there a board of governors or a BDFL or something like that effectively overseeing the whole project?
* What is the process for screening commits from people new to the project?
This whole post seems to be tinged with a bit of defensiveness on behalf of the most active committers to the project. But it wasn't the active committers who introduced this most recent bug.
Since meeting the revenue milestones of $1,253,241 in 2009,
$1,574,119 in 2010 and $1,681,101 in 2011, Tor has reached new
heights in 2012 with over $2 million in revenue (unaudited).[^3]
[^1]: List of sponsors/projects: https://trac.torproject.org/projects/tor/wiki/org/sponsors
[^2]: Example monthly report for SponsorF's project: https://lists.torproject.org/pipermail/tor-reports/2014-Marc...
[^3]: Tor Project Annual Report 2012, pg 8, https://www.torproject.org/about/findoc/2012-TorProject-Annu...
The fundamental facts are these: openssl contains a large quantity of code that, if I where to check into my company's repo, I would have at best a rough conversation with the cto and at worst I'd get fired. Plus a lack of good tests. These combine to create more than hypothetical problems; we've seen some severe security holes and there's almost certainly more to come.
The question that should be discussed is if openssl is, ala sendmail, unsuitable for purpose and, if so, what should it be replaced with.
It is 453,000 or so lines, more than five times the size of xv6. It is ten times as big as PolarSSL. Documentation and internal structuring is wildly inconsistent. Features that make static analysis annoying are widely used. The API is far too low level.
Do you believe this is acceptable in a security library? Do believe that aspiring to the security of qmail or OpenSSH is a reasonable goal, even at the cost of features? Why should I use OpenSSL for TLS termination when formally validated alternatives exist?
Oh please do share! (spoiler: alternatives which enable side channels because the pretty compiled-optimized-code (that is generated from source code that itself may feature immutable data etc. with no side effects) is ripe with leaks via cpu branching, caching, etc etc do not count.) This is not a spiteful rhetorical inquiry, by the way!
Side channel attacks once you've already got code running on the same operating system as the target are much easier. But if you can get arbitrary code to execute on the same machine running OpenSSL, you probably already have their keys.
So, I think the criticism to OpenSSL is valid. Why use it when it seems there are less bad alternatives? A lot of it comes down to network effects, inertia, and licensing. That's a much better answer than side-channel attack surface area, which all RSA implementations share :)
Before you write secure software, you have to consider who your adversaries are, and what they are capable of doing. Side-channel attacks over the network are definitely practical   . If you're making even a basic SaaS product, you should assume your adversaries can carry out the above. You should assume that by now, even lowly script kiddies have side-channel exploits in his arsenal.
 Relevant paper (2010) http://www.informatics.indiana.edu/xw7/papers/WebAppSideChan...
 Ed Felton's analysis on 
 Bruce Schneier's analysis on 
To remove side channels and preserve validation, you need to be slightly clever. You validate that a C implementation of one of the functions does the same thing as a function in the validated package, then swap them in the build output. This can be done for miTLS. Your validation of the C part is much more limited in scope than the whole thing.
Aha, this is interesting! Thanks for the link.
> OpenBSD has had two holes in a heck of a long time
> 453,000 or so lines, more than five times the size of xv6
> Do believe that aspiring to the security...even at the cost of features?
> Why should I use OpenSSL for TLS termination when formally validated alternatives exist?
I did not want to inject a pile of sloccount data in the main body of my comment but I do have some questions about
your counts. After a little research it seems that your numbers might be somewhat hand wavy which is unfortunate
because you gave no indication of this in your comment.
What versions of OpenSSL and PolarSSL are you referring to? My count for OpenSSL 1.0.1g puts it at 6.5 times the
size of PolarSSL and not the 10 times that you stated in your comment. I could not connect to OpenSSL.org so I do
not know if they have released a version since April 7th but I must say I am surprised that that the OpenSSL dev
team added 91,344 lines (25% increase) in the last seven days. And even if they did that still puts them 90,000
lines short of "ten times the size of PolarSSL."
My sloccount data:
# OpenSSL 1.0.1g:
Totals grouped by language (dominant language first):
ansic: 273820 (75.71%)
perl: 69192 (19.13%)
asm: 11015 (3.05%)
cpp: 4367 (1.21%)
sh: 3238 (0.90%)
lisp: 24 (0.01%)
Total Physical Source Lines of Code (SLOC) = 361,656
# PolarSSL 1.3.6-gpl
Totals grouped by language (dominant language first):
ansic: 52212 (94.71%)
sh: 2165 (3.93%)
perl: 748 (1.36%)
tcl: 4 (0.01%)
Total Physical Source Lines of Code (SLOC) = 55,129
# Linux kernel 3.14
Totals grouped by language (dominant language first):
ansic: 11828087 (97.00%)
asm: 275047 (2.26%)
!!!TRIMMED counts < 0.5%!!!
Total Physical Source Lines of Code (SLOC) = 12,193,312
Totals grouped by language (dominant language first):
ansic: 7022 (87.81%)
sh: 481 (6.01%)
asm: 301 (3.76%)
perl: 189 (2.36%)
lisp: 4 (0.05%)
Total Physical Source Lines of Code (SLOC) = 7,997
As for the rest of your comment, I have used OpenBSD on the desktop. I have never found the lack of a webserver running, or anything but sshd to be a hindrance to making a machine on which with a browser I can read papers, bank, listen to music, and program. sshd is of course attackable by anyone: good luck getting in though. Other OS's have followed OpenBSD, as do most hardening guides.
One of the features OpenSSL has is support for ciphers that long ago have been disabled, together with out-of-date TLS versions such as SSLv2(!). TLS is a specification, with multiple implementations. The specification is of poor quality, the protocol worse, but OpenSSL is in a class by itself, including implementations of national ciphers no one uses.
While Frankencerts were an issue, there was no realistic way to turn the two incorrect "OKAY" and the one "NOT OKAY" result of PolarSSL into an attack. You would need a CA to issue a cert with a start time in the future. It also does not take an enormous amount of code to fix this issue.