It prompted a lot of interest in their solution, which prompted them to release the patch.
edit:  by "sensitive information" we mean keys, if I understand the implementation correctly, you could still overrun into web server allocation and get back HTTP headers.
Randomized allocation makes it nearly impossible to forge pointers, locate sensitive data in the heap, and makes reuse unpredictable.
This is strictly more powerful than ASLR, which does nothing prevent Heartbleed. Moving the base of the heap doesn't change the relative addresses of heap objects with a deterministic allocator. A randomized allocator does change these offsets, which makes it nearly impossible to exploit a heap buffer overrun (and quite a few other heap errors).
The same reasoning applies to reads. If sensitive objects are distributed throughout the sparse heap, the probability of hitting a specific sensitive object is the same as the probability of overwriting the vtable in the above attack. The probability of reading out any sensitive object depends on the number of sensitive objects and the sparsity of the heap.
There are also guard pages sprinkled throughout the sparse heap. Section 6.3.1 shows the minimum probability of a one byte overflow (read or write) hitting a guard page. This probability increases with larger objects and larger overflows. You can also increase sparsity to increase this probability, at a performance cost.
The guard pages I believe help---but random guard pages just mean I won't know quite what's protected and what is not. This last week I benefitted quite a bit from being able to reconstruct year old server memory layouts precisely.
In this case, I want a marginal chance of compromise no worse than 2^-192, about the strength of RSA-2048.
Akamai patched the announced Heartbleed vulnerability prior to its public announcement. We, like all users of OpenSSL, could have exposed passwords or session cookies transiting our network from August 2012 through 4 April 2014. Our custom memory allocator protected against nearly every circumstance by which Heartbleed could have leaked SSL keys.
So: did the custom memory allocator exist already in August 2012? From reading the post this looks to be the case. Could it be that someone at Akamai took a look at the heartbeat (or other OpenSSL) code, decided that it could lead to memory leaks, and wrote their own memory allocator wrapper code to guard against this?
Edit: SSL -> OpenSSL
> This patch is a variant of what we've been using to help protect customer keys for a decade.
So it protects keys against heartbleed, but not other HTTP related data (urls, cookies, headers, etc).
Linux CryptoAPI seems to have an asymmetric key interface, RSA signatures are implemented, and there's even a x509/ASN1 parser. Shouldn't this be the default on Linux rather than furthering the NIH syndrome with crude library-local allocators? In the very least, shouldn't there be a secure malloc in glibc? If not, why doesn't OpenSSL use an existing secure malloc library?
Any extra library dependency makes OpenSSL less portable. It doubles the work needed to get it running on your new embedded system. shrug It may also increase the code size and attack surface (although probably a good trade-off).
I don't disagree that it sounds like a good idea :).
Given that it's likely that the private key can be derived from those parts, it would be good to know if this custom allocator is 'enough' (i.e. are the intermediate values stored in memory allocated using the same allocator)?
This week has been full of surprises. Has anyone tested this?
To test this belief, our engineers have rebuilt
every Akamai configuration since August 2012,
when we upgraded to openssl-1.0.1. This included
every kernel, hardware, and edge server software
combination; and then a careful inspection of the ways
in which memory was allocated, to see if any
non-long-term memory allocations might border
on our secure heap. Most of the configurations
were proven safe; but we found one configuration
that was not - there was an available memory block in
range of the secure key store.
This less safe configuration was active on our network
for nine days in March 2013
As someone who grew up in the Akamai ops environment... How else could you get anything done? I've taken this tool for granted since it was built in 2000. I expect every planetary scale computing company to do this. Isn't it what Heroku and similar git-push-to-deploy systems are supposed to get you?
If you load 1000 keys, can you extract anything past the 256th?
"This arena is mmap'd, with guard pages before and after so pointer over- and under-runs won't wander into it."
Doesn't that mean that this will only protect against overreads of a certain max length (such that the int16 length in heardbleed)? Seems like that wouldn't help with a length defined as a bigger int. I wonder if there's any better ways of doing this.
Wondering what the cost of putting each primitive (e.g., RSA private signing + key) in a separate (child) process would be? Mailservers (qmail and postfix) seem to do designs like this.
You could also imagine a page type that would only be readable to certain code segments (would take CPU support to do it.)
Isn't this what TPM is supposed to provide? It received an unfavorable reputation because of its association with DRM, but it addresses the same problem.
Context switching between applications is costly in terms of cache hit and CPU time.
The other problem is you then have to IPC the sensitive data to the other process, which will mean putting in temporary memory, which would expose it to stack crash, or heap attacks. You've also got to worry about synchronizing the different processes.
As for why they didn't "share" earlier, and assuming they didn't: the OpenSSL project would probably not have accepted this changeset anyways. It's extremely intrusive and the problem it addresses was, at the time this was written, speculative.
Yes I know that many open-source licenses do not obligate one to do this, but it still seems like the right thing to do to me.
"It [Affero GPL License] has one added requirement: if you run the program on a server and let other users communicate with it there, your server must also allow them to download the source code corresponding to the program that it's running."
I understand the reasoning behind AGPL and on the surface it seems like a good idea, to stop the parasitic behaviour many have towards FOSS. But there are many situations where you have very legit reasons to avoid it. Even in this case, AGPL would put you in a delicate position where you need to immediately disclose the changes you made without respecting the non-disclosure period for mitigation.
It's a licence that tries to solve the problem of the GPL going obsolete when many companies no longer distribute software, but rather services based on the software (like Google or Facebook) and they can basically get away with not giving anything back at all to the community work they built upon. But this is very hard to regulate and as a result AGPL is often so cumbersome that AGPL-licensed code is strongly avoided, and it's also extremely hard to enforce when the service simply doesn't release any code or software.
Enforcement is hard, but AGPL is strictly stronger than GPL which doesn't evenninvitr enforcement on sharing server code.
Yes, many companies avoid AGPL software. That isn't a problem for AGPL-leaning authors, that's the point.
There are many reasons why AGPL is used in extremely few projects of any relevance, compared to other GPL licences, or BSD, MIT, etc.
Now, to prove that someone is running AGPL software and that they have modified it, that seems pretty hard in many scenarios.
Software is protected by copyright by default, meaning you can't make a copy. A license is a limited waiver of copyright that grants permission to make a copy, if the conditions of the license are met.
I'd go so far as to say "nearly all of them."
iXSystems does a lot of cool community work via FreeNAS, PC-BSD and FreeBSD.
Github (Ruby), thoughtbot (Ruby), Google (Go, Python), Dropbox (Python), AT&T (C++, Ruby), Zurb (Ruby, Foundation), Twitter (Ruby, Scala), ...
missing a bunch and some platforms too.
Such decisions often happen at the engineer level, and it isn't for money management reasons, but rather "avoid purchasing and requisition BS, followed by licensing compliant BS" reasons.
Otherwise, the AGPL would never have seen the light of day.
So, it's also not true that "any employee of the company can publish the modications to the GPL-licensed code".
They're not distributing the code outside of their company, so the source license in this case is entirely irrelevant.
We could make a different argument if they were using AGPL'd code...
So we're stuck: we avidly use and consume free and open source software, but can only give back code by carving off bits here and there.
Now, my current employer... Email your manager: "I wrote this entirely in-house and would like to open-source it." Manager: "OK, let's talk to Legal" Legal: "OK, get at least one other person to verify that it doesn't contain any trade secrets and sign this." Upload to github. Done.
And submitting patches upstream - just a matter of code-review and sending it out. Now that's OSS-friendly.