
Scalable memory allocation using jemalloc - r11t
http://www.facebook.com/notes/facebook-engineering/scalable-memory-allocation-using-jemalloc/480222803919
======
krakensden
I'm impressed that there is still room to eke performance improvements out by
fiddling with the base data structure, the original author has been working on
refining it for quite a while*

* [http://t-t-travails.blogspot.com/2008/07/treaps-versus-red-b...](http://t-t-travails.blogspot.com/2008/07/treaps-versus-red-black-trees.html), [http://t-t-travails.blogspot.com/2010/04/red-black-trees-rev...](http://t-t-travails.blogspot.com/2010/04/red-black-trees-revisited.html)

------
ericHosick
40 points and no comments?

Just wanted to say it is great submissions like this that make me come to
Hacker News...

Thank you for taking the time to write this up and post it here...

~~~
liuliu
It is a long article and requires some time to digest. No comments is a good
thing that implies people do take time to read. I am still in the progress of
reading ...

~~~
JonnieCache
It also shows people here are unwilling to comment from a position of
ignorance; a smaller percentage of readers will be authoritative on such an
advanced topic.

------
Erwin
YMMV but for my Python web app executing mostly OO Python 2.5.x code, I got a
10% performance increase by using jemalloc compared to the malloc in RHEL 5.
It's as simply as LD_PRELOAD=/path/to/libjemalloc.so -- memory usage is also
better (millions of objects allocated than eventually released in a long
running process ended up with smaller amount of memory used when using
jemalloc).

~~~
ot
This is kind of surprising given that CPython uses its own ad-hoc allocator
(see [http://docs.python.org/release/2.3.5/whatsnew/section-
pymall...](http://docs.python.org/release/2.3.5/whatsnew/section-
pymalloc.html)) which resorts to malloc() only for large chunks of memory and
buffers... I wonder what's happening there.

------
amadvance
It would be interesting to see a performance comparison with nedmalloc. It
uses a trie instead a of redblack-tree for a faster information access.

<http://www.nedprod.com/programs/portable/nedmalloc/>

Some time ago I evaluated both of them, and they seemed the best available
choices. But I never found a comparison study.

~~~
jasone
I tried to include nedmalloc 1.0.5 in the benchmark, but it quite simply isn't
polished enough for actual use. To start with I had to remove an invalid
assertion in malloc() and implement posix_memalign(), only to find that it
crashes within moments of application startup.

------
dochtman
Why is this not the default malloc shipped with libc's around the world yet?

~~~
JonnieCache
Because if there is one library that needs to be conservatively packaged, its
libc. A critical flaw in libc pushed out around the world could cause
nightmares too awful to contemplate. Our entire modern world literally depends
upon it.

