

McDipper: A key-value cache for Flash storage - jonchang
https://www.facebook.com/notes/facebook-engineering/mcdipper-a-key-value-cache-for-flash-storage/10151347090423920

======
Titanous
Twitter open-sourced an implementation of this concept recently:
<https://github.com/twitter/fatcache>
<https://news.ycombinator.com/item?id=5202868>

I hope Facebook ends up open sourcing McDipper in the future.

------
erichocean
I find myself not working memcached (and variants) into new designs these
days, mainly because with Cassandra on the backend, there's no point -- better
to spend the money on the Cassandra machines and let it manage the caching,
in-memory indexing, and log-structured writes.

Memcached in that scenario just adds network overhead, devops complexity, and
increases the number of things that can go wrong.

For the times when Cassandra is overkill, I still end up using Redis -- with
the Lua stuff, it's ridiculously flexible these days.

Still, McDipper (and Twitter's fatcache) both look cool, and I'm glad people
are thinking about SSDs as something other than faster "disks".

~~~
daemon13
I've been chewing the idea of adding Cassandra to my stack for quite some time
as Cassandra seems like inpenetratable moster :-)

Any pointers how to start? Also, do you have any posts on how you added
Cassandra to your back-end?

------
justincormack
I can't work out if the bit labelled "hash buckets" is in RAM, while data
volume is SSD or if it is all SSD. I am guessing this is the case so it is
quite similar to eg Twitter's fatcache.

~~~
alexgartrell
It's actually all SSD by default, but you can pin the buckets into memory if
you'd like. We generally do this in the memcached-replacement use case and
specifically for CDN. However, we do have another use case not emphasized in
the article that has a mix largely consisting of smaller values which we store
in the bucket cache as direct records on SSD.

In general, if it fits in memory, it's faster to keep it there. If you don't
have space for whatever reason, you leave it on the SSD.

