
How to choose an in-memory NoSQL solution: Performance measuring - dsr12
http://highscalability.com/blog/2015/12/30/how-to-choose-an-in-memory-nosql-solution-performance-measur.html
======
_mikz
> Tarantool is an open-source NoSQL database management system and Lua
> application server developed in my.com. The first version of Tarantool was
> released in 2008 and the newest version is 1.6.7.

and later

> Through all tests we executed, Tarantool showed the best result for the
> count requests per second and for many of tests latency values on any type
> of examined workloads. Therefore, we can decide that for most of typical
> projects Tarantool suits them more that popular solutions such as Redis,
> CouchBase or Memcached. This is the basis of our decision to use Tarantool
> for our projects here at my.com.

So yeah. Obviously.

~~~
rtsisyk
tarantool.org is an open-source project which is supported by my.com __and
__many other commercial customers. my.com is an independent U.S. company that
choose products on competitive basis.

Anyway, the benchmark is fully open. If in doubt, you can always download disk
images and re-run benchmark.

// Disclaimer: [http://tarantool.org/](http://tarantool.org/) developer

------
frik
Why is memcached under-performing? Is it maybe their driver? Or their
benchmark? Shouldn't it be similar to Redis, if not a bit faster?

It's a bit fishy that the leading one is by the guys who wrote the article,
see
[https://news.ycombinator.com/item?id=10814318](https://news.ycombinator.com/item?id=10814318)

~~~
takeda
Those tests don't look reliable.

For example I'm quite surprised that no one pointed it out, but if you look,
the memcache its performance grows nearly linearly to number of threads. They
stopped at 256 when it was about to take over redis.

Now if you look at workload A and B, workload A supposed to have 50/50
read/write while workload B was 95/5 read/write. You see that memcached
performed terribly there. You would think that perhaps the access pattern is
different but then if you look at rest of databases they perform closely to
workload A. And memcached which was doing 15000req/s is doing 10000req/s on a
workload which supposed to be less work.

This looks to me that performance issue is most likely not in memcached itself
but in their test application, but we can't prove that because they did not
release code they used for testing.

------
hoodoof
Postgres is also a good in-memory NoSQL solution. Actually you can use
anything really as an in-memory NoSQL solution such as ArangoDB, MySQL,
MongoDB, RethinkDB or whatever.

~~~
Cieplak
Do you mean using postgres tablespaces backed by memory or do you mean using a
FDW?

If you do mean using a ramdisk tablespace, the postgres docs recommend not
doing that. However, if you really want to do that, make sure you attach xlog
(the WAL table) to that ramdisk tablespace as well, otherwise every
transaction will still hit the disk.

~~~
anarazel
Not the poster upthread. But if you have enough memory (shared_buffers) and
use unlogged tables, the effect is going to be that both reads and writes
pretty much only hit memory, no disk. Without going through the trouble of
moving tablespaces or anything.

------
coleifer
Sqlite is always a good option if you want in-memory and SQL. Kyotocabinet
supports a number of in-memory database types, as does its networked
counterpart kyototycoon. BerkeleyDB can be effectively run in memory if you
size your cache right, and as a bonus you can build bdb with a sqlite
frontend.

Anyway, cool post. For more lua, there's openresty (nginx and lua), and
kyototycoon supports lua scripts as well.

------
espeed
Apache Geode is a distributed, in-memory database
([http://geode.incubator.apache.org/](http://geode.incubator.apache.org/)).

See
[https://news.ycombinator.com/item?id=10596859](https://news.ycombinator.com/item?id=10596859)

~~~
rvncerr
I will try it. Thank you!

------
aristus
The hardware setup is not clear. Are you running YCSB client and the database
on the same single A3 instance?

~~~
rvncerr
No, I ran two instances close to each other (in one DC).

For example, Tarantool and CouchBase were installed on nosql-1 instance and
YCSB client on nosql-2 (There are links to *.vhd files in the article).

------
daschl1
Sometimes it is depressing how little effort people put in to run a level
playing field when doing numbers like this. The couchbase SDK used is from Mid
2013, with little effort the numbers could've been better..

~~~
rvncerr
I've used my simple Memcached module, not Couchbase SDK.

------
diziet
A couple of notes on usability: Having the labels of the charts (i.e., the
color to db matching) be defined only in one place (above the graphs) makes
comparing the charts in the article really difficult to read and interpret.

------
aps-sids
I'm quite surprised to see that there's no Aerospike in this comparison.

~~~
karmakaze
Similarly, I was hoping to find VoltDB in the mix.

~~~
karmakaze
Why downvote? How is VoltDB not relevant here?

~~~
drdaeman
No idea (I haven't voted here at all), but maybe someone didn't want to make
this a long thread naming myriad of NoSQL databases out there?

------
cihangirb
Hi folks, was this a test over a single machine? Also anyone know why the test
used an unofficial version of couchbase (4047)?

~~~
rvncerr
Yes, it is. When I was running the test, CouchBase 4.x hasn't been released.

