

Mongo Vs Redis, The Increment Battle - devdazed
http://devdazed.com/post/2737860983/mongo-vs-redis-the-increment-battle

======
antirez
I've no clue about MongoDB performances, and which of Redis and MongoDB is
faster incrementing integers, but for sure this benchmark is completely
useless, as it is actually measuring a mix of client library latency and round
trip time, being it single threaded, making no assumptions about the
durability requirements, and so forth.

For sure it seems like that MongoDB client library you are using has a smaller
latency, perhaps it is a C library? As Redis-rb is instead written in Ruby
itself.

Another huge problem of this benchmark is that it should try to increment a
random key per every iteration, out of a dataset with a few millions of keys,
as this is realistically what you need. You'll see how Redis with 50 clients
will increment this counters 100k times per second or more without any
performance degradation as the number of keys grows.

~~~
devdazed
I wouldn't say it's completely useless as it shows that Mongo is faster in a
single threaded process, using the most popular ruby client of each system.

If I were writing a high-concurrency multi-threaded system to handle millions
of keys at a time, Ruby would not be my first choice by far.

~~~
devdazed
Ok, I agree with antriz's statement. My results are very specific. I will make
note and be sure to be very thorough next time around.

~~~
antirez
Thanks for agreeing, however I must point out that since your blog post does
not accept comments, nor you rectified the content, this is misleading
information that will stay on the internet for virtually many years to come.

------
stevelosh
More accurate title:

(Mongo + Mongo Ruby Gem) Vs (Redis + Redis Ruby Gem), The Increment Battle
with No Useful Statistical Analysis Included

------
xal
You could make electricity from the amount of eyerolling this article is going
to generate here on HN.

------
viraptor
I think I heard a distant scream of Zed...
<http://zedshaw.com/essays/programmer_stats.html> He was so right for cases
like this...

~~~
true_religion
I'm a programmer, but not very good at statistics (as in I passed the class
with flying colors, but I can't really comment on new situations).

When it comes to things like benchmarking, I find it easiest to avoid making
absolute conclusions like "X is faster than Y", but to merely present the
data, all the data, and nothing but the data.

In this case, I'd have wanted to see the response times per request, and
perhaps the memory trend for the client. At the very least, I'd like to know
how Mongo and Redis were setup, and if either had begun to swap.

------
godi
Running your benchmark I get 11216 increments per second

Running the redis-benchmark from
(<http://redis.googlecode.com/files/redis-2.0.4.tar.gz>) with 50 parallel
clients 37800 requests per second

I would like to see the numbers for 50 clients to mongo doing the same.

~~~
antirez
interesting but 37800 is low too, either not Linux or is a Virtual Machine :)
That's what I get in my desktop system with Linux (Xeon E5520 2.27 Ghz):

INCR: 126743.98 requests per second

------
thinkbohemian
Can you give the mongo and reddis version numbers?

Just curious if you were using reddis 2.0 or possibly 2.2 ? Also what version
of the reddis client were you using? There were some performance improvements
recently.

Edit: thanks for sharing your experience, even if your test wasn't done with
the help of a nobel winning computer team, it can still be useful data.

~~~
antirez
To know the version may be interesting but not really useful, I bet that if
you change Redis with a dummy TCP server that is only able to reply +OK,
whatever it receives, you'll get the same numbers here.

Basically every kind of networked server with a simple request-response
protocol, in a busy loop over loopback, will show performances between 10k and
20k requests per second, mostly up to the speed of the implementation of the
client.

------
jonahharris
In his test, it appears the Mongo driver is just putting the message on the
wire, not actually waiting on the server reply or verifying that the command
was ever actually processed. Using the Redis in pipelined execution mode would
be a much closer test.

------
198d
My gut feeling tells me Redis should be faster at this, and as everyone else
has pointed out, the methods used to arrive at these conclusions are most
certainly questionable.

------
thomasknowles
Redis is asynchronous, try doing it with a few more connections.

------
mrinterweb
MongoDB is web scale.

~~~
ghotli
I think that meme is funny too, but lets keep meme only posts over on reddit.

~~~
mrinterweb
Sorry. I'll remember in the future that Hacker News is no place for humor.

~~~
nkurz
Thanks! That's a fine executive summary. The longer version is that original
humor is appreciated, particularly if insightful and non-obvious, but
regurgitated memes are not welcome here. If you are rushing to make the
obvious joke before everyone else does, please don't.

~~~
mrinterweb
Seriously. I did not know that the use of a meme is violates site ethics
guidelines. I checked the comment guidelines
(<http://ycombinator.com/newsguidelines.html>) I didn't see any guideline
referring to the use of memes. If using a meme is such an egregious offense as
this comment thread suggests, maybe this violation should be mentioned in the
guidelines.

Generally the popularity of a comment is a good enough indicator of how well a
comment is received. If I got 100 upvotes on that comment I may get the
impression that people like that. Considering I have < 0 on the comment, I can
take the hint. I don't need the proper patrol reprimanding me about a non-
stipulated rule.

------
soveran
Can you post the Redis configuration you are using?

------
smoody
of course none of it matters if the machine/vm crashes before the increments
are committed to disk. :-)

