
On Redis, Memcached, Speed, Benchmarks and The Toilet - julien
http://antirez.com/post/redis-memcached-benchmark.html
======
crad
memcached's ASCII protocol is known to be slower and less effective than the
binary protocol and using TCP for memcached is slower than using UDP. It's not
a fair assertion that redis is faster than memcached when you're picking the
slowest way to communicate with memcached.

Also, one should not expect to see parity in get/set ratios or you're doing it
wrong. Cache get with a good hit ratio/high effectiveness is the point of
memcached. While memcached gets effectively plateaued in his low concurrency
example, redis get performance degraded as you added more clients/concurrency.
In an unoptimized use of memcached, high transaction rate environment, I see
sets at 10% of gets.

Redis may very well be faster, but an effective benchmark should be closer to
real world use case and shouldn't hold one party back for parity in protocol
complexity.

~~~
antirez
Crad: I compared the same protocols, both TCP, both ASCII protocol. This is a
fair comparison for sure with TCP, as with UDP you have many tradeoffs, so
this is specifically a comparison of the TCP (reliable) protocol you can use
with Redis.

About the binary protocol: I think this is currently not the widely deployed
one, but the point here is that if the difference is about the protocol, even
Redis could switch to a faster protocol using a binary one. Using almost the
same protocol in both servers we are actually measuring the internals.

Anyway your reasoning can be applied the other way around. Many memcached
systems are currently using CAS extensively while you can perform just one
query with Redis atomic operations in many of the same real world scenarios.
So should I double the Redis numbers by two because there is a faster way to
do something in a Redis specific way?

About SET/GET ratio, I measured both SET and GET performances in an
independent way, so I don't get it.

~~~
crad
I get your reasoning about ASCII/TCP and it makes sense... apples to apples.

When asking for a performance benchmark, specifically about systems that are
very dependent on communication protocol for performance, I think that it's
more valuable to know how the best performing protocol of either side would
perform.

As far as atomic vs CAS, I'd be more interested in the "right way" than the
same way, so for my own interest, yes.

Re: Ratio, from your graph it appeared they were co-mingled. If each was done
on their own then my point is moot.

------
weixiyen
I'm Redis using it for:

\- session management

\- message queue (previously using 0mq pub sub)

\- memory cache (LRU argument = weak)

That's previously 3 technologies I had to manage, now only having to install
1. That means only having to worry about 1 driver, and only dealing with 1
API.

Less technologies in the stack is a good thing, esp if you get a performance
boost as well.

The current bottleneck is of course an easy way to scale, and 2.2 comes with
Redis Cluster.

The other problem some have indicated is the LRU implementation, which imo is
a non-issue in almost all use cases.

~~~
robhu
Are you using it as a LRU cache? I'd really like to use Redis for this -- are
there any examples out there of how to do it / could you share how you've done
it?

~~~
weixiyen
set max-memory inside the config, it'll automatically do it for you as long as
you are using expire on the keys.

The implementation according to antirez is that they sample 3 random keys, and
expire the soonest one, so put your high priority data with a longer expire
and you'll be fine.

~~~
robhu
Thank you ^_^

------
njharman
Really don't buy any of OA's weak arguments for speed over scalability. But I
do appreciate people taking shots at assumptions. Like mine that "memcached is
the fastest."

~~~
antirez
Not _over_ , we need systems that are both fast and scalable. But I suspect
than in five years since now most people will consider a much more reasonable
thing to shard client side, by hand, and in the most uncool ways a system that
can deliver 100k requests/second than an automatically sharding system with
10k requests/second. My guess is that currently the queries/watt figure is not
taken seriously enough.

With Redis we will shortly start working full time at "Redis cluster" so it's
not that we don't care :) Just I think the first goal should be to have a very
efficient "single" node, and then going the step forward trying to retain the
same speed if possible.

------
pvg
Ooof, antirez, get someone to help you proofread these things because in some
places they nearly make no sense.

"Stressing scalability so much put somewhat in shadow what in the 80's was one
of the most interesting meters of software coolness [...]"

"Unfortunately I also think that our dude is part of the problem, since
running crappy benchmarks is not going to help our pop culture."

"Apple to Apple, with the same mouth"

~~~
antirez
Sorry I'm trying to improve my English but progresses are really logarithmic
at this point...

Well also the fact that I try to write this blog posts in very little time (it
took me two hours at max, probably even a mother tongue should alloc more time
to end with an high quality blog post) does not help either.

~~~
nkurz
Your English is great. I'm a native speaker, and reading your post, I thought
you were too. A slightly sloppy one with a couple odd turns of phrase, but
perfectly clear and comprehensible. I understood the quoted parts just fine.

That said, please don't make excuses about how little time you spent on it. It
might be true, but it shows lack of respect for your audience. Like code, if
it's worth writing at all, it's probably worth writing well.

Keep writing, keep coding!

~~~
antirez
Thank you nkurz, sorry about the excuses about little time, I did not
argumented it enough: the fact is that I really _love_ to share what I'm doing
using blog posts, so I tend to work hard to achieve something that's worth
sharing, but then I'm so excited about sharing it that I write the post as
fast as possible!

I should try to resist some more hours in order to obtain a post that is more
consistent and easy to follow... instead I tend to push the "publish" button
and even when the post is already live I continue fixing it... Thanks for the
comment!

