

Redis 2.6 is near - thibaut_barrere
http://antirez.com/post/redis-2.6-is-near.html

======
pork
I'd like to pose a Redis question to the community at large. On HN, I keep
hearing about how awesome it is, how readable the codebase is, how efficient
and well thought out it generally is.

That's all fine, but there is _incredible_ resistance in many enterprises to
adopting new technology. I recently faced a similar problem trying to get
people to move to Redis, and the feeling is that there isn't enough
"enterprise social proof".

So my questions to anyone who has deployed a large enough Redis cluster (and
by cluster, I mean replication, since sharding isn't apparently supported
yet):

\-- What is the largest instance you have run? \-- How reliable has that
instance been? \-- How easy is it to maintain and administer? \-- What are
some issues you would warn other enterprise users about?

~~~
antirez
I can't understand this comment _now_ , after the youporn success story...
Isn't this enough for enterprise people?

Ok seriously now... Groupon did an event a few days ago about how they use
Redis with success, Twitter uses Redis for caching timelines, Github does a
good use of Redis, and similarly do many other non trivial companies, like
Craigslist, Stack Overflow, Digg, ...

Those are not "enterprise" companies, but the problems they solve are not
different, so IMHO the real problem is a cultural barrier. I think that this
is just a matter of time.

~~~
pork
Seriously, neither can I, and you're right that it's a culture problem. I am
genuinely trying to do my part to overcome that. Towards that, it would be
nice to see more in-depth expositions of setups, and above all, an honest
description of any issues and problems. I'm trying, antirez, I'm trying... :)

~~~
spenrose
One tried-and-true solution is to use it on "non-enterprise" projects (small,
invisible, failure-tolerant) within your enterprise, then mention that you've
used it, then use it on something slightly less obscure and mention it again
... worked for Linux in the '90's.

------
ajays
FTA: Redis ASCII art logo added at startup. This is where our major efforts
went in the latest months.

I knew it!

All kidding aside: Redis is an amazing tool. I'm looking forward to the new
version.

~~~
antirez
I did it myself with vim, looking at the logo. It's a long process ;)

~~~
njs12345
Emacs M-x artist-mode :)

------
JulianMorrison
What is the use-case for server side scripting? Does a running script block
readers and writers?

~~~
antirez
Every command blocks every other command in Redis, because Redis is single
threaded. However the implementation of scripting in Redis does a good use of
Lua, so a command implemented in Lua does not run too much slower than a
command implemented in C, for simple tasks (since we use a single Lua
interpreter, and cache compiled functions). So this is an example of what you
can do with Lua scripting:

    
    
        redis 127.0.0.1:6379> set foo '{"b":20,"a":10}'
        OK
    

The user wants to increment the value of "b" with a single query to Redis in
an atomic way:

    
    
        $ cat /tmp/script.lua
        obj = cjson.decode(redis.call("GET",KEYS[1]))
        obj.b = obj.b + 1
        redis.call("SET",KEYS[1],cjson.encode(obj))
        return obj.b
        $ redis-cli --eval /tmp/script.lua foo
        (integer) 21
        $ redis-cli --eval /tmp/script.lua foo
        (integer) 22
        $ redis-cli GET foo
        "{\"a\":10,\"b\":22}"
    

No back and forth from client to server, no locking needed. Basically you can
implement your specialized, atomic commands, in Lua.

~~~
kijin
The example you cited just looks like an awfully convoluted way to recreate
HINCRBY. Obviously there are much more powerful things that you could do with
scripting, but this example makes me wonder if all those H-commands might be
replaced with JSON & scripting (or handled internally by JSON & scripting)
some time in the future.

~~~
antirez
Well it's not really HINCRBY, it increments a field of a JSON object that is
otherwise not supported by Redis natively. It was just an example, but it is
not a strange one, because there are many users with JSON objects, and
similarly there are many users with specific needs: Do this if there is that
in the key, add this element only if the list is < N elements, and so forth.
With Lua scripting you can model all this without back-and-forth without the
client and server, in other words, with 1/100 of the latency (or less).

Given that we'll implement new commands only when they are "fundamental"
operatons. For instance there is some plan to implement list splicing. When
this happens it's worth to write it in C for max performances, so I think
we'll unlikely write parts of Redis in Lua itself. Scripting systems is
something I love (see Emacs for instance) but IMHO Redis is the instance of
system where this does not work: space and time are too important for Redis.

~~~
kijin
Thanks for the clarification. It's great to hear that scripting will reduce
the need to create new commands. I'm sure some people have been getting
slightly annoyed with the rapid proliferation of Redis commands, which already
number well over 100. Can't remember them all!

~~~
xnxn
That reminds me, I'd really like to see an official wiki (or just a simple
pastebin-like thing like the Flask snippets site) for Redis scripts, use
cases, best practices...

I've seen quite a few blog posts that highlight some amazing way to use Redis
and it would be very helpful to have them all in one place.

~~~
catwell
Nothing official but you can fork <https://github.com/mkrecny/redis-extend>
and make a pull request.

There was also <http://evalsha.com/> which looked cool but went nowhere...

------
shin_lao
I'm curious, why did you chose Lua?

~~~
antirez
1) 'make ansi'

2) Very fast for a such a small implementation.

3) C - Lua API makes sense.

4) Any algol-like language is easily picked by a programmer. So Lisp or Tcl
could not be idea, but Lua is fine syntax wise.

What I don't like about Lua (but was not enough to stop me from using it for
Redis):

1) 1-based indexes.

2) ~=, why not != that everybody knows?

3) A unique "float" type for all the numbers is not ideal.

4) A unique "table" type for maps and arrays is not ideal.

5) You want to be small, but C bit operations as native operators inside the
core are a good idea.

~~~
julian37
_You want to be small, but C bit operations as native operators inside the
core are a good idea._

Not being a Lua user, it took me a little while to parse that. You're right,
implementing these as library functions is silly, and having a dozen
implementations to pick from is ludicrous!

<http://lua-users.org/wiki/BitwiseOperators>

EDIT: Ok, less ludicrous now that they're shipping the "bit32" implementation.

~~~
antirez
Yes, my guess is that the "only floats" design resulted into the "no bit ops"
design.

------
i_am_a_human
I am continually amazed by Redis. I am involved in a project where parallel
processing and work distribution is achieved via 0mq and Redis acts as a sort
of "shared memory". The performance is outstanding, and it is a fun
architecture to work in.

~~~
dochtman
We use 0MQ and Redis together at work. We have an incremental flow of updates
that we distribute internally over 0MQ pubsub, and have one listener process
that updates the full state in Redis from each update message. That way, we
have both (a) a way to easily get the latest state (and fast), and (b) an easy
way to listen for updates to the state. It works beautifully, even with a high
rate of updates.

~~~
secoif
Forgive my ignorance, but why 0MQ pubsub and not redis pubsub?

~~~
ihsw
0MQ pubsub can be trivially distributed, whereas redis is currently limited to
master-slave replication (as opposed to master-master).

------
dmmalam
Is it possible to use a script to roll back redis transactions. Currently
transactions aren't like SQL transactions in that they don't roll back if
there is a problem executing the commands between the multi/exec.

Say if I add a new hash, and also increment a counter, and update a set to
track a relation. I currently do this within a multi/exec to ensure it's
atomic. It would be nice to roll back the commands, if say there was an
internal error between the exec/multi or if there was an external event like
losing a disk / bouncing the server by possibly writing to the AOF log.

Out of interest, how do people deal with this now?

------
ajtaylor
I'll be adding redis into our $work stack in the next couple weeks to add
pub/sub (email subscriptions) and a job queue (Resque). The ordered sets will
likely be very useful to help us scale sorting of search results, etc.

From what I've read, it's going to blow my mind and open so many new
possibilities. I can't wait for the clustering in 3.0 to come out!

------
bwarp
Good stuff!

However, another thing I'm not going to get to use whilst working in a
Microsoft monoculture :(

~~~
antirez
apparently Microsoft is working more to the win32 port, and now there is a
github repository: <https://github.com/Microsoft-Interop/redis>

I hope to see some good stuff in the next weeks.

~~~
bwarp
Unless it ships with Visual Studio, I doubt I'll ever see it.

~~~
benologist
You can still use Redis with C#, the only caveat I've found seems to be the
current drivers don't support authentication but that's not a huge issue if
it's not publicly accessible.

<https://github.com/ServiceStack/ServiceStack.Redis>

Edit: for the first time in my life I may be less than correct.

~~~
nknight
Whaaa? Authentication is one simple command, sent upon connecting. Why on
earth wouldn't they "support" that? It's not even a five minute task...

~~~
SoftwareMaven
I know nothing about this (neither redis auth nor MS' work), but if I were
managing the server products, I would want to make sure the redis
authentication doesn't confuse users of my platform. MS has a pretty clear
goal of making everything authenticate via ActiveDirectory. Keeping that
consistent may be more than a five minute task.

~~~
nknight
Well, first of all, he's not talking about a Microsoft-written library, he's
just speaking generically of C#/.NET libraries written by others.

But Redis's authentication is very simple in any case. There's a password. _A_
password, one. That's it. There are no accounts, no usernames, there's no
"infrastructure" or any way to hook it into LDAP or other insanity. There's no
accounting.

It's not _for_ the kind of authentication ActiveDirectory does. In practice,
it's more of an "Oops, did you really mean to connect to _that_ server?"
thing.

------
F_J_H
Would love to start playing with Redis but have limited time to do so. To save
me some time, has anyone found some good tutorials?

