

Simple Twitter clone using only the Redis key-value store as db and PHP - paraschopra
http://code.google.com/p/redis/wiki/TwitterAlikeExample

======
pierrefar
Is it true that Redis requires that the whole dataset fit in RAM?

I've seen that mentioned a few times and if so, it's a serious
consideration/fact to know about when deploying apps.

~~~
antirez
Absolutely true. This is why Redis can do things that with an on-disk storage
engine are almost impossible. This our reasoning: in order to really scale,
with every kind of DB, if data access is evenly distributed like happens in
many web 2.0 apps, the DB must fit in memory anyway (being this OS disk cache,
or directly used memory like in Redis). What really matters is _persistence_
and _replication_. Asynchronous disk dumps gives us all this: data is
persistent, replication is fast and non blocking and trivial to setup. And
because everything is in RAM we are free to do high level stuff without to
care about how data is stored on disk. On disk we have just a "stupid" dump
that we reload a startup.

In some way this is what Paul Graham described in some comment years ago: take
all the data in memory in your application, and just write a file in append-
mode to reload it later. Basically this is an ad hoc, informally-specified,
bug-ridden, slow implementation of half of Redis :)

~~~
spoiledtechie
Ya but what happens when Solid state hard drives come into play. And they will
be soon.

I, a asp.net guy will be able to compete with Redis then.

~~~
antirez
If RAM gets so cheap that Solid state hard drivers will come into play you'll
have your Linux box with 512 GB of Ram by default. Btw I'm happy with ASP guys
competing with Redis :) Also if someone stole what's good about Redis and
creates FooBaredDB that is just better I'll drop Redis and use FooBaredDB.

------
paraschopra
Redis actually looks pretty good alternative to traditional RDBMS. This
tutorial really shattered quite a few myths about key-value databases' real
world application. Anybody having experience with Redis? Anybody knows
practical implementations of key-value dbs on a large scale?

~~~
antirez
Hello, there are guys using Redis in production with 350 million keys for
server (64 GB ram for server). Also there are several other startups using
Redis in production. Of course I'm using it in production myself.

If you search 'redis' in search.twitter.com you can see a lot of guys hacking
with Redis in paywork environments. We are ready to release Redis 1.0 stable,
it'a a matter of one month now. Then we'll have two development trees, one
stable and one instable.

To deliver a rock solid product is one of my top commitment with Redis.

~~~
pipagiorgos
Question: (newbie here) if we assume that with 350 million keys for the 64GB
ram server we have reached the peak of it's capacity, if we double the number
of keys, do we have to get new hardware, like a second server or double the
ram?

Or is it something simpler? Like storing the current keys on disk (somehow)
and keep going with the server as it is.

To rephrase my question: keeping all in ram... is "all" like ALL ??? Would
this solution work for the "real" twitter, with the billion twits, if 350
million keys were created every hour?

~~~
antirez
If you have more data than a single server can hold the best thing to do is
hashing. For instance the Redis-rb client supports consistent hashing.

yes "ALL" in Ram :) this will work for twitter, facebook, everything: what
they are doing is to use only the RAM even if they have MySQL, with tons of
memcached around. With Redis of course the memcached layer goes away.

~~~
jjs
> If you have more data than a single server can hold the best thing to do is
> hashing. For instance the Redis-rb client supports consistent hashing.

Hopefully with a secret salt! It would be interesting if someone nasty managed
to take down a node by e.g. predictively registering certain usernames.

