
Redis Turns 10 – How it started with a single post on Hacker News - mrburton
https://redislabs.com/blog/redis-turns-10/
======
davidw
One of the funny things is that I've still never really used Redis directly,
only as a sort of opaque part of other things.

I posted that link because I've known antirez for a long time, worked with him
at Linuxcare, and when he said he was working on something cool - I believed
him!

~~~
turdnagel
I thought it was funny that the author of OP said that "...one suggested
Salvatore re-write Redis in Erlang" and it turned out to be you!

~~~
davidw
It was more of an excuse to get him to play with Erlang some... I always
thought he might find it fun to work with.

~~~
vorticalbox
Did he ever try it?

~~~
davidw
Not too seriously afaik.

------
EugeneOZ
Congratulations! Fast and stable DB.

In 2011 I wrote PHP client for Redis with features like tags, safe locks,
map/reduce. It was heavily tested and users had no issues - and since there
was no issues, it was removed from the list of clients after few years as
"abandoned". Nobody even tried to contact me :) Now I don't use PHP and barely
use Redis so it's all in past. But that experience taught me a lot of new
things.

Salvatore Sanfilippo, you are great programmer. Redis API is elegant, safe and
effective. I personally had 0 data losses all years I use Redis. And this DB
is always extremely fast.

~~~
arendtio
That is the irony of software development. In theory, good software could last
forever without any maintenance. In practice though, nothing has to be
replaced as often as software ;-)

What does that teach us about our software development practices?

~~~
jaabe
I think developers need to learn to build to last. We have medical software
that hasn’t been updated for two decades that runs just fine.

That’s often unobtainable with modern software development because we rely so
much on things that change too often, but it doesn’t have to be that way.

It’s a paradigm shift of course, but I think our business really needs to take
maintainability more seriously than it does. This goes for proprietary
software as much as Open Source, but with Open Source there is the added layer
of governance.

I work in the public sector and we operate quite a few Open Source projects
with co-governance shared between different muniplacities, and the biggest
cost is keeping up with updates to “includes”. It’s so expensive that our
strategy has become to actively avoid tech stacks that change too often.

~~~
matlock
While I don't disagree that we shouldn't strive for maintainbility, things
like medical software, airplane software or similar highly tested mission
critical pieces are specifically built to last for that long. Nobody is going
to pay us to build a WebShop to last for 20 years, thats just not a necessity
when getting it out quick is so much more important from a business
perspective than making it last forever.

> That’s often unobtainable with modern software development because we rely
> so much on things that change too often, but it doesn’t have to be that way

The reason we rely on things that change often is because we want to leverage
them to get products out faster. Many different layers of that (as every tech
stack is essentially a product by someone) and we have lots of updates to deal
with. The flipside of slow moving projects is bugs might not be fixed or new
helpful features might not be coming in, meaning you have to build it
yourself.

As a community we know and have known how to build mission critical software
for decades, but we actively often decide not to do it because it isn't that
important compared to other factors.

~~~
mikehollinger
So interestingly - the web shops you’re talking about do want to maintain
their client data, and do expect it to be available “forever,” somewhere. The
payment processors absolutely do at a minimum. Some of those layers are highly
hardened.

While the particular Etsy clone or t-shirt of the day, or customized shower
curtain site will certainly come and go, it’d be an entirely different problem
if visa, PayPal, stripe, swipe, or whatever payment processor packed it up and
went home at random.

~~~
phkahler
We need the foundations/infrastructure to be built to last. People need to
identify which kind of software they're making and treat the infrastructure as
unchanging. Changes in the basement need to be carefully considered with a
default stance of rejecting them unless justified by reasoning that has a time
horizon of many years.

------
azhenley
The original HN post only has 23 votes and 11 comments! Good reminder that HN
doesn't have to be crazy about your project for it to become successful.

[https://news.ycombinator.com/item?id=494649](https://news.ycombinator.com/item?id=494649)

~~~
bilifuduo
Reminds me of the time when Bitcoin was first posted to Hacker News and only
got 5 upvotes and 3 comments:
[https://news.ycombinator.com/item?id=599852](https://news.ycombinator.com/item?id=599852)

~~~
JrProgrammer
It's amazing how those three comments still reflect the main opinions about
crypto

~~~
anoncake
"This is an absurd waste of energy" is missing.

~~~
exolymph
It's not a waste of energy according to the miners who shell out for that
energy.

------
michaelangerman
For me, Redis is the most software valuable tool I use. I was hooked from the
day I started using it about 8 years ago. It somehow has a place in almost
every software project I work on. And it just keeps getting better. Now with
RESP3 on the horizon, it will enable me to even further benefit from the many
clients available in almost every programming language.

[https://news.ycombinator.com/item?id=494649](https://news.ycombinator.com/item?id=494649)

Here is the link from Feb 25, 2009

Note this number = 494649

Today = 19 247493

Nineteen (19) Million Posts etc later (mas o menos)

~~~
edpichler
Is it worthwhile to replace a MySQL with Redis? Am I losing something?

~~~
michaelangerman
I believe if you think of Redis as a data structure server, then there are
lots of applications you can build out that have simple commands like set and
get for both lists and hashmaps that are slightly different ways of thinking
about saving and retrieving your data. The concept of a set is very powerful
and easier to implement I believe than using SQL.

~~~
manigandham
Relational databases are based on relational algebra which is all about
operating on sets.

[https://en.wikipedia.org/wiki/Relational_algebra](https://en.wikipedia.org/wiki/Relational_algebra)

------
dvirsky
I can truly say that Redis has changed my life and career. I started using it
in 2010, got hooked immediately, and I've been a member of the community and
contributor ever since. I got to build so many cool things with Redis over the
years, it's amazing how versatile and powerful it is.

Having worked for a couple of years at Redis Labs, I got to work closely with
antirez, and that's been a transformative experience as well, which made me a
better engineer and open source contributor.

Thank you, Salvatore. Here's to the next 10 years.

~~~
badloginagain
And to that Erland rewrite!

------
atombender
What's the clustering story these days? Last I checked, Redis Cluster still
had lots of issues losing writes, and the design doesn't seem to have been
revised since Aphyr ran Jepsen tests on it back in 2013, making it (at least
from my perspective) practically useless for anything that requires
distributed consistency, which means most things that I look to use a
distributed data store for.

I've also been warned by fellow devs that much of the clustering logic is
actually done in the client, and there's historically been a lack of mature
clients for all languages. Even if you find a mature client, the complexity of
the implementation implies that not all clients may behave identically, or may
have different bugs. Then there's the issue of lock-in, where you become
dependent on a specific client library and its development lifecycle. I don't
know if all of this is true, but I also don't hear a lot of people talking
about Redis Cluster these days.

I know you can use Redis in master/slave mode via Sentinel + Twemproxy, though
even this solution seems to have some issues with data consistency. Running
all three also appears a lot more complex than an integrated system.

I see a lot of comments implying that Redis is mainly used in single-node
setups, so that might be where it shines?

~~~
antirez
Hello! Redis Cluster has yet the same tradeoffs, because they were basically
designed, and were not shortcomings. Btw Aphyr never tested Redis Cluster, and
does not made any sense to test it because he tests things for strong
guarantees so it should test Redis-CP instead, for instance, a consistent
store for Redis implemented into a module using Raft.

I believe that for the Redis use case the cluster tradeoffs make sense
because:

* Best effort consistency in the practice work quite well, even if it does not have any _guarantee_ , but does certain things to avoid losing writes in trivial ways.

* If you want to cluster Redis, you want Redis, not a cluster that automatically becomes a lot more slower, memory hungry, and so forth. So replication should be asynchronous.

However what I may change in the future, and there are plans for that, is to
add a failover strategy that does not just pick the replica that is more ahead
in terms of received writes: but even stop the failover if there isn't a
majority of slaves reachable. This improves certain properties, and if well
orchestrated can also show strong properties if writes are acknowledged only
after transferred to the majority of slaves.

Redis Cluster is used in many organizations right now. The next step is to
improve it (in different ways than having strong properties mostly), and
provide an official proxy for it.

~~~
atombender
Where can I find information about "Redis-CP"? Google isn't giving me
anything.

I don't really understand the nature of the "tradeoffs" you mention. As Aphyr
pointed out, Redis (with Sentinel) is not safe to use as a database, a queue
or even as a lock service. That really narrows the possible use cases. I can
absolutely see Redis being appropriate for many "lossy" applications: Caching,
web sessions, rate-limiting counters, precomputed analytics data, intermediate
outputs from distributed data processing pipelines, that kind of thing.

But the use cases where I'd reach for Redis seem a lot fewer than with data
stores that have high consistency guarantees, such as FoundationDB, TiDB/TiKV,
CockroachDB, or Cassandra/ScyllaDB. With the exception of TiDB, these are a
bit easier to reason about since there's no Redis/Sentinel/Twemproxy split.

On the other hand, I certainly appreciate the specialized data structures and
Lua support that Redis comes with.

~~~
Thaxll
Redis is safe to use as a locking mechanism, many people do that. You should
dig deeper than Asphyr said it doesn't work.

~~~
atombender
How do you do that in a distributed cluster with HA failover using Sentinel,
considering that Sentinel is susceptible to partitions and drops?

~~~
diminoten
Do you work for NASA, or are you overly paranoid about your infrastructure?

------
rkeene2
The first version of Redis:
[https://gist.github.com/antirez/6ca04dd191bdb82aad9fb241013e...](https://gist.github.com/antirez/6ca04dd191bdb82aad9fb241013e88a8)

------
hashhar
Redis is the best data store I've had the fortune to work with. No nonsense
data structures coupled with extremely reliable performance, beautiful API and
extremely easy to manage.

It also helped me earn a lot of praise for using redis-cli --pipe to bulk
insert data which bringed down a data import job to a few minutes from 4
hours. I eventually built a wrapper around redis-cli --pipe for using it with
a cluster.

~~~
techie128
You clearly haven't used it at scale with sorted sets.

~~~
lessclue
Can you elaborate on that please?

~~~
techie128
Try inserting more that 30K elements in a key which is a sorted set and watch
the insertion time, memory & cpu usage. Now try doing this to millions of keys
simultaneously.

~~~
darkr
We run several million sorted sets, but they are all short sets (100s), but do
thousands of writes/sorts per second without issue.

From memory - there was a setting to turn on/off gzip compression for a list
once it went beyond a certain size - do you have this enabled?

------
phowon
As someone who is fairly tech literate but not familiar with this tech stack -
in practical terms, what is Redis, and what is it used for?

~~~
phamilton
A lot of people are saying "k/v store", but I think there's an alternate
definition which I've stolen from aphyr: A shared heap.

If you have a cluster of machines operating on a dataset, you can store that
dataset in redis to get high performance reads and writes. In the simple case
of a cache, it's a key value store. But other complex cases exist: A priority
queue, an atomic transaction log, a lock server, and more.

It supports lua so if the data structure and operations you need doesn't exist
you can generally build it yourself.

~~~
phowon
What is it generally used for though? To me, a "k.v" store sounds like a very
generic but nice thing to have, but I still don't have a good sense of what it
is and what people think of it.

The one place I've run into it is in web development where it's used for
caching? In some tutorials I've read.

~~~
kopos
So lets consider one use case. We have a website where people login and we
create a session to track that user across different pages on the website. Now
the session will have data like user name (to display on the website), the
user's location or IP address, the user's preferences, etc. So whilst the
browser and the website communicate with a cookie (to identify this session),
we need to store this session data somewhere. You can use a file or a database
to read and write to this session data which works fine. But redis shines in
this example where the speed is very good and you also have the session id as
they 'key' and the session data as the 'value' that you can store.

Redis gives you amazing speed and because it provides a kv interface, the work
is very easy.

~~~
degenerate
Thanks, I never heard a practical example like this. I assume the "value" side
of Key:Value can be a giant heap of JSON? So theoretically you can retrieve an
entire collection of structured data as long as you know the key, correct?

And where the usefulness of K:V store is beaten by SQL is the point you would
want to run a "JOIN" or "GROUP" on the data, for example if you wanted to
count the number of keys containing a certain data point, correct?

~~~
kopos
Yes the big bag of data which is generally the value in the kv store is a json
(either as a serialized string or a map if it is natively supported by redis).

Yes, the usefulness of SQL always is the join or group but with something like
session, the idea is to just use it to dump values into the store which you
don't want to reach into the database every time. So different teams will want
to put in their own keys and data into the session object which means your DB
session store becomes extremely difficult to maintain over time.

On the other hand, the joins and groups based on this can be handled later in
time than in the req-response cycle itself.

------
dimgl
Redis is incredible. I built a matchmaker with it as the sole data store and
it has been a breeze to use. It's so simple to visualize data structures with
it and I just can't see myself using any other in-memory data store. Also
Redis Sentinel is fantastic.

~~~
MuffinFlavored
What does Redis Sentinel do in a nutshell?

~~~
33degrees
Provides high availability for Redis

------
whalesalad
Easily one of the most versatile and reliable pieces of software I’ve ever had
the pleasure of using. To ten more years!

~~~
incomplete
especially for bitcoin miners and unsecured redis instances!

~~~
whalesalad
I’m curious. Elaborate?

~~~
nickthemagicman
Around 75% of open redis servers are hacked.

[https://www.bleepingcomputer.com/news/security/around-75-per...](https://www.bleepingcomputer.com/news/security/around-75-percent-
of-open-redis-servers-are-infected-with-malware/)

~~~
antirez
Redis safe mode introduced in 3.2 reduced the problem but still folks actively
misconfigure Redis before putting it in a public IP... Now there are ACLs in
Redis 6 that will mitigate this even more, but it's a lost game, because
images are created with installations of Redis that are made completely
accessible on purpose.

~~~
nickthemagicman
Oh not blaming Redis in the slightest. Its one of my all time fav tools.
Things are only secure as ppl configure them.

I don't even want to know how many elasticache Redis servers data are just
unsecured on a public IP because it's so easy to configure that way.

~~~
incomplete
^^^ this exactly.

------
simonswain
I've been using Redis many years. It's served me well. It's simplicity and
vision is a testament to antirez's hard work. Thank you sir.

------
rb808
Has anyone ever had problems with Redis?

I expect it to always work and it always has. I really like it but am worried
I trust it too much now. Please tell me I'm fine to trust single instances!

~~~
Gigablah
Only problem I ever encountered was mainly my fault -- my Redis instance was
used to hack my server (the attacker manipulated Redis data and dumped it to
overwrite /etc/passwd, etc). I was an idiot and hadn't locked down my
installation. Luckily my provider had disk snapshots.

~~~
tpetry
Sounds interesting. Can you share how and what happened in detail?

~~~
Gigablah
There's actually a writeup of this technique on the Redis blog:
[http://antirez.com/news/96](http://antirez.com/news/96)

In my case they overwrote ~/.ssh/authorized_keys, /etc/group and /etc/passwd
as well.

------
dstroot
I was first exposed to Redis when I first started building NodeJS sites using
ExpressJS many years ago. There was a session caching layer that used Redis
that “just worked” and was fast. Later I had other opportunities to use Redis
and the more I used it the more my appreciation grew. However I echo some of
the well put discussions in this thread about Redis clustering. Thank you
Antirez!

------
kazinator
I like antirez's "Linenoise" library. I converted it to Unicode, and gave it a
multi-line editing mode with undo, visual copy and paste, edit in external
editor, parenthesis matching, incomplete syntax continuation, and more. It's
all in the TXR Lisp REPL.

~~~
antirez
Cool! You should reply here with a link to your fork.

~~~
kazinator
[http://www.kylheku.com/cgit/txr/tree/linenoise](http://www.kylheku.com/cgit/txr/tree/linenoise)

I've tried to keep it decoupled from TXR internals, and to abstract it from
the OS a bit, so there is a considerable "struct lino_os" interface now.

There is a dependency on a "config.h" which provides some HAVE_* constants.

There is a user guide to the REPL in the TXR Lisp manual; that provides
context for some of new interfaces, like what is the _lino_set_atom_cb_
function for.

[http://nongnu.org/txr/txr-manpage.html#N-025AAA27](http://nongnu.org/txr/txr-
manpage.html#N-025AAA27)

~~~
antirez
Wow a lot of interesting work. Thanks for sharing, I think I should compile a
list of the forks at the end of the README.

------
iokevins
Tcl was the first language used. I laughed, but only because I've developed in
Tcl and it has its time and place. Though I don't remember how to do an if
statement in Tcl off the top of my head. Update: ok, its squiggly brackets;
that's right.

~~~
fuball63
I've been using TCL more and more lately. It's ability to use its own
interpreter is super handy. I made a static site generator in it most
recently.

------
yeskia
In classic Hacker News fashion, the top comment is just suggesting you use
something else instead.

~~~
tom_
I wonder what the odds are of any new thing posted on HN staying the course
and turning out to be useful in the long run. Probably not very high.

~~~
alexanderdmitri
About 1 in 57 according to the random statistic generator app someone posted
years ago that I'm still using multiple times a day.

Super stoked the developer finally released the "output as odds" feature
(though my favorite output setting is definitely still "matter-of-fact tone
with a hint of strained patience").

~~~
fyfy18
Statistically speaking, I'm fairly sure it's safe to assume that had up-voted
negative comments too?

------
gadders
Just coming here to say congratulations to Antirez.

I've never had to use the software myself, but I always read Antirez's posts
and his interactions on here. I think it's a great example of technical
leadership.

------
mlthoughts2018
> “It takes guts to be a first follower. You stand out, you brave ridiculing
> yourself. Being a first follower is an underappreciated form of leadership”

There is a Mark Twain quote that reminds me of this:

“In the beginning of a change the patriot is a scarce man, and brave, and
hated and scorned. When his cause succeeds, the timid join him, for then it
costs nothing to be a patriot.”

(edited with the full quote)

------
saberience
Why would I use Redis over Couchbase?

~~~
therealdrag0
I don't know about Couchbase, but Redis tends to be lighter/faster than full-
fledged databases.

------
kawsper
I never realised it was Ezra that did the initial redis-rb implementation.

~~~
jaggederest
He talked about it at Railsconf 2009, as well. I remember that was the first
time I installed it.

------
jbverschoor
It's funny to see how HN has changed. People who are actually excited about a
project, creating a client for ruby within hours. Not the same anymore

~~~
tybit
People are still quick to build and add to new projects. Now it’s more likely
to be something related to Kubernetes or serverless, and those people are seen
as hype chasers not pioneers. Like I’m guessing people were in the past too.

------
nailer
Current Open Source Redis fork:
[https://goodformcode.com](https://goodformcode.com)

~~~
lessclue
Redis was/is open source. The licensing thing was with Redis labs modules
(plugins) for Redis.

------
aretaic
Slightly off-topic, but what makes Redis cluster preferable over something
like etcd as a K/V store?

~~~
antirez
They are at two extreme places of the spectrum basically. One is a very slow
CP KV without any feature over basic stuff. Another is a data structure server
with a KV shell, and weaker consistency guarantees and good performances.

------
tracker1
Doesn't load with ublock origin.

------
SpaceInvader
That dancing guy is rad tho :)

------
melenaos
Who can help me identify redis timeouts in azure mvc core project?

------
mosselman
Why is there JavaScript that prevents me from selecting any text? That is how
I read sometimes (subconsciously mostly). When I disable JS, I can select text
again. What are you trying to accomplish with this?

~~~
aosaigh
I always find replies like this amusing. It seems to be the habit of Hacker
News users to focus on minutiae while either missing or ignoring the focus of
a submission. It's baffling.

~~~
jasoncartwright
People who build websites analysing how websites are built. Pretty natural I'd
say.

~~~
mosselman
I agree. I can image architects going into buildings for various reasons other
than working on that building yet still analysing its architecture.

------
fastbmk
I guess 10 years is not enough to develop _concurrent_ in-memory data
structures storage.

Yeah, let's code like it's 1970's and multi-core CPUs are Sci-Fi ;)

~~~
vkazanov
Having no _threads_ is exactly what makes Redis so convenient in deploying it
to clusters: you have a node, and the node is a single process. It's like a
building block.

Also, Redis source code is so much cleaner than memcached!

