There are plenty of problems with BDFL-style projects, but I think there are a lot of advantages too. Redis is unique in my experience in that it works the way you'd expect - it doesn't cause outages, it is fast, and it has a vanishingly small number of gotchas. The feature set is well curated and for the most part fits together cohesively.
The most important thing Antirez did, in my opinion, was to say "No" to things. No to new features that didn't make sense. No to PRs that didn't fit the vision. Saying no is one of the most important jobs of a project maintainer. It's a thankless task that upsets a lot of people. But it's critical for a project to stay successful and achieve the leader's vision.
Maybe I'm a pessimist, but I predict after a few years of this new model we'll see weird features, more stability issues, and performance regressions as more cooks enter the kitchen. Time will tell.
Redis has had tremendous mission creep over the years. It started of course mostly as a volatile cache but I've now seen it also used as a message bus (in three different ways: BLPOP, PUB/SUB and Streams), for service discovery and as a general purpose database - something that Redis Labs (in my opinion: wrongly) encourages.
Memcache existed in 2009 when Redis was first released, also as a volatile cache...and still is just a volatile cache. Memcache is what "saying no" looks like. Redis is what "saying yes" looks like - and there are a lot of gotchas.
Redis has stayed the same, architecturally, from the beginning: it’s a keyspace where the values are arbitrary in-memory objects owned by their keys, and where commands resolve to synchronous function-calls against the objects (or ref-cells) at those keys.
Anything that can be done without changing that architecture, is in-scope for Redis. (Though if a data structure doesn’t have wide use outside of a domain, it’s best left to a module.) Anything that cannot be done without changing the architecture, won’t be done.
Much of the “fun” I’ve personally had in watching Redis evolve, has been seeing how Antirez has managed to solve the puzzles of getting features that intuitively wouldn’t be a good fit for this architecture, to fit into it anyway: changing technologies that are implemented one way everywhere else, into something else for Redis, that still work, are still performant, and still solve isomorphic use-cases—if not with the same steps you’d use to solve them in other systems. (E.g. using Streams vs. using a regular MQ; using Redis Cluster vs. using regular RDBMS replication; etc.)
I personally have not enjoyed all the strange and wonderful ways people have found to use Redis. Most of them are pretty fragile and (most dangerously) they often put all Redis uses in the same instance...with eviction turned on.
Maybe that's what Cal and his colleagues were using it for at the time, but the claim that it started as a volatile cache is just not true. Interesting datastructures and the fork based persistence model were there from the start.
Lists, hashes, sets and such that you have available in your programming language, but available as a networked server so any language/app/process can access the same data using the same useful structures.
But... things that are also just useful often and for what you'd need to spin off a separate service for, or make suboptimally in whatever (No)SQL you use to store your data.
So it is perfect if you just want to prototype whether given approach is viable, and "good enough" for many apps.
It works excellent as a message bus, though. And you can add HSET/HDEL to your list as the fourth way.
Lots of reasons why but here is just one because I'm short on time: all of these (four) ways of doing a message bus with Redis offer either a work queue mode or publish/subscribe - but not both (perhaps excepting streams which I've forgotten some of the details of). I have yet to see many real world messaging scenarios where you don't end up using both.
A "proper" message bus needs to have topics, exchanges and queues - like Rabbit. 50% of that functionality is usually not 50% of the value but 10% or less (especially when you consider all the other things about Redis you need to think about - like how eviction is configured).
This is a bit typical of Redis: for any X, you can normally do X in Redis, but is it really a good idea to do X in Redis? IMO, usually no.
Redis Streams actually does exactly what you describe and is basically a single-node fast Kafka replacement with multiple independent consumer groups, per-message acknowledgements, and queue semantics. We use it successfully in many high-load scenarios.
BDFL governance is extremely underrated in the opensource world. People attribute the success of the Linux kernel to its open contribution style, but I will argue that Linux is successful because there's a dictator at the top that enforces a direction, a long term goal, and most importantly, says NO.
Community based governance is a direct cause of that jwz's Cascade of Attention-Deficit problem one can find all over the open source world, especially Linux desktop related.
We don't have enough data points to determine if what you are saying is correct. I have also seen lots of BDFL projects smoke out.
Is the lesson that actual dictatorships only survive long term if 1) the dictator is a nice person and 2) they eventually get managed by an open democratic process?
I think you are actually describing the two clocks problem. Lots of BDFL projects have their own Cascade of Attention-Deficit issues, the project only goes in the direction the BDFL actually cares about.
There is no Steve Jobs in a democratic committee.
> To make a user-friendly system, the system must have conceptual integrity, which can only be achieved by separating architecture from implementation. A single chief architect (or a small number of architects), acting on the user's behalf, decides what goes in the system and what stays out. The architect or team of architects should develop an idea of what the system should do and make sure that this vision is understood by the rest of the team. A novel idea by someone may not be included if it does not fit seamlessly with the overall system design. In fact, to ensure a user-friendly system, a system may deliberately provide fewer features than it is capable of. The point being, if a system is too complicated to use, many features will go unused because no one has time to learn them.
* (This is quoted from the Wikipedia page as I don't have an online copy of MMM to hand).
Heck at one of Postgres conferences there was a shirt that said something like "Tom Lane reviewed my patch and all I got was this stupid shirt."
While he was not a BDFL style person, in the past, if he didn't like a change, it didn't make it in.
Keeping both of those balls in the air at once is why being a software coordinator like this is such an exhausting job.
The structure of a committee project doesn't make it better or worse than BDFL. It just makes it more average. Any greatness present is toned down. And substandardness is brought up.
This also main reason the "led by greatness" BDFL projects are better than the committee run ones and the "led by less than great" BDFL projects "smoke out". The committee's tend towards the middle of the bell curve, BDFL's are all over the place.
No, but it doesn't have as much success as Linux either
Democracy is just the least worse political system.
IRL you own a plot of land, or a home. Or you have a small factory, and there are a lot of network dependencies, who does you sell the supplies, where you sell your products. It the local BDFL get's mad it is very difficult to move. (Specially if there are some cash flow controls were you can't get out with the money that you get selling your home or that you have saved.)
Let's say you have a fix for some driver used on ARM. You'll submit it via Phabricator to get ARM maintainers to review it. If they don't, because eg they don't have time, then you'll probably commit it anyway, but if someone points out you did something wrong, you'll be expected to fix it.
Imagine an author has gotten started writing volumes 1 and 2, and those volumes became popular — maybe the Harry Potter books as an example. Now, adding more authors with equal "story-line decision making power", or a committee, for volumes 3, 4, 5, is a weird idea, right.
Linux is made because of many people all over the world, but it is successful because of the BDFL at the top. A consistent sense of mission absolutely benefits a project in the long term. There just isn't anyone that's going to get your idea better than you, when you're doing something that you know enough about to seriously get started on.
In essence, Antirez has protected the core of Redis from needing to bloat, by making it possible for anyone who wants to build on top of Redis to literally do so—build on top, rather than inside. As such, I expect future PRs to Redis Core to look much like current PRs to Redis Core: just fixing bugs, adding stability, and resolving infrastructure-level interoperability concerns.
As for the future of Redis I choose to be more optimistic. I think the Redis philosophy (thanks to Antirez) has become ingrained. The benefits of a minimal feature set are hopefully well-established.
In the very least I suspect this will last much longer than many expect before the bureaucrats take over.
(This is also referenced and linked in the article.)
This is hard. It makes you feel bad inside to tell people "No", especially when they open PR and clearly did a lot of work adding some feature.
Reminds me of Brian Goetz: "We have to say no to almost anything. If we didn't, things would pretty quickly degenerate to the point where the system collapses on its own weight (...) if we did this, which of these eighteen possible features might we foreclose on, and might we want to leave those possibilities open, at the cost of saying no to this thing here." (https://www.infoq.com/presentations/java-history-present-fut..., 24m+)
Redis has succeeded because of antirez. I really wish it can continue doing so for the long run without him. More than anything it reminds me of Steve Jobs' death (although gladly this is in much happier circumstances and I wish antirez many more decades of hacking). It's true that a decade later Apple is still doing great, but you just know things would have been different if he was still around.
It seems that this model is not so common? At least the list on Wikipedia (https://en.wikipedia.org/wiki/Benevolent_dictator_for_life) only lists Django as an example with 2 BDFLs.
I was active in a smaller open source project (http://www.openlierox.net) where it mostly was like that, i.e. we were 3 main developers, and all major design questions were discussed among us. But this example is probably not so representative, as there rarely were further contributions by other people. But anyway, I was very happy with this model, that there is not a single leader, but 3 people.
And a project that starts with just 1 BDFL, and works fine & builds sth people "love", it'll continue working fine with that 1 BDFL.
If, however, replacing one of those 3, or adding another BDLF to a one person project — I'd think that by default, that won't work.
... Unless the new person has worked next to the earlier BDFLs for years, so the previous BDFLs can be ok certain s/he is "the right one"?
Whatever works fine, works fine — the risky thing, is introducing a change (a new person)?
Is forking no longer a solution to this problem? Or are you just wanting to wrestle control away from Linus?
Without the legal obligation to upstream, the majority wins what happens afterwards.
Why is that a problem?
Erhmmm, why does his age or colour matter in this context? Debate can happen about people and their leadership style, (I'm not a fan of Torvalds) but when the first criteria someone sees for criticizing another person is their age, colour, or race, that seems a bit shallow and biased.
> Racism: prejudice, discrimination, or antagonism directed against a person or people on the basis of their membership of a particular racial or ethnic group
I was then [regrettably] tempted to write something with a "mic drop" feel, just to express my disagreement in an acceptably pithy way.
And then it occurred to me that this whole sub-thread is Twitter-length comments. And I was about to add one more. And I realized that maybe the moderation of HN actually prevents us from going as deeply on this topic as we do on everything else, and so that moderator preference has essentially reproduced a pithy and unproductive Twitter dynamic for topics like this.
Pithy and shallow self-censored comments in reply to pithy self-censored comments.
EDIT: But also, moderation is really hard <3 Just wanted to try to put words to a small dynamic that's perhaps playing out here.
I'm not aware of any moderator preference against talking about this subject. Again: huge thread on it yesterday that persisted on the front page all day.
However, I don't think every single off-hand mention of race or gender needs to devolve into a subthread of 50 multi-paragraph comments. So I appreciate your decision to avoid it in this instance.
Yeah, I was a moderator on one of the first university-run online forums, and it was so stressful and difficult...! All the first-year students just spoke off-the-cuff, but "monitoring" all that speech was SO challenging...!
There's no such preference. What made you think there was?
The question of political topics on HN is a hard one because some political stories are on topic, some are off topic, and even the on-topic ones tend to turn into flamewars. I've written a lot about how we approach this: https://hn.algolia.com/?dateRange=all&page=0&prefix=false&qu.... Some good threads to start with might be https://news.ycombinator.com/item?id=21607844 and https://news.ycombinator.com/item?id=22902490. Also https://news.ycombinator.com/item?id=17014869, which shows how far back political discussion goes on HN, as well as the argument about politics on HN.
If anyone reads those explanations and still has a question that isn't answered there, I'd like to know what it is.
> What made you think there was?
I recalled a post way back when trump was voted in, asking the community not to speak about politics. (I'm sure I'm misrecalling the specifics, but that's the fuzzy take-home that's nestled in my mind after all these years.)
Ah yes, found it. Apparently it was just one week, but I remembered it as a standing request :) https://news.ycombinator.com/item?id=13108404
And redefining a word so is "bad for everyone but X" doesn't make X innocent of doing it, it makes X guilty and a bully.
Also, the world is much bigger than the US.
Also, congratulations to moving off of Redis. In open source, there is a thing where people look at retiring from a project as some sort of failure of either the project or the maintainer, but I think the opposite is true.
We all have a finite amount of time on Earth, and everything has its natural beginning and end. Moving on to something else means creating the opportunity for yourself to find the next big project for you. And for the project itself, it means bringing in new eyes and perspectives.
Change is scary, but healthy.
I normally don't write about what I come across during my work, but in aggregate I can tell you that Redis, Linux and MySQL are the most common recurring elements across 150+ jobs looking at different companies, and using it rarely if ever leads to trouble.
So even if I don't use it myself directly quite a few of the companies we have invested in do, and an indirect 'thank you so much' is well deserved. I am very curious what it is that you will do next besides blogging. Linus had 'git' as his second major project, arguably it has had just as much effect on the world of free software as Linux did, you've definitely raised the bar for yourself :)
Much good luck!
Saw it multiple times at multiple jobs just break on its own.
Most memorable one was a bug where certain pattern of data caused mysql think data is encrypted, crash and refused to start until data was restored from backup. It took quite time to fix it because it happened randomly and initially we assumed it was broken hardware.
I started using Redis around 2010 and I learned to appreciate so many things from you along the way:
* Data structures are fun.
* In-memory is fast and can be safe!
* Append-only logs are awesome.
* Databases can be more than MySQL.
* Complexity analysis is worth making clear in the documentation.
* Hybrid L1 cache-friendly data structures beat complexity analysis for small data.
* There are only so many hours you can work in a day.
* It's cool to sit by the pool.
* It's cool to have a screen name.
* Above all, code is poetry, not dependencies.
I've been using redis since I think 2.2 and have learned so much from your posts over the years! More than just about redis. Are you able to post links on your site to the videos you're making? I don't speak Italian but I'd love to learn from subtitles.
I'm really excited for you and look forward to whatever fun things you decide to do next!
Greets from NZ
This entered my list of favorite quotes! For this, if not for your huge contribution to OSS, grazie!
I also want to thank you for making this decision while you were still active, Redis didn't skip a beat. Right up to announcing this you made sure that the community made progress. As we at GitLab detail in https://about.gitlab.com/blog/2020/06/24/scaling-our-use-of-... in April you personally responded to an issue https://github.com/antirez/redis/issues/7071 within an hour.
I glad you are choosing for what makes you happy, you more than earned it.
Best wishes for whatever it is you choose to do next.
Also, good decision, and happy to see that you now can finally take a real vacation if you want.
Also, it is good news for everyone, because it means you will be able to concentrate your energy on new creative pursuits if you want. Which, for example, leads to beneficial things like video tutorials, etc.
It feels like people may still kind of be on your back a little bit but this time they may be asking you what the next big project is. I hope you can shake them off and just follow your passion without needing to make a commitment to something new
(especially not anytime soon). This will be best for everyone in my opinion.
Your work that’s had the most impact on me isn’t Redis but kilo! It taught me how to have fun hacking on C back in university. Me and a couple friends cloned the repo and started adding fun features.
Here’s to more 1000 line whatevers! <3
I first used it as the datastore for a flash sale about 8 years ago now. Even though close to 130k people were trying to book rooms at the same time, it performed just as I hoped it would.
I was going to say that I haven’t used it in a few years, but really I’ve just forgotten that it’s running a critical part of our system day in day out because it never causes me any trouble.
Good luck with your next adventure, here’s hoping it’s as fruitful for the community as your last one.
Good luck exploring your other options
Just wanted to offer you my thanks. I've seen the project a bit from the sidelines over the years out of interest, and I think you should be proud of it.
Hope you will do something that excites you next. My only tip is : don't worry about what other people think is exciting - follow your gut.
Thank you for Redis, and I hope you can relax and have some fun with whatever is next!
I did have a post that basically said "I relate," but it wasn't popular (I guess it came across as self-promoting. My apologies).
In any case, I can relate, and completely support you in your future endeavors. I am not a redis user, but I have seen nothing but good said about it.
> I'll just do whatever every morning I want to do for a long time.
I am glad to read that you made a choice, as difficult as it might have been, and I hope it will make you happy (or happier).
Redis is simple and straightforward and a joy to use. Best of luck :)
Looking forward to reading whatever you do next!
Wishing you the best in whatever you do next.
Staying tuned :)
> However I never wanted to be a software maintainer.
And nothing say that you have to be. There's this perverted view that anytime someone creates a popular FOSS project, they need to dedicate every waking minute to maintaining it. That's neither economically feasible nor psychologically reasonable.
> Redis was the most stressful thing I did in my career, and probably also the most important. I don’t like much what the underground programming world became in recent years, but even if it was not an easy journey, I had the privilege to work and interact with many great individuals.
What is "underground programming world"?
The programming world that nobody else sees.
Everybody uses shiny FOSS tools, and some even make big $ using them, but nobody wonders where they come from and how much pain went into building them.
But I'm unsure.
Dude, the problem is just Twitter. Everybody shouts on Twitter, it's a system that encourages mob reactions. "Back in the day" people had to subscribe to a ML to yell at you, so fewer people did; and we all knew which lists were cesspools of trolls and flamers, making it easier to avoid the bulk of it. Twitter instead gives you the whole firehose, unfiltered and amplified. It's not a problem restricted to opensource or programmers, we see it everywhere at the moment.
I'm glad I'm not alone experiencing this.
One of the core open source projects I use has taken a no comments stance because "the code speaks for itself". It does if you're a core contributor, but if you dive in less frequently it takes a while to grok what's going on and re-build the mental model. Comments would reduce that.
Thanks for all you've done Salvatore, and for your blogging. Your enthusiasm puts the fun back into my programming!
Thanks for explaining.
> Now you can't say anything, if you don't respect a good practice (LOL) people yell at you on Twitter. Even saying that commenting is a good idea is a problem. Not cool.
Just because they're loud does not mean they're correct or that they're the majority. I can understand not wanting to deal with any of it though.
The larger than life you become in your space, they more your words, actions, and non-actions get parsed and twisted for political intent. It's either step away entirely or hide behind the mask of a pseudonymous alt.
They're not. They're the mediocre developers with nothing substantive to contribute. Think of your developer heroes. Almost none of them will ever engage in such nonsense. They're too busy building wonderful things to bother with petty nonsense that gets nothing done.
Thanks for mentioning the T- website specifically, it takes a lot of courage to call it out directly these days, because all it takes is one of them with a lot of followers and then you get swamped with hate. It's fucking unreal man.
There's a ton of us down here programming and having fun, free of politics and bullshit. Look forward to seeing you (again).
But there is DEFINITELY something wrong with the fact that, seemingly, NOBODY wants to be a software maintainer.
There are lots of other people who want to maintain it. Let them, and if they screw it up that is not his fault. He finished his project. Give him a break.
I think everybody wants it on their resume, but the practical day to day work tends to be more filtering poorly written GitHub issues than writing any interesting code. For a job that doesn't pay much (often at all), that's not something I would want to spend my time doing.
If you have to ask... ;)
HN commenters are encouraged to post feedback in the threads about what's good and bad for HN comments.
it's where you program in a basement.
I say this frequently both online and when discussing system design with newer devs, but will repeat here: of all the production issues I've debugged, the culprit has has never been redis. In fact, redis has been a critical piece of achieving cost-effective scaling. It is one of only two pieces of software (along with postgres) that I blindly recommend without any caveats. From following along here and on your blog about how you approach things and think about the software, I think its clear that you and your vision for the project are a large factor of why it has been so reliable.
Thank you antirez!
You'll love it until you don't.
The scaling and clustering story is not nearly as nice as the quick start.
It's definitely worth recommending... with caveats.
A half-ton pickup truck works well until you need to haul bigger loads. At some point, you either need to haul smaller loads or change to a different, bigger truck.
That seems to me an argument that should be pointed at cloud providers rather than Redis itself ;)
This would be a significant down side to using redis at all, except you can get away with not caring if you out source the problem with your credit card.
@sethammons "... sharded to (currently) six clusters ..."
I wonder, when did you start sharding? How many GB memory did your previously single-machine-Redis-instance use, before it was time to shard? How much memory does each "sharded node" have?
(It wasn't the CPU (Redis single threaded) that forced you to shard? But because you needed more memory? (Or sth else?))
> "nearly 200 redis nodes"
How much memory in each node, if I may ask?
Sharding is always something you do on day 1 for every project, because a single-machine-Redis-instance is a SPOF and won't pass even the most basic operational readiness checklist.
> (capacity planning questions)
The answers to these questions don't generalize, they depend on your workload. You should figure out some approximation of your data types and request profiles, and use those to get a rough understanding of what one Redis server on a given machine class is capable of delivering. This usually means applying a few different classes of read/write loads against a small cluster at steadily increasing RPS, and documenting how CPU, memory, and latency characteristics change. From these numbers it's possible to derive a capacity plan.
The last time I did this, for my workload and machine class, one Redis instance (one core) delivered 250-500k RPS, invariant to memory used. We'd use the conservative end of that range, combined with the RPS and data set growth rates we predicted, to provision for ~12 months of growth. Operationally, we would deploy (cores-1) or (cores-2) instances per host (I forget exactly) on 32- and 64-core machines. I think they had like 64-128G of RAM, and we made sure to leave enough memory overhead so the AOF or whatever persistence option wouldn't lock up the box. But even the choices of what class of machines to use is a function of your use case, if you have really large dataset with relatively non-costly (CPU) operations, you want a totally different machine profile than a relatively small dataset with complex operations. Availability SLOs also factor in.
All of this is basic operational stuff, and with Redis the answers are pretty well-understood and highly predictable. Completely opposite to systems like Elasticsearch, which was a total nightmare to predict, provision for, and operate.
Cannot agree with that. If one does that or not, would depend on things like Service Level Agreements (SLA) — and one can have a pretty high SLA uptime %, without sharding, e.g. if there's an underlying pretty stable hosting provider that live migrates if there's a hardware failure.
Thanks for writing about the last time you used Redis. Interesting to hear that, in that case, the machines had sth like 64 – 128 RAM, and 250k – 500k RPS. Yes I agree that I'd need to benchmark and think about what type of machine(s) to use (some time later — a bit too early for that now). Sounds as if you are / were a pretty large company / project, needing that much memory and machines :- )
If you have one instance you better be damn sure that downtime doesn’t cause an an outage. IE. no redis means services still run.
The “it’s fine, the SLA covers outages” is just a) laziness and b) negligence.
You don’t do that with web servers, you don’t do it with databases. You don’t do it with your redis instance either.
...well, I suppose some people will do the it anyway; but you get what you get if you do.
If a major outage gets you fired, you have no o w to blame but yourself.
Sorry, no. A single instance is never acceptable from any risk perspective, no matter what guarantees the hosting provider claims.
Redis is one such tool--its clustering "story" may not be ideal but even a small, minimal cluster will be more than sufficient for any use-case that a distributed memory cache is fit for at a scale that will cover 99% of business' needs.
~90k/sec on a single core of L5520. Scaling Redis is a premature optimization for 99.99% of the use cases. While it may be sexy to talk about millions of ops per second that one's project does in reality the number of projects that have that as a requirement is probably barely in triple digits globally.
In any case, I've had the same experience with only two programs - CouchDB and Redis.
Takes a rare person to say this, made me smile :)
But don't worry Antirez, Redis is far from a bad work of art, quite the contrary.
And I think he's proven himself to be a superior programmer. In that in almost every category of engineering skill and knowledge, he is better than 90% of the people criticizing him. If he didn't start out that way, over the years his knowledge and skills grew to make it so.
But also maybe it reflects a bit that there is always some new "best practice" or something that some months ago no one heard about, but now many people are unfortunately using the adoption of as a proxy for programming skill because they are unwilling or unable to actually judge something or someone on actual merit.
I think he's done a great job with Redis. It goes to show sometimes that you need to let go of the thing you created for the good of the community. If I were in his position, I would have made the same move
I have a huge respect for Redis, and Antirez, but I have to say my respect doesn't include RedisLabs. They're the ones who started a commercial endevour to capitalize on Redis initially without Antirez, and then later offered him a position. They're the ones that Kyle@Jepsen says is making claims about ACID, not Antirez. I always heard that Clustering was a feature Antirez was very reluctant about.
I hope Redis is, fundamentally, taken away from RedisLabs.
Antirez, you say you want to express through code and open source, and I can't square that with your association with RedisLabs. You've been taking their money for quite a few years now to allow them to put up a billboard saying "the home of redis".
I'm going to stop right there. I've cut my own path through the wild jungle that's Open Source, and it's not yours, and I respect the years you put in to make the product with your vision. I'm sure there are some other first coders of foundational databases on this list, but it's a small club.
As an employee of RedisLabs, I don't think you are free to say what you'd like to say, I hope you follow whatever path does allow you to freely express in the future.
Good luck always.
a previous post of mine got 500 points and stay #1 for a few hours.
traffic from it looks like this
First screenshot looks like Google Analytics to me; I'd say it's more likely a lot of us have blocked it and Cloudflare's number is more accurate.