Hacker News new | comments | ask | show | jobs | submit login
Redis 4.0 (groups.google.com)
588 points by fs111 on July 14, 2017 | hide | past | web | favorite | 164 comments



"9) Active memory defragmentation. Redis is able to defragment the memory while online..."

I'm so amazed that this is a thing.


Memory fragmentation is largely related to the allocator you're using (ie: glibc malloc, jemalloc, tcmalloc) and previously it was up to the OS to manage this (ie: freeing up unused memory).

Now with active memory defragmentation things are a bit more pleasant, specifically with high delete load actually freeing up unused memory in a timely manner without impacting performance too much.

Previously, to fully recover unused memory, you would have to restart the Redis server. Obviously this is not feasible but when Redis is using >50% more memory on a 120GB machine than it should then you will have to consider this an occasional housekeeping option -- now, as mentioned, this ridiculous task is no longer necessary.


>Now with active memory defragmentation things are a bit more pleasant, specifically with high delete load actually freeing up unused memory in a timely manner without impacting performance too much.

Is Redis managing this itself using simple calls to Jemalloc, or is Jemalloc doing it on its own because it has better algorithms than the OS?


With regards to better algorithms than the OS -- jemalloc is not quite there yet.

Here is an illuminating discussion between Redis and Jemalloc devs regarding this:

https://github.com/jemalloc/jemalloc/issues/566

Redis will perform its own housekeeping, hence the usage of the term 'active.' AFAIK, rather than metadata being stored for this, there is a periodic active scan and manual measurement.

See more here:

https://github.com/antirez/redis/pull/3720


120GB on a single thread, yeah. Redis has been abused for quite some time but 120GB is way out of reasonable reach for current CPU arch if you use a single core only (even if you switch off hyper threading)


Call me a slowpoke, but I wasn't aware that Redis was single core!


It makes it great for storing distributed locks, semaphores, and thanks to Lua scripting it can store/update cache invalidation lists - eg https://github.com/Suor/django-cacheops


just randomly curious here - why? hash time gets absurd?


Presumably, time complexity for most operations correspond directly with CPU usage.

See here for an example with RPUSH: https://redis.io/commands/rpush

RPUSH is O(1) which the best you can possibly get.

ZADD on the other hand is O(log(n)), which is quite good but at a large scale it becomes easier to run into performance limitations, especially on high workloads. Performing O(n) operations on large data sets is out of the question unless you're comfortable with Redis being unavailable for multiple seconds/minutes.

That said, you will have to limit yourself to operations that are computationally simple (ie: O(1) or O(log(log(n))) or pre-shard across multiple instances (or run a Redis cluster).


Redis on MOSIX might be an interesting experiment.


That's only part of the "os" if you mean part of the installed libc, as opposed to part of the kernel. Managing heap is done in userspace.


It is one of those things where people say "Java can be faster than C/C++ because it can compact memory" and the C/C++ lady says "here, hold my beer"


To be fair only a compacting GC can do that


That isn't strictly true. You do not need to do garbage collection to compact memory. The data needed to do memory compaction and garbage collection largely overlaps, so you may as well do both, but you don't have to. To implement either you need to do a scan of all pointers in memory - for a garbage collection you save a currently in use list, for compaction you need to a map of where they are so you can change them latter. What you do with that data is different, but only the first step is the same. (though garbage collection can feed the empty places into the compactor which makes that work easier)

I've actually been thinking about how to do compaction in C++. The reason you cannot do it in general is code like "socket.send(new object, sizeof(void*)". The remote is then expected to send back that pointer which you cast to object and reference. Evil code for sure with a lot of failure cases that result in leaks or security holes, but possible.


Compaction is rather trivial with stop the world phase. Doing it without is a lot more challenging.


With forwarding pointers not. Which is only a problem with an FFI, which needs a traditional mark & sweep for its external callbacks anyway.


And the Java GC is NUMA aware. The C compiler is not.


Memory fragmentation can be avoided by construction, e.g. using trees allocated in linear storage (e.g. like std::map is implemented, or the maps in libsrt [1]). For strings a binary tree (or a B/B+ could be faster), and it would not require defragmentation nor rehashing (and worst access time would be log(n), instead of O(n)). Make growing the tree using realloc() is "free" on systems with virtual memory map/remap.

[1] https://github.com/faragon/libsrt


Could anyone explain me why this is needed with virtual memory? Wouldn't the OS take care of handling gaps of free space?


Seriously, that sounds magical


@antirez: Congrats! Are you going to modularize disque now that v4 is ready?


Yes, one of the top items in the Redis 4.2 roadmap is to port Disque as a Redis module, so the steps are: 1) Implement a Redis Cluster modules API (I've already some draft design, basically there are low level stuff to enlist nodes, broadcast messages, send one-to-one messages, and have callbacks called when we receive a message), and then port Disque as Redis module on top of such API, so that we can both validate that the Cluster API is good enough with a real project, and can make Disque a first citizen of the Redis ecosystem without all the code duplication, that in the end, made the project extremely hard to sustain. Thanks.


I look forward to Aphyr scrutinizing it. No disrespect to you, his posts are always fun to read and show how hard it is to get this stuff correct.


I'm very excited about this!


If you want to try out some of the modules already available: https://redis.io/modules


Maybe with the new modules support there will emerge some explicit way to do robust worker/job queues? So you don't have to remember your BRPOPLPUSH/LREM dances (or whatever it is) just so.


Antirez has talked about moving Disque to a Redis module, which should solve that problem neatly. I'd love to drop our RabbitMQ dependency and move queueing into Redis.


Why?


One less service to manage, plus RabbitMQ's options for dealing with network partitions are all terrible. If you want to run a RabbitMQ cluster, you don't have any option that avoids data loss.


Can you even avoid data loss with partitions in a AP system like RabbitMQ?


RabbitMQ is not strictly an AP system. Its clustering functionality supports several different modes for handling partitions (all of which allow for data loss) and is CP. Federation, which you should use if your network between nodes isn't super reliable, is AP.[0]

What I'd like to see is clustering with support for merging data between nodes. You'd probably end up with duplicates, which I'm OK with, but you'd avoid the data loss that happens with other modes like Pause Minority, where messages get dropped on rejoining a cluster.

[0]: https://www.rabbitmq.com/distributed.html


one less dependency to take care of, I presume


Redis Cluster connecting to nodes via DNS instead of IP would vastly simplify deployment on K8s.


How do you solve this problem? Can you use headless K8S services for that?


You can use a StatefulSet which gives a fixed hostname per pod. Then you can have a sidecar pod that monitors the cluster, discovers the IPs of all pods, and uses redis_trib to create and maintain the cluster. It's a hassle to make it robust though.


Don't see why you would do that, instead of a headless service.


Is there a Redis PPA for Ubuntu 16.04 that is supported by the redis team?


Redis is of the the very few applications , that I install manually almost every time.

trick is to use the utils/install_server.sh


hey, I wrote that little script years ago, glad to hear someone (besides me) is still using it! :)


docker images should be ready soon in case that is of interest. PR just went in - https://github.com/docker-library/redis/pull/98


Anyone know when we might see this on AWS (Elasticache)?


I don't like the Elasticache offering of Redis at all. It is severely limited if you are not just using Redis as a cache. There is no way to scale up/down the size of the node without putting it down(this is possible if you run it on EC2 as well as you have access to commands related to replication that are not allowed on Elasticache). We have had master restarting instead of failing over to slave and in the process coming back with no data. Even the snapshot is only taken once in a day.


Probably not soon because they use a fork of Redis.


LFU policy sounds really interesting!


Thank you, there is more info in this blog post: http://antirez.com/news/109


Awesome write up. I can't wait to try out LFU with our usage. I have a suspicion that our cache hit ratio will see a nice bump.

I've said this before and I'll say it again...thank you for all you do!


The point around visualizations for measuring and tuning is an excellent one with a great example. Thanks for the link -- great writeup.


It seems it even supports nearest search for lon/lat points by default... Quite nice since most RDMS don't even support it be default.

Although I'm curious to know what algorithm it uses for nearest search, it doesn't talk about it in the doc.

I don't really understand what redis should not be used for, I guess it's not for complex queries? Conventional RDMS really seem to belong to the hard disk drive age. So the difficulty resides in having well designed data schemes.


I adore redis, I use it for tons of stuff, but I still like to keep my single-point-of-truth for critical data in an ACID, transactional relational database.

I frequently denormalize that data INTO redis so I can answer certain classes of queries quickly. I'm also very happy with redis for caching, rate limiting, inter-process-communication, distributed locking and tons of other fun use-cases.


The entire redis geo capabilities are based on geohashing.


Any news on when this will be on Azure Redis Cache ?

https://azure.microsoft.com/en-us/services/cache/


Wouldn't expect it in anything less than 3-6 months tbh... depends on how anxious MS is to git this into place.


Interesting coincidence (or maybe not...) Redis was discussed on the Floss podcast that I listened to earlier today and now I have an inkling of what Redis is. My first exposure to Redis was to ponder where it came from after I tried to run Gitlab on my puny (J1900, 4GB RAM and spinning rust) file server. It was spectacularly non-performant with most page loads timing out. I suppose it was because Redis had insufficient RAM for operation. Redis may be scalable toward large busy systems but seems less so in the other direction. I thought it would be cool to have a real Git server but this one was not it.

During the podcast the Redis guy mentioned that 4.0 was on the verge of being released.


> I suppose it was because Redis had insufficient RAM for operation. Redis may be scalable toward large busy systems but seems less so in the other direction.

This is a pretty weak conclusion to draw. Did you try running Redis on its own without Gitlab? If not, I would encourage you to try it out as it takes up much less RAM than you think. The Raspberry Pi is even a supported platform...


Any reason why you lay blame on Redis and not Gitlab? Gitlab is what's in charge of serving and rendering the pages, after all.


The issues you had may have come from GitLab not having enough resources available to handle everything, as opposed to Redis itself; anecdotally, I have an application with a couple thousand keys in it that only uses ~5MiB of RAM.


Yeah, I'm in the same boat - I have a redis I use for caching auth tokens and some other values for a small service, and it doesn't take much ram at all. (5-10 MiB)


[flagged]


We detached this subthread from https://news.ycombinator.com/item?id=14770477 and marked it off-topic.


Is this satire? "man" here is short for "manual".


it's an attempt to mock an old discussion about the use of "SLAVE" / "MASTER" terminology.

These threads fill with reactionaries who think marginalized people are just complaining about nothing or by reductionist leftists who think so-called "identity politics" detract from class struggle.

Edit: It's worth noting that antirez has, as is usual for him, dealt with this issue wonderfully by making a call without downplaying the importance of the problem.


> It's worth noting that antirez has, as is usual for him, dealt with this issue wonderfully by making a call without downplaying the importance of the problem.

He was doing okay until he wrote that everyone is a slave because having to follow "politically correct rules" is itself a form of slavery.


lol. i hadn't seen that ;_;


[flagged]


I was so focused on the aspect of misogyny that I didn't realize how ableist this term is.


Meh.. manual does not mean hands.. or if you were trying to be satirical, please don't turn HN into reddit.


Nitpick: yes, it does. man- is a root for 'hand', and 'manual' as a noun was a term for 'handbook'.


Oops.. looks like my etymology has long been forgotten. thanks for correcting.


[flagged]


I'm not for the change in terminology at this step, because it is very unfortunate to change the terminology of a project that is here for years, however I tried to use the fact we have this terminology to remind people of what slavery looked (and still looks in certain parts of the world) like, so:

1) We have a SLAVEOF NO ONE command which is very popular.

2) We have a note about slavery here: https://redis.io/commands/slaveof

For new projects I suggest to pick other terms as well, even because master and slave are kinda inelegant IMHO other than linked to bad things.


I was looking for an issue to see if the project had addressed this at all before I posted this (failed to find it until after). I also had not read the SLAVEOF man page. So, I apologize for bringing up something I'm sure you're less than excited to have keep coming up.

I can respect (and disagree with!) the backwards compatibility argument.

I've always found you to be a considerate and thoughtful maintainer (no easy thing) whose work I really admire. So, I'm very glad to see that you've taken the steps you have on this front.

Thanks for your reply.


It's admirable you and the project edit yourself in this way. Although some will say they are only words, and only have the meaning we give them, the legacy of suffering the words bring to mind are nearly unbearable. Speaking for myself here obviously.

It's a choice whether or not to use these words, and any project can use them or not as they see fit. They are common in the field after all. I definitely think there's a high road here, and you've taken it, cheers!


antirez, this is easily one of my favorite comments I've seen on Hacker News in a number of years. In looking into what you've said here, I've learned that this is a bigger movement in software than I was aware of with projects like Drupal[1], CouchDB[2], and Django[3] taking similar stances. The github issue[4] spawning your change contains even more profoundly good ways of thinking about how software can do more, in the code itself, to "be the change you want to see."

Bravo and thanks for pushing the envelope forward.

Regardless of whether you agree with the changes, the fact that these discussions are occurring about the code that we write is really great. I love seeing how we can take code to a much different level than the 1s and 0s it is often reduced to in the mind.

I'd love to see some other examples of similar movements in software whereby we are taking great care of the code we write as an author, historian, or any other writer would take care of their own words.

[1] - https://www.drupal.org/node/2275877

[2] - https://issues.apache.org/jira/browse/COUCHDB-2248

[3] - https://code.djangoproject.com/ticket/22667

[4] - https://github.com/antirez/redis/issues/3185


Here's another example of changing terminology in software for similar reasons to master/slave: Changing PetSet to StatefulSet in Kubernetes: https://github.com/kubernetes/kubernetes/issues/27430


Maybe a simple solution is to leave SLAVEOF in place for compatibility, and implement a REPLICAOF command as a synonym. Update the docs to use the synonym. Not sure what to do about the INFO output, though.


Neat, I didn't know that.

Worth mentioning that "ciao" originated from "I'm your slave" - maybe in v5 we can have "ciao <master_ip>" :)

> The word derives from the Venetian phrase s-ciào vostro or s-ciào su literally meaning "I am your slave". https://en.wikipedia.org/wiki/Ciao


It's excellent to see you're taking the issue seriously.

Comparing this carefully considered reply to all the others also gives me some more evidence that those that excel at technology are often remarkable people in many other aspects as well.


> because it is very unfortunate to change the terminology of a project that is here for years

...is it? i guess my interpretation of saying 'unfortunate' here is that there is some kind of sentimental attachment or something.

are there legitimate practical issues?


A huge backwards incompatible change


  If slavery is not wrong, nothing is wrong.
  -- Abraham Lincoln
I've been trying to get an intuition (make myself feel good) about the material conditional logical connective and this seems to satisfy the case for a false antecedent and false consequent (¬p→¬q). The equally true statement seems to check out:

  If slavery is not wrong, something is wrong. (¬p→q)
However this also evaluates to True which annoys me:

  If slavery is wrong, something is wrong. (p→q)
And this evaluates to False:

  If slavery is wrong, nothing is wrong. (p→¬q)
I get that the formal definition of → is different from English's if/then but that just makes understanding proofs so much harder. Perhaps someone could better paraphrase Lincoln in first-order logic?


I'm not sure if you're trying to make a joke. In any case, quotes such as this are ill-suited for learning formal logic, because they rely on a lot more than their literal meaning.

In this case, you cannot see the statement as being about boolean true/false values. It only works on an ordinal scale, and our calibration of the cutoff for "wrongness". Slavery here is assumed axiomatically to be "really really bad". In fact, among all the bad things, it is the worst. But is it wrong? That depends on how bad something must be to be wrong. But if you do not consider slavery to be wrong, but merely bad, nothing can be wrong, because everything is less bad than slavery.

So this is basically Lincoln trying to argue for AUC as a better metric than sensitivity/specificity.


Applying logic to pithy quotes is generally going to be painful, because there's tends to be a lot of implicit meaning. But let's have some fun and break this down.

(Bear with me, it's been a while since I've done rigorous proofs.)

Let's say there's a function R that outputs a numerical value of an actions morality.

Let's define an action as wrong if R(action) is below some threshold M.

So what Lincoln is saying is:

R(slavery) >= M -> R(action) >= M for all actions

If we assume Lincoln's statement to be true, that must mean that R(slavery) <= R(action) for all actions.

Which can be reduced to R(slavery) = min(R(actions))

Translated to English, Lincoln is saying that slavery is most morally despicable act.


If at least one thing is wrong, slavery is wrong?


The words only have meaning if you give them meaning, and while a smaller progressive org might care, it doesn't seem to be a fight worth having. No one thinks your technical terminology supports the historical subjucation of a race, and I wish we could be adults and get on to real work.

Outrage fatigue is a thing. I just want to build, and save the politics for my off hours.

Edit: mods, feel free to detach as off topic. I did not expect this to go off the rails, and I don't want to take away from antirez's achievement


[flagged]


Except in nearly every case, this is the reality of the situation: http://i.imgur.com/erINvSI.png

It isn't that the people are offended - it's that others are trying to get offended "on their behalf" as a form of virtue signalling. Also see most of the "cultural appropriation" movement. Where non-Japanese people get offended that other non-Japanese wear kimono while nearly every actual Japanese person I've seen on the issue expresses that they are fine and even fully supportive of foreigners wanting to wear kimono.

There's a difference between fighting for those without a voice and speaking on behalf of others who do not actually approve of what you are saying. Both can be a result of empathy - but it is important to recognize when you're doing the latter and when you're doing the former.

The number of black people I've seen who are offended that people would even think that they would take offense to the master/slave issue outnumbers the number of black people I've seen who think it is an issue to be fixed. Almost every individual I see calling to "fix" this issue is a white person speaking on "behalf" of black people.


> Almost every individual I see calling to "fix" this issue is a white person speaking on "behalf" of black people.

Could it be that black people are put off by people fighting for their right to use words that have no significance for them, but need to remain just "on principle", and that they're put off by such behaviour, avoid the tech sector, and therefore aren't there to be visibly offended whenever you're looking?


It's common well outside the tech sector. My (black) ex frequently complains how white people try to speak on black peoples behalf about what should offend her, for example.

To the point that she more than once have been in rage over white people accusing her (to her face) of being offensive to black people for saying something that they just assumed black people in general should find offensive.

She's also part of the diversity board of a major bank, where she sits on a panel that is mostly old white men talking about what issues affects their minorities and women, without a hint of recognition from any of them of the irony in that they've not recruited more minority representatives and women from their staff to be on the board itself.


It "could be" but it just isn't. The reality of the situation is that minorities are being silenced or told they have "internalized racism" and whenever these actual minorities speak out on the "issue", non-minorities tell them that they're not sympathizing with minorities that it "could" effect. It's always a "could", because "could be's" don't need to exist in reality.

Of course, if any of these people fighting on "behalf" of minorities actually had minority friends this would become self evident. The coddling and speaking on behalf of the minority is almost always more offensive to the minority then whatever non-problem the non-minority is tripping over.

Imagine how a woman would feel if she was told by a man to be more sympathetic towards other women on some specific issue, because clearly the man is better able to empathize with a women's problem than the woman herself who is trying to tell the man that it isn't actually an issue and that he's making a mountain out of a mole hill. He insists in his correctness and declares that she has internalized misogyny and that he is better qualified to speak on behalf of women than she is. Swap out "woman" with "person of color" and "man" with "white person" and it has a different result?

If you fail to see how that situation is offensive then I think you're the one with the empathy issue, because you aren't actually seeing from their point of view but how you imagine their point of view to be. I'd be bloody fucking pissed if a cis person tried to educate me on issues surrounding and within the LGBT community or told me I should be offended over something (eg: being called trans, something I call myself for fuck's sake) - and of course that's happened multiple times in the past.


Sir or Madam, I am one of the most emphatic humans I know (on my scale as a pleb). I care for the suffering of all people, and am willing to give up large chunks of my life remaining for social causes. I live below my means so people I've never met can live a better life.

But arguing over terminology used by a data store? Let's not discuss window dressing patterns while the neighborhood is burning to the ground.

Pick the hill you want to die on wisely. This is not that hill.


[flagged]


No, it's all about feel good, especially about YOU feeling good that you did something. Kind of like people signing Internet petition or wear awareness bracelets.

The problem with it is that you did ABSOLUTELY NOTHING, except making yourself feel better.

Want to help? Contribute to something that actually matters.


To each their own. Appreciate the discussion.


> I care for the suffering of all people, and am willing to give up large chunks of my life remaining for social causes. I live below my means so people I've never met can live a better life.

You say this, but it doesn't jive with your original comment. Choosing words and conduct carefully are the "low hanging" fruit in this world, they cost you nothing, but have material impact on the groups you are a part of.


Arguing for not offending others by saying to someone that they seem to lack empathy is gutsy.


I'm not opposed to offending others. : ) Just be aware when you do, and why you're doing it. The parent post simply seemed blithely unaware of how deeply hurtful the phrase can be to people. (Imagine typing that phrase out every day when literally your own great great great grandparents were dragged here in chains and were considered the personal property of someone. That's simply f'ed up and would suck beyond belief, and not even being willing to acknowledge that harm is crazy to me.)


So, I have clinical depression, which I take medication for. I've worked in neurology, my formal education is in psychology and neuropsych, and growing up I've known more than my fair share of people with some form or other of mental illness. In short, I live it, I've studied it, I know it.

Your comment reminds me of someone on a chat channel who had a nagbot that banned the word 'crazy' (amongst others). Why was 'crazy' banned? Because it might offend crazy people. Being one of those, and knowing people like that, absolutely no-one who would be labelled with the moniker 'crazy' would be offended by the word unless it was directed at them. The idea that a schizophrenic friend would be 'deeply hurt' if I said 'the traffic this afternoon was crazy' is utter nonsense. This forbidden word was forbidden by someone who clearly doesn't associate with the kind of people that were being defended.

And so it is when your black American engineer types the word 'slave', suggesting that their self-identity is so wrapped-up with the identity of a person that their grandparents grandparents barely knew, that they would find a non-directed under-the-bonnet technical term deeply hurtful. It's disturbing that you consider these people you're supposedly defending as being so incredibly emotionally fragile.


Honest question: Is that how you feel when you type those out?


Yeah, I actually dislike it every time myself, though that's not hugely important to me, nor why I commented.

There was some interesting commentary in this thread about cultural appropriation and whatnot, which seems to me to be beside the point. My intent was only this: if there are other people rightly upset by something, it doesn't matter if you're not, it doesn't even matter if the majority are not, it's still valuable to have empathy for those those that are. That's all.


Question then: who decides whether someone is "rightly" upset? There is some implication there that we don't need act on every offense taken.

For instance, I don't expect others to act (i.e. should be forced to change) because something offends me.

Notes:

Don't confuse this question with me arguing that we shouldn't be empathetic.

I'm also not from the U.S. so my approach and offenses are different culturally than your own.


[flagged]


My changing my tech terminology isn't going to fix systemic racial tensions. It's not going to fix white officers getting away with killing unarmed black civilians. It's not going to fix the mass incarceration of black males due to broken public policy over the last few decades. It's not going to fix systemic policy issues that lead to a higher chance of poverty for minorities, preventing class mobility.

Pick the right fight. How many man hours have we all wasted on this discussion already?

My time is valuable. Through my dollars earned I contribute to positive social change wherever possible. What's your contribution besides berating me?


If it's any consolation, a lot of my peers, myself included, share this view. I wouldn't bother taking any of your valuable time responding to people who refuse to acknowledge your points.


> My time is valuable. Through my dollars earned I contribute to positive social change wherever possible. What's your contribution besides berating me?

Dunno. But that antirez guy up there agrees with the original criticism, and I think I may have seen the name before, somewhere. But maybe my memory is faulty...

Since you're complaining about the time such discussions waste, I'm wondering what you see as the end-game here? What do you believe to me more likely: those complaining now to throw up their hands and never mention it again, or the resistance against change slowly eroding until the terms disappear?

Because it's kinda tautological to use the discussion itself as the main argument for your side of a discussion. If that's the #1 argument against the change, actually making the change would solve it with absolute certainty.

And if you accept that there's a problem with "white officers getting away with killing unarmed black civilian", you shouldn't discount the power of words and symbols. Stuff like this may seem insignificantly small, but those problems are ultimately rooted in culture, and culture is nothing but ideas, which are nothing but words.


> you shouldn't discount the power of words and symbols

Words are not static entities. Language evolves in unpredictable ways. Master/slave became jargon based on a truly despicable thing, unfortunately, but the original meaning is an irrelevant detail in our technical context.

One can attack the idea of "context", but I think it is not very effective, or even honest. It is a leap of logic to imply that a single expression used in very specific moments is somehow related to atrocities happening right now. The words (and symbols and culture and ideas) argument is a killer abstraction and anything can be said with it.

I am black and I live in an extremely racist society. The habit of discussing this kind of thing can have a negative impact on what really matters. Even empathetic people could become tired.

Ps: my english is not that great.


I think you're good at heart, but you are wrong here. Words carry meaning, not only the meaning we each attach but also the meaning that is imposed on us. By your logic, white folks should be able to say nigger without black folks being upset about it, which is an absurd result of your premise.

Constantly maintaining a clear separation of symbolic meaning (words) and actual meaning (reality) is what monks and yogis do in a sense. It's a life-long discipline to do it perfectly, and it takes a lot of energy. The energy expended to constantly separate words from meaning is an unfair burden on certain classes.


[flagged]


>"Edit not saying you are, but if someones ancestors were slaves, and the IT dept sits around talking about masters and slaves, how would that make you feel?"

https://en.wikipedia.org/wiki/Slavs#Ethnonym https://en.wikipedia.org/wiki/Slavery#Terminology

I feel just fine about it, thank you. I'm more disturbed by the fact that we're even having this conversation at this point in our society.


Okay then, be mad that "slave" isn't a technically correct term either. "replica" or "follower" both describe the behavior much better.

> it doesn't seem to be a fight worth having

Technical correctness is a fight worth having. Human decency is a nice side effect.

> the words only have meaning if you give them meaning

That's not how words work.


The context of how I use the word "fuck" dramatically changes what it means. Whether you're offended by the word is entirely up to the listener as well.


Do you understand the phrase "technical correctness" or is that open to interpretation as well?


In this case, the "technical correctness" of your argument is up for debate.


You're debating the concept of words themselves. Which have meaning. It's the mechanism through which words work. They have both a meaning in context and out of context. They have meaning both intended and unintended. Good technical documentation should strive to minimize unintended interpretations by using the most clear and concise words possible.


This terminology has been used for a long time in computing. It's immediately recognised by everyone and in no way endorses human slavery. I'm getting quite fed up of the OTT political correctness movement in tech.

Out of interest, what is your opinion on male/female USB connectors? Does one deserve to get fired if they make a joke about "big dongles"?


The actual act of slavery existed as an accepted institution for longer than this computing term has. If we can make slavery a thing of the past (in the western world), we can also update to a more technically correct description of the behavior.

The term "slave" isn't even a good technical term. It would be better to call it a follower or replica which better approximates it's actual role. Not only is it "politically correct" it's also "technically correct", the best kind of correct.


We will not have "made it a thing of the past" until the words loses the kind of power over people where they are still seen as offensive by anyone.


True, but it has also been common practice for at least 20 years in the US to not use the term slave. Source: I worked on replication since 1996 and we were asked to use "supplier" and "consumer" at that time. Actually I've been surprised to see slave making a come back recently. Wondered if people hadn't received the memo or perhaps were outside the US cultural influence.


Worked with US teams most of my career (since '95 or so) and never had any of them complain about master/slave nomenclature in any of the software we used. I suspect that rather than being "common practice in the US", it's been common in whatever niche you work in or whatever part of the US, or even just in the specific companies.

Of course it might be I who've worked in such a bubble too, but given the number of people here who seem to find it annoying that people are even suggesting not using it, I'm inclined to believe the use of master/slave has remained more common that you think.


"outside the US cultural influence"

Where is this magical place you speak of?


Eastern Europe sees discussions like this being done by the "bored westerners" and finding it great fuel for right. When Django was moving away from "blacklists", the general attitude was to mock it, or keep quiet about it:

https://translate.google.com/translate?sl=pl&tl=en&js=y&prev...


There are other ways to offend someone from Eastern Europe, or more generally almost any country. It's not unusual for some painful recent history to be still a raw nerve.

Slavery is a significant topic in the US. It was a prelude to the civil war; it was abolished within recent memory; and the echoes of racist sentiment still affect the lives of millions: people alive today remember segregation, and both casual and systematic discrimination against African-Americans continue to occur.

This is not a big deal in Poland, which has no such extensive recent history with slavery.

But - for example - in 2015 the Russian ambassador in Warsaw was excoriated in public and summoned to the foreign ministry for his remarks partly blaming Polish policy in the 1930s for contributing to the start of WW2 and the subsequent death of millions of Poles. Now that was a raw nerve.

A different standard does not mean the absence of standards. People who fail to see that, and then mock others, are simply demonstrating a lack of self-awareness.


>A different standard does not mean the absence of standards.

But that was the point I was arguing for. That US standard is not everyone's standard.


Then we are in agreement. My remark was not a rebuttal to yours.


> I'm getting quite fed up of the OTT political correctness movement in tech.

Is it over the top though? It's a references to a deplorable practice that humans do; why shouldn't we strive for something more civil, and more technically correct? The slave of a database isn't the property of the master and subject to its every whim; it's a replica of the data that can do separate, independent things from the master with that data.

> Out of interest, what is your opinion on male/female USB connectors? Does one deserve to get fired if they make a joke about "big dongles"?

As for gendered connectors, the terms themselves don't carry (or shouldn't :() any judgment nor does it's usage. A valid cable, pipe, &c doesn't have to be heterogeneous/hermaphroditic and can be homogeneous. I would hope that the LGBTQ community wouldn't be offended that only M/F ends can "mate", as biological "mating" (i.e. copulation for the purpose of procreation) can only happen between M and F ends, and see it as a judgment on non-procreation intercourse.

However, if there was an issue about the term M/F being a judgment call, it would be responsible for the community (well, all communities, as the terms M/F for connectors goes beyond just electrical connectors) to listen and (slowly) introduce better names.

There is no reason to make a large portion of the population uncomfortable when it literally costs us nothing to change it (over the long run).


> deplorable actions humans do.

So when do we rename 'kill'?


'Bash' is too violent as well. We should rename it to 'hug'.


`bash` is also pretty evangelical ;)

On a serious note, I've never actually understood where the term "hung" comes from in computing. Why is the process "hung" and not simply "stalled" or something else.


Hung is used like that outside computing. For example in the terms "hung jury" and "hung parliament". I have no idea how this word became common in computing though.


The real reason is because most of the concepts surrounding multi-process computing come from the telephone industry.

The modern interpretation is that system operators need to feel like they have a big penis.


> The real reason is because most of the concepts surrounding multi-process computing come from the telephone industry.

ah

> The modern interpretation is that system operators need to feel like they have a big penis.

I actually didn't even think of it in that regard. I regard hung as in something that is hanging (dangling) from something else.


Humans kill many things over the course of their existence. For instance, we slaughter animals for food, pluck weeds to help our crops, and preserve food by boiling or chemical means (e.g. salt).

Killing is not deplorable. Murder is deplorable. (I'm a vegetarian, so I find slaughter for food deplorable too, but I'm in the minority here.)


...or killall


Happy to provide the flame you're expecting.

Master/slave describes the relationship between the services and has been standard terminology for decades. I have yet to hear of any IRL victims of slavery that are miffed by the use of this terminology.

Do not be offended on someone else's behalf.


But it's ok to assume a lack of offense on the part of every other person? I think we all get what the terms mean In This Context but there are other options available that don't require that kind of boxing and unboxing. Other projects have done this successfully[0] and the Redis maintainer is right here in the thread saying he understands the concern. The thing I never get with these objections is, what is the harm in raising the point/ making the change? Who loses?

[0] https://code.djangoproject.com/ticket/22667


It's pretty "lame" to push established projects to make confusing terminology changes to address whatever words the "peanut gallery" is upset about this week. Try not to be so "uppity", "guy".


"Slave" is technically incorrect. There are better terms to describe the behavior such as "follower" or "replica".

If someone has been around long enough to understand what a "slave" is can't figure out what a "replica" or a "follower" does, maybe they shouldn't be in tech. The transition cost for documentation is essentially zero.

Naming things is hard, and better names are better for everyone. Antirez said he would change the name if he could start over, hence the hill you're dying on is not shared by the author of the library you're posting on.

Edit: For clarity of argument.


It's not just a replica, it can work offline for a while and push back. Leader/follower is also flawed in my opinion. It's master and a slave even if it offends your sensibilities. These arguments are a waste of everyone's time.

>This hill you're dying on looks awfully lonely.

Ironic coming from a grey response to a 15 point comment.

I will change my tune if even one victim of slavery comes forward and speaks about the use of the terminology.


> It's not just a replica, it can work offline for a while and push back. Leader/follower is also flawed in my opinion. It's master and a slave even if it offends your sensibilities.

I'm glad that your concerned that "follower" and "replica" aren't perfect terms. We should be concerned with terminology.

I humbly suggest that "slave" DBs are not physically whipped by their "masters", therefore are an even worse metaphor.

> These arguments are a waste of everyone's time.

Then why are you wasting your time? If you don't care, don't argue back. Boom, you've got your time back. We all get a better name, everyone is happy.


I agree that the terminology doesn't matter, but consensus does. Master/slave are the commonly understood terms.

Here's a question: if "slave" is offensive, should "kill" (as in 'kill 69' to kill a process in SQL Server) be considered offensive as well? Simply as a consequence of being something 'bad/immoral'? I'm sorry, and I'd be glad to be corrected, but I genuinely find this argument...disingenuous. And people who see things the same way I do see that kind of argument and think that you've got other motives.

EDIT: I want to say (to prevent this discussion from flying off into Trump-land) that I'm liberal on most issues except gun control. I just have a hard time relating to the 'micro-aggression-y' brand of liberalism I've been seeing of late.


Master/slave was also a commonly accepted role of business for far longer than it has been a computer term. We managed to change that in the west. I believe in us, we can do it.

> I genuinely find this argument...disingenuous

My argument is that the term is not technically correct. How is that disingenuous?


I'm arguing specifically against the term being offensive. EDIT: and oddly, I can't seem to find the word anywhere in your comment that I replied to. Am I losing my mind, or did you take that part out?


I had it in there to emphasize how un-technically incorrect it is. Apparently people saw the word "offensive" and they didn't read the rest, so I took it out to add clarity. I forgot that HN doesn't provide a diff of edits, so went back and added a note in the comment that it was edited.

Edit: Edited this note to mention I updated the previous note with an edit note.


I'm fine with an argument based solely on its technical merits. But then you're going to lose out to things like backwards compatibility and the sheer force of momentum - everyone understands these terms.

There are probably hundreds of ill-fitting terms in the tech world that now mean what we mean them to mean (lol). This one receives blowback not because it's technically unsound (which it might be), but because some paint it 'offensive'. That's why everyone seized on this part of your (now edited) argument.

EDIT: thanks for being honest btw


For most projects it seems like a pretty easy change. You could add new keywords, alias the old methods/commands to them. You don't even have to deprecate them, just make the docs point to the newer terminology.

What is easier having to have this conversation EVERY time the term gets used for the rest of eternity, or making the change once and being done with it?


>What is easier having to have this conversation EVERY time the term gets used for the rest of eternity, or making the change once and being done with it?

That's not very compelling. Just because certain people make a lot of noise doesn't mean we have to do what they want us to do.

>For most projects it seems like a pretty easy change. You could add new keywords, alias the old methods/commands to them. You don't even have to deprecate them, just make the docs point to the newer terminology.

...for every project that uses these terms across the entire industry. You know what's easier? Not making the change. There are two arguments:

1. The technical gain by changing these terms outweighs the drawbacks (legacy code, consensus/convention, etc.). I doubt there's much to gain here.

2. The technical aspect doesn't matter since it's offensive. I've already addressed this.


It depends on how expensive your deprecation process is. I don't think we can generalize for entire ecosystems. Some langs and products it would be fairly trivial to update, and retain back wards compat. Others might be harder and stay constant for longer.

Antirez said if he started a new project it would be named something different. I can only assume some other future databases will will take this approach. Maybe Antirez will write one of them. When that happens now we have a new problem: divergence in the ecosystem.

My preference would be for a big project to adopt a separate term, "secondary", "follower", "replica", whatever. So that when other new DBs look for an "alternative term" there is somewhat of a standard. Really though, that's not even that bad of a problem, because when i say all those terms, "secondary" etc. it's pretty obvious in the context of a DB what i'm talking about.

Do you have to do something __just__ because people are making your life hard? No. Do you have to make a change just because something is technically incorrect? No. Do you have to make a change because something is offensive to a large group of people? Still no. Maybe a better question to ask is, at what point would the tables be turned? What would push you over the edge to say it should be changed?


>It depends on how expensive your deprecation process is. I don't think we can generalize for entire ecosystems. Some langs and products it would be fairly trivial to update, and retain back wards compat. Others might be harder and stay constant for longer.

Agreed - I was talking more 'sum' than 'average' cost.

>Antirez said if he started a new project it would be named something different. I can only assume some other future databases will will take this approach. Maybe Antirez will write one of them. When that happens now we have a new problem: divergence in the ecosystem.

Also agreed, divergence breaks the 'consensus/convention' I mentioned earlier. That definitely has consequences, although in this case it wouldn't be too bad, as you mention next:

>My preference would be for a big project to adopt a separate term, "secondary", "follower", "replica", whatever. So that when other new DBs look for an "alternative term" there is somewhat of a standard. Really though, that's not even that bad of a problem, because when i say all those terms, "secondary" etc. it's pretty obvious in the context of a DB what i'm talking about.

Also true, and (assuming we're making the change) that seems to be the path of least resistance. At least we'd only have one new term, lol. And they are fairly clear.

>Do you have to do something __just__ because people are making your life hard? No. Do you have to make a change just because something is technically incorrect? No. Do you have to make a change because something is offensive to a large group of people? Still no. Maybe a better question to ask is, at what point would the tables be turned? What would push you over the edge to say it should be changed?

It's hard to quantify the factors imo. You kinda have to balance consensus/convention & migration cost vs. 'cost of incorrectness' & 'human cost of offensive term'. Cost of incorrectness is a big one if newbies struggle to understand the tech due to confusing terminology, but I don't see that here. If the term were truly offensive I'd agree that that'd tip the scales. I just don't see it.


[flagged]


[flagged]


That guy was an asshole btw


The people who have to change their code to accept the changes lose.

For Django the change was much simpler because it was mostly a docs/tests change. However, here, we'd have to change Redis' API (or include more than one way to do the same thing).

Raising the point is fine. Discussion is important. But making the change, in this case, has some clear downsides that antirez has carefully weighed against the pros.


> The thing I never get with these objections is, what is the harm in raising the point/ making the change? Who loses?

A good practice should be to try to find the most convincing voice (online, not just here) with an argument that answers this question.

That's the best route to honesty. If you can't do this then you're not up for debate.

This is something we all have to do with all of our arguments or questions.

If a question is not investigated it becomes a statement. Just because the person that you're asking cannot come up with a good answer does not mean that a good one does not exist.

So you win by make a statement and not asking an honest question.


We lose linguistic ground to the neo Marxist ideologues that seek to tyrannize the use of the English language.


It is rather sad that most discussed part of the announcement of a pretty amazing piece of software that probably had multi-billion dollar effect on the computing is concentrated on the perceived offensive usage of the words "master" and "slave" by people who never experienced it or had contacts with anyone who had experienced it.


Seriously... someone so stupid to not even be able to place words in a context would probably always find something to rant about, whatever censorship you could think of...


How is changing documentation to more accurately reflect its behavior censorship?

Do you think whitespace commits are censorship? Both make for better reading and more accurate representations of our technology.


That is not what is being done here. A small minority is feigning outrage at the potential for offense because of a commonly used term that everyone understands. Let's call a horse a horse folks. Don't shift the goal posts here. You and those on your side are attempting to police language in a space where it is clearly not needed solely to show others how virtuous you are. It's disgusting, it's immoral, and there is no place for it here in this discussion. This discussion is about software, not social justice.


Volunteer to fight against ISIS if you really care about abolishing slavery on earth.

Edit: Donating to FBM could be a way: http://www.freeburmarangers.org/


Well this post certainly went to hell.


Escalated quickly in a manner I wasn't expecting. Anyway, congrats to antirez on 4.0.


I edited my GP post to ask mods to detach.


I think the m/s terminology is not technically correct.

The actual behavior of a "slave" would better be described as a "follower" or a "replica". In an IRL master/slave dynamic the master does not first do the work only to be copied by a slave. The slave does all the work while the master whips them.

It's better to choose a more technically correct metaphor/terminology.


You chose the wrong definition of slave to make your argument...

https://en.wikipedia.org/wiki/Master/slave_(technology)

In the computing world, this is what master/slave means. And has for 40 years. It doesn't matter what the words can also mean, just like it doesn't matter that the word whip can mean a kind of desert, or to stir.

You need a dictionary.


"Slave" has a meaning in the non-computing world as well and the two concepts diverge. Which is my point. Why is the term "variable" good, it's because it can change, it's variable it describes the concept well. When programming specialty lingo does not diverge from non specialty lingo it makes the concepts easier to reason about and work grok.

The old term is confusing to newcomers, and changing it to something else would not break the brains of any tech oldtimers.


They have a note on it in the docs for SLAVEOF: https://redis.io/commands/slaveof

They stuck with it to avoid breaking backwards compatibility.


Many hydraulic systems have master and slave cylinder configurations for brake and clutch, for example, in automobiles. I'm afraid that use is ubiquitous and not likely to change.


Admin, this entire subthread is going to digress to Oblivion.


There it is. Post modernism in clear form.


And now we've gotten to the meat of it. When will these people stop? Do they need to be in control of absolutely every discussion? They are taking over the development community.


This sums up Hacker News so beautifully. On a post related to the release of a new version of a piece of software for caching data would the topic of slavery come up.


master/slave is terminology that's readily understood by almost everyone in computing.


may I use color names as values? is black and white allowed?? is this correct politically?


I've said it before and I'll say it again: I am so smitten with antirez (and redis)! One of my favorite projects for sure.




Guidelines | FAQ | Support | API | Security | Lists | Bookmarklet | Legal | Apply to YC | Contact

Search: