
Redis 5.0 RC1 is out - dsr12
https://groups.google.com/forum/#!topic/redis-db/9FcwPtutyyQ
======
koolba
Streams ([https://redis.io/topics/streams-
intro](https://redis.io/topics/streams-intro)) are going to be awesome.
Blocking sorted set pops will also be quite useful. Looking forward to this
being widely deployed enough to start using them.

I don't see TLS support in the release notes though I remember antirez
mentioning reviewing a PR for it. Is that being handled separately and
possibly back patched?

~~~
antirez
Unfortunately there was no way to put TLS so late in the game. We tried, but
after I and my colleague Fabio reviewed the code, it was clear that in order
to be a complete implementation the TLS support hooked and modified many
things in the Redis internals, including sensitive things like replication.
We'll try during Redis 6, now that we have time to do the things in a proper
way.

~~~
koolba
That sounds more than reasonable as "measure twice, cut once" is the right
answer for something as critical as a data store.

I've said it before and I'll say it again, thanks for Redis!

~~~
antirez
Thanks! Btw I really hope we'll be able to go forward with the TLS patch ASAP.
For the same reasons as above it will be great if we could merge/adapt it in
the early stage of Redis 6, so that there will be a lot of time to test it.

------
ralusek
I made a similar comment when streams were first announced, but didn't get
feedback.

My question is about using streams for event-sourcing, in similar cases to how
Kafka is frequently used. In such cases, the expectation for persistence is
paramount, as the event log is ultimately the primary source of truth for the
entire application. The actual utility of having the event log in memory is
only really important while the consumer groups are reading it, after which it
mostly just needs to be archived.

So my question is, are the common strategies used by Redis to persist to disk
going to try to keep the whole event log in memory, or will they be able to
intelligently offload to disk? Otherwise, is the expectation that clients will
instead write to the stream, and consumers will be responsible for consuming,
writing to some persistent layer like s3, and then removing from the stream?

~~~
antirez
Hello ralusek, the idea is to provide more the tools for the application layer
to move data from/to stream. I don't think we'll have direct support for
persisting streams on disk. Honestly streams as implemented in Redis are very
adapt to this use case, because of the macro-nodes that are swappable as a
single entity, however some time ago I decided that I wanted to focus on the
on-memory aspect.

~~~
ralusek
Cool, thank you for response.

------
antirez
This Twitter thread contains more information / pointers:
[https://twitter.com/antirez/status/1001480569442783240](https://twitter.com/antirez/status/1001480569442783240)

~~~
jazoom
>Yes I totally appreciated your Tweet indeed, don’t get me wrong. I was just
trying to communicate that in general I try to ignore stupid rules even if
they are pretty damn real in the industry. Thanks!

Right on!

------
kzrdude
Side track, how is the Windows port of redis doing? Is it likely to be
updated?

~~~
antirez
AFAIK Microsoft abandoned the Win port, and AFAIK they run the Linux version
on Azure. However with Windows 10 Linux subsystem you can run Redis for
development without issues, just "make" and it's going to work out of the box.

~~~
breakingcups
Do you know why they abandoned it? Or why they started it?

~~~
antirez
They started the port after seeing I had no interest in writing a port for
Windows. They stopped it IMHO when they realized, basically, the same things
that prevented me from starting the project at all, that is: it runs very well
on Linux and is very hard to port to Windows in a serious way, because of
Windows lack of fork(). If you ask me Windows should absolutely get a serios
POSIX layer ASAP and just let the Windows users use all the OSS that is out
there.

~~~
WorldMaker
Which is what the Windows Subsystem for Linux does.

If I recall correctly, Azure uses some combination of Windows Docker container
and WSL support now to run Redis on Windows.

Though from what I heard (as scuttlebutt/rumor/lay person reading the tea
leaves), that isn't why they abandoned the port (as native ports are still
handy for performance, and encouraged by Microsoft as much as they can), I
heard it had more to do with the usual OSS reason of abandoning a port: not
enough upstream support/enthusiasm for merging necessary, underlying patches,
and not enough time/energy to devote to keeping a fork up-to-date with a
wildly diverging upstream in such a situation.

(Compare Git for Windows or Node for Windows, which have been officially
recognized upstream, and directly merge a lot of patches upstream from
Microsoft's contributions.)

I don't have a dog in the fight and think you (antirez) are doing a good job,
and Microsoft independently are doing a good job with OSS the last few years,
and while I'm disappointed there wasn't more Reese's Cup peanut-butter-and-
chocolate collaboration here between y'all, I'm okay eating my peanut butter
and chocolate separately. That's OSS, in all its bazaar beauty.

~~~
antirez
The Windows Subsystem AFAIK is not meant to run things for production, but I'm
not a windows expert indeed.

About the abandoned port, yep we are saying the same thing in different terms
probably: the reason why merging from upstream was so hard was probably
because the windows port, for lack of POSIX interface, had to reinvent the
persistence of Redis in user-space terms. That was why it was so hard to keep
the patch in sync AFAIK.

~~~
WorldMaker
IIRC, WSL hit "general availability" status in a recent release, including on
servers, but I believe that yes the guidance is still that WSL is primarily
"for developers" and that running production services on it is, in one of the
greatest OSS traditions, "at one's own risk", but "allowed" now in a "at least
it won't break your warranty" sort of way.

------
erulabs
Congrats & thanks Salvatore! Redis Streams is an awesome feature, and bringing
Redis Cluster management tools into redis-cli is a world of improvement!

Keep up the great work, hope to see you at next RedisConf!

~~~
antirez
Thanks! See you soon, there will be a Redis event in London btw in case anyone
is interested. Should be November 2018 AFAIK.

~~~
atomi
Will you be providing docker images for the rc releases?

[https://hub.docker.com/_/redis/](https://hub.docker.com/_/redis/)

~~~
antirez
I'll ask my colleagues at Redis Labs if they can do it. Thanks for the hint.

~~~
omginternets
I would like to second this request. Thank you very much for your
responsiveness :)

------
dstroot
#1 - Congratulations! #2 - I am interested more in where clustering is going
and running Redis inside K8S. Is there something someone can point me to to
read on best practices for Redis 5?

------
nasmorn
I have a bad time with redis hosting from time to time. It is surely my own
fault. What are your favorite resources/recipes to configure/host redis on
your own?

~~~
esterly
We've found Redis installs with no issues see:
[https://redis.io/topics/quickstart](https://redis.io/topics/quickstart) or if
you are on windows
[https://github.com/rgl/redis/downloads](https://github.com/rgl/redis/downloads)

On Ubuntu this guide works well
[https://www.digitalocean.com/community/tutorials/how-to-
inst...](https://www.digitalocean.com/community/tutorials/how-to-install-and-
configure-redis-on-ubuntu-16-04)

Only issue I've had is remembering to add the port to iptables

~~~
antirez
Thanks, a few other tips are here:

[https://redis.io/topics/admin](https://redis.io/topics/admin)

[https://redis.io/topics/latency](https://redis.io/topics/latency)

In general our operations documentation does not still reflect all the good
things put into Redis 4, we'll update everything shortly.

------
alberth
Has the Redis codebase stabilized?

I mean that with no disrespect but the 4.x branch seemed to have a number of
CRITICAL bug

[https://raw.githubusercontent.com/antirez/redis/4.0/00-RELEA...](https://raw.githubusercontent.com/antirez/redis/4.0/00-RELEASENOTES)

Again, many thanks to Antirez who’s mostly a one-man-show.

~~~
antirez
Hello alberth,

I would not say Redis has an history of being unstable, actually it is
regarded as not failing and not crashing often. However at the same time, some
people believe that it is operationally challenging, which is a different
problem. That Redis operations are harder than the ones of other DBs is in
part true because the fact of having the data in memory, with different
mechanisms to backup the data on disk, means that in case of an operation
error (not necessarily a bug, just also an admin error), the outcomes can be
worse. However in the filed of operations Redis improved year after year.

However this does not answer your question about critical bugs. Basically most
of the critical bugs you'll see in the Redis 4.0 changelog is about the new
version of replication that was put in Redis in Redis 4.0 itself. The new
protocol, called PSYNC2, was harder to implement and has many corner cases
because it is able to restore the replication link with the master after it
was broken. To do so requires to take the state of the replication intact, and
given that slaves can do that (partial resynchronization) after an RDB
restart, we found many issues in this respect: after an RDB restart there were
cases where the state was not fully restored.

Most of the time it was very hard to trigger such bugs, and in most cases
those issues were found analytically and never reported. Yet I feel like I
should mark every of such bug as CRITICAL in the release, regardless of the
fact nobody was affected for a log time.

~~~
chandanrai
Wow, humbleness embodied! As great is redis,so is the maker.

------
didip
redis-trib.rb is finally ported to redis-cli. Thanks Salvatore!

Stream data type looks very interesting as well. Is the goal to be competitive
to Kafka-like use-case?

~~~
antirez
Hello didip,

I've a non-competitive style when the matter is software :-) I simply think
that Redis streams could be very useful, in some way I borrowed Kafka ideas,
in other ways I totally refused the Kafka implementation / approach, and in
general Redis streams have a lot of cons and pro at the same time in a mixed
way, due to the total difference between Kafka end Redis. So I think it's the
users that should apply the right tools for the right jobs, I'll avoid any
comparison. I just gave Kafka credits in the Redis doc because certain ideas
are clearly from Kafka or popularized by Kafka.

~~~
lossolo
Hello Salvatore, in which cases you would not use Redis streams and used
Kafka/other solutions ? And in which cases you think Redis streams shine
compared to other solutions ?

~~~
antirez
Hello lossolo, I hope that the community will figure this out. I'll try to
make videos, blog posts and programming examples about Streams in Redis, but I
don't like to make direct comparisons and I don't want to become a Kafka/other
solution expert as well in order to make sure to make fair comparison :-) If I
think at Redis Streams, I think they work very well in a set of domains. The
_king_ use case IMHO is time series. Having fast range queries is pretty
interesting for this use case. Then the second one is events and event
sourcing of all the kinds when the number of events-per-second is very
sustained. However I've some feeling that also thinks like delayed jobs that
are currently implemented on top of blocking lists may be rewritten to use
streams and offer better overall features/operations.

~~~
sertorius
Just wanted to affirm this - I saw the streams announcement last year on the
same day that I was having issues with Kafka and I implemented it into
production for processing IoT time series data right away.

It was probably a bit too early for production use, but it worked perfectly!

One nice advantage of it vs. Kafka is that given that it's so lightweight, a
Redis instance with streams can be dedicated to each customer with its own
port and firewall setup and give you a really solid security story for clients
who are paranoid about such things. (Kafka security is possible too of
course...but harder to explain for a layman.)

Thanks antirez - it's great. I've written a python Redis iterator that I'm
looking forward to PR'ing to the python client to encapsulate the XREAD
timestamp tracking bit and manage timeouts.

~~~
antirez
Thanks! Glad to hear a real world experience about it. And thanks for the
Python client PR plan.

