
Redis PSYNC2 bug post mortem - antirez
http://antirez.com/news/115
======
benburleson
Thanks for all you do antirez!

------
snewman
> the tests never try to mix PSYNC2 with replication of Lua scripts

This sort of testing gap is so hard to avoid. Any complex system has an
inexhaustible number of potential feature interactions. It's very difficult at
best to guess which combinations need to be tested, and often difficult to
even write a test which exercises a particular combination.

What techniques do people use to ensure test coverage of complex cross-feature
interactions like this? Two things I emphasize:

* Emphasize integration tests, and try to use multiple features in each test. This can help drive out interaction problems; the downside is that these tests are harder to write and (especially) maintain. It can also be difficult to exercise specific code paths.

* Write randomized tests, that exercise as many features as possible. The challenge here is identifying failures. "The program didn't assert or crash" is often a good start. A gold standard is to run a simple reference implementation alongside your real implementation, and compare the output... but that's not always possible.

~~~
cortesoft
If you mean randomized tests as in adding random choices to your tests, I
think that is a very bad idea (that I have been bitten by before).

You NEED tests to behave consistently. If you run the tests twice against the
same code, it needs to return the same result. Otherwise, how will you know
you have fixed the issue the tests caught?

Speaking from experience, soon after you introduce randomness to tests, you
will start to get intermittent test failures. When that happens, people will
start to ignore failing tests because, 'hey, this is probably just a random
failure'

~~~
anonacct37
Seeded RNG.

~~~
cortesoft
So not really random, then? What is the point of adding an RNG if it is going
to output the same values every time?

~~~
jeanmichelx
Because then the testing stimulus is not hard coded anymore, but generated.
It's then trivial to run longer sequences, or tweak the test.

In hardware design, where reliability is key (making a chip costs multiple
millions and 6 months time), there is only random testing (with seed for
reproducibility), not "that bunch of case that the person testing thought of
that day"

------
michaelmcmillan
We're all human – appreciate the write-up.

------
frik
Is Redis a one-man show? (plus contributors)

I am still using memcached, but consider Redis in future.

I am looking for a Redis-based high-performance message-queue that can be
filled from Nodejs and consumed with PHP. Basically a high-performance
message-queue that doesn't need dozens of servers to start with.

~~~
why-el
Try [https://github.com/antirez/disque](https://github.com/antirez/disque).
Written by the same guy behind Redis.

> Is Redis a one-man show? (plus contributors)

Well, yes, started by one man, and he continues to lead it, with many
contributions from the community.

Update: I fixed the wrong URL.

~~~
gizzlon
> Try [https://github.com/resque/resque](https://github.com/resque/resque).
> Written by the same people behind Redis.

Huh? I don't think it is, Redis is by antirez ..
[https://github.com/resque/resque/graphs/contributors](https://github.com/resque/resque/graphs/contributors)

~~~
why-el
Oh damn, I meant
[https://github.com/antirez/disque](https://github.com/antirez/disque), not
resque. Apologies.

~~~
detaro
FWIW, it appears the current plan is to not continue it as a standalone app
but turn it into a redis module:
[https://gist.github.com/antirez/a3787d538eec3db381a41654e214...](https://gist.github.com/antirez/a3787d538eec3db381a41654e214b31d)

