
Creating a Redis Module in 15 Lines of Code - dvirsky
https://gist.github.com/dvirsky/83fc32366d5ad82fc3dca47ed2704377
======
jasonjei
I'm trying to be cautious yet open-minded and optimistic with Redis modules.

Redis is such a well-written C software that I can't help but think modules
might become an antipattern to Redis. I like the idea that the option to use
modules exists, but I hope it doesn't spiral into dependency hell as it is
with the Ruby ecosystem. I hope library developers think very hard before
employing and requiring a module.

Redis is fast, easy to use, and extremely flexible, and makes it easy to use
any of the hosted options (Redis Labs, RedisToGo). Its allure is that Redis is
batteries-included yet small enough like a Swiss knife. I wouldn't mind
installing a module if it brought in significant optimization for a specific
use case. I personally hope most library developers will pretend modules don't
exist, and that only app developers will use them. I have this selfish desire
that most libraries stick to the vanilla Redis features, because for most
things you can use app code to manipulate the features you need without
modifying Redis and compromising performance. Modules might add to the heap of
things to think about, such as special Redis deployments or the things that
can go wrong with the module code.

My fear is that instead of years of using the simple Lego blocks that is
vanilla Redis, modules will start a pattern where we need to install special
modules to use certain libraries.

TL;DR, I'm worried for the potential for abuse with Redis modules. My love for
Redis was how simple and powerful it is, and yet now Redis has grown up, ready
for modules.

~~~
spinlock
The only other C project I know of with a module system in Linux. I've
actually found that Linux modules make development _much_ easier because you
can isolate your code from the rest of the Linux code base. Of course, the
last time I was professionally developing with the Linux kernel, git hadn't
been invented yet and you really couldn't keep the entire kernel in version
control without going insane. So, maybe things are different now.

~~~
elcapitan
Are modules not pretty common? Apache and Nginx have modules like that as
well.

~~~
themartorana
Nginx's modularity and tolerance/embrace of plugins is absolutely amazing. I
use a combination of openresty[0], lua scripting, and third party modules in
our API servers (coincidentally, one that talks directly to Redis) and it's
still one of the fastest, best pieces of software in our stack.

I understand the concern, but if it ends up being half as flexible (and I
already love me some Redis) I can only see good things.

------
spinlock
What projects are people using Redis for? I'd love to know how others are
using Lua or plan to use the new module system specifically.

We're using redis as a queue (Sidekiq) and also to store the index.html page
for our SPA. We have an "active" key in redis and its value is the key for the
current index.html. This let's us push new versions of our app into redis
(each generation of our asset files has a unique name so a new index.html
gives you a new version of the SPA). We can then add a get param to our URL
and pull a specific version of the index.html out of redis. Once we've smoke
tested, we update our "current" value to point to the new index.html and,
boom, zero downtime deploys.

What's your story?

~~~
jasonjei
We use Redis as a record level distributed lock on top of pub/sub and a cache
and a queue. We also take advantage of expiring keys to detect if users are
connected or have activity. We used to use it a session store but switched to
encrypted cookies. I know some have used it as a semaphore as well.

~~~
doomrobo
What did the semaphore usage look like? Did it use pub/sub or something else?

Also what encryption algorithm are you using on your cookies. I ask because
certain algorithms produce malleable ciphertexts, which could potentially be a
huge vulnerability.

------
bndw
There are also lots of examples from Antirez himself in the source:
[https://github.com/antirez/redis/blob/unstable/src/modules/h...](https://github.com/antirez/redis/blob/unstable/src/modules/helloworld.c)

~~~
dvirsky
There's also the RedisModulesSDK repo that has a few helpers, plus a skeleton
module with a makefile:
[https://github.com/RedisLabs/RedisModulesSDK](https://github.com/RedisLabs/RedisModulesSDK)

------
shin_lao
I think it's a very good idea because it will reduce the pressure on the core
development team about feature X or Y. "You really want this feature? Write it
as a module."

Then it can serve as an incubator before importing functions into the core
library.

The only problem with modules is that poorly written modules may cause
unstability. One way to mitigate this is to have some sort of fuzzer/stress
tester for developers to test their modules against.

~~~
dvirsky
Totally agree. It's really easy to break redis if you're not careful. When I
wrote my module I made sure to do lots of random input testing, stress
testing, and running things in valgrind to make sure everything is kosher.

------
networked
That looks a lot like a Tcl extension [1]! Maybe I shouldn't be, considering
[2], but I still was pleasantly surprised.

[1] Example:
[http://www.omanurkka.fi/files/test.c.txt](http://www.omanurkka.fi/files/test.c.txt)

[2] [https://tcl.wiki/9497](https://tcl.wiki/9497)

~~~
dvirsky
Yeah, not surprising :) Also, the API sort of reminds me of the Python C API.

------
molecule
Obscuring links to code w/ shortened URLs is off putting and seems
unnecessary.

~~~
runchberries
Yeah. I wasn't going to click the link, but you got me curious. The link is:
[https://raw.githubusercontent.com/antirez/redis/unstable/src...](https://raw.githubusercontent.com/antirez/redis/unstable/src/redismodule.h)

I don't see why you would hide a link to the official redis repo.

~~~
molecule
This gist inspired me to learn that you can preview bitly links by appending a
+ to the URL

[http://support.bitly.com/knowledgebase/articles/136551-can-i...](http://support.bitly.com/knowledgebase/articles/136551-can-
i-preview-a-shortlink-before-clicking-on-it)

~~~
biot
Sure, but you're better off not using mystery meat links from Libya in the
first place.

~~~
dvirsky
I've added the real links beside the bitly links, thanks.

------
briankwest
I really like these types of examples but to fixate on the fact its 15 lines
of code, forgetting about the many thousands of lines of code below it that do
the actual work. Don't forget to thank your developers for making an API that
can show you how to use the library in 15 lines.

~~~
dvirsky
You are misreading my title. As someone who actually helped with building it,
the point was not to boast about it, but show how little you need to get a
module stub up and running.

Usually I hate those titles too because they show some "import gravity" type
of thing, but here the point was to just show that the boilerplate is roughly
15 LOC.

------
JustSomeNobody
What's the point of being so proud it only takes 15 lines of code? If it had
taken 20, would people have groaned and walked away? Examples like this aren't
"real world" anyway, so...?

~~~
sophacles
It's small enough to get a rough feel for how complicated a module would be.
It's small enough that it's like "hello world" \- test your toolchain (etc)
install. It's simple enough to start understanding the process without having
to read a bunch of docs. It's straight-forward, so there's no need to both
puzzle out the module interface AND someone's module logic.

There are probably plenty more reasons I'm not thinking of ATM.

~~~
JustSomeNobody
Ok, then say "Small redis module example to test your toolchain".

Exclaiming it only took 15 lines of code is mostly irrelevant.

~~~
nkozyra
"Small" is relative and doesn't tell me anything. "15 lines" tells me this is
quickly digestible and I can take a minute or two to look it over.

