

Improve your beta-testing with Nginx, Lua and Redis - transmit101
http://devblog.mixlr.com/2012/06/26/how-we-use-nginx-lua-and-redis-to-beta-ify-mixlr/

======
runningdogx
With a modern pcre library version and nginx, instead of

    
    
        location ~ ^/([a-zA-Z0-9_]+)/ {
          set $username $1;
    

you can use named captures

    
    
        location ~ ^/(?<username>[a-zA-Z0-9_]+)/ {

~~~
newman314
Newer versions of pcre and nginx also include the ability to use the pcrejit
which provides a nice speedup for regex.

------
richardv
This is a fantastic explanation. We have started to do something similar but
your LUA rewrites is a lot more effective. I wasn't even aware that you could
do something like that.

------
eli
I've done very similar things with Varnish reverse caching proxy. And as a
bonus, Varnish is crazy fast if your pages or assets are cacheable.

Its scripting language is very simple (not quite as nice as Lua, IMHO), but it
supports multiple backends and rewriting arbitrary headers:
<https://www.varnish-cache.org/trac/wiki/VCLExamples>

~~~
mqzaidi
We have also done similar things to test our new platform, using nginx and
cookies, by sending part of the traffic to the new platform to be beta tested.
Didn't need to use lua though. <http://qzaidi.github.com/2012/03/11/nginx-ab/>

------
jperras
We've done this before, but by adding an additional load-balancer that
forwards requests to a different port (e.g. 81 instead of 80) depending on the
subdomain that was accessed. The request is processed by nginx, which has
separate server blocks and associated upstreams for processing requests that
come in over the configured ports. The only application logic that was
necessary to accommodate the changes was a check to determine if the user
should be part of the beta, and redirect them to the appropriate subodmain if
necessary.

While it's not as clean as having the check done in nginx itself via Lua
script + Redis, it's relatively low friction. That said, the Lua script
embedded in the nginx config is very slick, and I had been contemplating
writing something similar using the nginx memcache module. Nice work.

~~~
transmit101
Yes that works too as I mentioned in the first couple of paragraphs of the
post.

However having a beta.mixlr.com domain a) introduces a lot of friction and
seriously reduces the ease with which you can test features b) may alter the
perception or expectations of a visitor, making testing less valid.

We've got a lot of users and we want to make the roll-out process completely
transparent and inclusive.

Hence this solution.

------
grogs
Good write up for a sensible solution to a problem which should effect lots of
sites.

Personally, I would probably take the approach of settings a cookie per user.
This would be very suitable when beta testing is enabled/decided by users,
rather than a subset being chosen A/B style based on a value in the database.
This would have the advantage that nginx would not have to make any database
requests; just check a cookie on the request, likely speeding up the speed at
which nginx handles requests. However, it would require some changes to the
application (versus [small] changes to the database which the slug approach)
to set the cookie etc.

I wonder how easy this is to do in apache.

~~~
transmit101
Cheers. Yeah a cookie approach would be suitable for a lot of uses, including
A/B testing. It's easy in Nginx/Lua:

local is_some_cookie_set = nginx.var.cookie_SomeCookie ~= nil;

Then you don't need Redis at all. For us though, we want to separate users by
the URL and then a check at an application level variable, so I think this is
the right approach still.

------
projct
Couldn't you use a cookie and map, rather than URLs? Sounds like users could
end up on different rails back ends unintentionally based on what URL they
use, with your method...

[http://serverfault.com/questions/268633/controlling-nginx-
pr...](http://serverfault.com/questions/268633/controlling-nginx-proxy-target-
using-a-cookie)

~~~
transmit101
As I explained in the post, that's the intended behaviour.

We want to allow our users to enable beta version for all visitors to their
Mixlr page, not just for themselves.

But a cookie approach would be easy too - see my other comment for a simple
example.

~~~
projct
ah, I see. I guess I didn't quite understand the product enough. Neat idea,
and thanks for clearing that up.

However I wonder if map is faster than using lua just for cookies. I'll have
to test it. :)

------
davnola
Lovely. We do something very similar in front of EC2 clusters with
<http://openresty.org/>

------
Androsynth
rather than: if (-f $request_filename)... you should use try_files

<http://wiki.nginx.org/IfIsEvil>

<http://wiki.nginx.org/Pitfalls#Using_If>

~~~
transmit101
Thanks, it's a good point, I will update the post later.

