

Adding Concurrency to Rails Apps with Unicorn - craigkerstiens
https://blog.heroku.com/archives/2013/2/27/unicorn_rails

======
siong1987
This sounds really cool but one thing that this blog post didn't mention is
that there is memory constraint on each dyno. So, this model might work if you
have a Rails[1] app that doesn't use up a lot of memory. And, since Unicorn
forks as worker processes, so, 2 unicorn workers will use up double the memory
of a typical Rails app. If you have a really huge Rails app, you could still
end up using only one unicorn worker for your Rails app, which is the same as
just using any webserver on one dyno.

One interesting thing about the post is the mention of Puma[2] as the
webserver for Rails app. Puma is thread based, so, its memory footprint is
significantly less than unicorn. So, in theory, you should be able to handle
more concurrent requests(more threads) per dyno with Puma.

Btw, don't just blindly follow the post. Make sure you test your app and see
whether your app use up more memory each dyno allows. If your dyno runs out of
memory, Heroku will just silently drop all your requests. So, you might end up
having a worse problem.

[1]: In general, Rails uses up quite a lot of memory on cold start. [2]:
<http://puma.io/>

~~~
btilly
_And, since Unicorn forks as worker processes, so, 2 unicorn workers will use
up double the memory of a typical Rails app._

Thanks to copy on write, the amount of memory needed for 2 processes is
strictly less than 2. Just make sure to preload as much data as you can before
you fork. And in many use cases memory slowly comes unstuck (change anything,
and a whole page of data comes unshared) so it can make sense to kill workers
every so often and let them refork with everything properly shared.

~~~
pixelcort
Will Unicorn automatically kill older forked processes periodically and fork
off new ones, or is that something that would need to be done separately?

~~~
swampthing
One of the comments on the Heroku article mentions this gem, which does just
that:

<https://github.com/kzk/unicorn-worker-killer>

~~~
kawsper
Regarding memory, I think you can do something like this in your Unicorn
config:

after_fork do |server, worker| size_in_bytes = 600 _1024_ 1024 # megabytes
Process.setrlimit(Process::RLIMIT_AS, size_in_bytes)

------
scottshea
I am glad to see them mentioning this. We use Unicorn, and have for quite a
while, as a means of lowering the bottlenecks and slow response times. I
strongly suggest looking at Unicorn Worker Killer too
(<https://github.com/kzk/unicorn-worker-killer>) given the cap on memory for
the dynos

~~~
eminh
For those interested in doing monitoring outside of the process itself, we run
this command every minute via cron

ps -o rss,pid= -U username | sort -k 1 -nr | head -2 | awk -F: '{FS=" ";if
($1>limit*1000) print $2}' limit=`jot -r 1 144 192` | xargs kill -QUIT

It will gracefully kill 2 most memory-consuming unicorn workers above random
range of 144-192MB of memory. It proved to be extremely efficient and simple
solution to our needs.

------
cmelbye
Is it just me, or is this not at all the same kind of node.js-style
concurrency that makes sense Heroku's new router? Using Unicorn simply means
that you're firing up a few more workers, it's only slightly better than
buying more dynos.

~~~
jrochkind1
It is not at all node.js-style concurrency.

It is probably about the same as buying more dynos -- I would think that
unicorn set to fork three times is actually about the same as three dynos with
thin in standard configuration -- as you say, either way it's three processes,
each of which can only handle one request at a time.

It is _probably_ not a solution to the bad worst-end latencies that can occur
when you have random (or round-robin) routing, and relatively high variation
in request durations.

Using a multi-threaded request dispatch (NOT what unicorn does, but what they
start hinting at towards the end) _may_ be a solution to that though. Although
multi-threaded concurrency is STILL not "at all the same kind of node.js-style
concurrency", as node.js is evented, not multi-threaded. But multi-threaded
concurrency seems likely to make sense with random or round robin routing too.
Although more research and data is called for (at least more than I have,
which is zero).

But there's no reason to assume that node.js-style evented concurrency is the
only thing that can possibly make sense with random routing. And if it is,
that's very inconvenient. Trying to do node.js-style evented concurrency with
a Rails app... or any app that wasn't built from the ground-up (including all
it's dependencies) for evented-style concurrency... or even if it WAS... it's
non-trivial.

In contrast, multi-threaded request dispatch requires not a lot of change to
your code -- basically just avoiding shared in-memory state between requests
(basically, global/class variable access).

In addition to puma, thin in fact does have a multi-threaded mode, although
it's poorly documented.

I think they are actually passing the buck suggesting "We can't tell you which
way to do multi-threaded request handling, it depends on your app!" Well,
sure, it always depends on your app, but that applies to their suggestions to
use unicorn as well, but that didn't stop them. There are still often general
best practices for typical web apps.

I think it's true that they can't tell us the right way to do multi-threaded
request handling -- because it requires more research and analysis and
experimentation (and possibly a patch or two) to figure out the best practices
here, the community hasn't put enough into it yet. But if anyone's got the
resources to put into it, it's heroku, and it would be rather good for their
business to figure this out and educate the community. Heroku became so
respected because they seemed to really know what they were doing, to be at
the top of the game -- if they can't manage to figure this out either, it
lowers our trust in them.

~~~
bitcartel
I think the OP is referring to the fact that Node is also single-threaded and
you have to use the Node Cluster module to fork child processes.

------
habosa
So after reading this I can't see a single disadvantage to using Unicorn. Is
there any reason why I shouldn't immediately switch from thin?

~~~
bretthopper
Unicorn is only meant for "fast clients" and they specifically say that "slow
clients" should be served by a reverse proxy like nginx. I put those terms in
quotes since they're subjective. But I've read that Fast Clients are only
those on a LAN. So basically, Unicorn should NEVER be directly exposed to a
user.

Problem is, Heroku doesn't run a reverse proxy anymore on its Cedar stack.
Rainbows[1] is a server based on Unicorn but designed to also handle longer
requests/slow clients.

[1]<http://rainbows.rubyforge.org/>

~~~
habosa
Why does the speed of the client matter?

~~~
bretthopper
This is a good read about it: <http://unicorn.bogomips.org/PHILOSOPHY.html>

------
malyk
If you are using the NewRelic add-on you can go to the "dynos" tab and see the
memory footprint of your application over time. The Heroku memory cap is
512mb, so if you stay a decent amount of space clear of that then you are
likely to be ok running multiple workers per dyno.

For instance, our app which I'd say is a medium sized rails app, generally
consumes between 175mb and 210mb of memory. So we /should/ be safe running 2
workers per dyno which would effectively double our processing power.

HOWEVER, if you hover over the memory graph it'll tell you the min/max/avg
consumption over that particular time period. Looking at that I see we
sometimes spike the max over 350mb. The question is, what is that spike going
to look like running unicorn. Unfortunately, I can't think of many ways to
accurately test/predict that without simply giving it a try.

One other thing that was mentioned in the comments of the article is that you
can set the unicorn backlog to a very low number (default appears to be 1024)
which will apparently tell the heroku router that a particular dyno is "full"
and to look somewhere else to process the request. One comment recommends
setting the backlog to 25. I wonder what setting the backlog to something like
#workers*2 would do.

------
hayksaakian
Check out the unicorn-rails gem if you'd rather not fiddle with config files.

~~~
nwienert
I was just checking it out but I saw there were a ton of forks and a number of
seemingly important issues. I was hesitant to use it after seeing that.

~~~
molecule
_...there were a ton of forks and a number of seemingly important issues..._

I see no open issues, 2 closed issues, and 5 forks, in which two or three
configuration settings are tuned.

<https://github.com/samuelkadolph/unicorn-rails>

~~~
nwienert
Ah, my bad. Confused with this:

<https://github.com/sosedoff/capistrano-unicorn>

------
nifoc
Since the Unicorn documentation explicitly states that you should not let it
handle slow clients (everything outside your datacenter) directly, using
Rainbows![1] might be a better solution.

[1] <http://rainbows.rubyforge.org/>

~~~
markov_twain
Heroku apps at the very least sit behind the heroku router, which I would
assume is in the same datacenter. If the information in this quora answer
<http://www.quora.com/Scalability/How-does-Heroku-work> is still valid, your
app is also behind a front-facing nginx reverse-proxy.

I'm not sure that actually means it's handling slow-clients, though.

~~~
nifoc
I think the important part is that the reverse-proxy should buffer the
request, but I can't find anything about wether or not the Heroku router does
this.

------
maxpow4h
GitHub wrote about this back in 2009.

<https://github.com/blog/517-unicorn>

------
obilgic
This (concurrency) makes heroku's routing system little more bearable...

