

Preforking echo server in Python, inspired by Unicorn-is-Unix - RyanMcGreal
http://jacobian.org/writing/python-is-unix/

======
sant0sk1
RyanMcGreal must not have taken too much time reading the linked article,
because this is NOT Unicorn ported to Python. It is a port of the very simple
example server written in Ruby in Ryan Tomayko's article about Unicorn.

See: <http://news.ycombinator.com/item?id=865306>

At the end of Jacob's post he states that he'll be spending his evening
_reading_ Unicorn's source. Not porting it.

~~~
RyanMcGreal
You're quite right. I've corrected the title to reflect this.

------
jrockway
I'm mostly commenting on this article's linked article, but...

Prefork is great if you have about two or three concurrent connections, and if
you aren't using a dynamic language like Perl, Python, or Ruby. Otherwise,
lightweight threads will serve you better.

One problem that people have (had?) with persistent preforked mod_perl apps is
that copy-on-write also copies on reads for certain Perl structures. Say you
have "my $foo = 42" and then you fork. The Sv associated with $foo is shared.
But when the child says "if($foo eq ...)", the Sv is no longer the same; it's
changed from an SvIV to a SvPVIV. No more sharing. (Maybe Python and Ruby are
more clever here, but they pay a speed price on non-forked applications for
this.)

Eventually you end up with two completely different memory images for the same
app instead of the 1 copy of the parent shared with the child.

As for the "about two or three concurrent connections" part, I think you will
find that "select" exists for a reason. ("select" is just as much a part of
UNIX as "fork", BTW.) If you have a bunch of mostly idle filehandles to read
from, select (or rather, its modern replacements) will perform much better
than a bunch of forked processes. Imagine you are writing a push messaging
server; do you really want one process for each of your 1 million users? No;
you want a lightweight thread for each.

(Try this sometime; create 1,000,000 lightweight threads. Then create
1,000,000 processes that do nothing. Then note which one requires you to
reboot your computer by yanking the power cable.)

~~~
btilly
Um, no. Your advice is completely backwards.

Perl does NOT have a lightweight threading model. Period. Literally every time
you spawn a thread, Perl copies all of its data structures to avoid sharing
stuff that is not threadsafe. If your Perl developers try to tell you
otherwise, they are incompetent. The standard way to build a high performance
website in Perl is to use prefork and then put a reverse proxy in front of
them in httpd accelerator mode. See
<http://perl.apache.org/docs/1.0/guide/strategy.html> for an old, but still
accurate, description of httpd accelerator mode if you don't know what it is.

Please note that Windows does not have good support for forking. Therefore I
have never heard of a competent team of Perl developers choosing to try to
deploy a high volume website on Windows.

To the best of my knowledge both Ruby and Python do support lightweight
threading, unlike Perl, but both have a global interpreter lock that
synchronizes very frequently, and severely limits scalability. I don't know
what strategies are common in those languages, but personally I suspect that
the right strategy on Unix with those languages is, like Perl, to use pre-fork
with a reverse proxy in httpd accelerator mode.

However both Python and Ruby have implementations on other platforms, such as
the JVM. I _believe_ (verify this before relying on it) that those
implementations support lightweight threads without the drawbacks of the C
implementation. Therefore if you want to use lightweight threading with that
class of scripting language, use one of those implementations.

~~~
jrockway
Perl's lightweight threading is provided by Coro.

~~~
btilly
Coro is just a cooperative asynchronous framework on par with, say, POE. You
do _not_ get to use multiple CPUs. You do _not_ get to use a database. You do
_not_ get to use a normal multi-threaded webserver.

I would be suspicious of anyone seriously trying to create a high volume Perl
website using Coro.

~~~
jrockway
You are right about multiple CPUs. For that, fork and load-balance. Instead of
each forked process handling one connection, it handles a few thousand.

If your app is CPU intensive, I have to wonder why you'd use Perl for that.
You get a 2x speedup by using multiple cores, but a 50x speedup by switching
to Haskell or Common Lisp for the critical section. (You could also use C or
C++ or Java, but that's just being crazy.)

As for databases, most real databases have non-blocking interfaces; this means
database queries won't stall your threads. (Postgres and BDB are known to work
well. MySQL requires hacks.)

And yes, you don't get to use a normal multi-threaded webserver. I am not sure
how that works.

This all scales quite well.

~~~
btilly
Yes, you can use some databases. But I don't think you can use the classic DBI
interface.

More problematic, though, is that a single poorly coded function call on a
seldom hit page can seriously impact responsiveness for a large fraction of
your website. With real threads or processes you can protect against a
function with a memory leak with resource limits. (You can set that up
BSD::Resource with Perl on Unix systems.) If you try that with Coro you risk
taking down a large fraction of your website each time a bad function runs.

If you have a small team and absolutely trust their work, this works. But as
you scale up in complexity, mistakes will happen. You will have problems. And
your choice of cooperative multitasking will look worse and worse.

Cooperative multitasking is not a new idea. In fact it is usually the _first_
thing people try. Windows through 3.1 (continued under the hood in the 9x
line). Apple did it through OS 9. Ruby did it through the 2.8 line. Yet in
every case people run into the same problems over and over again and conclude
that they are better off with preemptive multitasking. Even badly done
preemptive multitasking such as Windows 95 or Ruby 2.9.

Again. I would be suspicious of anyone trying to create a high volume website
these days in Perl relying heavily on cooperative multitasking. I'm not going
to say that they won't succeed. But they are setting themselves up for
problems down the road.

Furthermore it isn't as if there is a real problem that needs solving here. A
single decent server properly set up can serve enough dynamic content from a
single machine to put you in the top few thousand sites. Buy more machines and
you can scale as far as you want.

~~~
jrockway
We have preemptive scheduling in Coro. See:

<http://github.com/nothingmuch/coro>

Also available from CPAN is EV::Loop::Async, which allows events to be handled
even when Perl is busy. (It uses a POSIX thread for this.)

(The key to success with Coro is using the right libraries. You write what
looks like blocking code, but the libraries make it non-blocking.)

Anyway, the end result is that you use a lot less memory to handle a lot more
clients. This may not be an issue if every request is CPU-bound, but you'd be
suprised how often your process is blocking on IO, and how many resources a
process-per-connection model consumes.

Don't let this become another lecture on functional programming:
<http://www.perlmonks.org/index.pl?node_id=34786>

~~~
btilly
That looks cool, but it looks like that patch is not in the CPAN version. I'm
not sure how much I'd trust it. Particularly if you'd loaded some badly
behaved XS code. Or run a disaster RE. For instance I ran into one last week
which took down Perl 5.8. Losing one mod_perl process occasionally was only an
annoyance. Losing a good fraction of my site capacity would be much worse.

EV::Loop::Async lets you handle events, but won't solve the problem of, "I
loaded an external library, and it didn't return control for 10 seconds."

Neither addresses the problem of protecting yourself against badly behaved
functions that have a fast memory leak.

BTW you're assuming wrong when you assume that I'd be surprised at how often
my processes are blocked on IO or how much resources they take. I am painfully
aware of both factors. However it is easy to plan for that. I've personally
seen 2 servers pump out a million dynamic pages/hour with real traffic on a
website with only obvious optimizations. I know for a fact that the
application code had memory leaks, bugs, and the occasional segfault. I'm
happy to buy 4x the RAM to go with an architecture that makes those non-issues
to the overall function of the website.

~~~
jrockway
Maintaining a hacked-up-piece-of-shit is a different problem from starting
from scratch and Doing Things Right. In the situation that you're in, you
probably made the right decision -- throw RAM at the problem so you never have
to think about it again.

When writing an app from scratch, though, you have some control over the
quality of the code, and can aim to serve more users with less hardware.
System administration is hard, and the less systems to administer, the better.

~~~
btilly
Assuming that your codebase will continue to be a work of elegance is
challenging. Particularly if you're loading CPAN modules that are written and
maintained by other people to a different standard. Of course if you reject
those CPAN modules, then what's the point of writing Perl?

But, you say, we'll just limit ourselves to high quality CPAN modules? The
real standard ones that everyone uses? Surely nothing will go wrong?

Fine. Last week I ran into a segfault coming from the Template Toolkit
triggering a regular expression bug in Perl. (I am waiting on the bug report
until I get official permission to submit the patch with the bug. I'm careful
about copyright these days...) That's about as standard as you can get. Assume
that an extremely popular pure Perl text manipulation module on top of Perl
works as documented and enjoy the core dump.

The moral is that unless you are personally writing the whole software stack
you're using, you never know what will trigger a bug somewhere. And no sane
web company is going to rewrite their whole software stack. (For the record
the most painful bugs in the application I described previously were at the C
level, and none of that code was touched by anyone in that organization.)
However there are architectures that let you mitigate classes of problems
before they come up. What is that protection worth to you? Given how much
traffic you can get per server, what lengths do you need to go to to optimize?

------
ktharavaad
If you want to know more about forking processes vs threading. You should look
at TCPServer.py in the standard libary. You can look at the ForkingMixin class
and the ThreadingMixin class.

A good Python server source code to read is the CherryPY WSGI server. You can
read it in the web.py git directory here.

[http://github.com/webpy/webpy/blob/master/web/wsgiserver/__i...](http://github.com/webpy/webpy/blob/master/web/wsgiserver/__init__.py)

------
pwmanagerdied
Anyone who knows the standard libraries could do this in a few minutes, this
post is clutter. :(

