

Pancake HTTP Server faster than nginx running PHP - Hawkee
http://pancakehttp.net/

======
cheald
"Pancake is a fast, simple and lightweight HTTP-Server written in PHP with
acceleration for PHP-powered websites"

So...it's written in PHP, so it loads up a PHP application and serves it
faster than running through a real webserver to a PHP backend via FCGI. Duh?

Now go ahead and add in high concurrency and static asset serving and add PHP-
APC to the workers and don't benchmark on an OpenVZ VM (which doesn't
guarantee resources!) and let's see how things shake out.

Pancake seems to be touting this "CodeCache", which is great, except that PHP
already has APC, which does this already, and I'll wager it does it better.
This smells like a false benchmark, where Pancake wins because it's avoiding
work that the poorly-configured nginx stack has to do. If you were to tune
both of them to their maximum potential, I would be shocked if nginx didn't
eat pancake for breakfast.

~~~
pp3345
I am the developer of Pancake. I've found here after I got a mail saying that
Pancake was mentioned here.

I have to admit that benchmarking on an OpenVZ VM probably isn't the best
idea. However, at the time of benchmarking, my VM was the only one on the host
node and I've made sure that no other service on my VM could create high load.

I believe that CodeCache is more efficient than APC and easier to use. In a
future version I will also improve the CodeCache, allowing more code to be
cached than now possible. I am also planning on having a feature for automatic
configuration of the CodeCache.

~~~
cheald
Well, I was curious and had a bit of free time, so I set up a benchmark. I
just used Apache + mod_php with my all-the-extensions-and-the-kitchen-sink PHP
install versus Pancake with a pristine PHP install (since it needs special
compile flags) and the vhost provided for the nginx benchmark on the Pancake
website.

Results are here: [https://docs.google.com/spreadsheet/ccc?key=0Aqxu-
bQf_3O5dFd...](https://docs.google.com/spreadsheet/ccc?key=0Aqxu-
bQf_3O5dFdtT2ZZQUdCbE43X1BSelM3enZYV2c)

tl;dr: Pancake falls over completely at higher concurrencies. It even starts
dropping requests at -c2, and it just gets worse as concurrency gets higher. I
even tried running it with 16 workers rather than 4 (which should cover up
through -c9 easily), and it was still failing to return valid results.
Pancake's throughput goes up as concurrency goes up, but it's because it's
just dropping requests rather than serving them.

mod_php easily saturates my two cores with -c2 and remains consistent at ~285
req/sec. Pancake never really achieves better than 90 req/sec, and completely
fails to handle high concurrency.

Remember, this is Apache and mod_php, which is renowned for being the slow and
creaky way to run PHP. If I were to set up nginx and php-fpm, I'm pretty sure
I could push that chart even further.

I call bull on "faster than nginx".

~~~
pp3345
Okay, I've taken a look at your benchmark. The failing requests are due to a
typo in the configuration. It should be "phpsocketbacklog" instead of just
"socketbacklog". Pancake uses unix sockets for the internal communication
between RequestWorkers and PHPWorkers. When a RequestWorker tries to send a
request to a PHPWorker while the internal socket backlog is full, Pancake will
abort the request due to the lack of available resources. Therefore the
backlog is needed on high concurrency. I've updated the configuration at
pancakehttp.net including some additional performance optimizations.

Even though requests won't fail anymore with the fixed configuration,
throughput on high concurrency still isn't quite satisfying. This seems to me
like a bug as the PHPWorkers seem to process the requests perfectly concurrent
and finish each in about the same time, but then the delivery of the result to
the client is somehow delayed, probably due to a problem with the internal
communication. I've tested Pancake with php-fpm (Pancake supports FastCGI) and
high concurrency and it seemed to scale quite good.

I'm investigating on this problem and hope to provide a fix soon.

Regarding static file throughput, I know that Pancake does not really reach
the speed of other webservers here. Pancake was designed to execute PHP as
fast as possible and I still have a lot of ideas in my mind on how I could
further improve PHP execution speed. PHP itself isn't quite fast and therefore
it is difficult to develop a webserver in PHP that reaches the same speed when
serving static files as webservers written in languages like C. I am currently
experimenting with a new feature that will hopefully increase the speed of
static file serving but I still don't know if it'll really work.

~~~
cheald
I just don't think you're ever going to be able to beat nginx on the webserver
end of things because you have to contend with the PHP garbage collector and
nginx does manual memory management. And...well, C is a lot faster than PHP.

I fixed the backlog setting and re-ran the test, and pancake doesn't drop
requests anymore, but it's not improved at all on speed.

<https://gist.github.com/4063785>

I was watching top during each test, as well, to get a sense of what was
happening; pancake was pegging the CPU at close to 100% user and reached a
load average of ~14, whereas the mod_php test was something like 85% user/15%
system and reached a load average of 5. This would seem to indicate to me that
Pancake is handling something that Apache/mod_php is able to hand off to the
kernel, and since it's stuck in userspace, processes end up waiting around for
their turn to do stuff.

90 req/sec really isn't bad - especially considering that without APC, my
install gets ~35 req/sec - but given that it seems to get most of its boost
from avoiding re-evaluation of the source files, and given that APC does the
same, I don't think it's really all that fair to compare Pancake to a PHP
install not running APC, especially since Pancake requires a specially-
compiled version of PHP and has to be run as root, so anyone that can run
Pancake can install and use APC. Additionally, APC transparently works across
all your source files already - it has no manifests to maintain, so turning it
on is literally as easy as "pecl install apc" and enabling the extension in
your config. That's a difficult combination to beat. If you can do it, I'd
love to see the results, but given what I know of the architecture of these
various pieces, I'm skeptical that it can be done with pure PHP.

Finally, I think that trying to contend with nginx from a marketing standpoint
is silly, because web applications are composed of much more than just PHP
files; I have to be able to serve images, stylesheets, Javascript, and the
like for a full web application, so I have to consider the performance of the
webserver as a whole in terms of its ability to fully deliver a page to a
client to evaluate it properly; a webserver that runs the script end of
things, but then struggles to serve up my assets (and even worse, blocks
workers serving assets while I have script requests pending) isn't going to be
very useful in a practical scenario. I performed my benchmarks with my
development Apache install; I can tell you from my own experience that nginx
will trounce it solidly, so the gap to overcome is almost certainly
substantially wider than in my benchmarks. Considered as a whole, from first-
byte to request-fully-delivered, I don't think you're going to be able to
approach nginx, let alone beat it, without moving major parts of the
application out of PHP and into a lower-level system.

------
mxcm
The license for this web server is quite unusual,

<http://pancakehttp.net/license/>

I would expect a more sensible license.

P.S, that license page is so well hidden.. only possible to find by going to
the github page

~~~
pp3345
I am not a lawyer, I don't have a clue about law and licenses. Even though it
is prohibited to use Pancake for commercial websites with this license I won't
kill anyone because he is hosting a small blog making a few dollars with
AdSense or developing some small scripts for friends for little money. Just in
case someone would like to use Pancake for a online shop or something making
bigger amounts of money I'd like that person to contact me.

------
Adirael
Interesting but: This license does not allow using Pancake for commercial
websites. If you want to use Pancake for such sites, please contact support
[at] pancakehttp [dot] net

------
devicenull
Can we really trust numbers returned from a benchmark run on a VM? It seems
like the other load on the system during each benchmark would effect the
results.

------
ddorian43
The benchmark is testing a php application.

~~~
Hawkee
Thank you, updated title.

