
Snap: A Haskell Web Framework: Benchmarks - aliasaria
http://snapframework.com/benchmarks
======
acangiano
Let me state once again that the Rails benchmarks are absolutely way off. Snap
would still be faster - much faster - but you can expect several thousands
req/s from that Rails app on that kind of hardware.

~~~
rufugee
I did my own testing, but I didn't publish them because I left Haskell
completely unoptimized and didn't really have the time to try to optimize it.
I also added play (<http://www.playframework.org>) into the mix, because I
admire the framework and how closely it mimics Rails, but in Java (and Scala).

Still, I saved 'em, so here you go:

On a 2.27Ghz Xeon dual quad core with 12 GB ram, using Snap's tests from
GitHub:

    
    
      Test Pong:
      | Rails Webrick | Rails Mongrel | Rails Thin  | Rails JRuby (glassfish) | Snap          | Play!         |        
      | 155.0 req/s   | 1940.5 req/s  | 619.0 req/s | 3606.0 req/s            | 11347.0 req/s | 19602.2 req/s |
    
      Test File:
      | Rails Webrick | Rails Mongrel | Rails Thin  | Rails JRuby (glassfish) | Snap          | Play!         |        
      | Didn't test   | 188.3 req/s   | 718.0 req/s | 6633.3 req/s            | memory leak   | 606.6 conn/s  |
    

The full results are here:

Pong: <http://gist.github.com/414724> File: <http://gist.github.com/414728>

Note, Snap had a mem leak when performing the file test that caused a machine
with 12 GB to swap, but that may be because I used the Ubuntu Lucid version of
Haskell.

Also note, Play is damned fast. And relative to other Rails options, JRuby
with glassfish is as well.

~~~
mightybyte
You need to look at the average reply rate instead of the request rate.
Requests/sec can be misleading. The replies / sec tell quite a different
story. Here they are for pong:

Rails Webrick: 153.9 Rails Thin: 33.3 Rails Mongrel: 90.7 Rails JRuby
(glassfish): 3,350 JRuby Mongrel: 31.3 Snap: 11,309 Play: 19,796

~~~
rufugee
You're absolutely correct. Sorry. Here it is, updated properly:

    
    
      Test Pong:
      | Rails Webrick | Rails Mongrel | Rails Thin  | Rails JRuby (glassfish) | Snap          | Play!         |        
      | 153.9         | 90.7          | 33.3        | 3350.5                  | 11309.5       | 19796.3       |
    
      Test File:
      | Rails Webrick | Rails Mongrel | Rails Thin  | Rails JRuby (glassfish) | Snap          | Play!         |        
      | Didn't test   | 179.9         | 758.0       | 6146.9                  | memory leak   | 1617.9        |

~~~
barmstrong
Didn't realize Glassfish was that fast. Thanks.

And hadn't heard of Play! Will check into that.

Also, just as a counterpoint to all this - as PG pointed out in his book, with
Moore's law, speed of execution is no longer the limiting factor in most
software development. It is programmer time and productivity, where (I'm
guessing) Rails still wins this test. I'd expect that to be an ongoing trend.
But of course speed of execution is still necessary in special situations, so
this is still valuable.

~~~
cx01
I think Play+Scala should be competitive with Rails in expressiveness, while
still being a lot faster. Of course, if you prefer dynamic typing you might
disagree with me here.

~~~
rufugee
Personally, I've found it's very expressive and easy to follow, although I'm
using Java play. And having reliable autocomplete and ctrl+click navigation is
something I really missed in Rails, but have in Play.

------
rubyrescue
interesting. i would like to see a comparison to yaws or mochiweb (perhaps
w/nitrogen).

------
angusgr
Although these seem like useful (and impressive) baseline benchmarks, won't
the really important benchmarks come from measuring actual dynamic content
generation, and/or caching performance for generated dynamic content?

~~~
prb
When I did experiments with RoR versus Haskell via FCGI, Haskell had around a
two orders of magnitude performance advantage under load and only required a
single OS process.

------
warfangle
Would love to see a comparison to Node.js on here, as well.

~~~
matthijs
Indeed, for example the express framework appears to be optimized for high
performance <http://expressjs.com/>

