

Node.js Versus LUA "Hello World" HTTP Server Showdown - s1m0n
http://simonhf.wordpress.com/2010/10/13/node-js-versus-lua-hello-world/

======
silentbicycle
It's _Lua_ , not _LUA_. Not an acronym. (It means "moon" in Portuguese.) Could
you correct the title?

Also: Serving "Hello World" is not a very informative benchmark, especially
for an async web stack. Something like making an asynchronous database request
and then sending the response may be more representative. If you want to
eliminate the database response time variable, then write a C program that
listens for incoming connections and responds with "Hello world" or something,
but either way, make it actually do some work server side.

~~~
simonhf
silentbicycle, thanks, I updated the name as suggested.

So do you have any ideas for a more interesting benchmark program? Ideally it
should be something which needs to keep state as you suggest and makes use of
some kind of simple business logic, and is only a few hundred lines of code at
the very most. What about a simple chat server which handles people and rooms?
The benchmark might handle, say, max. 10k people chatting in max. 1k rooms.
What do you think? Any better ideas?

~~~
silentbicycle
I think a server that takes a login and password, sends the pair to another
core (or server) to do a bcrypt (<http://www.openbsd.org/cgi-
bin/man.cgi?query=bcrypt>) check, and then responds with a pass/fail would be
a good benchmark.

An async web stack benchmark server should juggle loads of concurrent
connections while making internal requests to another process or server, and
then sending the response (when available). That's representative of a lot of
common server tasks, yet can control for the work itself.

I'm suggesting bcrypt in particular because the amount of CPU time per client
can be easily controlled. (In a nutshell, bcrypt is a hash function which can
be made arbitrarily slow to deter password cracking.) Offhand, I'm not sure
which systems have bcrypt besides OpenBSD, though.

FWIW, I have a simple bcrypt wrapper for Lua
(<http://github.com/silentbicycle/lua-bcrypt>).

There's also a Ruby bcrypt wrapper (<http://github.com/codahale/bcrypt-ruby>)
which includes a copy of the bcrypt source from OpenBSD. I feel vaguely uneasy
about including that with my Lua wrapper (it really should be its own library,
and more widely available!), but may eventually do so.

~~~
simonhf
silentbicycle, I like your idea. I wonder if it will be considered 'enough' by
those who poopooed the "Hello World" benchmark? On a dual core or better box
and assuming the bcrypt server replies immediately, I would predict that
performance of the main server would halve. Why? Because the main server can
only handle n events per second and by communicating with the bcrypt server
we're doubling the number of read events which must happen within the same
wallclock time. What I've found is that handling e.g. epoll events is slow
compared to e.g. regular function calls or other IPC mechanisms on the same
box. So the fastest way to do the bcrypt idea on a production server would be
to use non-socket-based-IPC (e.g. shared-memory-based which is an order of
magnitude faster than socket-based-IPC). Anyway, assuming we go with socket-
based-IPC for the purposes of the comparison, what's the fastest way for the
node.js program to maintain state and handle a pool of connections to the
bcrypt server? It would be great if a node.js guru would take this on... and
then I would take on the SXE and SXELua implementations. What do you think...?

~~~
silentbicycle
bcrypt doesn't reply immediately. You could set it to require ~10ms of CPU
grinding each time it checks whether a password is valid, for example. Having
latency while work happens on another core or server shouldn't impact the
total number of concurrent connections the server can handle, though - the
benchmark should measure that, specifically.

Oh, also - the reads and writes to the bcrypt server will be really short, and
could use persistent connections. Something like
"$2a$07$8BzFlUuprZN4FSBpDx3ZAuPWOu4CZ3uv8Awa4EjAZhNmnIY59nh2e" -> "OK". And
yes, it would increase the number of events, but a real server is probably
going to be communicating with a database, memcached, etc. instead.

FWIW, this comment thread is getting buried deeper and deeper in my threads
page - if you want to keep the conversation going, my email address is in my
profile. Have a good weekend.

------
mrb
His fastest "node.js+net" case is not optimal. I could easily come up with an
even faster implementation by using Buffers instead of strings by not calling
setEncoding(), which makes it 30% faster on my machine. This should translate
to about 37500 queries/sec on his machine:

    
    
      var net = require('net');
      var server = net.createServer(function (stream) {
       stream.on('connect', function () {});
       stream.on('data', function (data) {
          var l = data.length;
          if (l >= 4 && data[l - 4] == 0xd && data [l - 3] == 0xa && data[l - 2] == 0xd && data[l - 1] == 0xa) {
           stream.write('HTTP/1.0 200 OK\r\nConnection: Keep-Alive\r\nContent-Type: text/html\r\nContent-Length: 13\r\n\r\nHello World\r\n');
         }
       });
       stream.on('end', function () {stream.end();});
      });
      server.listen(8124, 'localhost');

~~~
mrb
And this other version that doesn't check for "\r\n\r\n" is 47% faster than
his "node.js+net" case. This should translate to about 42000 queries/sec on
his machine:

    
    
      var net = require('net');
      var server = net.createServer(function (stream) {
       stream.on('connect', function () {});
       stream.on('data', function (data) {
           stream.write('HTTP/1.0 200 OK\r\nConnection: Keep-Alive\r\nContent-Type: text/html\r\nContent-Length: 13\r\n\r\nHello World\r\n');
       });
       stream.on('end', function () {stream.end();});
      });
      server.listen(8124, 'localhost');

~~~
kurokikaze
Why empty callback on 'connect'? It will go to event loop for every client,
and yet do nothing of value.

~~~
mrb
Good point. I inherited this design flaw from the snippet he posted on his
blog.

~~~
simonhf
mrb, please note that the connections per second and queries per second are
handled individually in the benchmark. So the connect handler should make no
difference to the queries per second.

------
jasonwatkinspdx
I'm frustrated by Lua, because it seems such a productive, pragmatic language
that is performant to boot (let alone the numbers LuaJIT is posting), but yet
it never seems to gain momentum.

Why has Lua adoption been so slow outside the game world?

~~~
silentbicycle
Lua _is_ gaining momentum, but it's been a very slow burner. There are a lot
of programs (mostly games) that use it, and they aren't required to be open
about it. Lua also improved quite a bit (IMHO) between 4.0 and 5.1, but 5.1
has only been out since 2006.

Some things omitted from Lua's std libs because you'd typically just do them
in C anyway, and it keeps Lua very small. While you don't _need_ to know C to
use Lua (and there are community libraries for them, see LuaRocks:
<http://luarocks.org/repositories/rocks/>), it really shines in symbiosis with
C.

Most web devs don't know C, though, so it misses quite a bit of the free hype
that would come from word of mouth in _that_ community (look at e.g. Ruby),
especially compared to word of mouth in the (NDA-infested) gaming industry.

FWIW, I don't think it would take long for somebody proficient in Javascript
(or Python) to learn Lua - This comment
(<http://news.ycombinator.com/item?id=1786280>) is a good summary. The
languages are very similar in overall design (prototypes, JSON/dict-style
objects, etc.), but where Javascript has numerous design bugs frozen in its
spec (thanks to the browser wars) and "The Good Parts" telling how to
sidestepping them, Lua had 15 extra years to evolve and _fixed_ them.

Recommended intro: Ieursalimschy's _Programming in Lua, 2nd ed._
(<http://www.inf.puc-rio.br/~roberto/pil2/>). The first edition is free
online, but covers 4.0, and the language changed a lot (most notably, the
packaging system).

~~~
pygy_
Nitpick: The first edition of the PiL Book covers Lua 5.0 which is very
similar to 5.1.

The most noticeable changes are the vararg handling, the module system (as you
said, but it will be phased out in v5.2), and some library changes.

The differences are listed here: <http://www.lua.org/manual/5.1/manual.html#7>

That said, I purchased the second edition to understand the module system,
only to decide not to use it anyway :-). I don't like the way the `module`
function pollutes the global namespace.

~~~
silentbicycle
5.0, not 4.0 - you're right, thanks. Missed the edit window. :/

And yes, the 5.2 environment handling sounds like an improvement to me as
well.

~~~
pygy_
I liked setfenv() for its versatility.

I implemented a toy markaby [1] clone in Lua using it.

    
    
        tpl = html(function()
            head(function() --> tag names should be non-locals. They will be rebound at runtime.
                title "Barbazor"
                link {rel = "stylesheet", type = "text/css", href = "style.css"}
            end
            body(function()
                h1 {"Big title"; id = "title"}
                div {class = "foo"; function()
                    for i=1,5 do
                        p( "counting: " .. i )
                    end
                end}
            end
        end
        
        render( tpl ) --> returns clean, well indented HTML.
    

I don't think it can be done with the new environment system without
reimplementing setfenv. You could achieve the same effect by passing an
explicit param to anonymous functions, but it would look worse IMO.

This example also highlights a weakness in the language syntax: lambdas are
ugly. Functions are first-class, semantically, but not syntactically, which is
sad.

[1] <http://markaby.rubyforge.org/> , just in case ;-)

~~~
silentbicycle
Hah. I did exactly the same thing. I usually use discount instead, though. To
my understanding, it'd work to set _ENV's metatable's __index to a table w/
HTML tag generators. I've read about _ENV but haven't actually used 5.2-work4
yet. (I started using 5.2-work2 but they went and changed it again, so now I'm
waiting until it's a sure thing.)

I agree that the syntax for first class funs could be more concise, but that's
true of any language that requires a "return" keyword (i.e., is statement-
based rather than expression-based). I think "fun x -> x + 1 end" would be
better, but maybe that's just the OCaml talking.

~~~
pygy_
Using _ENV would not be that simple.

    
    
        * The library and template files may have different _ENVs. 
        * The _ENV of the tempalte file could already have an __index.
    

You'd have to add something like `_ENV = luhtml.makeENV(_ENV)` before your
template definitions to take care of this.

The only way to have a clean template file would be to emulate setfenv using
the debug library.

\--

My main gripe with the current syntax is that, when scanning the code,
function declarations and invocations are too similar.

I'd like to have somthing like

    
    
        :[x] print(x); -> x+1 end 

or

    
    
        :]x[ -> x+1 ]

Both are unambiguous, and would allow to pass a lambda to a function requiring
a single argument without parenthesis, like you can already with strings and
table litterals.

    
    
        somefun :]x[ dosomething( x ) ]

I prefer the second syntax because of the initial happy smiley, although it
the reversed brackets are alien and would cause an uproar if they were to be
added in any popular language.

\--

At last, what is discount?

~~~
silentbicycle
Discount (<http://www.pell.portland.or.us/~orc/Code/discount/>) is a very
fast, C-based implementation of Markdown
(<http://daringfireball.net/projects/markdown/>).

I have a Lua wrapper for it, but just noticed that I don't have it on my
github page. I'll post it soonish. Somebody else has a Lua wrapper for an
older version of discount, but includes the discount source in it (!), and
it's been updated due to security-related bugs, so that's out.

Talking about reversed brackets, look at J (<http://www.jsoftware.com/>). The
only language I've seen that uses ], [, {, and } as unpaired operators. Pretty
serious heresy. Cool language, though.

~~~
pygy_
Thanks for all that :)

------
malkia
LuaJIT 2.00 calls "C" functions a bit slower than reference lua or luajit
1.00. Mike Pall explained that in the newsgroup, if I'm not wrong. But the
idea with LuaJIT is to use "C" much less.

~~~
pygy_
IIRC, code sections with foreign call cannot be traced by the compiler... Or,
if they can be, the performance would be worse than not doing it (my memory is
a bit foggy).

A custom API for LuaJIT is in the works, to alleviate this problem. Extensions
will have to be written twice though, if one wants to support both runtimes.

------
amix
What versions of node and Lua are used in the benchmark? Where can SXE be
found (Google can't seem to find it)...?

------
rbranson
Let's remember that Node.js is more about predictability, scalability, and
(most of all) simplicity of development, than performance. Lua(JIT) is indeed
impressive, but having to learn another programming language is a huge
downside.

~~~
KirinDave
Attitudes like yours make me sad, on many levels.

It also makes me mad, because by every metric save the what-you-already-know,
Lua is _at least_ comparable, if not outright superior.

~~~
maushu
There is also the code reuse factor. Why learn/use yet other language if you
can use javascript for both server and client?

~~~
gurraman
Better yet, let's go back to using ALGOL or something. Imagine how many
reusable components we could have mustered up. We'd best stop all this
innovation right now before it's too late.

~~~
maushu
Hmm, browsers don't use ALGOL though, better stay with JavaScript.

~~~
gurraman
Yes, but if the old geeks had been arguing like you did, the browsers would
probably only support whatever language was popular on the server-side at that
time.

