
Lwan: A high-performance and scalable web server - jxub
https://lwan.ws/
======
kjeetgill
Jeez this is amazing. Just end to end, rock solid engineering. I came to chime
in that this basically hits all the same design beats as go with it's standard
library (relatively low foot print, compiled binary, async user-space
threading, etc.) But that sells this work short.

This is amazing work!

~~~
le-mark
I agree with the caveat I didn't read anything similar to go's work stealing
thread scheduling, useful for tasks with synchonized IO for example.

What appeals to me is the Lua integration. I'd love for a Lua application
server to become widely available. The http servers out there now don't really
fit the bill for (Xavante, Luvit for example).

~~~
kraftman
What's wrong with openresty?

~~~
le-mark
FYI "what wrong with X?" is not the same conversation as "Are you familiar
with X? It has these great features ...". Great that you want to promotoe
you're favorite tech, but there are more effective ways to do it.

------
nine_k
> Low memory footprint (~500KiB for 10k idle connections)

> Small disk footprint: x86-64 executable has 110KiB (~52KiB if packed with
> UPX)

> No copies between kernel and userland for files larger than 16KiB

Looks perfect for embedded stuff (and generally impressive).

It needs a separate TLS terminator, though.

~~~
acidx
Yes, the small footprint was a deliberate choice, specifically for the
embedded scenario. One of the things I did back in the day was to run Lwan+Lua
on an Intel Galileo board, without an SD card (it has only 4MB of flash
memory, which has to fit the Linux kernel, some minimal userland, and
Lwan+Lua).

TLS is hard given the current architecture, but I've been investigating kTLS,
so maybe it'll happen someday. I would appreciate help on this front, though.

~~~
petee
If you haven't seen, openbsd added a new api to simplify tls programming, "as
a response to the unnecessary challenges other APIs present in order to use
them safely."

I don't know how portable it is yet though

[https://man.openbsd.org/tls_init.3](https://man.openbsd.org/tls_init.3)

~~~
acidx
Ah, yes, I'm very aware of libtls. The API is indeed nice, but it would
require some changes in Lwan that I'd rather avoid; for instance, every socket
call would need to be virtual, and it would not be possible to use something
like sendfile() or mmap() a file and writev() it to the socket.

I'd like to use BearSSL, which is a nice, clean implementation of TLS, makes
no memory allocation (so it fits Lwan architecture very well)... but only for
the initial connection handshake; I'd then hand over TLS to the Linux kernel
with kTLS
([https://netdevconf.org/1.2/papers/ktls.pdf](https://netdevconf.org/1.2/papers/ktls.pdf)).
However, there's not much beyond some tests in that front.

~~~
bwindels
BearSSL is a great library, I also implemented TLS support with it recently in
my low-resource rust webserver project wwwee [1]. It's one of the nicest
designed C libraries I've ever seen. I hope more projects start using it so it
gets out of beta.

For static files, I ended up using the aio syscalls in Linux as they work
quite well for reading files on a supporting filesystem (mainly ext4, xfs). I
also couldn't figure out how to guarantee that sendfile doesn't block, but
didn't look into it too deeply.

Very nice project, seems very mature.

1: [https://github.com/bwindels/wwwee/](https://github.com/bwindels/wwwee/)

~~~
acidx
Thanks!

I don't think there's a way to guarantee that sendfile() won't block on Linux.
On FreeBSD, you can specify a "go as much as you can without blocking, and
then return" flag; Linux doesn't have a flags parameter for sendfile() (maybe
it needs a sendfile2() syscall with that?).

You can maybe use readahead() and keep the file open, but that may also block
as it does its work behind the scenes. A thread just for this kind of blocking
calls would be nice. But even then, this doesn't seem sufficient; so you might
want to use multiplex sendfile() calls in bursts.

Also, yeah, Linux aio syscalls are interesting, but they're quite tricky to
use and not very portable. I'll check out your project, thanks for linking to
it -- should be useful reading material while I learn Rust.

~~~
bwindels
Right, wrt sendfile, I remember coming to that conclusion as well. I really
wanted a thread-less design, mostly for simplicity's sake and to support
single core SOCs well. Is that what you do in Lwan for large files? Use a
dedicated thread for socket writes with sendfile? I guess mmap + madvise could
also help, but there is just no way to know for sure, so you sort of have to
assume the worst?

Which ever way you do it, it's so easy to end up blocking without realizing,
strace is great here to see how much time you actually spend in the kernel.

How much the aio calls block is highly filesystem-dependent. I was hoping to
use btrfs for my project, but when testing aio support, I saw my demo was
spending around 80% in io_submit and less than 10% in epoll_wait, where as
with ext4 it was more than the opposite.

Interesting stuff, and doing it in Rust was a pleasant experience I must say,
although if you are trying hard to avoid copying and allocating, the borrow
checker can get a bit naggy. Worth it though IMO, and should improve in the
future.

~~~
acidx
No -- in Lwan, files that will be sent with sendfile() are kept open for a
while; when they're first open, the same thread that opened (and this might
block) will call readahead(). Files are sent in 512kB bursts. It might use
mmap()+madvise(), but only for small files (<16KB)... but, yeah, there's not a
lot of control here. I'm not that worried, though; this is but a toy. :)

Registered I/O (RIO) or I/O Completion Ports (IOCP) in Windows seems to be
better in this regard, but I don't have a lot of experience in that platform.
Trent Nelson, from the PyParallel project, has a few talks about it; it's
quite neat.

------
MrBuddyCasino
Note: this is not related to the phony GWAN project, except maybe to poke fun
at it.

Previously:
[https://news.ycombinator.com/item?id=11373719](https://news.ycombinator.com/item?id=11373719)

~~~
agumonkey
Also note the weird initial drawn term... must be a very fun oriented team

~~~
acidx
Yes. Team. [http://www.commitstrip.com/wp-
content/uploads/2014/05/Strip-...](http://www.commitstrip.com/wp-
content/uploads/2014/05/Strip-Vision-Open-source-650-finalenglish.jpg)

------
dom96
> For Round 10, Lwan has taken the crown. But we expect the other top
> contenders won't leave this a settled matter.

Am I just blind or is Lwan missing here:
[https://www.techempower.com/benchmarks/#section=data-r10&hw=...](https://www.techempower.com/benchmarks/#section=data-r10&hw=ph&test=json)

~~~
acidx
It's a known issue with TWFB. I don't know if they're planning on fixing it,
but most frameworks that are not in newer rounds are not appearing in older
rounds any longer.

~~~
dom96
Oh, that's a shame. Why is lwan no longer in the newer rounds?

~~~
acidx
Because I failed to move the benchmark harness from the Lwan repository to the
TWFB repository. Also, I'm not really spending a lot of time on Lwan any
longer; there are so many things I'd like to do, and so little time. :)

------
cntlzw
I am only a spare time C coder myself and was surprised that I can follow the
code. Nice! Any reason why typedef struct is avoided?

~~~
acidx
Thanks! Code readability is important: not only for others (it's an open
source project, and contributions are welcome!), but for myself... I have
terrible memory. :)

The reason struct typedefs are avoided is because I prefer to be explicit (the
"Zen of Python" isn't just for Python!). It's also common practice in most of
the projects I work on, and it's not much of an issue for me as far as a
little bit of typing goes: I'm a fast typist.

------
maxpert
Amazing I am planning to shift RaspChat on this. Would be nice if same
engineering effort can be put into WebSocket. This will make it much more
desirable everywhere to build pub/sub systems.

~~~
acidx
There is SSE, which, although unidirectional, can be used in lieu of
websockets for a chat application (receive messages/events via SSE, send via
another endpoint).

However, as I said before, this is an open source project; if you have an
itch, you can scratch it. PRs are open. :)

~~~
saganus
What is SSE in this context?

~~~
acidx
Server-sent events: [https://en.wikipedia.org/wiki/Server-
sent_events](https://en.wikipedia.org/wiki/Server-sent_events)
[https://developer.mozilla.org/en-US/docs/Web/API/Server-
sent...](https://developer.mozilla.org/en-US/docs/Web/API/Server-
sent_events/Using_server-sent_events)

------
paulsutter
Seastar does up to 8 million connections per second, performance due in part
to a user mode TCP stack

Previous discussion:

[https://news.ycombinator.com/item?id=9440886](https://news.ycombinator.com/item?id=9440886)

~~~
acidx
The authors of Seastar used to use Lwan to benchmark their HTTP server
implementation. When Seastar isn't using DPDK/usermode TCP stack, and drops to
the POSIX API, Lwan is still faster.

~~~
paulsutter
Yes. And in other news a bicycle is faster than a Ferrari with no engine

EDIT: touché! i agree!

~~~
acidx
Curious that you compare Seastar with a Ferrari: those cars are known for an
internal finish that's not compatible with their price tag... and Seastar is
nothing like that. It's a seriously well engineered piece of software, and it
shows: its readability and code quality is superb.

------
tpaschalis
Impressive all and all, even more for being written in (grokkable!) C. Nice
work.

------
equalunique
Is this in any way related to the proprietary G-WAN high-performance and
scalable web server? [http://gwan.com/](http://gwan.com/)

------
mitchtbaum
Lwan used to be at the top on Techempower Benchmarks, but I don't see it
anymore. I do however see a Rust webserver, called Actix-Raw, which tops the
list. acidx, what do you think of that project?

~~~
acidx
I'm not aware of it. I'm learning Rust (just bought the "The Rust Programming
Language" book, FWIW), so I will check it out.

------
bullen
Cool, if you want something similar but in Java:
[https://github.com/tinspin/rupy](https://github.com/tinspin/rupy)

------
ksec
I wonder how does this compare to H20.[1]

[1][https://h2o.examp1e.net/index.html](https://h2o.examp1e.net/index.html)

~~~
acidx
Depends how you measure it.

A long time ago I benchmarked it, and Lwan came out on top, performance-wise.
However, performance isn't the only thing that matters in a piece of software
like this.

H2O is a great project; in fact, the authors were tightly involved in the
definition of HTTP/2.0, and it is one of the first servers to support this
version of the protocol. It is well written, and easy to grok the code. Some
of its byproducts include a nice (and reusable) HTTP parser library, and
clever tool named qrintf (that pre-parses the format string passed to
printf()-family functions and generates optimal code for that).

Also, there's a company behind it -- Lwan is just a toy I work on the weekends
when I feel like it.

------
MaxBarraclough
Looks neat! Is HTTPS support in the works?

~~~
acidx
In the works? No. Planned? Yes. I've been playing around with kTLS/BearSSL but
so far I haven't made any stride. PRs, my inbox, and the IRC channel are open,
though!

------
openbasic
Would you recommend this to serve a C++ webservice, or should we stick with
boost::asio?

------
LearnerHerzog
Lwan... that's the best name they could come up with?

~~~
acidx
Author here. It's a pun on another similarly-named web server. It's a terrible
name, and I should have searched the web before, but it's too late now.

~~~
agumonkey
what about the l a m e thing ?

~~~
acidx
This whole project started as a joke, and I still see it as such.

~~~
kiddico
And the praise for the code quality makes the joke even funnier.

My favorite kinds of projects are ones that end in a (relatively short)
punchline, but required tons of time investment.

~~~
agumonkey
Similarly Flask was also a joke at first. And now a pillar.

