ajax.cc is probably the most interesting - it's an example of long-poll AJAX with jQuery and Lacewing::Webserver as a backend. Lacewing uses epoll/kqueue/IOCP rather than a traditional select/poll approach, so it should scale quite well.
I've been developing Lacewing pretty much in private for quite a while, now - it's quite daunting when I start thinking about having to write all the documentation and such, but this morning I just thought "screw it, it's going on github, undocumented or not". It might as well be public while I'm getting it all together...
Some parts need improving, mainly because of how long I've been working on it - there were many gaps in my knowledge when I started. I don't think I've learnt so much from any project!
Thanks for sharing! I was actually looking for a lightweight embeddable C/C++ web server not long ago, perhaps this will fit my needs. Only thing I'm wondering about is does it (or will it in the future) have windows and mac support? Ideally I am looking for something cross platform for what I have in mind.
(My current solution is JVM based (over JNI) and uses Jetty, but I really want to eliminate the JVM dependency - I use it for more than just the web server, but the other stuff is easy enough to replace with C++ solutions)
Yes, I've used it on both Windows and OS X, as well as Linux and FreeBSD servers.
It uses IOCP on Windows and kqueue on OS X. There's a project file in the 'msvc' folder which should build out of the box on Windows - on OS X you probably need macports with autoconf/automake/libtool to build it.
I suppose it's just meant to be easily embeddable into just about anything. It could be used to add webserver functionality into an existing application (think web control interface for a daemon), or it could even be used to write a complete C/C++ web-application or backend.
I don't want to make any claims about the performance of Lacewing (aside from anything else, it's too early for that) but one would assume a backend written in C/C++ would be a little faster than one written in a scripting language. Although probably quite a pain to write..!
From a technical point of view, I found that most webservers were tuned for high performance on Unix, but on Windows they wouldn't run very well at all. The news earlier about Node.js having Windows support on the roadmap (which is mainly what prompted me to release Lacewing today) suggests that it's worth supporting, and Lacewing has used IOCP and other tricks on Windows from day one.
It would certainly be interesting to try - do you know if kqueue is available on iOS? If not, I guess the only thing to do would be writing a select() or poll() fallback for the EventPump class. I might have a look at it sometime.
Having said that, I'm not sure how much use the server stuff would be on iOS, and I haven't yet written the Unix implementation of the Client class (not that that will be very much work).
I have some reason to believe you can get libevent working on iOS with minimal drama; I think there's at least one app that embeds it.
Which, by the way, is what I would suggest you do with your code; if you're so dependent on kqueue that it's a portability issue, you have little to lose and lots to gain by ditching the custom event loop.
I support epoll and IOCP, too, and I don't consider event handling something complicated enough to justify bringing in an external library.
I quite value the freedom of dependencies Lacewing currently has (except the obvious OpenSSL, etc - although it doesn't use OpenSSL on Windows, it uses native Windows SSL).
misses the case where the double dot is not followed by a slash (eg http://example.com/foo/..). Currently you're almost certainly okay (read(2) will _generally_ (OpenGroup says "implementation specific") return EISDIR) but beware if you should implement a mod_autoindex-alike.
ps. Somebody else said it already but: nice read :)
Lacewing splits input lines on "\r\n" but browsers often happily accept "\n" as a line delimiter (see 19.3 of RFC 2616 for Postel-esque tolerance discussion). So the client can send a request header like:
Cookie: foo=bar\n<naughty stuff>\r\n
to manipulate the server response, cf "response splitting attack".
This is not discussed in the 37 pages(!) of the new cookie RFC 6265 :-/
It doesn't automatically send back any cookies the client sends, only the ones that have been modified by the application code (eg. via Request.Cookie("name", "new value")).
Making the parser split input lines on both "\r\n" and "\n" is a good idea. Thanks!
> only the ones that have been modified by the application code
Sounds good (after a quick skim again I still can't see that but I'll take your word for it!).
I've had two attempts to build and test but unfortunately failed: currently can't build (src/unix/Client.cc:242: error: cannot convert ‘sockaddr_in’ to ‘const sockaddr’) and hello_world.c dumped core on me this time yesterday (Ubuntu 10.10).
"Work in progress" as you say... but it may just scratch an itch. I will come back again in a couple of months :) Thanks.
Someone actually just submitted a pull request to fix this on Github, which is now merged :-)
I just cloned the source and compiled it successfully on Debian, and it all seems to work alright (I can't get hello_world.cc to crash.) If it happens again, could you possibly get me a stack trace?
By the way, if you look at the cookie output code:
The strcmp() line checks the value of the cookie when received from the browser (Client.Input.Cookies) against the value of the cookie with any modifications (Client.Cookies).
In src/webserver/Webserver.Incoming.cc you have an ad-hoc HTTP parser that internally buffers partial request data and does some questionable things like recursing on each request header line.
Consider using a finite state machine parser for HTTP like this one [1] or this one [2], which will reduce your overhead, and help you push the buffering and eof problems back up to the main event loop code.
I remember I used to do something like that in C++ because throwing exceptions on error situations gave me zero debug information about what happened. But forcing a segfault instead you can get the stacktrace.
As someone else commented: kudos for you code! It's really clean and well laid out.
Encouraging to see more development in this space. Thanks. I'm curious, though. How do you think this compares with the mongoose (http://code.google.com/p/mongoose/) project. Other than the fact that it's C++-based what would you say are the relative advantages/disadvantages?
At a glance, I'd say it looks both more lightweight and more high-level than Lacewing (Lacewing won't host anything at all unless you add in a handler which calls SendFile or whatever, whereas Mongoose seems to have a concept of a document root and things like that).
Mongoose seems to use select() and nothing else (I only had a quick read through the source, mind), whereas Lacewing opts for things like epoll/kqueue/IOCP, which should mean that Lacewing might scale better to many clients.
Mongoose also appears to use multithreading, which Lacewing currently avoids in favour of the reactor-esque Lacewing::EventPump. Lacewing has, however, been designed with multithreading in mind, and it's something I'll be adding as an option for the Server and Webserver classes in the near future.
By the way - Lacewing does provide a C API too, despite being written in C++.
Mongoose is only selecting for the listening socket, so the comparison there isn't particularly valid. Mongoose scales as well as threads on its platform scale.
Mongoose also does a fair bit more than your software does, including SSL, CGI, HTTP Authorization, and file uploads.
Neither of them, frankly, are particularly trustworthy in their present state (Mongoose has apparently actually been audited by someone, but it too stores lengths in signed u64s). But Mongoose, for all its extra functionality, is actually somewhat smaller than yours.
I'm glad you're playing with this and wish you the best of luck, and would only urge caution about advocating that people adopt this code (or Mongoose). Most developers would almost always be better off with a better tested server, or one implemented entirely in a high-level language.
The advice about developers being better off with a better tested server is very true, though, of course. I hope Lacewing might fit into that category one day, too.
I honestly believe that you should avoid multithread support and instead focus on the current single threaded reactor pattern.
Any developer found easily just attach something like a zmq ipc queue in order to support multithreaded handling, given that the request object is thread safe.
This approach would just leave you with work which ensures that the protocol parser doesn't trump any I/O overhead.
I think a question people should often ask themselves is: Have you considered if you need a build system at all? Is this something I can properly build with a plain old Makefile?
It's possible - I can't say I gave it much consideration, I just used automake because I knew about it and it's widely available.
I shouldn't need to include autogen.sh or require people to have automake/autoconf installed to build it, but I had problems using a configure script/Makefile generated on one system on another. I guess I'll sort it out when I come to understand the tools a bit better.
ajax.cc is probably the most interesting - it's an example of long-poll AJAX with jQuery and Lacewing::Webserver as a backend. Lacewing uses epoll/kqueue/IOCP rather than a traditional select/poll approach, so it should scale quite well.
I've been developing Lacewing pretty much in private for quite a while, now - it's quite daunting when I start thinking about having to write all the documentation and such, but this morning I just thought "screw it, it's going on github, undocumented or not". It might as well be public while I'm getting it all together...