
Mongrel2: State Machines, HTTP-0MQ, Events - ihodes
http://sheddingbikes.com/posts/1278464593.html
======
iwasphone
I read that post twice and created an account just to leave a comment. What a
brilliant guy--saves app developers from having to hack up their webservers to
support their choice of backend AND provide incredible insight into (and
granular control over) connections.

~~~
zedshaw
Thanks! I appreciate the compliment. That's the general goal, but with all
things, it remains to be seen how easy it is to actually do this. It's really
easy for me to do it in C code, but the key is how to make it easy for other
people without much fuss.

------
necubi
Zed makes a big deal about getting long-polling for free, but with evented web
servers (built on EventMachine, Node, Twisted, etc), you also get long-polling
for free, since requests are just objects that get thrown around until they're
completed. This architecture makes long-polling trivial, because when a
request comes in you can put the request object into a hash or array, then
when the data or computation you're waiting on finishes you look up the
request and send it back to the client.

Regardless, there are a lot of interesting ideas here, and I look forward to
playing around with it when it's more done.

~~~
zedshaw
Well, this is long polling from any backend, not from within mongrel2. So your
0MQ handlers can respond to any connected HTTP or JSSocket, which other
servers can do given the right design.

However, what I was more referring to was that by using a state machine I
didn't have to actually implement anything. I just wrote the code according to
the state machine, and what fell out was a ton of features simply because of
design serendipity.

------
vault_
It seems to me that the logic behind the problem solving going on in Mongrel2
has been 'how can I make this into a problem that's already been solved?'

I think that seems like a pretty good methodology in a lot of cases actually.

~~~
zedshaw
Exactly, or another way to do it is, "I have problem X, how can I make it go
away without solving it." Either by designing the thing so I don't need to
solve X, or by finding someone else who's already solved it for me.

------
po
Does a finite state machine where you can modify the events to cause any
transition you want defeat the purpose of the FSM? It's interesting that while
the states are limited, by adding code you can cause a transition from any
state to any other. You can look at the pretty picture to understand the
default FSM, but as soon as you install any additional code, you don't know
what transitions it will take. Adding modules effectively rewrites the FSM.

Is that the intent?

Also, this reminds me a bit of the design behind Varnish.

~~~
zedshaw
Well, you can't cause any transition you want, you can just change the events
in the stream. So, if you're in the middle of proxying you can get out of it
and go into a handler, but you couldn't totally screw up the FSM and go
someplace weird.

So, adding modules doesn't rewrite the FSM. You'd need Ragel and a C compiler
for that. ;-)

------
megaman821
Currently I have one server running haproxy, which distributes requests to
three servers running nginx (which serves static content) and proxies
application requests to apache/mod_wsgi. How would Mongrel2 simplify this?

~~~
zedshaw
Well, knowing haproxy user, it is embed into your infrastructure like a tick,
so it probably wouldn't help you. My experience has been, if it ain't broke
don't fix it.

But, if I were to do an alternative, I'd basically ditch haproxy and nginx and
just have Mongrel2 talking 0MQ or HTTP to arbitrary backends directly, rather
than through multiple layers. With 0MQ you don't have a single point of
failure by design, so you could (theoretically) run tons of Mongrel2 servers
in front and have them talk to big clusters of 0MQ in back.

That's the goal anyway.

------
deweller
This post and comments here have convinced me I need to learn more about
Finite State Machines and how to apply them.

Can anyone recommend some good educational material about using finite state
machines in software development?

My Google searches find theoretical papers and not practical applications.

~~~
zedshaw
Check out "Practical State Charts in C/C++" by Dr. Miro Samek.

~~~
johnzabroski
Notes:

The 2nd Edition is better than the 1st Edition, because it does not have the
distracting analogies to quantum physics. For me, the best part of Samek's
book was Chapter 5, State Patterns. However, he also covers Standard State
machine Implementations (and their flaws) in Chapter 3. Chapter 6, Realtime
Framework Concepts, covers WHY you want to take special care in implementing a
state machine, such as avoiding corrupting the current event in order to
preserve RTC semantics. You will not get a more coherent presentation of these
issues anywhere else. You could spend years of your life reading CS papers on
the subject, implementing state machines, etc. and not come to the mastery
Samek has. My favorite quote from the whole book has to be pointing out
deviating from a simple Moore FSM, by introducing guards, can cause big
trouble. He says overuse of guards is "[the source of] architectural decay" in
complex reactive systems.

The one thing I disagree with Samek about is that transitions have any state
context associated with them. For me, transitions occur between states. This
approximates the design of Mealy actions going between Moore actions., and is
closest to the semantics of object-orientation, since the sender does not need
to know the receiver directly. Samek presumes actions are dependent on the
source context, but if there are two outgoing transitions from a source
context, they probably will not have the same action. You move to a state
before entering that state, and you move out of a state before moving to
another state.

------
megaman821
The more I read about Mongrel2 the more interested I am in it.

I have never really looked at the code of a web server before for comparison
but it is kind of interesting to read about Zed's approach to building a web
server.

~~~
nupeter
I think Mongrel2 has some cool ideas in its design, but using sqlite for
configuration seems very heavy, creates more dependencies and you can't edit
the data in a text editor which moves away from how you traditionally
configure a unix service. Bad choice.

~~~
zedshaw
I think the traditional way of configuring a unix service manually using a
random config file format that hides the underlying data structure is a bad
choice. I'm trying to provide a way to automate things, and the best way to do
that is with a universal data structure and langauge that modifies it
accessible from every programming language.

sqlite3 fits that goal.

In other words, try doing .schema on your nginx.conf.

~~~
alnayyir
Initially I didn't like the idea of using sqlite for the config, but actually,
this makes total sense and I agree with it as long as you don't make it into
something Lovecraftian :D

------
mronge
I find his use of Ragel generated state machines fascinating. He uses them in
ways I wouldn't typically consider, very clever stuff.

I'm on the lookout for nasty code of my own that can be replaced with a state
machine.

------
signa11
erlang also provides a number of 'behavior' modules e.g. gen_fsm/gen_server
etc. which takes out the drudgery of writing boilerplate network client/server
code. w.r.t your fsm, are there timeouts also happening at some states ? might
be required to avoid holding onto precious resources if either end becomes
unresponsive for whatever reason...

~~~
zedshaw
I had them in there, but was having a problem getting the base state machine
right, so took them out. Cool that you noticed though.

------
CoffeeDregs
Hey... wait a minute ... what's up with this non-ranty, non-swearing,
respectful-reply Zed... What have you done with the real Zed?!

~~~
huwshimi
I believe it may have something to do with the non-ranty, non-swearing,
respectful questions.

------
zaph0d
Go Zed! You rock.

