

Python and the Real-time Web - joecarpenter
http://mrjoes.github.io/2013/06/21/python-realtime.html

======
lvh
Nice, thorough article. Kudos for showing all the alternatives. That said, I
think you had a strong preference for one, and that shone through. For
example:

With greenlets, you say: Why is greenlets are great? Because they allow
writing asynchronous code in synchronous fashion. They allow using existing,
synchronous libraries asynchronously. Context switching magic is hidden by
greenlet implementation.

Greenlets don't magically make synchronous libraries asynchronous! That's
gevent! Greenlets are just the coroutines. Plus, you're forgetting that
"magically" making sync libraries async doesn't actually work in plenty of
cases, and the way that it works generally involves monkeypatching half the IO
bits in the standard library!

Also, there's a bit of a false dichotomy. You claim that there's two options:
coroutines or callbacks. That might be true, but then you need to keep in mind
that callbacks doesn't necessarily mean something-looking-like-the-Tornado-
callback-demo-code. That mind seem like a nitpicky implementation detail, but
that's the sort of thing that lets Twisted do inlineCallbacks (where you write
generators instead of coroutines) or Corotwine (a third party package where
you get actual coroutines) or geventreactor (a third party package where the
event loop is done by gevent -- since the other way around isn't actually
supported).

The inline callbacks equivalent, given the same API, would be something along
the lines of:

    
    
        data = yield get_more_data()
        return make_response(data)
    

The twisted equivalent of this wouldn't even look like data = yield
get_more_data(); Twisted's API calls you when there's data, so it looks even
simpler:

def dataReceived(self, data): return make_response(data)

Also, txsockjs now integrates great with the Twisted Resource API, so it will
live neatly side by side with existing web stuff you're serving from twisted,
which _can_ include e.g. WSGI apps (since twisted comes with a wsgi server :))

I have prepared a talk that deals with Twisted Mixing, which I hope to submit
to PyCon. The work in progress is here:
[https://chiselapp.com/user/lvh/repository/TwistedMixing/inde...](https://chiselapp.com/user/lvh/repository/TwistedMixing/index)

I'd love to have a cogent argument for the things you didn't like about
Twisted, but two out of three appear to be more or less the same thing
("complex", "hard to learn"), and, on top of that, subjective. I'm sorry you
had a hard time. It shouldn't have been. If you have any specific issues, I
would like to address them. I'm assuming that by "not PEP-8" you mean mostly
"uses camelCase", in which case I'll give the usual apologist answer:

\- Twisted predates PEP-8's recommendation of snake_case ;-) \- It's actually
PEP-8 compliant: the PEP says to do what the code around you does, and
something about consistency being a hobgoblin ;-)

Also, you mention that you can run Twisted on Tornado, but not Cyclone. Is
there a particular reason for that? From all points I can tell, they get you
the same result (mixing Twisted and Tornado code), but the reactor that ships
with Tornado just gives you fewer event loop options (and generally inferior
ones).

Disclaimer: I'm a twisted dev. Can you tell? ;)

~~~
joecarpenter
> Greenlets don't magically make synchronous libraries asynchronous! That's
> gevent!

That's right - I wrote that they allow writing code in synchronous fashion and
I didn't say that it happens automagically though.

> That might be true, but then you need to keep in mind that callbacks doesn't
> necessarily mean something-looking-like-the-Tornado-callback-demo-code.

Yep, I also provided how Futures and generator-based coroutines can be built
on top of the callbacks.

Just in case, Tornado supports both out of the box.

> The twisted equivalent of this wouldn't even look like data = yield
> get_more_data(); Twisted's API calls you when there's data, so it looks even
> simpler

Very similar to some Tornado API too - method of a class that gets called when
something happens. For example, sockjs-tornado follows this convention.

> \- Twisted predates PEP-8's recommendation of snake_case ;-)

Yes, I'm aware of it, but before I started playing with asynchronous
libraries, I had some Python experience and got used to underscore naming
convention. While it is easy to switch between both, I'd prefer consistent
code style. Especially when I mix typical Flask Web application with real-time
portion in same application.

No doubt, Twisted is mature and featureful framework. When I was investigating
different options, Twisted was first one I tried. However, I also tried
Tornado and found it easier to start with. And because Tornado worked for me,
I decided to stick to it.

------
csense
I like the blog post format better than a Google Hangout. When there's a
conversation, unless (or perhaps even if) it's heavily moderated, things get
lost in the noise.

I was really confused about the word "polyfill," which the author used without
explanation. Since I've written code to render filled polygons before
(concavity is a fun corner case), I assumed this was what it was referring to,
and I was really confused about how that's even remotely relevant. Wikipedia
finally straightened me out [1].

[1]
[http://en.wikipedia.org/wiki/Polyfill](http://en.wikipedia.org/wiki/Polyfill)

~~~
pronoiac
When did "polyfill" replace "shim," I wonder?

~~~
the_cat_kittles
I think of "polyfill" as filling many different holes to make it so you can
forget about the different cases (browser compatabilities in this case). A
polyfill pours concrete over a bumpy surface and makes its smooth. "Shim", to
me, is essentially synonymous with monkey patch, i.e. something you shove in
there to make something work- shoving a folded napkin under a table leg to
make it flat is shimmy to me.

~~~
yareally
> I think of "polyfill" as filling many different holes to make it so you can
> forget about the different cases (browser compatabilities in this case). A
> polyfill pours concrete over a bumpy surface and makes its smooth.

Kind of like a façade then[1]? A shim would probably be synonymous with an
(very informal) adapter.

[1]
[http://en.wikipedia.org/wiki/Facade_pattern](http://en.wikipedia.org/wiki/Facade_pattern)

[2]
[http://en.wikipedia.org/wiki/Adapter_pattern](http://en.wikipedia.org/wiki/Adapter_pattern)

------
jisaacstone
I'm working on porting a node/websockets game I wrote to python using the
twisted websockets branch. (not integrated in core yet but fully functional so
far)

Instead of using socketio I rewrote the client side code in pure js. It was
not difficult at all, and nearly all browsers support websockets now anyway.

Anyway - if you've written a twisted server before then writing a websocket
backend is trivial, and the frontend is not much more difficult. I might do a
full writeup when the project is complete.

------
rubiquity
Wow this is an incredible wealth of information in regards to the Real-time
Web. I recommend anyone read this regardless of whether you use Python or not.

------
emehrkay
I haven't read the whole post yet, but is the author saying that a WSGI+Gevent
is better at real-time than Tornado alone?

~~~
joecarpenter
I'm just sharing my personal experience with both. And no, I prefer Flask
(WSGI) and Tornado working side by side, in separate processes.

------
VeejayRampay
As a Ruby programmer, I face the same sad state of affairs in Ruby frameworks
unfortunately.

Nice writeup about all the alternative, well done.

------
the_cat_kittles
After stumbling around this problem for ~6 months, making half-solutions, this
is such a great article. Thank you!!!

