
Open Source Is Really About Documentation – Twisted vs. Tornado - fogus
http://www.littlehart.net/atthekeyboard/2009/09/14/open-source-is-really-about-documentation/?utm_source=rss&utm_medium=rss&utm_campaign=rss
======
Rantenki
I have built several twisted apps which are in use in production environments,
including web and jabber/xmpp apps. I love twisted, and use it whenever
possible...

...and I agree completely with the op.

Twisted is powerful, flexible, well architected, and reliable. It is also damn
near impossible to get over the steep learning curve and to actually USE it.

Show me a competent Twisted developer, and I'll show you someone who had a gun
jammed roughly in their mouth, and was FORCED to learn it at some point. We
are all glad we know it _now_, but that learning process SUCKED.

~~~
caffeine
[http://twistedmatrix.com/projects/core/documentation/howto/t...](http://twistedmatrix.com/projects/core/documentation/howto/tutorial/index.html)

What's wrong with that? Seems quite comprehensible to me...

~~~
moe
It's indeed the starting point to learn the fundamentals - but becomes almost
hilarious after step 4 when they begin to introduce Components, Adapters,
Perspective Broker and other things without losing many words about what all
that stuff does, much less how to re-arrange it for custom needs.

The best investment in terms of mindshare the twisted project could make at
this point would be to simply sit down and write documentation.

Grab sphinx (or another tool that makes doc-writing fun) and start out by
writing a page about each Class that is being used in these examples. Why does
it exist, what role does it have and don't forget the link to the API doc.
Then I'd suggest to make a high-level overview, perhaps even with a little
diagram, that explains how everything plays together in the various
application types. Hyperlinks to the previously mentioned detail-pages should
then work wonders for anyone exploring the project.

Writing _good_ documentation is hard, but you only have to do it once. The
lack of documentation on the other hand forces _every_ new user to wrap their
heads around innocent lines like this:

    
    
       internet.TCPServer(79, factory).setServiceParent(
           service.IServiceCollection(application))
    

I'm surely not the only one who put off learning about that whole Service
metaphor and stuck with the simpler examples until I absolutely couldn't avoid
it any longer. That's not the way you want users to approach your framework.

~~~
masklinn
> The best investment in terms of mindshare the twisted project could make at
> this point would be to simply sit down and write documentation.

Making documentations (and contributions to it) a first-class contribution
with its bugs & discussions & refactorings, and fun to write (the way the
Django project handles it) would probably also go quite a way towards that.

~~~
moe
Yup, agreed. Creating and endorsing good documentation is an artform in
itself.

Currently the barrier of entry is too high and I guess too few people are deep
enough in twisted to explain all the why's of even a simple baseline example
_properly_.

It's kind of a deadlock situation because without that lowlevel insight it's
also hard to get confident enough to contribute high-level docs.

~~~
masklinn
> Currently the barrier of entry is too high and I guess too few people are
> deep enough

Thing is, through checking out Django I realized you don't really need to be
"deep enough" to contribute to that stuff. The thousand-cut effects applies,
and when you're trying to make heads or tails of the doc, you're bleeding from
these thousand cuts so you know and understand the doc's pain points. If doc
contribs are first-class, then right as you understand the issues and what the
doc tries to say, you'll be steered towards opening a documentation bug (and
maybe even contributing a patch) for that part.

Maybe it's just doing to be a bit of wording, maybe it'll be a link or two to
other relevant parts of the doc; maybe it'll be things you'll have figured
through your sweat and tears, maybe something you'll have learned by asking a
question on IRC.

But even as a beginner, you'll have added a little stone and made the doc
slightly better through your current understanding. And it's enough.

I barely contributed to Django myself, but my first ever contrib was simply
pointing out that the `now` tag's format character `n` produced an RFC 2822
date, not an RFC 822 one as written in the doc at the time. It was almost
nothing, but it still very slightly improved the documentation for the next
guy reading it, and it actually made me feel involved in the project. Even
though I didn't have anywhere near the knowledge of the codebase to contribute
to the code itself.

edit: and re the thousand cuts effect, that's the great thing about making
beginners contribute: "old timers" don't notice or remember these small issues
(which pile up), because they fill in he blanks or missing parts through their
knowledge.

------
tdavis
Not all of Twisted's documentation is bad. And if folks can't figure out the
docs, Jean-Paul and Glyph are available to help in IRC and know everything
about Twisted (since as far as I know they represent 90%+ of the current
contribution and maintenance).

Besides, Tornado can afford to have (seemingly) more comprehensive
documentation because it is far smaller in scope and support. Twisted has a
large number of robust libraries for doing all sorts of things asynchronously,
from web servers to ORMs to SSH and everything in between. It's like comparing
an orange to a bushel of apples and it's silly. Twisted has never supplied a
web framework, which is exactly what Tornado supplies.

The Twisted docs are lacking in many ways, I just don't understand how Tornado
is objectively _better_ because it doesn't currently have that problem -- they
aren't the same thing!

~~~
grumpycanuck
(Disclaimer - I wrote the blog post)

"The Twisted docs are lacking in many ways, I just don't understand how
Tornado is objectively better because it doesn't currently have that problem
-- they aren't the same thing!"

Of course it's the same thing. If your documentation is lacking, don't
complain when people point it out. It's awesome that Jean-Paul (who commented
on the blog post) is available to help people...but if there was more
documentation then the simple questions disappear and you are left answering
the really tough ones. Easier said then done, obiously.

Upon reflection, this is really a comparison of Twisted.web with Tornado. The
rest of Twisted got dragged into it, perhaps unfairly.

------
mahmud
Blogging Is Really About Whining.

There seems to be a segment of the programming society that's just content
with spewing whinage about not being able to get what they want, instead of
using what they find and improving it to fit their needs.

I am no Python programmer, but i have looked at twisted and found it trivial
to grok. I saw how they have wrapped the Unix system calls and put a layer of
abstraction over it .. I didn't like it, and went back to using Common Lisp's
iolib. I must have missed an opportunity to blog there :-(

~~~
aaronblohowiak
I think you're taking a reductionist view, and I think you've missed the point
the author was making. The author first expressed his understanding of the
influences on individual behavior and then he made specific suggestions (with
examples) of how twisted had failed to optimize for a specific outcome within
that framework.

I consider this analysis and feedback, not whining.

You seem to suggest that the author is whining in lieu of being self-
sufficient. I can't speak for the author, so I will just speak for myself. I
wholeheartedly believe in the self-sufficiency that programming culture
promotes. However, I also believe that the easier you make it for others to
achieve self-sufficiency in your open-source library, the more successful it
will be.

I'm happy that you found it trivial to drink deeply of twisted's essence. I
also think that you are right that you have an opportunity to blog! I for one
would _love_ to read about why you think that CL's iolib is a better
abstraction layer than twisted. You haven't missed the opportunity -- I'd
upvote that over most of the stuff on the front page!

------
wehriam
As usual, a simple "this versus that" comparison is only useful as an
intellectual exercise.

Tornado is a web framework, in many ways closer to Django than Twisted. Beyond
that the underlying approaches are different - Tornado does not use deferreds.

There are a number of great projects that use Twisted. Soon many new projects
will use Tornado. But the frameworks' respective documentation does not tell
the story.

~~~
masklinn
> Tornado is a web framework

Not really. It's the combination of an asynchronous HTTP server and an
optional web framework. And work is being done [to add WSGI
support]([http://github.com/facebook/tornado/commit/8ca616088cfb26ff19...](http://github.com/facebook/tornado/commit/8ca616088cfb26ff19fcc6f359d654fef905b8da))
so other frameworks can be plugged into the server.

At its core, Tornado is a server not a framework.

~~~
wehriam
The Tornado front page refers to itself as a framework six times, but we're
splitting hairs.

Twisted is, at least for now, more general purpose. Can't wait to see where
Tornado goes though.

------
cesare
It's obvious, documentation is always important. It's even more important for
proprietary software, since you can't even study the source. But it seems that
neither of this two projects has good and extensive docs/tutorials.

You could argue that if Twisted was better documented the FriedFeed guys
wouldn't have started their own project. I don't buy it. There could have been
many (technical and non-technical) reasons to do this anyway.

I (mostly) agree with a comment on the blog: _This is just pure NIH syndrome.
It happened because it’s easier to write code than to read it, and it’s more
fun to create than to fix._

------
_glass
I'm sorry, but <http://twistedmatrix.com/trac/wiki/Documentation> is the
documentation, where I started and it shows nice examples, a high level
introduction -- everything needed to dive in. Actually I learned a whole lot
back in the days when I had to use this fine documentation.

------
dhotson
I think a bit of friendly competition will help both projects.

The Twisted devs may have some hurt feelings, but I think they can learn a lot
from this.

------
gluegadget
Well, I think when friendfeed developers were writing tornado they didn't mean
to opensource it or even talk about it in public, so they went for the name
"Tornado".

I agree that documentation is really important in Open Source, and so is
_finding_ the documentation. Googling for such a generic names makes things
hard. Considering all the hype Tornado got recently the homepage is not still
listed in the first 5 pages of Google's results for "Tornado".

So I think it's a good advice to think twice (or thrice) before naming your
next project.

------
antirez
I don't think the "documentation sucks" can be used to model very well the
issue, there is another very important thing: is it simple to use, and the
base case is obvious, or is it over engineered? I think in the Twisted /
Tornado issue there is some of this elements too.

A lot of libs around here are, technically speaking, well coded, reliable,
fast. But the API sucks. An example is the libpng, if you have some experience
with it you'll understand. When an alternative with a clean API arise it's
very hard to resist.

------
c00p3r
Just remember MySQL - its success was about documentation (the most),
community (as a result) and some PR.

It is so strange to create reusable components without detailed guides.

The nginx had this very problem. If it could get a user-friendly documentation
it will earn more adoptions.

~~~
alecthomas
I also think this was one of the factors in PHP's popularity. Its
documentation was (is?) excellent.

------
californiaguy2
Every time I've thought about using Twisted for something, I go back to their
documentation and remember why I wrote it myself with asyncore last time.

