
Rewriting Reddit (2005) - amalantony06
http://www.aaronsw.com/weblog/rewritingreddit
======
jedberg
web.py never really worked well, so we ended up rewriting it again. The
interesting thing was that Django still wasn't up to the task when we did
that, mostly because it's templating engine was too slow. So we chose Pylons
instead.

It should be noted that Django has since fixed that issue.

~~~
Igglyboo
Do you guys wish you were using Django right now?

~~~
aquilaFiera
No, the reddit project itself is still based on Pylons. However redditgifts
(the reddit marketplace) is based in on Django so reddit does have Django.

~~~
thalesfc
Do you guys know what they felt about Flask?

~~~
keysersosa
At the time that we switched to pylons and decided to punt django, flask
didn't exist. I believe it was some time on '07 or '08\. I'd check, but I also
think it predates our switching to git from subversion...

Personally I like flask a lot and it's come a long way. Also, when spez and I
moved on to hipmunk we started using tornado.

~~~
jedberg
The switch to Git happened one day before we open sourced on June 17, 2008, I
remember that day well :) So yeah, the history of when we switched to Pylons
is probably lost forever sadly. But yes it was right around when I started in
March '07 that you guys started working on the transition.

------
_stem
For all people still thinking that Django is not "lean" or flexible enough for
their next project, let me just leave this here:
[https://speakerdeck.com/mstepniowski/django-best-python-
micr...](https://speakerdeck.com/mstepniowski/django-best-python-micro-web-
framework)

A single-file web application in Django:
[https://github.com/mstepniowski/microdjango/blob/master/poll...](https://github.com/mstepniowski/microdjango/blob/master/polls.py)

~~~
INTPenis
>A single-file web application in Django:
[https://github.com/mstepniowski/microdjango/blob/master/poll...](https://github.com/mstepniowski/microdjango/blob/master/poll..).

That still looks very complicated compared to most microframeworks I've used,
including web.py.

And all that django-specific logic that needs to be in there makes it harder
to read unless you're familiar with the django ecosystem.

~~~
coldtea
> _And all that django-specific logic that needs to be in there makes it
> harder to read unless you 're familiar with the django ecosystem._

What "django-specific" logic? Those are standard imports and a few helper
objects, the kind of which you'll find in any web framework.

[edit: importants => imports]

~~~
INTPenis
I've been using flask lately and I could compress four of those imports into
one import in flask. And the settings aren't even necessary.

All these things, when compared to a true microframework, contribute in
legibility of the code.

~~~
_stem
You're right. Settings are not even necessary in my example, as we should be
fine with defaults in most cases. The only lines you really need are these
four:

    
    
      from django.conf import settings
      settings.configure(ROOT_URLCONF='polls')
      from django.core.wsgi import get_wsgi_application
      application = get_wsgi_application()
    

You lose some of the Django niceties (like being able to provide your own
custom settings on the command-line) but flask doesn't have them anyway.

------
joesmo
I think you can leverage these criticisms at about 90% of the framework
software out there written in just about any language. Most developers like to
create complex, complicated software mainly to show off their "cleverness"
without thinking through the implications for the people that have to use it.
It's as if they think that by cramming in every design pattern they can think
of and using many different libraries together for no perceptible reason makes
the software better when they could've achieved the same thing much more
simply and cleanly if they had put a little thought into it. Such over-
engineering is equally a problem as the spaghetti code of programmers who
don't know better.

~~~
sbilstein
I'm not too familiar with Django but with several other frameworks and I
definitely feel like this applies to lots of them out there.

In my experience though, I've worked on teams with developers who criticize
frameworks for those reasons, choose a minimal or no framework at all, and
spend a ton of extra, unnecessary time reinventing a worse version of a
framework they decided 'sucked' once they realize they need security, error
handling, form processing, an ORM etc... Over-engineering is an epidemic.

~~~
LeafStorm
I almost went down this route on a project once. As soon as I noticed myself
writing an identity map, I slapped myself in the face and just downloaded
Symfony.

~~~
girvo
Funnily, this is the reason I adore Slim so much. In the PHP world, it gives
me exactly what I require in a framework, and nothing else. Slapping it on top
of a well-defined class heirachy becomes a simple exercise. I like it so much,
that I'm porting it to Hack/HHVM[0]!

[0]
[http://github.com/LeanFramework/Lean](http://github.com/LeanFramework/Lean)

------
temuze
The first Python web framework I used was Django. I'm really glad I came
across it first because I was inexperienced at the time and it forced me to
adopt a lot of good practices.

However, now that I'm more experienced, I can relate a little to what Aaron is
saying here. Learning all of this boilerplate stuff is annoying! Recently, I
started contributing to a friend's rails app and I found the process of
learning how to use each separate little tool tiresome.

This is why I like Pyramid - it starts off dirt simple. Hello World is almost
as short in Pyramid as it is in Flask. As you grow, you can start organizing
and adding different things as you need them. It has absolutely 0 magic. I
find Pyramid to be a happy medium between Flask's minimalism and Django's
extensibility.

~~~
Yuioup
I wonder. How popular is Pyramid? I ask because I found Pyramid to be quite
intimidating the first time (I'm not an exprienced Python developer). It took
me four attempts to finally 'get' it - the first three times giving up in
frustration and leaving it alone for 2-3 months before trying again.

Now that I get it though, it seems like an interesting framework but not a
productive one. I've only dabbled in Pyramid with my free time and have never
built a serious project with it, but I can't imagine building anything large
scale with it.

(I'm a .NET developer BTW, MVC is what I work with in my day job)

~~~
sontek
We ([https://www.surveymonkey.com/](https://www.surveymonkey.com/)) use it for
all our services. I can't imagine using another framework! Pyramid is great
for what we do.

I also know [http://www.cars.com/](http://www.cars.com/) is heavily powered by
Pyramid.

------
coldtea
> _The Lisp newsgroup, comp.lang.lisp, was upset about the switch that they’re
> currently planning to write a competitor to reddit in Lisp, to show how
> right they are or something._

That must have ended as a huge success, because as we all know the important
thing behind a website is the backend language...

~~~
Perceval
Isn't Hacker News written in Lisp?

~~~
dllthomas
I think you miss the point of the parent comment, which is that backend
language is an implementation detail and generally shouldn't matter to users.
You can disagree with that point, it just is a very different point than
anything addressed by noting that HN has a Lisp backend.

------
mikepurvis
Note that reddit has rewritten at least once since the web.py rewrite. cf.
[https://news.ycombinator.com/item?id=51078](https://news.ycombinator.com/item?id=51078)

I recall reading a more scathing post from spez at one point about
frustrations with web.py, but I don't seem to see it now. It may have been
removed. In any case, Flask seems to be the inheritor of the "write webapps in
Python" description.

~~~
thearn4
> In any case, Flask seems to be the inheritor of the "write webapps in
> Python" description.

Flask definitely fills that role for me. It's very simple to get up and going
with it, and deploys easily.

~~~
thalesfc
Me too. This post is making me so happy to see that others like Flask as I do.
:)

------
wcummings
What Lisp webserver was used, originally? Hunchentoot? What web servers/stack
are Lispers using these days?

~~~
orthecreedence
I use Wookie a lot
([http://wookie.lyonbros.com/](http://wookie.lyonbros.com/)) but I'm biased
because I built it. It's a non-blocking CL app server.

------
mox1
web.py is still my go to for lightweight webapps. I won't start a big
discourse on the pros and cons vs. other technologies, but I will say it is
incredibly easy to setup and use.

The current github page is here:
[https://github.com/webpy/webpy](https://github.com/webpy/webpy)

If you are looking to get up and running quickly, I would recommend forking
this skeleton: [https://github.com/jzellman/webpy-
skeleton](https://github.com/jzellman/webpy-skeleton)

Finally, the built in database framework is a bit simplistic. I like to pair
this with Peewee (included in the webpy-skeleton).

~~~
rdtsc
Yeah same here. It has the cleanest mapping from REST resources to Python
class + method. It just made sense to me:

The router block at the top plus:

    
    
        class MyResource:
    
           def GET(self,...): ...
    
           def POST(self,...): ...
    
           def PUT(self,...): ...
    
           def DELETE(self,...): ...
    

Looks really nice. Haven't tried serving or rendering templates from it, so
never tested that code.

~~~
tiglionabbit
Check out Flask's MethodView

~~~
rdtsc
Oh Cool. Hadn't seen that before. I like MethodView dispatch pattern.

------
linuxydave
Whatever happened to that Lisp clone?

~~~
swartkrans
I'm not sure if linuxydave is joking, but for those not in on the joke, Hacker
News is written in arc, a Lisp variant. Hacker News resembles early versions
of reddit quite a bit and Reddit was a ycombinator funded startup. Early on
there was a big overlap in the two communities, although Hacker News was quite
a bit more exclusive and limited in scope.

~~~
27182818284
I knew HN was written in Arc, but I wasn't aware that it is _the Lisp clone_
referred to by this piece. Was it?

~~~
nocman
When he said "Whatever happened to that Lisp clone?" I'm pretty sure he was
referring to the part of the article that mentions the folks on comp.lang.lisp
who were discussing writing a competitor to Reddit in Lisp.

The discussion was not about writing a new Lisp or Lisp-like language, but a
clone of Reddit.

The fact that HN is a site that performs a function very similar to Reddit,
and happens to be written in a dialect of Lisp is completely coincidental :-D
(unless there has been some comment by pg about his motivations for making HN
that I missed, but I kind of doubt it).

~~~
linuxydave
>I'm pretty sure he was referring to the part of the article that mentions the
folks on comp.lang.lisp who were discussing writing a competitor to Reddit in
Lisp.

Yep, spot-on. It's the same with those guys who wanted to make a Facebook
clone focused on privacy or whatever. Lots of talk and then you never hear
about the project again.

~~~
ryanfreeborn
You mean Diaspora? (or maybe there's another clone I missed out on). The
founder of Diaspora committed suicide. I think the project mostly died with
him.

~~~
dbdr
Wow, I didn't know about this suicide :(

Is the project really dead though? I have no personal experience, but these
stats show a growing number of active users:
[http://pods.jasonrobinson.me/](http://pods.jasonrobinson.me/)

~~~
ryanfreeborn
Oh interesting. I didn't know it was still moving along. It had a lot of
momentum a few years back, but hasn't been making nearly as much noise since.
Not sure if the founder's suicide actually correlated with the dropoff in
media attention.

------
euphemize
Used Tornado, django and flask quite a bit, web.py only once. As others have
mentioned, it didn't work that well, and the community around it is smaller
than other frameworks. Again today, Django seems a bit cumbersome, Tornado
works well but has a very low bus factor.

Pretty sure flask is currently the front runner in "python microframeworks".

~~~
gsk
What's bus factor?

~~~
jms18
If the author got hit by a bus, there's no one else around to maintain it.

Generally refers to "how many people can continue to develop/support this
software if original maintainer (and close co-developers) get hit by a bus."

------
tim333
For another take on the Reddit rewrite from Steve Huffman see:

[http://www.pyvideo.org/video/234/pycon-2009--keynote--
reddit...](http://www.pyvideo.org/video/234/pycon-2009--keynote--reddit--
steve-huffman-and-al)

25m 40s

------
closetnerd
Its incredible how intelligent this kid was.

~~~
ocb
Just 19 when he wrote this!

------
pshc
We currently use web.py + gevent to run our API where I work. It's really easy
to work with since it's so tiny and simple, and since we only need JSON in and
out for now, it's perfectly sufficient.

Before web.py, we were on a scary Tornado implementation. The API was
rewritten to web.py in a week with a minimal delta. We won't be on web.py
forever, and gevent sometimes kicks up minor fusses, but I think web.py has
given us a nice malleable form that helped us get out of a terrible place, and
we can easily hop off once we decide where we want to be.

RIP aaronsw.

~~~
quasse
I'm not very familiar with Tornado (we've switched to mostly Flask now) but
I'd be interested to know what kinds of problems cropped up that lead you to
switch to web.py

~~~
pshc
Tornado requires asynchronous I/O. Almost all of the existing I/O was done
synchronously. Yeah.

------
rpdillon
I've used CherryPy on a number of projects, and I really like it, but rarely
see it mentioned in discussions of Python web programming. It's light and
minimal, and seems equally deft for writing APIs as conventional web page
driven apps. Flask is mentioned all the time, however. Does anyone have
insight into the popularity of Flask vs. CherryPy?

------
general_failure
Is reddit that simple to rewrite over the weekend?

~~~
wcummings
In 2005, very probably

~~~
keysersosa
Yup that was the case. Though, I believe it was also a long weekend -- I think
it might have been Veteran's day because I remember putting the finishing
touches on it around Thanksgiving.

------
gprasant
They say hell hath no fury like a lisp fan scorned. I wonder if they hadn't
been acquired, would they have still used Lisp ?

~~~
keysersosa
We switched to python about 4 months into the startup and a full year before
getting acquired.

~~~
raldi
And some people still haven't forgiven you.

------
83457
Would clojure fare better now?

------
andyzweb
RIP Aaron ;_;

------
geerlingguy
Just wondering (as someone who only uses/touches Python indirectly, through
some projects that use it, but is not currently a 'Python developer' by any
means); was this posted in response to some recent drama in the Django
community, or as an argument against Lisp?

I'm just wondering why it seems this story is quickly getting upvoted on the
front page of HN, when it seems there are no other frontpage stories remotely
related to Django, Reddit, or Lisp?

~~~
rrradical
I upvoted just because it's a neat, historical document that I missed the
first time around. I have no particular stake in Lisp or Python.

~~~
bhhaskin
Yep, I knew that Reddit was originally written in Lisp and made the change to
Python at some point. It is neat to learn exactly why and how.

~~~
general_failure
What did you learn? That lisp is not maintainable? The article just glosses
over the details.

------
gcb0
> "It means readable HTML with the proper HTTP headers."

said all the other frameworks's authors a few months before him that are now
churning out deprecated headers just like that one will in a couple months,
while gallantly trying to protect the user from handling headers.

~~~
voltagex_
Huh?

