

Python For The Web - Mizza
http://gun.io/blog/python-for-the-web/

======
benatkin
> Python is the best language in the world for interacting with the web, and
> I'm going to show you why.

I disagree. I didn't need to read it, but I did, and I still feel that way.

The statement says _language_ , but the actual points made are about the
libraries and the community (i. e. the ecosystem).

I think the ruby ecosystem is actually better, for a few reasons:

* Deep integration with JavaScript (ExecJS and TheRubyRacer are way ahead of what is available in Python, Perl, and PHP)

* Many different excellent choices for talking to HTTP APIs, including the highly innovative Faraday

* A top-notch JVM implementation (JRuby has had more traction than Jython over the last five years)

* Lots of good auth libraries: OAuth (for authenticating with a plethora of outside services), Devise (pre-built auth with confirmation emails, password resets), and OAuth2 provider (server) implementations

* Ruby has evented programming too. Goliath is pretty good.

Python is great but I take issue with saying that it's the best web dev
language out there.

~~~
pyre
It also depends on what you need to do with the web. Perl is currently the
best language for interacting with Unicode, for example. If you web needs
include Unicode, you might have trouble with Ruby or Python, unless you're
willing to accept half-ass, partial, or just plain wrong Unicode
implementations.

~~~
ashrust
I agree, I love Python but I've spent untold hours dealing with its unicode
issues.

~~~
codygman
Everything unencoded is unicode in 3 isn't it?

~~~
lloeki
Strings are unicode. Bytes are encoded data.

With Python 2 strings you have to be proactive about it and \- always use u""
unless you know what you're doing \- segregate non-unicode data to boundaries
by decode()ing early and encode()ing late, or you're guaranteed to shoot
yourself in the foot sooner or later

but if you do it it's quite smooth sailing.

Also,

    
    
        #/usr/bin/env python
        # -*- encoding: utf-8 -*-
    

helps a lot.

All that just brings you closer to Python 3 anyway and really helps when using
2to3.

~~~
ashrust
Agreed, however I've run in to compatibility issues with 3 a few times and so
resolved to stick w/ 2, deal with the text parsing issues myself, keep the
libraries I like and leave the ops team alone.

------
_mattb
Flask is also awesome for python-based web development. I found it very easy
to jump in as the docs are great: <http://flask.pocoo.org/>

Here's the post's Django example written using Flask:
<https://gist.github.com/1296926>

~~~
Mizza
This is really, really cool. Thanks for making this.

I really dig the Flask project (love the website and the docs, and everybody
who uses it raves about it) and I hope to play with it more in the future, but
I don't think I'd recommend it to somebody who is a first timer, largely
because of one issue: Data. Flask leaves you to sort it out on your own, which
is great if you're capable of that, but Django holds your hand, which is more
appropriate for a beginner.

~~~
akavlie
Data? Can you explain that? Not sure what you mean.

~~~
anthonyb
I'm pretty sure he means that there's no database support built in, ie. you
need to go download something like SQLAlchemy, SQLObject, Storm, etc. If
you're already going to do that anyway it's not so much of an issue, but for
people who are just getting their feet wet, it's yet another obstacle to
overcome.

~~~
akavlie
Ah, of course.

And it makes sense -- I think Flask is a much easier introduction to config
and the VC of MVC development... but once you get to the M, well... SQLAlchemy
is great, but the learning curve is steeper. And even though Flask has a well-
documented extension for that, it's still another package and a separate piece
of documentation.

------
mekoka
Very nice article and thanks for the intro to Requests. Though, I'm not sure
we would agree about Django being _the best thing to learn in the long run_.

I undertook to move to Python for web development about 8 months ago and
embarked on a thorough quest for _the one true_ framework. Obviously Django
was the first and most recurrent recommendation. Having briefly toyed with it
in the past, I really could see myself commit to it for the long run. The
various raving reviews also made the choice all that simple. I was all ready
to give it my seal of approval, when I encountered my first complain, which
was so compelling that it raised an eyebrow.

It was about an experienced Python programmer explaining that as time went and
as he progressed with Django, he found himself increasingly swapping parts out
of it for external libraries. SQLAlchemy, WTForms, Jinja2. In the end he had
only the routing module and the admin, which wasn't that big a deal for him.
He was asking what was the point of using a full-stack framework not
necessarily designed with interchangeability in mind, if you end up just using
it like a glue mini-framework?

As I dug deeper, I found more similar complains, all from similarly
experienced developers, who all ended up adopting something else with a _light
plugable base_ approach. I heard of Repose.bfg, Pyramid, Werkzeug and a slew
of other ones, that allow you to get down and dirty fast, while still allowing
you to get big _in the long run_.

Just as you, I was recently asked by a friend wanting to get into web
applications development to recommend a platform to work from. I also did
point to Django, but explained that it wasn't because it's necessarily the
best, but rather because it's the gentler introduction. It comes with
batteries, crutches, first aid kit and a nice box of goodies, perfect for
someone who has no clue what they're doing.

Note that I'm not dissing Django or relegating it as an amateurish framework.
I agonized on my decision and still sometimes experience some Django envy (FYI
I adopted Flask and don't regret it one bit). Nonetheless, it's hard to deny
that it does a particularly good job of introducing newbies to good concepts
fast, while at the same time being notorious for getting in the way of more
experienced developer than some other frameworks.

~~~
anthonyb
The issue that I have with that attitude (as an experienced Python and Django
developer) is that there's a lot more to it than just "swapping out Django's
DB and replacing it with SQLAlchemy". You miss out on most of the
infrastructure around Django's database stuff that makes it so easy and
worthwhile to use.

For example, fixtures. In Django, you can include them in your tests and have
it All Just Work(TM):

[https://docs.djangoproject.com/en/dev/topics/testing/#fixtur...](https://docs.djangoproject.com/en/dev/topics/testing/#fixture-
loading)

You can also use them outside of testing, too:

[https://docs.djangoproject.com/en/dev/howto/initial-
data/#pr...](https://docs.djangoproject.com/en/dev/howto/initial-
data/#providing-initial-data-with-fixtures)

Fixtures in Flask/SQLAlchemy? Not so much - you have to roll your own (crappy)
implementation:

[http://farmdev.com/projects/fixture/using-fixture-with-
pylon...](http://farmdev.com/projects/fixture/using-fixture-with-pylons.html)

[http://flask.pocoo.org/mailinglist/archive/2011/3/6/flask-
sq...](http://flask.pocoo.org/mailinglist/archive/2011/3/6/flask-sqlalchemy-
and-fixtures/)

Also note that, while I'm picking on fixtures here, Django also has a bunch of
other database related features, like introspecting a pre-existing database,
and generating a bunch of Model classes from it. Combine that with South and
you're 90% of the way there when migrating the data from a legacy system.

This has turned into a bit of a rant, but I've seen a lot of half-baked
reimplementations in "pluggable" architectures of stuff which Django just gets
right.

~~~
mekoka
It's not a matter of _attitude_ , it's about having tools, evaluating them for
what they're doing for you and making a choice. There's always a conscious
trade-off.

You can't focus on what's good about Django, as if someone picking SQLAlchemy
or Werkzeug is a fool at a loss. The same goes for the other libraries that I
listed.

In my own case, I'm not particularly a fan of opinionated frameworks. I've had
my share of griefs with them. Django looked nice, but I easily could identify
with the pains those developers went through when dealing with its lack of
flexibility at certain corners. I was new at Python, not at web development, I
did not need the hand holding, no matter how nice and clever the code was.

This was not a one evening process, I read blog posts, forums, perused
StackOverflow and even HN. It took weeks. Feel free to take the journey, then
tell me it didn't give you pause.

Flask is a small framework that is far from pretending to be what Django is.
It's not even at version 1, but with its flaws, I'm quite happy with what it's
allowing me to do.

I don't mind you ranting about my post, just don't assume that I would use
Django the way you do. Stuff that you relish might be what turns me off and
the beauty of it all is that it's all acceptable.

~~~
anthonyb
I really don't understand the "lack of flexibility" thing myself. I've made
Django and Django's ORM and admin do all sorts of weird stuff, and it's much
easier to build on top of that than to reroll everything, which is what you
have to do with Pylons/Flask/Bottle/...

And database/fixture set up is something that you _will_ have to do, assuming
that you test your app, and that your app is more sophisticated than "I have
some strings which I want to upper case". Ditto for working with legacy
databases, migrating data, bla bla blah.

The default option from what I've seen tends to be to set your DB up once and
hope for the best, or else repopulate it after every test. Both of these
options work great(ish) when you first start your project, but then grind you
down six months later when your test suite takes an hour to run. A decent ORM,
along with in-memory SQLite and fixture setup is generally what's settled on
for most integration suites that I've seen, and Django does that out of the
box.

Also - a minor nit, but WTForms and Jinja2 are based on Django's form and
template libraries. Switching to them doesn't get you that much - they
certainly don't replace any of Django's core infrastructure. And I've found
SQLAlchemy to be basically unusable unless you use the new 'declarative base'
stuff, which looks suspiciously similar to... Django's ORM :)

~~~
zzzeek
> And I've found SQLAlchemy to be basically unusable unless you use the new
> 'declarative base' stuff, which looks suspiciously similar to... Django's
> ORM :)

I can relieve your suspicions as I never looked at Django's ORM _at all_ when
designing declarative. It's merely poking normal SQLAlchemy attributes, all of
which existed before Django was ever released, onto a class. I can assure you
active-record style class mapping is not an idea Django invented.

~~~
anthonyb
The same design pressure (to make the simple stuff easier) is at work in both
cases. Much like, say, Bottle and Flask looking very similar.

The "basically unusable" is probably a little harsh, but I've never really
understood the motivation behind some of SQLAlchemy's design decisions. eg.
separating one class/table definition into interacting table, model and
mapping classes makes no sense at least 99% of the time - that seems like
something which should be hidden internally (but accessible if you really need
it).

~~~
zzzeek
well if you read our docs you'll see that they're entirely in agreement with
that. separate mapping/table design is called "classical mapping" and was
years ago superceded by the declarative style. Update your sqlalchemy
knowledge before commenting on it.

interestingly enough a ton of users still prefer the mappers/tables to be
separate.

------
Mizza
I wrote this for a friend who is just getting started out in python/web stuff.
It might be a little n00bly for lots of people here, but it could be
interesting for people who aren't pythonic who are thinking about making the
switch!

~~~
sixtofour
Nicely done, very clear and concise.

A question: is lxml what we would use today instead of Beautiful Soup?

~~~
waterside81
lxml is definitely faster, but I've found BSoup to be more forgiving with
poorly formatted DOMs

~~~
kenneth_reitz
BeautifulSoup is poorly maintained — you have to be very specific with which
version you're using.

Note: Lxml has a number of repair modes that allows it to parse virtually
anything. Cpu cycles and memory go up quite a bit when they're activated, but
it's still better than BeautifulSoup.

------
mccutchen
Note: In Python 2.6+, you can just `import json` instead of needing to install
and import simplejson.

~~~
stock_toaster
I believe that the python2.6 json module (basically forked form simplejson) is
much slower than the current simplejson version. The speedups (simplejson
rewrite) didn't get included until 2.7 as I recall (memory is vague on this
point).

~~~
pjscott
According to my un-reproducible benchmark, simplejson was 27x faster than
built-in json. It's not a small difference.

~~~
stock_toaster
Yeah, I remembered it being significant, but wow. That is larger than I
recalled it being. Probably due to a very unscientific test at the time. ;)

------
jroseattle
This is an article professing love for a specific library in a given language.
Move along, nothing to see here.

------
llambda
First, why is the author recommending SimpleJSON? Just use json, it's built
into the standard lib! (Also if I'm not mistaken the json implementation
included in Python may even be based on simplejson, the APIs are very similar
even if not.)

Second, I disagree that Django is the best web framework. It _might_ be the
best web framework, but it depends on what you're doing. I've come to prefer
Flask for its simplicity and overall the way it feels more Pythonic.

That said, requests cannot be recommended enough! It is an awesome package
that should not be missed if you're doing web programming in Python.

~~~
pyre
requests comes close to Perl's LWP::UserAgent in terms of usability, but
LWP::UserAgent has been around for years. I don't know how that makes Python
the 'best language for interacting with the web.'

~~~
draegtun
And also LWP::UserAgent has two younger whipper snappers biting at its heels!

* HTTP::Tiny - lightweight useragent that comes with 5.14
    
    
      use HTTP::Tiny;
      print HTTP::Tiny->new->get('http://gun.io')->{content};
    

* And the all singing and dancing Mojo::UserAgent
    
    
      use Mojo::UserAgent;
      print Mojo::UserAgent->new
              ->get('https://github.com/timeline.json')
              ->res->json->{repository}->{name};

------
ez77

      import requests
    
      r = requests.get('http://gun.io')
      print r.content
    

_That's it! In only three lines of python, you can grab a whole webpage and
print it to the screen. Awesome!_

Well, that's a poor selling point that unduly diminishes the credibility of
the article. Your audience is likely to know snappy alternatives such as

    
    
      curl gun.io

~~~
blhack
Or even just urllib?

    
    
         import urllib
    
         foo = urllib.urlopen("http://gun.io).read()
         print foo
    

I mean...maybe urllib (or urllib2) are old and unhip at this point, but
they're _heavily_ documented.

I use urllib all over my projects and have never encountered any problems.

~~~
phzbOx
In fact, it's even _better_ than what the author said:

    
    
      import requests
      print requests.get('http://gun.io').content
    

Only _two_ lines!

~~~
kenneth_reitz

        import requests; print requests.get('http://gun.io').content
    

my god, python is cloud scale.

~~~
rbonvall
In one statement:

    
    
        print __import__('requests').get('http://gun.io').content

~~~
kenneth_reitz
+1

------
hartror
I was a little surpised that lxml is used in this example over Beautiful Soup.
Any reason?

~~~
pjscott
It's much faster than Beautiful Soup, better maintained, and does an excellent
job of handling crazy broken HTML.

------
gbrindisi
Why on earth I didn't heard about Requests before? How does it compares
against urllib2?

~~~
mekoka
Where have you been indeed? It's been posted on HN at least twice and made
page 1 both times:

<http://news.ycombinator.com/item?id=2882301>

<http://news.ycombinator.com/item?id=3094695>

------
sethish
In my experience I have to reference the local path to a python file when
using manage.py

`python manage.py runserver` becomes `python ./manage.py runserver`

But your mileage may vary.

~~~
rglullis
Are you doing anything funky with the file or the file system? Windows, Linux
or Mac OS? There should be no need to do what you are saying.

Perhaps you are trying to run "./manage.py runserver" without calling python?
In this case you'd be right that you need to have "./" as part of the command
call, but you still would have to add the shebang to manage.py and make it
executable.

------
eliben
I'm not sure why the avoiding of the Python standard library.

Why import an external json library, when there's one built in?

------
jay_kyburz
But how to I get Python in my browser. Python on the back and front end is my
dream.

~~~
gbrindisi
I don't understand - you can generate html easily with python. Can you
elaborate?

~~~
narag
I guess he means as a replacement for Javascript.

------
5partan
also you have to write: from lxml import html to import the html-parser

~~~
Mizza
Noted!

------
tripzilch
The whole article is a bit heavy on the keyword stuffing, isn't it?

------
phzbOx
To parse Html and XML, BeautifulSoup is _awesome_.

------
nashequilibrium
great post, I enjoyed reading it and following along!!!

~~~
jQueryIsAwesome
You got downvoted for a simple compliment? Many people here are really
smart... but many of them are not nice at all.

