
Requests: HTTP for Humans - DanielRibeiro
http://docs.python-requests.org/en/latest/index.html
======
watmough
I don't know much Python, but I'd likely be correct in assuming there are
quite a few proven http client libraries in existence.

This looks like another reason to believe that if you build a better solution,
even when the space is well-covered, that you can still win adoption, so
maybe, a lot of people can stop worrying about being second.

~~~
SoftwareMaven
Being second can often be beneficial. You get to learn from the mistakes of
the first without being locked into those mistakes.

------
waterside81
Awesome stuff. How many hours I've wasted futzing around with urllib2 and
creating my own Request handlers and all that other nonsense. This lends
itself nicely to working with 3rd party APIs (Facebook, Twitter) in
particular.

~~~
espeed
I used httplib2 (<http://code.google.com/p/httplib2/>) for the REST module in
Bulbflow -- httplib2 is by Joe Gregorio of Google, and it's not bad.

~~~
RyanMcGreal
httplib2 has a very nice API, but be warned that it's not thread safe.

------
smiley325
I've noticed that the layout of the site is uncannily similar to flask's
documentation (<http://flask.pocoo.org/docs/>), down to the "Fork me on
Github" in the top right corner. Is there some sort of software producing this
kind of documentation?

~~~
forsaken
Yes. They're both created using Sphinx: <http://sphinx.pocoo.org/>

Requests is hosted on a site called Read the Docs, which does documentation
hosting for Sphinx projects: <http://readthedocs.org/>

Sphinx has become the defacto documentation tool for Python projects, and a
lot of other projets as well. Notably a number of PHP projects, and Varnish
the web cache use it also.

~~~
ludwigvan
It should also be noted that Sphinx was developed by Pocoo, the same group
that develops Flask, Jinja, Werkzeug.

<http://pocoo.org>

~~~
lightcatcher
And also Pygments (probably the most popular syntax highlighting libraries)
and they run <http://paste.pocoo.org/> , an excellent pastebin.

<http://www.pocoo.org/projects/#project-hub> shows all of their projects.
Pocoo significantly improves the lives of all Python developers. Thanks pocoo
team!

------
izak30
For those interested in getting to know requests vs urllib2, I gave a talk @
PyOhio, covered some simple examples in each (and why requests is super great)
[http://python.mirocommunity.org/video/4388/pyohio-2011-consu...](http://python.mirocommunity.org/video/4388/pyohio-2011-consuming-
web-apis)

------
ludwigvan
Another useful Python library by Kenneth Reitz is tablib, "Python Module for
Tabular Datasets in XLS, CSV, JSON, YAML, &c. ". See
<https://github.com/kennethreitz/tablib>

~~~
ez77
That looks extremely useful. Thanks! Project webpage: <http://tablib.org>

------
alexatkeplar
Using this with DictShield (<https://github.com/j2labs/dictshield>) or
Colander (<https://docs.pylonsproject.org/projects/colander/dev/>) would be
very cool methinks

------
rubergly
I'm almost frustrated to see this. I just learned to use urllib2, and found
that it took a great deal of effort to get any reasonable behavior. I wish I
had known about this before. It really looks great and I will try to use it in
any future work.

~~~
kenneth_reitz
You could be a contributor then ;)

------
joeshaw
A coworker introduced me to Requests a couple of weeks ago. It makes life a
whole lot easier.

It does highlight my biggest gripe about Python, though. I love the language,
but the class library is hit-and-miss, and anything to do with HTTP is a
pretty big miss. httplib, urllib, urllib2, cookielib -- yuck. I like that
Requests has a nice API, but it's still built on top of pretty bad class libs
underneath. (And the Requests API does expose some warts, like having to use
cookielib's CookieJars.)

I'd really love to see someone's take on a clean break from the standard
modules with a simple, maleable API.

~~~
kevingessner
If you're looking for a "clean break", Google's httplib2 [1] might be what you
want. It's somewhat lower-level than Requests, but adds a bunch of advanced
stuff like caching support and keep-alives. As you might guess from the name,
httplib2 is more like httplib, while Requests appears to be more like urllib2.

1: <http://code.google.com/p/httplib2/>

~~~
acdha
httplib2 is part of why you should use requests: it's far more respectable as
a client but not as well documented and it still takes _way_ too much code for
basic operations. I appreciate what httplib2 is trying to do, that there's a
ton of hard low-level annoyances in building a modern HTTP client, but really,
just use requests instead. Kenneth Reitz is very motivated and he gets the
degree to which simple things should be simple whereas httplib2 feels more
like an academic exercise than something people should use to build production
systems[1].

Disclosure: I'm listed in the requests AUTHORS file but can claim credit for,
oh, about 0.0001% of the awesomeness.

1\. <http://code.google.com/p/httplib2/issues/detail?id=96> is a good example:
an annoying bug which affect many people, there was a fix available for
months, which worked great when I applied it in a fork and pounded a couple TB
of data through it, but it took over a year to make it into trunk and even
longer to make it onto PyPI where any other project which required "httplib2"
would get the working version.

------
dshah
Have been using this for a little while now. Totally awesome.

The simple example on the home page should have one more line of code that
shows that you use r.content to get to the content that came back from the
request. That would seem to be the most common use case.

~~~
beaumartinez
So that's why I couldn't get it to work—have a look at this[1].

Thanks for teaching me what the documentation failed to—it's neither in the
front page example, or in the API documentation for Response.

[1] <https://github.com/kennethreitz/requests/issues/105>

~~~
kenneth_reitz
Fixing now :)

------
sayrer
This library seems like a good idea. I hope it is or will be based on Joe
Gregorio's httplib2, so it gets close to correct.

Also, I can't believe it includes PATCH support. Surprising, but good, to see
that out in the wild.

~~~
StavrosK
Why not just include arbitrary verbs? The rest of the protocol doesn't change
at all...

~~~
izak30
There is support for arbitrary verbs, just more direct API support for the
common ones.

here's the api for a generic request, get is right under it:
[https://github.com/kennethreitz/requests/blob/develop/reques...](https://github.com/kennethreitz/requests/blob/develop/requests/api.py#L21)

------
stock_toaster
This looks neat. I have been using restkit[1] for a little while now, but I am
certainly going to check this out.

[1]: <http://benoitc.github.com/restkit/>

~~~
jokull
+1 for restkit. It has a Pythonic interface, shortcuts and completely replaces
urllib, urllib2 and httplib all of which I've had some beef with before. Love
restkit!

------
danielpal
Fascinating, I am really glad that "we" the new wave of programmers are not
only conserned about code quality but are also deeply concerned with
"usability". Everyday, as I go back and forth using old libs vs new libs, its
obvious that the quality of old libs is pretty damn good but they have crappy
"usability". New libs have a bit more bugs, being that they are also "newer",
but they are a pleasure to use.

------
simonw
I wish someone would write an equivalent of this for Python's myriad of
horrible process management functions. popen, popen2, popen3 etc were a
nightmare, and the newer subprocess module still doesn't have an API that I'd
call memorable or obvious.

~~~
kenneth_reitz
I am planning on this actually! HTTP lends itself well to running a process.

It will likely be very call+result oriented at first (no interaction w/ a
running process)

------
dickeytk
Wow I was just about to start going through the pain of using urllib2 for the
millionth time right before I read this.

------
maxogden
equivalent in node: <http://github.com/mikeal/request> equivalent in jquery:
<https://github.com/iriscouch/request_jquery>

~~~
losvedir
How do those handle SSL and user authentication? I skimmed your links but
didn't see anything.

Python's urllib2 isn't _that_ bad for the simple stuff shown in your links,
but it's a mess when you have to try to log in somewhere. Seeing in the
example that logging in is as straightforward as

    
    
        requests.get('https://api.github.com', auth=('user', 'pass'))
    

just blew me away.

~~~
EiZei
Yeah the authentication parts of urllib2 remind me of the horrible 20-letter
classes glued together in Java, just the thing I wanted to escape from by
using Python. Seeing that I'm trying to make this script interface with a
website that uses both cookies and http auth this library seems like a
godsend.

------
bialecki
Quick question on the code. I'm looking at this commit:

[https://github.com/kennethreitz/requests/commit/773630b0109c...](https://github.com/kennethreitz/requests/commit/773630b0109c7671930b7ab38bb71c22ac8a94c5#L0R16)

and wondering what "from .status_codes import codes" does?

Specifically the fact that it's not "status_codes" and has a dot prepended. Is
that something special in Python (guessing no) or some convention that is
used? Just curious.

~~~
bialecki
Just needed to look at the docs:
[http://docs.python.org/reference/simple_stmts.html#the-
impor...](http://docs.python.org/reference/simple_stmts.html#the-import-
statement).

The relevant bit:

"When specifying what module to import you do not have to specify the absolute
name of the module. When a module or package is contained within another
package it is possible to make a relative import within the same top package
without having to mention the package name. By using leading dots in the
specified module or package after from you can specify how high to traverse up
the current package hierarchy without specifying exact names. One leading dot
means the current package where the module making the import exists. Two dots
means up one package level. Three dots is up two levels, etc. So if you
execute from . import mod from a module in the pkg package then you will end
up importing pkg.mod. If you execute from ..subpkg2 import mod from within
pkg.subpkg1 you will import pkg.subpkg2.mod. The specification for relative
imports is contained within PEP 328."

Cool stuff.

~~~
dheerosaur
Really cool stuff indeed. Thank you.

------
st3fan
This is a great library. I've used it in a couple of projects now and it is
great.

This is the kind of stuff that I would love to see in the Python standard
library.

------
pestaa
Quickly judging by the look, I thought "hey, Armin Ronacher just released
another library!", but this one seems as well thought out as if it was
designed by Armin himself.

------
tsycho
Does any one know of a good Ruby equivalent of such a library?

~~~
kennystone
<https://github.com/archiloque/rest-client>

------
eddieplan9
The example written in urllib2 instantly reminds me of Java and the classy
query joke [<http://ejohn.org/blog/classy-query/>]. Just a thought: are most
Java-related APIs over-engineered, and does Java have a bad influence on many
API design work?

------
perfunctory
This seems to be a fantastic project. I wish more libraries were designed this
way. Library's usability is just as important as performance or feature
completeness.

------
sdfjkl
This was much needed.

------
serichsen
By the way, Common Lisp has Drakma for this: <http://weitz.de/drakma/>

------
abecedarius
From the title I thought this would be some kind of mechanical-Turk webserver.

------
anoopks
Does anyone know of such an API in Java - HTTP for Humans in Java?

~~~
ollysb
Restlet is a good attempt at writing an api that matches the http spec well.
You can find docs for a client at
[http://wiki.restlet.org/docs_2.1/13-restlet/21-restlet/318-r...](http://wiki.restlet.org/docs_2.1/13-restlet/21-restlet/318-restlet/320-restlet.html).
It's still more verbose than it might be, would be a good starting point for
writing a decent api though.

~~~
nl
Restlet is a server framework.

I think the parent poster is looking for a client API that is nicer to use
than java.net.Url and/or the various versions of HTTPClient

~~~
ollysb
From the restlet home page "Do you want to blend your web services, web sites
and web clients into unified web applications exposing and consuming RESTful
web APIs?"

The link I provided was to an example of using restlet on the client side. I
would agree that the api is not as human friendly as it could be but it does
describe interactions in a restful which is why I suggested it would be a good
starting point for writing a friendly api.

------
notmyname
how does this compare with webob
(<http://docs.webob.org/en/latest/index.html>)?

~~~
ianb
People will rightly tell you that it does not. But I must admit that this did
get me to thinking about a few small changes to WebOb along with the
integration of WSGIProxy, at which point there would be some real
similarities.

------
marcamillion
Not to start a language pissing war...but that syntax looks awfully like Ruby.

That means it's pretty elegant and nice :)

