
Httpx: A Ruby HTTP library - damir
https://gitlab.com/honeyryderchuck/httpx#httpx-a-ruby-http-library-for-tomorrow-and-beyond
======
0xFACEFEED
Never understood why people prefer APIs like this:

    
    
      page1, page2, ... = HTTPX.get("/page1", "/page2", ...)
    

Why encode parallelism into a single polymorphic function? Why not use
composition instead? Now you're stuck with this API forever. And if you want
to customize parallel fetches (eg: batching) you'll need another different
looking API.

~~~
chucke
Hi, I maintain httpx. I'll try to answer your concerns.

The API was not a question of preference. If you look at most http lib APIs
(python requests included), a single call returns a response object. My goal
was to keep compatibility with this API to ease the migration effort, while
baking in support for concurrent requests, which was the whole point of the
lib.

There is no parallelism in that call btw. It's in the Readme. If it's http/2,
it'll multiplex requests. If it's http/1, it'll try to pipeline, and if
doesn't work, it'll fetch them one by one (with or without keep-alive). If
they're different origins, the requests will be managed inside an event loop.

> Why not use composition instead?

You'll have to explain with an example of your own. However, there's already a
ticket for implementing an event-based request handling, I just never
prioritized it because no one asked for it yet, no one has shown interest in
contributing, and frankly, I haven't needed it yet.

> Now you're stuck with this API forever.

It's still early 0.x days, so I might still remove APIs I don't think are
future proof. That said, a lot of libs have lived with similar APIs all these
years, and I don't see any reason for removing it.

~~~
michaelbuckbee
Hey, very impressed with httpx. I'm not in a spot to contribute code but would
like to sponsor (if that's an option). My email's in my HN profile and
apologies for dropping this into the thread I couldn't find contact info
elsewhere.

~~~
chucke
Thx for the kind words. I thank, but will not accept financial contributions
from individuals. Feel free to use it, modify it, or talk about it to your
colleagues :)

------
hartator
We are using http.rb at [https://serpapi.com](https://serpapi.com)

It’s interesting to see that the Ruby ecosystem doesn’t seem to find its
“requests” library. There is so many very good competitive libraries: native
net/http, HttpClient by Nahi, http.rb, HttpParty, RestHttp, RestClient, excon,
em-http, faraday, curb, etc., and now Httpx!

~~~
VectorLock
Its almost emblematic of Ruby's mindset vs. Python's "there is one right way
to do something" philosophy.

~~~
hartator
Except for the language itself: Python 2.x vs 3.x are still battling choices
after 10 years :)

~~~
dragonwriter
Ruby had the same problem for a while, and only got over it more easily
because of lower volume and diversity of legacy code (because it had far less
penetration hang Python and almost all of that in Rails at the time of the
1.8/1.9 split.)

------
thunderbong
I've been using httparty, since forever.

Am I right to assume the primary difference with this library is concurrent
connections?

[https://github.com/jnunemaker/httparty](https://github.com/jnunemaker/httparty)

~~~
chucke
httparty is a dsl wrapper around net/http, so it's also limited by it. For
instance, it doesn't come with keep-alive support (I'm using httparty in my
dayjob legacy code).

httpx is more than concurrent connections. It's actually about concurrent
requests done right, advanced http features (it can do connection coalescing,
alt-svc), and it comes with a baked-in plugin system which makes it easy to
extend.

Go through the wiki and give it a try when you can.

------
BerislavLopac
Hmmm... [https://www.encode.io/httpx/](https://www.encode.io/httpx/)

~~~
chucke
That one seems more recent. Would be cool to know if ruby's httpx was the
inspiration behind it (I can only judge looking at the name :) ).

~~~
florimondmanca
Python HTTPX co-maintainer here: AFAIK, Ruby's HTTPX did not inspire Python's
HTTPX.

Eg. the name came from a totally unrelated discussion, and research did not
yield results about Ruby's counterpart back then [0].

In fact, Python HTTPX started with requests-async [1], which was then spun up
into httpcore, which was then renamed to http3, and finally HTTPX.

Still very funny to me that these two projects (Python HTTPX and Ruby HTTPX)
have so much in common: HTTP/2 support, API compatibility with the current de-
facto HTTP library of the language, etc.

[0]: [https://github.com/python-
http/discussions/issues/1#issuecom...](https://github.com/python-
http/discussions/issues/1#issuecomment-510014839)

[1]: [https://github.com/encode/requests-
async](https://github.com/encode/requests-async)

~~~
uranusjr
LOL I was almost posted a ref to HTTPX until I saw this thread :p Thank you
for HTTPX. I only learnt about it recently, but it’s now my go-to tool moving
forward. Excellent work!

------
pbreit
It's astounding to me that so many platforms/languages still have such
mediocre http libraries. Surely basic http capabilities should be built-in or
in standard libraries?

~~~
jsnk
It is part of core Ruby. [https://ruby-
doc.org/stdlib-2.6.5/libdoc/net/http/rdoc/Net/H...](https://ruby-
doc.org/stdlib-2.6.5/libdoc/net/http/rdoc/Net/HTTP.html)

People develop other HTTP packages usually for ergonomic reasons, but the HTTP
from core Ruby works fine.

------
damir
Also, rubyweekly #474 [1] has HTTPX gem listed on top of the issue. Congrats
to HTTPX author on well deserved attention.

1: [https://rubyweekly.com/issues/474](https://rubyweekly.com/issues/474)

------
acejam
This looks interesting, thanks for sharing. I've always had good success with
httprb[1], but I'll give this a look over too.

[1] [https://github.com/httprb/http](https://github.com/httprb/http)

------
kawsper
Does anyone have any plans/thoughts/visions for how we should deal with SRV-
records in our standard libraries in different languages?

Requesting a DNS-server for a SRV-record gives you a prioritised list of IPs
and ports, and it is up to the implementor to pick the right one.

As it works currently you would have to do a lookup, pick a endpoint, and then
feed it into your HTTP library or TCP library.

This is fine in code that you control, but there are so many libraries out
there that expects a hostname or an IP-address with a port number, and in a
highly containerised world endpoints are more short lived than ever, so we
need retry mechanics as well: Try the next server:port pair in the prioritized
list.

------
sandstrom
Great library!

Although choice and experimentation certainly has its value, I'm curious to
hear if there was any attempt to build the missing features into any of the
existing libraries? (for example excon or http.rb which this is based on)

------
conorh
It wasn't until I read a comment here that there was a wiki that I found most
of your documentation. I'd link this higher in your README as it is what I was
looking for when I went to the project (lots of examples):

[https://gitlab.com/honeyryderchuck/httpx/-/wiki_pages/home](https://gitlab.com/honeyryderchuck/httpx/-/wiki_pages/home)

------
damir
I found this gem (pun intended) only after having an issue with original
awesome http.rb gem not being able to talk http2. Happy httpx user ever since.
Ping me if you decide to start accepting donations/support payments.

------
obahareth
This looks like a pretty awesome library and very well-written and documented
library. I'll definitely try it out in some of my Ruby projects and I would
love to see an adapter for Faraday.

Kudos to the author(s).

~~~
chucke
Thank you. And there's already one :)

------
save_ferris
Why would I choose this over Faraday?

~~~
davefp
Faraday is a wrapper around lower-level libraries like this one.

Faraday would likely need some modification to support the concurrency aspect,
but eventually you could be using Faraday _with_ httpx, rather than instead
of.

~~~
save_ferris
Good to know, thanks!

------
fouc
Random comment - I find it interesting that I have such a strong association
of open source = github, so when something is on gitlab, I automatically
deduct mental points, even though that's not fair at all.

I think gitlab's sidebar is probably not helping with my association, it's too
similar to bitbucket or other "enterprise-y" type websites.

~~~
midgetjones
Personally I find the UI and terminology so willfully - almost obstinately -
different to github's, that it's an absolute pain to navigate. I'm sure I
could get used to it, but just now I just wanted to jump to the code, and it
took roughly 30 seconds to find it hidden inside the 'repository' tab.

~~~
manojlds
The link is already in a page similar to the main page of repo on Github. Open
and link and scroll up?

~~~
midgetjones
Ugh, it's a link to the `README`, with a fixed sidebar. I partially take back
my original comment, but I'd still argue that it's not the most helpful UI.

~~~
sytse
How can we make it more helpful?

