
Curl says bye bye to pipelining - okket
https://daniel.haxx.se/blog/2019/04/06/curl-says-bye-bye-to-pipelining/
======
sohkamyung
Maybe the headline should be edited to "Curl says bye bye to HTTP/1.1
Pipelining" to prevent giving people scanning the HN headlines a heart attack.
:-)

~~~
karmakaze
I had to read a many comments to see how this could be interpreted as
alarming. I guess I assume more context of HM readers.

------
skunkworker
Note, this is about HTTP/1.1 Pipelining being removed not *nix Pipeline.

    
    
       HTTP/1.1 Pipelining was badly supported by curl for a long time in the sense that we had a series of known bugs and it was a fragile feature without enough tests.

~~~
markmm
*nix pipes are part of POSIX and the OS/Shell implements them, if the app takes input and produces output it can be a component of a pipeline.

In short, the Curl developers couldn't remove pipes without crippling the app.

~~~
DougMerritt
Everything you say is correct, and may even be a word to the wise, as it were
-- but there are always _new_ command line apps made by people who haven't
been thoroughly exposed to the true Unix/Linux/Posix way, and need to have
that kind of thing explained. [1] [2] [3]

Which is to say that your comments are not self-evident to everyone, by any
means.

Example: less(1) was eventually fitted with the -R flag to allow highlighting
to be passed by pipe, to make it friendly to piping from highlight-producing
apps rather than its default highlight-unfriendly stripping of input
controls/escapes.

Benign intentions with regard to files sometimes require a little more thought
to be simultaneously friendly to pipes, except for the simplest of plain text
apps.

(I'm not trying to make a big point, but I just rediscovered 'less -R' out of
necessity today. So, I was pondering some of these issues)

Worth a re-read for everyone (and I worry that these things may be in the
process of being forgotten):

[1]
[https://en.wikipedia.org/wiki/Unix_philosophy](https://en.wikipedia.org/wiki/Unix_philosophy)

[2]
[https://en.wikipedia.org/wiki/Unix_philosophy#Eric_Raymond's...](https://en.wikipedia.org/wiki/Unix_philosophy#Eric_Raymond's_17_Unix_Rules)
and the book:

[3]
[https://en.wikipedia.org/wiki/The_Art_of_Unix_Programming](https://en.wikipedia.org/wiki/The_Art_of_Unix_Programming)

[4] Raymond didn't invent any of this, but he explained it very nicely, IMHO.

Edit: P.S. the above is somewhat meandering, and is sure to be criticized
because simple equivalence of piped and file input is the most important
thing, which I agree with -- but I was mixing that with a secondary point
because of what I was doing today. Sorry. "I did not have time to make this
letter shorter" etc.

------
tomchristie
Good call, given that it's not enabled in modern browsers, and is unreliably
supported in proxies and servers.

It's for the best that it becomes widely treated as a non-feature of HTTP/1.1.

~~~
iforgotpassword
Still somewhat baffling that it's apparently so buggy across servers and
proxies that it was for the best to disable it across browsers. In comparison
to http2 multiplexing it seems trivial to get right, yet posts like this one
suggest that http2 works fine everywhere and in every regard, and http1.x
doesn't.

~~~
rakoo
It's because real life http2 makes encryption mandatory, so that middle boxes
can't fiddle with the stream. Thanks to that everything arrives as it was
intended on the server

------
amelius
Why isn't pipelining part of TCP/IP, so the OS can handle it instead of the
application?

~~~
vbezhenar
Because TCP was designed as a single-stream protocol. There's SCTP which
operates with messages rather than streams and can send those messages
simultaneously.

~~~
amelius
Sounds interesting. Is that implemented in kernels? And why wasn't HTTP1.1+
built on top of that?

~~~
Volundr
Pretty much any modern OS supports SCTP. HTTP wasn't built off it because HTTP
came long before SCTP, but you won't find many protocols built off SCTP
anyway, because it's poorly supported by middle-boxes and firewalls. Just look
at any cloud firewall for example, you can open TCP or UDP, but not SCP or any
of the "lesser known" protocols. It's currently in an unfortunate chicken and
egg problem, where crappy firewalls don't support it because it's rarely used,
and it's rarely used because crappy firewalls don't support it, but are
everywhere on the public internet.

HTTP3 actually does propose changing the transport protocol, but to QUIC,
which is a transport over UDP.

~~~
lttlrck
SCTP can be tunneled over UDP - which WebRTC does.

It didn’t get the right traction. It isn’t built into windows, it was even a
second class citizen on Linux not that long ago requiring lksctp to be
installed (maybe it still does).

