
HTTP/2 all the things - matsuu
https://docs.google.com/presentation/d/1l9c9ROjLTD8clOL0yFufAOMbxNC0D-19zCiXMgqtY-M/present
======
teddyh
No mention of SRV records. Of course.

[https://news.ycombinator.com/item?id=8404788](https://news.ycombinator.com/item?id=8404788)

It really is no surprise that Google is not interested in this, since _Google_
does not suffer from any of those problems which using SRV records for HTTP
would solve. It’s only users which could more easily run their own web servers
closer to the edges of the network which would benefit, not the large
companies which has CDNs and BGP AS numbers to fix any shortcomings the hard
way. Google has already done the hard work of solving this problem for
themselves – _of course_ they want to keep the problem for everybody else.

This is going to bite them big time in the end, because Google got large by
indexing the Geocities-style web, where everybody _did_ have their own web
page on a very distributed set of web hosts. What Google is doing is only
contributing to the centralization of the Web, the conversion of the Web into
Facebook, which will, in turn, kill Google, since they then will have _nothing
to index_.

They sort of saw this coming, but their idea of a fix was Google+ – trying to
make sure that _they_ were the ones on top. I think they are still hoping for
this, which is why they won’t allow a decentralized web by using SRV records
in HTTP/2.

~~~
personZ
Of the varied bizarrely negative posts this submission has earned, yours is
the one that perplexes me most. How is HTTP/2 going to "bite" Google (even if
we assume that Google is the only initiator of the HTTP/2 spec, while they're
far from it)?

Is the web, right now, failing? Are those little sites somehow not existing?
You seem to be positing that a new standard not supporting your pet feature is
suddenly going to destroy the web, which seems a little hysterical.

Interesting to note that as a relatively small shop we've already seen huge
benefits from SPDY (the precursor) because it allows us to serve from a single
site in the US, offering a much, much better experience for our users in
Singapore (the benefits to high latency connections are significantly higher
than the results commonly reported).

~~~
teddyh
I thought I expressed myself reasonably clearly. The web is currently slowly
centralizing and turning into Facebook. Google could fight this trend by using
SRV records for HTTP/2\. If, otherwise, this trend continues, Google will
eventually have nothing in the open web to index, and will therefore get no
value from selling ads in their search service. Therefore, Google is doing
itself a disservice by not using SRV records in HTTP/2.

I further speculated about the _reasons_ Google might have for acting the way
the do – i.e. not supporting SRV records. I proposed two reasons: First, SRV
records do not solve any problem which Google has not already solved for
themselves, and they do not want anybody else to simply get the solution for
free, since that would devalue their previous investment. Second, Google
assumed that _they_ would be the ones that the Web would centralize into, and
even since Facebook has appeared, Google created Google+ and tried to push
_it_ as a centralization point, using every pressure point they could muster
(YouTube, etc.) to make this happen. If they could make this happen, they also
would not benefit from a more decentralized web as would be enabled by SRV
records in HTTP/2.

~~~
personZ
_The web is currently slowly centralizing and turning into Facebook. Google
could fight this trend by using SRV records for HTTP /2._

I fail to see how HTTP n.n has anything to do with the web centralizing
towards Facebook, nor have your posts provided any clarity on that.

~~~
teddyh
A new version of HTTP is a perfect (and, in fact, the _only_ ) opportunity to
introduce SRV record usage into the HTTP protocol. The fact that they have
chosen _not_ to do this, even though SRV is essentially made for this – an
extension of the MX record system into a generalized system for any and all
protocols – requires an explanation.

------
insertnickname
It's interesting that modern web developers don't consider that maybe websites
are slow because they keep stuffing them to the brim with every conceivable
script and other resources from 30 different domains. No, of course that's not
why the web is slow. The web is obviously slow because HTTP is too slow.

~~~
acqq
Even worse, to have the delivery faster with HTTP2, the developers will have
to optimize it even more than now. And we know that that phase of optimization
was traditionally ignored by most of those who aren't Google, because it's
hard and the benefits for most of the producers non-existent or non-obvious.
It's economics that will work against the speedups, just like it was without
HTTP2.

"Hey boss, I need people to work on optimization of the delivery of every page
of our site."

"Do we save something? Do we win something?"

"No, but to some of our clients the content will appear in three instead of
four seconds, if don't use the Google analytics and the Facebook button."

"Go away!"

~~~
grey-area
_" Do we save something? Do we win something?"_

Yes, you win at least two things by removing social links or other third party
hosted code:

Security - your site is no longer liable to be hacked via the js you're
including on every page. This _will_ happen to one of these major social
services, perhaps after they run out of money or are acquired by some large
corporation and security becomes an afterthought. At that point millions of
websites across the web will have js injected which steals
data/code/credentials because they are hot loading js on every request from
several domains. I expect it to be a major scandal when it hits and make
people rethink including live dependencies from all over the web on a website.

Speed - users get faster page loads, which directly translates into more
sales/engagement etc.

These are not at all intangible or without measurable results.

It's quite possible to have simple social buttons/links using a url and
traditional web which function in pretty much the same way but send you to a
page to share/like/comment instead of taking over your page with a little
social popup. The real reason services like FB etc want you to include their
js on every page has nothing to do with improving your user experience or
making your website better, it's so that they can improve their tracking, so
you'd also be doing your users a favour from that perspective too, as well as
benefiting the business.

~~~
acqq
I know about buttons, but the site owners do want to run the adds, and the
adds typically don't come from the same site.

And the discussion is about the benefits of HTTP2 optimizations.

------
kator
The first 100ms to 160ms of the Yahoo call is RTB[1] delay. I don't see how
HTTP/2 will fix that. Also notice all the various analytics beacons that will
still be third party in the future. I know many people hate this stuff but
when you're Yahoo! and most of your income is from advertisements on your web
site you will still have to sell that inventory and prove to third parties
they got what they paid for via various "pixels".

I don't doubt there is a lot of room to improve, clearly a lot was learned
from SPDY and other projects. But I do worry that what we're actually doing is
rewarding large companies who can bring all the stuff into one stream and the
small sites will be yet more disadvantaged. I could see a world where this is
used as a way for PHB's[2] to justify "just move our stuff to
Google/Amazon/Apple". These sort of initiatives may incentivize more
centralization and IMHO that is not a good thing.

Also being an old guy(tm) I worry about loosing the ability for humans to talk
to these services or see the conversation in a textual way. Before you explode
on that comment think about the balance of JSON -vs- various binary
serialization solutions. How many of us have chosen a serialization protocol
because "we can read it". I'm not arguing that as a perfect argument, but if
you're already pushing TLS and HPACK then why not at least leave the textual
data in there? I guess I worry that now we head down the RFC hell where only
large bodies can get extensions in the protocol because we only have 256
options on this header and each bit counts. To be fair I've not deep dove the
protocol yet, I suppose theses are just rantings of a guy who read a slide
show.. :)

[1] [http://en.wikipedia.org/wiki/Real-
time_bidding](http://en.wikipedia.org/wiki/Real-time_bidding) [2]
[http://en.wikipedia.org/wiki/Pointy-
haired_Boss](http://en.wikipedia.org/wiki/Pointy-haired_Boss)

~~~
Lukasa
> Also being an old guy(tm) I worry about loosing the ability for humans to
> talk to these services or see the conversation in a textual way.

This is definitely a loss. It's real, it's unfortunate. But the binary-vs-text
change is actually pretty minor: a well-crafted binary protocol permits simple
tools that can be used to debug it.

The actual problem here is that HTTP/2 is highly stateful. Connection state,
stream state, and header compression state are all required to actually
reliably debug a HTTP/2 flow. In practice, that means either your HTTP/2
implementation needs to be able to dump its state, or you need to capture the
entire session to understand it.

For my part, I will not mourn the ability to telnet to a port and type HTTP.
In practice, that approach lacks value in most modern cases. However, I will
mourn the loss of tcpdump as a single HTTP debugging tool. Your
implementations now need to make debugging possible in their own right.

~~~
saurik
> But the binary-vs-text change is actually pretty minor: a well-crafted
> binary protocol permits simple tools that can be used to debug it.

I think the fundamental change is that, given a classic Internet protocol, I
can look at it and learn how it works without having to already have a tool to
understand it. I can look at a stream of POP, SMTP, HTTP, FTP, IRC, NNTP,
IMAP... the list really does go on and on... and even if I've never seen the
protocol before, even if I don't know what the protocol is called, I can sort
of figure out how to speak it.

The downside, of course, being that people then think they can speak these
protocols, and build horrible "almost sort of" clients for them that fail to
interoperate correctly :/. BUT, there is still this interesting egalitarian
concept that these protocols are designed for people in some very real way. I
love network protocols. I've implemented a ton of them over the years. I got
into them at a very young age. Part of the reason why I got into them is
because they were just so darned accessible. It makes me a little sad to see
this aspect go away.

------
angersock
People do realize that this is going to mainly benefit large companies, right?
Anybody who still scatters their resources across many CDNs and whatnot is
probably not going to see a tremendous benefit.

I'm not convinced that the 30-40% improvement in performance (in some cases)
is worth the additional complexity. There are some nice features, but I can't
help but think that this is something being pushed mainly by Google et al.
because it benefits their server farms as opposed to being an objectively good
idea.

At least it should be relatively easy to setup on a server.

~~~
enneff
You don't think the web being almost twice as fast is worth it? Really?

~~~
m_eiman
30% faster... IF you've got your page split up into 78 requests on 12
different hosts using 1.2 MB.

Btw, do they still have the special case handling of headers and whatnot that
hardcodes today's implementation details and "best practies" into the protocol
we're supposed to use for decades ahead?

~~~
Nullabillity
My understanding of the presentation was that the "78 requests over 12 hosts"
thing was a hack that the everyone would benefit from, but only big players
could afford, with this just making it free/cheap/default.

------
mjevans
"push 'tombstone' record to invalidate cache"

For that /alone/ this is technically superior and that's frosting on the cake.

Now if only we could also get DNS servers to reply with similar packages of
useful data.

~~~
Arnt
They do, it's called "additional data". There are issues about cache poisoning
and a bit more, but the concept exists and is used.

------
byuu
Having written an HTTP server+proxy recently, I haven't been super impressed
with HTTP/2 so far. There is some good in it (basically everything I'm not
mentioning), but also a lot of bad.

First, Firefox (and some others) are forcing TLS to use HTTP/2 :
[https://wiki.mozilla.org/Networking/http2](https://wiki.mozilla.org/Networking/http2)
; that's a big deal breaker for a lot of people. Yes, encryption is all well
and good. I'll all for it! But SSL certs either cost money, or you get them
from companies that will want cash to revoke them for you if compromised.
SSL/TLS errors are still a royal bitch (and pop up with less popular
authorities), with browsers warning you of your impending _undoing_ if you
choose to continue (sometimes damn near requiring a blood contract to
override.) They also require extra CPU resources. This can be a problem for a
site that is only hosting kitten pictures or video game news. It's also a
barrier toward people like me experimenting with it, since I now _also_ have
to learn how to use TLS if I just want to toy around with the protocol.

Second, I don't really agree that using a new, custom-made compression
algorithm is a smart way to do headers. We are talking about ~300 _bytes_ of
data per header ... are the bandwidth gains really so superior to outweigh the
CPU costs in having to compress the data, and to overcome the added
programming complexity in working with these headers?

Third, it's really a fundamentally different way to do things. Like the slides
said, you're going to have to really redesign how servers and website packages
serve up content to be optimized for this new model, or else performance may
even be worse than HTTP/1.1 ... having seen the way the real world works, I'm
not very confident that web developers are going to take this seriously
enough, and we'll likely see a lot of "HTTP/1 over HTTP/2" behavior anyway (eg
not taking advantage of server push.) The servers like Apache and nginx can
only go so far toward doing this for you.

Fourth, since it's not backward-compatible, we're pretty much not going to be
able to use HTTP/2 exclusively for another 5 - 10 years. Which, of course,
doesn't mean we shouldn't ever upgrade HTTP/1\. It's just kind of crappy that
we have to basically run two very different HTTP engines that serve content
very differently for the next decade, waiting for people to upgrade their
browsers.

I would have liked to have seen an HTTP/1.2 intermediary step that added a few
extra headers, like 'Server-Push: "filename", ETag'; and perhaps a
specification rule that no HTTP/1.2 request could ever ask for /favicon.ico or
/apple-touch-icon.png. Just that would have reduced countless extra wasteful
connection requests -> 304 Not Modified responses that we have today on
HTTP/1.1, without having to resort to max-age and not being able to instantly
update your site anymore. And it would just silently keep working for HTTP/1.1
users (obviously without the 1.2 benefits.)

...

Also, all of these slides are always pretty sparse. Given that the new header
format is binary, does anyone know how clients are going to go about
requesting HTTP/2 capabilities? Is there a special HTTP/1.1 header? Because
Apache will respond to "GET / HTTP/2" with an HTTP/1.1 OK response at present.
(In fact, it responds with 200 OK even to "GET /
ITS_JUST_A_HARMLESS_LITTLE_BUNNY/3.141592" ...)

~~~
Lukasa
These are all really good points, let me just address one or two.

> We are talking about ~300 bytes of data per header ... are the bandwidth
> gains really so superior to outweigh the CPU costs in having to compress the
> data?

The commonly-cited reason for pursuing header compression is making the very
first request of a session, when your initial congestion window is small. The
goal is to make the request small enough that it can fit in one round-trip so
that you don't have to wait for all the ACKs.

Reducing duplication is great and will help for a lot of cases as well, but my
impression from the WG has been that it's less important than that first round
trip.

Note also that HPACK is really quite efficient in terms of CPU cycles. It
takes relatively few to compress the headers very heavily.

Finally, if you really don't like it, you can turn HPACK off. That gives an
out for constrained devices that genuinely can't spare the cycles.

> Does anyone know how clients are going to go about requesting HTTP/2
> capabilities? Is there a special HTTP/1.1 header? Because Apache will
> respond to "GET / HTTP/2.0" with an HTTP/1.1 OK response at present.

Yes, because there's a draft spec. I invite you to read it[0]. The summary is
that you have three options:

1\. For plaintext HTTP/2, you use a HTTP Upgrade: header, just as everyone
expects you to. 2\. For HTTP/2 over TLS you use ALPN, a feature of TLS that
lets you request a protocol in the TLS negotiation (to avoid an extra round
trip). 3\. Alternatively, you might be able to use HTTP Alternative Services
(a WIP draft).

The behaviour Apache is using is clearly a bug.

[0]: [https://tools.ietf.org/html/draft-ietf-httpbis-
http2](https://tools.ietf.org/html/draft-ietf-httpbis-http2)

~~~
byuu
Thanks for the response!

> The goal is to make the request small enough that it can fit in one round-
> trip so that you don't have to wait for all the ACKs.

Thank you, that makes sense. I definitely see how bloated some of these
headers can get these days with super-long user-agent strings, x-foo values,
accepted modes, etc. If this is really going to make a substantial number of
initial requests and responses fit into a single round-trip, then that sounds
like a decent rationale.

> Note also that HPACK is really quite efficient in terms of CPU cycles.

Is there a C implementation of it? I had quite the bitch of a time to
implement INFLATE in C, taking about 10KB of super-nasty code at a minimum. I
hope that pain isn't repeated here. Something like just plain LZSS would be a
real treat because that's maybe five lines of code. Of course, compressing
with LZSS isn't very efficient in the naive case. I'm guessing HPACK is
probably an entropy coder ... I'll Google around for more on HPACK, thanks for
providing the name to me.

> Finally, if you really don't like it, you can turn HPACK off.

That's really interesting, and good to hear. So that would have to mean the
initial request from the client cannot be compressed (eg your Upgrade header),
and the server then must decide what to send back. Hopefully we don't end up
with any clients that ignore you and start sending their subsequent request
headers compressed anyway.

> 1\. For plaintext HTTP/2, you use a HTTP Upgrade: header, just as everyone
> expects you to.

So is this going to be yet another "your PC boots in 16-bit real mode in
2014", where servers have to keep switching from HTTP/1 to HTTP/2 on each new
startup forevermore? :/

I mean, I don't know what the alternative would be other than header
compatibility with 1.1 ...

In this case, will the server and client be able to leave that initial HTTP/1
Upgrade: connection open, and just start using it as HTTP/2, eg sending data
back and forth in turn? (I can't see why not.)

~~~
Lukasa
No problem!

> Is there a C implementation of it?

Currently there's no good standalone version, because we're still at interop
stage. Take a look at nghttp2[0], however. They've got a HPACK layer that I
believe can be separated out. I've got a plan to actually do that work for
them, but haven't had time to sit down and do it.

Nevertheless, theirs is the best implementation around at the moment in terms
of compression efficiency. My own HTTP/2 library can optionally use it.

> Will all clients respect this, or will they start trying to send you HPACK
> requests anyway?

So we need to be clear on what this means. The actual binary format of headers
via HPACK cannot be turned off. However, all the _compression_ can be. This
isn't optional: the other party must respect your preferences.

> So is this going to be yet another "your PC boots in 16-bit real mode in
> 2014", where servers have to keep switching from HTTP/1 to HTTP/2 on each
> new startup forevermore?

Hopefully no. _Hopefully_.

You're allowed to start straight away with HTTP/2 if you have some form of
'prior knowledge' that HTTP/2 will work. What 'prior knowledge' actually
consists of is not entirely clear-cut, but in principle you can go straight to
HTTP/2.

Note also that Upgrade: is only a problem for plaintext HTTP/2\. If you use
TLS, going to HTTP/2 is exactly as fast as going to HTTP/1.1 (because the
negotiation is done at the same time as the TLS negotiation).

However, servers are clearly going to have to support HTTP/1.1 for a long time
to come. This is unfortunate, but unavoidable.

[0]: [http://nghttp2.org/](http://nghttp2.org/)

~~~
mohamedattahri
When you consider headers per request, we're talking bytes. But at the
Internet scale, it's gigantic.

Accept: image/gif, image/jpeg, image/pjpeg, application/x-ms-application,
application/vnd.ms-xpsdocument, application/xaml+xml, application/x-ms-xbap,
application/x-shockwave-flash, application/x-silverlight-2-b2,
application/x-silverlight, application/vnd.ms-excel, application/vnd.ms-
powerpoint, application/msword, _/_

This is an Accept header on IE8 with Office installed. Back when it had over
55% of market share, Google alone estimated that as much as 38Gb of useless
garbage internet traffic was sent its way every single day. And that was back
when the Internet population was half was it is today, web apps were were
primitive, and mobile and the network constraints were nascent.

~~~
lucaspiller
For the majority of requests I'd guess the Accept header isn't even used. Do
people still write applications that send a PNG if supported, falling back to
a GIF otherwise? Or use this to determine the user's language preference? I
expect in most requests they are ignored and the server sends whatever it has.
APIs would be the only thing I can think of, but even then you could get
around that by having different endpoints for different formats. At the cost
of saving a few TB (if not more) of Internet traffic a day it seems a fair
trade off :-)

~~~
wmf
Content negotiation is getting more popular with WebP, although mostly on
large sites like Google and Facebook.

------
_stephan
The current specification of priorities in HTTP/2 seems problematic (and
apparently was agreed upon by coin toss):
[http://lists.w3.org/Archives/Public/ietf-http-
wg/2014OctDec/...](http://lists.w3.org/Archives/Public/ietf-http-
wg/2014OctDec/0450.html)

------
higherpurpose
It's a shame they dropped mandatory encryption. Now "HTTP2 all the things"
doesn't mean "encrypt all the things" anymore.

~~~
Lukasa
In practice it very nearly does. Only IE plan to support plaintext HTTP/2, all
other browsers will only support it over TLS.

Opportunistic Encryption is still on the table as well.

~~~
higherpurpose
Ah, once again Microsoft is the one that supports _less_ security than the
other major companies. Microsoft needs to distance itself a bit more from law
enforcement agencies.

~~~
Lukasa
I got the impression that Microsoft is backing this because of pressure from
large enterprises, which do not want to deploy TLS in their intranets. I don't
believe IE will _prefer_ plaintext HTTP/2 to TLS HTTP/2, and it will certainly
deploy both.

Put another way: I don't believe Microsoft is worse than Google in this
regard.

------
BorisMelnik
Wow what a great history lesson. I had no idea HTTP 1 (or .9) was basically an
"idea framework" for the www. Sounds like this will help solve some major
issues and organize the interwebs a bit better.

------
adam-a
I'm curious about how the Header tables are supposed to work. How does the
client reliably know which request was last received by the server? In the
example

> method: GET > path: /resource > ...

can be followed by just

> path: /other_resource

but how do I know a badly behaving router didn't delay a DELETE request from
earlier? Do I have to manually table all my responses to make sure there are
no potentially dangerous requests on the wire?

~~~
Lukasa
There are a few key points here.

First, Ilya's slides are out of date. The reference set got removed (it was a
complexity nightmare), so there's a bit less efficiency in deltas but a
substantially simpler algorithm for header sets.

As for your badly behaved router, there are two options. The first is that
it's a router, not a proxy. In that case, the strict ordering of TCP ensures
that there's no problem here. A router cannot cause the server to miss a
request without knowing about it.

Assuming by 'router' you meant 'proxy', the key here is that HPACK compression
state is hop-by-hop, not end-to-end. That is, HPACK compression state is only
synchronised at each end of a TCP connection. The strict ordering of TCP
ensures that those two ends cannot be confused about the order of requests.
Across TCP connections, there's new HPACK state.

~~~
adam-a
Ah thanks for explaining it. I'm so used to thinking of http as a set of
concurrent requests, but as a single stream optimisations like this make
sense.

------
dsr_
What's the number one thing people who run web servers are concerned about?

It's not performance.

It's not customer experience.

It's reachability.

When a person with a browser clicks, they have to receive the page they
clicked on before anything else can happen. Seems obvious, right?

In order to do that, they need a web server that they trust. Most people opt
for Apache, nginx or IIS. Which of those has well-tested and trustworthy
HTTP/2 implementations?

~~~
wmf
All of them will.

------
est
I wonder what's QUIC's role in this? An UDP based multi-home roaming protocol
looks so much better for today's mobile internet world.

~~~
Lukasa
HTTP/2 is redesigning the application layer protocol for the modern internet.
QUIC is an attempt to redesign the transport layer. The two should work hand-
in-hand. It'll be interesting to see how it works out!

------
devanti
i'm assuming clients and web servers that start to support http2 will also be
backwards compatible with http1.1 which seems necessary

------
hnmcs
Hell yeah, Ilya Grigorik.

[https://www.igvita.com/archives](https://www.igvita.com/archives)

"A word to the wise is sufficient." When he speaks, it's quite often relevant
at a quantum level beyond your average tech post or presentation.

