
Varnish author criticizing HTTP/2.0 proposals - cgbystrom
http://lists.w3.org/Archives/Public/ietf-http-wg/2012JulSep/0172.html
======
rickmb
After the whole IPv6 story, I'm surprised the author ignores the political
dimension of designing a new protocol. As Mitch Kapor said "Architecture is
Politics". It's not just about what solution is best from a technological
perspective, it's about what we want our future to look like.

The internet has become way more important than back when these protocols
first became standard, and every time a protocol or standard is up for debate,
political and commercial forces try to influence it in their favor. Some of
the concepts they tried to shove into IPv6 were downright evil, and would have
killed the internet as we know it. Personally, I'm relieved all that is left
is a small, un-sexy improvement which albeit slowly, will eventually spread
and solve the only really critical problem we have with IPv4.

I really dread subjecting HTTP to that process. Although I fully agree with
the author's critique of cookies for instance, the idea of replacing them with
something "better" frankly scares the crap out of me. Especially when the word
"identity" is being used. You just know what kind of suggestions some powerful
parties will come up with if you open this up for debate, and fighting that
will take up all of the energy that should be put towards improving what we
already have.

As techies we should learn to accept design flaws and slow adoption and look
at the bigger picture of the social and political impact of technology: HTTP
may be flawed, but things could be way, way worse.

~~~
maggit
You don't go to the author of Varnish for political advice, you go to the
author of Varnish for _technical_ advice. I'd be disappointed if he took
advantage of his good name to push political advice when he got his reputation
for being good at technical matters.

After all, is any of his technical advice invalid due to political concerns
that are not wild speculation on your part?

\----

Not to say that politics doesn't enter into it, just that it should be brought
to the table and discussed by other actors. And those actors should probably
be all ears about the technical issues.

~~~
batista
> _Not to say that politics doesn't enter into it, just that it should be
> brought to the table and discussed by other actors._

How so? You seem to be distinguishing between politics actors (politicians?)
and technical actors.

In a democracy it is not just important but essential that ALL have their say
on policy, not just "political experts".

------
sciurus
His proposal is at [http://phk.freebsd.dk/misc/draft-kamp-httpbis-
http-20-archit...](http://phk.freebsd.dk/misc/draft-kamp-httpbis-
http-20-architecture-01.txt)

It comes with the caveats that "Please disregard any strangeness in the
boilerplate, I may not thrown all the right spells at xml2rfc, and also note
that I have subsequently changed my mind on certain subjects, most notably
Cookies which should simply be exterminated from HTTP/2.0, and replaced with a
stable session/identity concept which does not make it possible or necessary
for servers to store data on the clients."

~~~
pdeuchler
If I'm interpreting that correctly, that could be a huge win for privacy

~~~
quotemstr
How so? A unique identifier is a unique identifier.

~~~
ars
If the browser created the unique identifier instead of the server it could
create a new one as often as necessary.

I think that would actually work very nicely.

~~~
rmc
Browsers can do that now with cookies, by just clearing old cookies.

------
pilif
While in general I understand where he is coming from, I believe his main
argument about adoption is flawed.

What do you think is more likely going to be adopted? A protocol that's not
backwards compatible at all (heck, it even throws out cookies) or something
that works over the existing protocol, negotiating extended support and then
switching to that while continuing to work the exact same way for both old
clients and the applications running behind it?

See SPDY which is a candidate for becoming HTTP 2.0. People are ALREADY
running that or at least eager to try it. I don't think for a second that SPDY
is having the adoption problems of ipv6, SNI issues aside.

Even if native sessions would be a cool feature, how many years do you believe
it takes before something like that can be reliably used? We're still wary of
supporting stuff that a 11 years old browser didn't support.

~~~
nl
_I don't think for a second that SPDY is having the adoption problems of ipv6,
SNI issues aside._

Google being able to modify both the client (Chrome) as well as few fairly
significant server installations has kind of helped there a little bit...

~~~
obtu
The key difference with IPv6 is that SPDY is opaque to routers and only needs
changes at the endpoints. IPv6 has the chicken-and-egg problem that it only
becomes usable once everyone (hosters, ISPs, core networks, home networks, web
frameworks, client devices) has deployed it, and most of those actors don't
bother with things that bring no immediate benefit.

~~~
phkamp
You do know that when IPv6 roll-out was initially contemplated, it would
happen on IPv4-convertible addresses only, until everybody had converted. That
would have allowed seamless IPv6/IPv4 interop and made migration a breeze.
Lacking any "must have" features, nobody could see a reason to bother. Now we
have to do the migration without the benefit of seamless conversion.

~~~
comex
Luckily, HTTP only requires the browser (which tend to be frequently updated)
and server to be upgraded, not some random aging routers, there is little
reason to fear adoption, there is minimal harm in keeping both protocols
around for some time, and there is something much closer to a "must have"
feature - speed.

~~~
ryanpetrich
A lot of hardware (proxies, DPI gear, hotel wifi hotspots, cellular carriers,
etc) sniffs/mangles HTTP in transit and must be verified to be compatible with
new HTTP features. HTTPS doesn't have this constraint.

------
regularfry
How protocol adoption happens, in three steps:

1\. Working code. 2\. Publicity. 3\. Ubiquity.

That's it. Kamp is making a lot of the right noises here, but he's already
lost ground to SPDY just because they've shipped code. No amount of sitting
round tables bashing out the finer details of a better spec will help as much
as getting code written - even if it's just a placeholder for an extensible
spec, as long as that placeholder does something useful.

~~~
gioele
> he's already lost ground to SPDY just because they've shipped code.

Is SPDY good enough now and fixable enough in the future to leverage this
gained ground?

As more and more people deploy SPDY, they will understand its problems and
there will almost no chances to change it, differently from what Google did at
the beginning (they have gone through 1 big change and many small changes in
the protocol). When the authors of the main servers (varnish, apache, nginx)
will start feeling its limits, will they have to keep it around just for sake
of compatibility?

Please note that SPDY on the server is not a requirement, just an opportunity.
Nothing will change for your users if you want to remove support for SPDY in
your server after you deployed it and used it for some time. There are no
"http+spdy://example.org" URL around, so supporting HTTP only will always be
sufficient. Maybe not as much performant as SPDY but 100% supported.

------
rdsubhas
(These are just my opinions as web developer. Please feel free to downvote if
my expectations are wrong)

While I can completely agree with the technical merits of this proposal, there
are some very two-faced statements.

Author begins by pointing out the painfulness of IPv4 to IPv6, says that the
next HTTP upgrade should be _humble_. But then proceeds to kill cookies and
remove all the architectural problems in HTTP. Isn't that the same what IPv6
was? Wouldn't such an approach produce the same amount of pain to the
implementors (that is us, the web developers)?

Any upgrade will certainly have some backward-incompatible changes. But if it
is _totally_ backward incompatible, I don't understand why it still needs to
be called HTTP. Couldn't we just call it SPDY v2 instead, or some other fancy
name?

Cookies are a problem. But the safest way to solve that problem is in
isolation. Try to come up with some separate protocol extension, see if it
works out, throw it away if it doesn't. But why marry the entire future of
HTTP with such a do-or-die change?

I blindly agree with the author that SPDY is architecturally flawed. But why
is it being advocated in such big numbers? Even Facebook (deeply at war with
Google) is embracing it. It's because SPDY doesn't break existing
applications. Just install mod_spdy to get started. But removing cookies? What
happens to the millions of web apps deployed today, which have $COOKIE and
set_cookie statements everywhere in the code? How do I branch them out and
serve separate versions of the same application, one for HTTP/1.1 and another
for HTTP/2.0?

More doubts keep coming... Problem with SPDY compressing HTTP headers? Use
SPDY only for communication over the internet. Within the server's data
center, or within the client's organization - keep serving normal HTTP. There
are no bandwidth problems within there. Just make Varnish and the target
server speak via SPDY, that is where the real gains are.

I could go on. I'm not trying to say that the author's suggestions are wrong.
They are important and technically good. But the way they should be taken up
and implemented, without pain to us developers, doesn't have to be HTTP/2.0.
Good ideas don't need to be forced down others throats.

~~~
phkamp
No. Ipv6 is Ipv4 with bigger addresses, it didn't try to solve any of the
other problems of IPv4 (or attempts to solve them were killed by ISPs).

One example was multihoming (having more than one ISP) serveral smart
proposals were floated (anycast, nearcast etc) but they were killed by ISP's
who protected a lucrative business.

If Ipv6 had made multi-ISP multihoming possible without all the trouble of
BGP, business would have killed to get it back in the late 1990ies.

Cookies only disappear from the wire, they are trivial to simulate on your
server (see my other reply here).

~~~
jerf
"Ipv6 is Ipv4 with bigger addresses,"

Yeah, I used to think that, then I participated in some IPv6 conversions and
watched some others. I don't think that any more. IPv6 may not be the Glorious
Solution to All Network Problems Ever, but it's not just the obvious
incremental improvement on IPv4 either. It's a new protocol.

(I do sometimes wonder if an IPv4.1 that simply set a flag and used 8 bytes
instead of 4 was proposed _right now_ if it could still beat IPv6 out to the
field even with IPv6's head start. Note, I'm not saying this would necessarily
be a good idea, I just find myself wondering if IPv4.1 could still
hypothetically beat IPv6 to deployment.)

~~~
romaniv
What practical problems, aside from address range exhaustion, does it solve?
I've read some technical articles about benefits of IPv6, but most of them
keep returning to address size problem.

~~~
AnIrishDuck
I think a big part of this is address range exhaustion is the root cause of
many other problems. For example, IPv6 effectively obsoletes NAT, which
removes all kinds of complexity from many protocols (off the top off my head:
IPSec, many P2P protocols).

IPv6 also brings saner (IMO) protocol headers, and introduces a variety of
other incrementally improved protocols (see ICMPv6, DHCPv6) that have been
tweaked with the benefit of years of deployment experience.

------
moe
I have a lot of respect for phkamp, varnish is an impressive piece of
engineering.

I disagree with the stab he takes at cookie-sessions here, though. He seems to
ignore that sessions are not only about identity but also about state.

Servers should be stateless, therefor client-sessions (crypt+signed) are
usually preferable over server-sessions.

Having a few more bytes of cookie-payload is normally an order of magnitude
cheaper (in terms of latency) than performing the respective lookups server-
side for every request. Very low bandwidth links might disagree, but that's a
corner-case and with cookies we always have the choice.

Removing cookies in favor of a "client-id" would effectively remove the
session-pattern that has proven optimal for the vast majority of websites.

~~~
phkamp
No, servers should hold _all the state_ and clients none. If I add something
to my shopping basket from my mobile phone, I want to be able to add more from
my browser, rather than have two shopping baskets.

Servers storing stuff on the clients is just plain wrong, and it is wrong from
every single angle you can view it: Its wrong from a privacy point of view,
it's wrong from a cost-allocation point of view, it's wrong from an
architecture point of view and it's wrong from a protocol point of view.

But it was a quick hack to add to HTTP in a hurry back in the dotcom days.

It must die now.

~~~
moe
_No, servers should hold _all the state_ and clients none._

That's the opposite of the general consensus in the webdev-community.

Client-state is not only vastly more efficient in many cases but it also
usually leads to cleaner designs and easier scaling.

Many of the modern desktop-like webapps would be outright infeasible without
client-state. What's your response to that, should we just refrain from making
such apps in the browser?

 _If I add something to my shopping basket from my mobile phone, I want to be
able to add more from my browser_

And at the same time you probably appreciate when on your slow mobile-link the
"add-basket" operation happens asynchronously, yet doesn't get lost when you
refresh the page at the wrong moment.

I'm a bit confused here. You know better than most how critical latency is to
the user-experience. Saving on server-roundtrips or hiding them is a big deal.

Yet you promote this dogma without providing an alternative solution to this
dilemma.

~~~
phkamp
Cleaner for who ? Easier for who ?

That kind of cookie usage just makes it Somebody Else's Problem instead of
your problem.

~~~
moe
_Cleaner for who ? Easier for who ?_

For the webapp-developer, which results in a faster and cheaper experience for
the user.

I'm still baffled at your persistence given you sit pretty much at the source.
You have probably written VCLs for sticky sessions yourself and pondered the
constraints wrt data-locality and single points of failure? Sticky sessions
are just not a good design when the alternative is so readily available; it's
the first time in a long time I hear anyone disagree with that.

 _That kind of cookie usage just makes it Somebody Else's Problem instead of
your problem._

And who would that "somebody else" be?

Users certainly don't care about a few hundred extra-bytes that their browser
sends with each request, especially since that trade-off usually makes their
browsing faster than the alternative would be.

The privacy concern is valid but boils down to developers using cookies
_wrong_ (without encryption). If we were to remove all technologies that are
used wrong by incompetent developers then the internet would be a pretty empty
place.

------
patrickaljord
> See for instance how SSH replaced TELNET, REXEC, RSH, SUPDUP

> Or I might add, how HTTP replaced GOPHER[3].

telnet and gopher were used by a few thousands servers only and were not
consumer facing technologies (for the most part), it doesn't make sense to
compare that to IPv4 and HTTP that are used by millions (billion?) of servers.

~~~
hollerith
>telnet and gopher were used by a few thousands servers only and were not
consumer facing technologies

The card catalogs at most university libraries and most libraries of any
national or international importance were reachable by telnet in 1992. And I
think card catalogs count as a "consumer-facing" service.

The vast majority of internet client software in 1992 was text only. The first
exception to this that I am aware of is the WWW, which most internet users had
not started to use by the end of 1992 (email, newsgroups and ftp being the
most widely used services). The way most connected to the internet or an
intranet from home was by sending vt100 or similar protocol over a dial-up
link -- with a Unix shell account or VMS account at the other end of the link.
Repeating myself for emphasis: in 1992 most people accessing the internet from
home or from a small office used a modem and IP packets did not pass over that
modem link. The point of this long paragraph is that the vast majority of the
machines on which these shell accounts ran were also reachable by telnet.

Finally, the telnet protocol in 1992 was a "general-purpose adapter" similar
to how HTTP is one today. For example, the first web page I ever visited I
visited through a telnet-HTTP gateway so that I could get a taste of the new
WWW thing without having to install a WWW browser. Note that this telnet-HTTP
gateway is another example of a "consumer-facing" telnet server.

In summary, there were probably more than a few thousand telnet servers in
1992 -- and many of them were "consumer-facing".

I am almost certain there were a few million users (certainly so if we include
college students who used it for a semester or so, then stopped) of the
internet in 1992, and most of those users used telnet.

~~~
DougWebb
IRC was pretty widely used back in the day too, and it was and still is text-
only. There have been a number of non-text replacements for IRC (SecondLife
being the most successful) but most died off. IRC's grandchildren, Twitter,
SMS, all of the instant messaging and chat services, are all still text-only
or close too it.

~~~
hollerith
OK, but I feel the need to stress that when I wrote "text only", I was
referring to the _user interfaces_.

(And the reason user interfaces are relevant here is that telnet was the main
way to export user interfaces over the internet in 1992.)

------
dhruvbird
Excellent judge of the protocol. It's interesting that no one is trying to
solve the cookie bloat problem with http.

------
comex
Removing cookies from a protocol which is otherwise fully compatible with
HTTP/1, in the sense of being able to be interposed as a proxy or substituted
in the web server without breaking apps, is a terrible idea.

> Cookies are, as the EU commision correctly noted, fundamentally flawed,
> because they store potentially sensitive information on whatever computer
> the user happens to use, and as a result of various abuses and
> incompetences, EU felt compelled to legislate a "notice and announce" policy
> for HTTP-cookies.

> But it doesn't stop there: The information stored in cookies have
> potentialiiy very high value for the HTTP server, and because the server has
> no control over the integrity of the storage, we are now seing cookies being
> crypto-signed, to prevent forgeries.

Anyone with a grain of skill is capable of using cookies as identifiers only;
it's hard to see what cookies vs identifiers has to do with "notice and
announce" or security. An explicit session mechanism could provide benefits
over using cookies for the same purpose, but what exactly would _removing_
cookies achieve other than breaking the world?

~~~
dangrossman
Why must HTTP 2.x be backwards compatible with 1.x? Should SSH have not been
created because you can't talk to it with a telnet client? If a new protocol
offers sufficient benefits, it would be worth having to make minor changes to
apps to support both.

Cookies suck, from a technical and regulatory-compliance standpoint. Plus,
I'll finally stop having to clear my cookies every month or so just to log in
to my PayPal and American Express accounts. Both sites keep creating unique
cookies on every login until there are so many that they pass their own web
servers' max header length limits.

~~~
comex
> Why must HTTP 2.x be backwards compatible with 1.x?

Because the only benefit of removing cookies is a tiny bit of simplicity which
could theoretically allow removing (a small amount of) code browsers will
already have to keep around for probably at least a decade to support existing
websites. If cookies are mostly unused by the time HTTP/3.x rolls around, we
can talk...

> Cookies suck, from a technical

Agreed, but...

> and regulatory-compliance standpoint.

I don't understand this point. Surely the need for regulation of user tracking
by websites doesn't depend on whether cookies or an equivalent mechanism are
being used? If people start using Not Cookies(tm), they will be unregulated at
first, but the law will be changed if the effect is the same.

Edit: Similarly, any protocol that gives a website a persistent identity token
without its explicitly requesting one is a bad idea - cookies do provide a
modicum of visibility to the user regarding who's tracking them. Not sure
exactly what Kamp is proposing.

> Plus, I'll finally stop having to clear my cookies every month or so just to
> log in to my PayPal and American Express accounts. Both sites keep creating
> unique cookies on every login until there are so many that they pass their
> own web servers' max header length limits.

Hah, no you won't. I strongly suspect legacy codebases will remain on HTTP/1.1
approximately forever, at least if 2.0 is backwards incompatible.

~~~
mbreese
> and regulatory-compliance standpoint

One of the main reasons why people can't just turn off Cookies is because they
are needed for session management. This makes it very difficult to just
disable. If there was a dedicated session management method in HTTP/2.0 then
that would remove a lot of the need for Cookies. Then they could be used for
what they were intended (local persistent state). This would also give users
better methods for managing them (or just disabling them).

~~~
comex
Eh... maybe eventually, once nothing uses cookies anymore (including existing
HTTP sites). But surely this can be solved today by having browsers force
cookies to expire with the session?

------
bifrost
Frankly many problems addressed by these proposals are not things that should
be solved in a protocol like HTTP. Do you want every HTTP request to be slowed
down with a crypto token exchange and verification? unlikely. In special cases
you definately want it, but all the time? Absolutely not.

~~~
obtu
It may be counter-intuitive, but with the right cyphers, the crypto costs of
SSL and SPDY are negligible.

~~~
phkamp
No, they are not. For one thing you have to terminate all your SSL on your
loadbalancer in order to distribute the traffic. That makes SPDY a no-go for
web-hotels/web-hosting where each customer has their own certificate.

Second, there are perfectly valid legally mandated circumstances which forbid
end-to-end privacy, from children in schools to inmates in jail and patients
in psych. hospitals, not to mention corporate firewalls and the monster that
looks out for classified docs not leaking out of CIA.

~~~
comex
> No, they are not. For one thing you have to terminate all your SSL on your
> loadbalancer in order to distribute the traffic. That makes SPDY a no-go for
> web-hotels/web-hosting where each customer has their own certificate.

That's what SNI is for.

> Second, there are perfectly valid legally mandated circumstances which
> forbid end-to-end privacy

Then install spyware on the user's computer, or add a trusted SSL key and mitm
all the things.

------
sophacles
One huge benefit I think people are missing about the notion of a protocol
standard session mechanism, is that we can use HTTP auth much easier, and
perhaps get away from this notion of every site having to redo the login
process. Browsers can handle the "remember login" settings, and logging out is
as simple as tab closing. No "remember this computer" or "don't remember this
computer" checkbox confusion. No random sites saying "remember me always" and
requiring manual logout on a borrowed computer. It certainly helps with the
password wallet concept too.

Sure all that stuff has become semi-standard as it currently exists, but it is
ugly, hacky, and sometimes doesn't work, and other times opens doors for
hilarious malfeasance.

------
jorangreef
"Overall, I find all three proposals are focused on solving yesteryears
problems, rather than on creating a protocol that stands a chance to last us
the next 20 years."

------
NHQ
"We have also learned that a protocol which delivers the goods can replace all
competition in virtually no time."

This is an argument for websockets eating protocol lunch.

------
ajankovic
In my opinion it is always a good idea in these kind of situations to set a
goal we should strive to.

I wonder what the ideal web protocol would look like. If, for example, we
didn't have a burden of billions of servers and Internet reliance on HTTP/1.x
protocol.

What would be the most ideal solution that would suite emerging use cases for
the Web? Are there any research papers on this topic?

~~~
squidi
I agree with this way of thinking. phkamp has come up with some innovative
ideas, but if it doesn't come easy for the rest of us there are lots of
techniques to help e.g. asking, "if I was superman and could implement any
solution, what would it be"
[http://personalexcellence.co/blog/25-brainstorming-
technique...](http://personalexcellence.co/blog/25-brainstorming-techniques/)

------
borlak
load balancers aren't meant to just be "HTTP routers". they can definitely be
used as such for smaller applications and do a good job at it, but a real load
balancer needs to be quite complex, being able to adapt to the underlying
applications that make use of it.

if your goal is to only route HTTP requests, then you're only solving the
first step of an increasingly complicated field of computer science (namely,
web applications).

Cookies aren't going to go away. if you want to improve the protocol to deal
with cookies better, that makes sense, but acting like they are some kind of
evil on the internet that should be forgotten isn't going to work. it's a bit
self-defeating to argue that some protocols failed because of failure to
provide new benefits and then argue against Cookies in HTTP!

~~~
nl
_load balancers aren't meant to just be "HTTP routers". they can definitely be
used as such for smaller applications and do a good job at it, but a real load
balancer needs to be quite complex, being able to adapt to the underlying
applications that make use of it_

I think Poul-Henning Kamp is fairly well qualified to discuss what load
balancers do.

(And yes, load balancer _are_ fundamentally HTTP Routers. Yes, sometimes they
do content manipulation etc, but all those features are add-ons to the basic
use-case)

~~~
youngtaff
Perhaps Kamp is qualified to discuss what some 'load-balancers' do...

The proxy developers have always had their doubts about SPDY (you can see them
when @mnot first proposed it as a starting point)

Terminating SPDY at the HTTP Router makes a lot of sense architecturally but I
know some orgs don't terminate SSL at the load-balancer due to the licensing
costs.

Ultimately we need load-balancing options and someone to develop the
opensource proxies (haproxy, varnish etc.) into more sophisticated offerings.

Perhaps they'll be a SPDY module for Traffic Server

------
mike-cardwell
I can think of a few organisations that benefit financially, a great amount
from the fact that cookies allow them to track users. Ad companies like Google
and Microsoft spring to mind. The same organisations building our major
browsers... Conflict of interest? You better believe it.

~~~
SeoxyS
Imagine if browsers reported a user-resettable uuid sent over on each
connection open. Advertisers get the benefit of being able to track users, do
it more securely since no data is ever stored on the client, and must eat the
cost of actually storing the key-value pairs, instead of passing it down to
users and expending massive amounts of bandwidth bouncing the data back and
forth.

It's not about killing cookies, it's about finding a better solution that
solves the problem better.

Note: I work for a startup that "benefits financially" from tracking users—a
feature without which we would not have a business.

~~~
sirclueless
This already exists in spirit, in the form of "Incognito" or "Private"
browsing. This doesn't solve the protocol issues of cookie bandwidth and
insecure storage, but it does provide a user-resettable store of cookies.

