
Just one QUIC bit - okket
https://blog.apnic.net/2018/03/28/just-one-quic-bit/
======
mhandley
I was in the IETF meeting. It was actually a very civil discussion, and I
think almost everyone in the room could see both sides on the issue. On the
one hand, we've seen increasing problems with middleboxes making assumptions
about end-to-end protocols, and this making it very hard to then deploy new
end-to-end functionality. The problem is real; we measured this effect in this
paper:

[http://www0.cs.ucl.ac.uk/staff/M.Handley/papers/extend-
tcp.p...](http://www0.cs.ucl.ac.uk/staff/M.Handley/papers/extend-tcp.pdf)

Indeed, one of the changes the IETF group has made since taking on QUIC is to
also encrypt the QUIC sequence numbers, so middleboxes can't play silly games
by observing them (they were already integrity protected).

However, the flip side is that network operators do use observations of
network round trip time gained from passive observation of traffic so as to
discover if traffic is seeing excessive queuing somewhere. An inability to do
this with QUIC may either lead to worse network behaviour, or to them using
other methods to gain insight. If you're in a location where you can observe
both directions of a flow, you can easily do this by, for example, delaying a
bunch of packets by 200ms, then letting them go. When you see a burst of
(encrypted) ack packets return, you can deduce the RTT from your observation
point to the destination. I'd really like to avoid operators thinking they
need to do such hacks just to measure RTT, and the spin bit lets a passive
observer see the RTT.

In the end, I was not convinced entirely by either argument, and neither was
the consensus in the room. It's not a clear-cut decision; there are reasonable
arguments either way. Such is engineering.

~~~
anameaname
Realistically, are network operators going to add artificial delay to packets
going through their network? If they don't get their bit, and they start
slowing down communication, it would seem like market correction would solve
it.

~~~
bsder
Sarcastically: who will even notice a 200ms hiccup, nowadays, given the
enormous waste of time that everybody shoves into their Javascript ad
networks.

Personally, I don't want them to expose _anything_. The RTT on my packets is
yet another piece of metadata capable of being abused.

In addition, I hope that the whole QUIC thing is a nice reboot of actually
getting end-to-end connectivity back on the Internet so we can get some
protocol experimentation moving again.

~~~
slededit
That's 200ms that could be used loading more tracking scripts.

------
qalmakka
Middleboxes have been tampering with packets and headers for way too long.
It's most definitely time to move back control to the two ends of the
connection, to avoid fossilisation and the same mistakes made in the past.
IPv6 and QUIC are a cardinal step in this direction, and we cannot afford to
let myopic decisions screw up the opportunity to fix what has been broken for
so long.

~~~
whatupmd
"Who needs Firewalls and NAT," he loudly declares while kicking down the walls
of the house to expose himself to the natural elements.

------
anameaname
If you give an engineer a bit:

[https://mailarchive.ietf.org/arch/msg/quic/yQfBIAOnUEKIWjZhF...](https://mailarchive.ietf.org/arch/msg/quic/yQfBIAOnUEKIWjZhFW8CYPOcEKc)

You can kind of see how this might get out of hand and cause the discuss to
digress and go in circles.

------
breatheoften
There are still environments where active policies described by network
operators to manage traffic flows is _required_ for users to make any
productive use of the network ... I used to work on research ships — 50 people
at sea for months at a time with satellite internet having only 256kbps (800ms
latency).

It was abysmal to watch as over time everyone became completely dependent on
internet connectivity for their computing needs as all services moved to the
cloud. Trying to provide a useable internet experience in that context
involved blocking as much as we could that users machines were doing (usually
without their awareness) in the background that simply couldn’t happen from
that network — making the trade offs to try and select for things that _could_
work was a huge challenge.

It seems like QUIC offers better fundamental performance primitives but no way
to solve for protecting users from themselves in this kind of environment ...

~~~
tangent128
The"correct" answer there is probably to make the network limits explicit, by
explicitly requiring a proxy server or organization CA installed in the
client. QUIC should still work with middleware that actually terminates the
connection instead of meddling from outside.

~~~
marshray
Sounds like Android at least can be told via DHCP it's on a metered
connection:

[https://www.reddit.com/r/networking/comments/4wk4rw/dhcp_opt...](https://www.reddit.com/r/networking/comments/4wk4rw/dhcp_option_to_signal_metered_connection/)

------
lumisota
My takeaway from the meeting was that it isn't clear that the spin bit is
actually useful. Leaving aside the arguments that RTT shouldn't be exposed (it
is anyway, with the handshake): does the spin bit actually provide useful
information?

It seems to be a fairweather metric: OK resolution when the network is
operating normally, but providing no useful information when something has
gone wrong (which is when you'd want it most).

------
wpietri
Nice. I especially like this bit: "Sometimes difficult choices admit to no
sensible compromise between them and the process simply has to make a
contentious decision one way or the other. While individuals make such
decisions all the time, the collective process that enlists a large group of
diverse perspectives, interests and motivations finds such decision making
extraordinarily challenging."

------
pdkl95
> The client echoes the complement of the last seen bit when sending packets
> to the server.

Since neither the client or server care about this big, there doesn't appear
to be anything that forces the client to actually implement this behavior. The
client could always set it to 0 or even set it (pseudo-)randomly on each
packet.

The bit shouldn't exist, but if the IETF did add it to the standard, would
hostile middleware boxen actually start drooping packets if they don't see the
"spin bit" change?

~~~
silotis
This was my first thought as well. I would love to see the reaction from the
pro-spin-bit camp if Google announced that Chrome will never set the spin bit.

------
Asdfbla
As someone only superficially familiar with networking, I think it would be
interesting to know what impact the current packet inspection practices by the
middleboxes have on network performance. After all, the article mentions that
those middleboxes want to control the message flow and use that as
justification. The assumption behind QUIC seems to be that there's no "global"
benefit in that specific kind of network management and that it's mostly
selfish interests of local network operators that motivate messing with the
packets?

Basically, what are the incentives of the middleboxes to inspect packages and
what do they really stand to lose?

~~~
signa11
> interesting to know what impact the current packet inspection practices by
> the middleboxes have on network performance.

well, all (almost ?) current boxes can do that at line rate for minimal sized
packets.

> incentives of the middleboxes to inspect packages(sic)

incentive is for charging / billing / steering packet flows etc.

~~~
Asdfbla
I probably expressed that wrong, I was more wondering if the packet sniffing
had any beneficial impact on the performance in the sense of QoS or congestion
control or something like that. After all, they have to do it for a reason.

But your second points mentions it's for billing and such, so I guess that's
my answer.

~~~
woah
I don’t think it’s for billing most of the time. The middleboxes usually
provide some kind of immediate benefit to the network, while breaking core
networking assumptions and thus contributing to a weird form of technical
debt.

------
bogomipz
The author states:

>"Many network operators use the IP and transport packet headers to perform
traffic engineering functions, packet interception and forced proxy caching."

I'm guessing this referring to transparent hardware caches looking at Layer
7/HTTP headers?

And then:

"Various forms of middleware may reach into the TCP control fields and
manipulate these values to modify session flow rates"

Does the author mean TCP flow control here? So there exists middleware which
changes the ACK and RWND values in TCP headers? Does anyone know what
middleware vendors do this? I'm guessing this might be done as as part of
network "accelerator" hardware devices like Riverbed. Is this correct?

------
datalist
I am not sure I understood the article to be honest.

Does QUIC encrypt the actual UDP packet (meaning from IP down, including the
UDP header)? If that is the case it is not UDP anymore and there will be
"problems" in getting to anywhere in the first place.

If it only encrypts the UDP payload, where does NAT come in? How would that be
different from what TLS does over TCP?

Did I miss anything? Thanks.

~~~
iknowstuff
The article only mentions NAT as a showcase of what IP packet parsing is used
for. QUIC does not encrypt the UDP header.

TLS 1.3 actually run into a lot of issues due to network ossification. QUIC
has been developed to bypass TCP's ossification problem and enable quicker
iteration and deployment to improve latency, congestion mitigation etc.

You could argue that if QUIC's principles were in place for TCP/IP 20 years
ago, middleboxes would not have made protocol revamps as difficult as they are
today. Perhaps NAT would have never been developed and we'd all be on IPv6
already.

~~~
datalist
Thank you, though giving an example that does not even apply to the case is
not the best approach when one wants to make a case :)

If it does not tamper with UDP I fail to recognise the issue though to be
honest.

~~~
pilif
> If it does not tamper with UDP I fail to recognise the issue though to be
> honest.

the problem is that if you expose _any_ kind of information in addition to
just the IP and UDP headers, then _future_ middleboxes will start to use this
information and will start dropping packets they can't parse.

If a QUIC packet is just random noise (aside of the UDP and IP headers), then
a middlebox can't make assumptions about the inner workings of QUIC. It's a
very common (but unfortunate) practice by "security" appliances to drop
everything they don't understand because they treat that as potentially
malicious.

Let me make a (hypothetical) example: Let's say QUIC packets had some publicly
available "version" field. The implementation as it's used now is setting that
to 1.

Now middleboxes start "gaining" support for QUIC and as "everybody" knows, the
only widely deployed version is 1, so these middleboxes start to treat every
other value of that field as a possible attack and drop such packets.

Now, years later, we want a new version of QUIC, but unfortunately, the most
widely deployed (and never updated) middleboxes out there assume any version
but 1 to be malicious.

Which leaves us with a "version" field that practically _has_ to be set to 1,
so now we need another way to flag the new packets. Maybe a "real_version"
field? Who knows? We'll have to try various things until the majority of the
middle-boxes currently deployed are fooled.

Also, it will likely be impossible to fool them all, so even when we get
around the majority of the boxes, we'll still exclude some people from being
able to reach QUIC 2 servers. Sure - it will be a very small amount, but it
won't be zero.

This isn't just theoretical. We had this problem just now with TLS 1.3. Since
the beginning SSL and then TLS had version fields in order for clients and
servers to negotiate the SSL version to use.

Unfortunately, because of precisely this problem, that field stopped being
useable years ago where even the 1.2 negotiation had to happen using a
workaround which then promptly also stopped working for 1.3.

By not exposing anything but random noise as part of a QUIC packet, the
protocol designers aim to prevent this from happening. If all a "transparent"
proxy sees is random noise, they can decide to not support QUIC at all or to
support all of it. They can't decide on a "safe" subset and burn that into the
internet for all eternity.

~~~
datalist
Thanks for the detailed response, but I am "with" you :).

The bit (no pun intended) I do not understand is why there is such a
discussion in the first place? Why do they insist on a plaintext status bit?

Thats why I wondered if there was any encryption outside of the UDP payload.

~~~
pilif
The discussion about exposing the bit was to allow network administrators to
detect retransmissions. The bit is supposed to flip constantly, if it doesn't,
then retransmissions happen and thus something might be wrong with the
network.

Because it's just one bit and it can have two values and both are valid and
both values are seen with about the same frequency, a middlebox won't be able
to just drop a packet if the value is either 0 or 1.

This is why there is even a discussion happening. It's felt to be reasonably
safe to include to provide some actual value to tools.

The debate is though whether it's really safe and/or whether it provides
enough value to go through the trouble.

If you ask me personally, in my professional life, I have been bitten by
protocol ossification way more than by not being able to make sense out of a
packet stream, so I personally would absolutely not expose anything.

But then again, I'm an application level developer and not a network
administrator.

~~~
cesarb
> The bit is supposed to flip constantly

> a middlebox won't be able to just drop a packet

You do know that someone somewhere is going to make a middlebox that just
drops a packet unless that bit flipped in the precise sequence that the
middlebox developer believed was the correct one, right?

Then someone proposes an enhancement to QUIC which happens to change the
sequence of the flips (perhaps some multipath thing, or an enhancement in the
way it treats reordered packets), and it breaks...

------
xtrapolate
This proposal is a compromise. Wouldn't a specific metadata/heartbeat
"packet", routinely dispatched by the involved parties, serve a better
solution?

My personal stand is to prohibit exposing any metadata on the connection
whatsoever, but given the counterarguments, I don't think the proposed
solution is ideal.

~~~
no_identd
>My personal stand is to prohibit exposing any metadata on the connection
whatsoever

[https://grothoff.org/christian/habil.pdf](https://grothoff.org/christian/habil.pdf)
Christian Grothoff - The GNUnet System [October 2017, 181 pages, PDF]

------
pcunite
I disable QUIC in my network because I'm doing QoS at my router and utilize
TCP flow control to slow things do to where I want them to be. QUIC makes the
incorrect assumption that the network never wants to slowdown or to even allow
for a configurable speed. Don't be too clever.

~~~
zlynx
Just ruthlessly drop QUIC packets that are over the flow control limits. It
will adapt.

Playing games with TCP settings is fun I guess, but simple RED with ECN and
drop work well with fewer bad side-effects.

Every Internet protocol expects packet drops on overload. What was that about
being too clever?

------
gene91
This idea of forbidding middlebox from messing up things sounds great. I have
an honest question here though.

What stops a middle box from terminating all QUIC connections (and therefore
get access to decrypted content)?

~~~
floatboth
QUIC uses TLS, so it's pretty much the same situation as any other TLS
connection.

Of course in a corporate environment you can trust the middlebox's CA on all
the work machines. That's actually fine. What everyone seems to hate here is
middleboxes that _don 't_ terminate connections and try to mess with them in
other ways.

------
bullen
Again the solution is not to reimplement TCP on top of UDP but to make TCP
like UDP:

[https://www.ietf.org/id/draft-add-ackfreq-to-
tcp-00.txt](https://www.ietf.org/id/draft-add-ackfreq-to-tcp-00.txt)

~~~
tialaramex
A nice thing about the I-D system is that we don't have to waste RFC numbers
on this sort of crank nonsense. Once upon a time the IETF would end up
publishing this sort of thing as an RFC and then everybody would just ignore
it, but that used up the numbers and short memorable numbers are nice, which
was a shame. Now we can publish them as "drafts" which just quietly expire
once their interest moves on to investigating which metal foils can be made
into hats that best resist government mind control rays or whatever.

~~~
no_identd
>to investigating which metal foils can be made into hats that best resist
government mind control rays or whatever.

You joke, but Allan H. Frey already solved that in the 1960s, just add a 2"x2"
wire mesh near the temples, over the temporal lobes. Blocks all RF interaction
with the brain, no tin foil hat required.

...of course, this doesn't block 'mind control rays', the only thing it
actually blocks consists of the Microwave auditory effect, and the only target
audience for that consists of Radar techs walking around in front of hugeass
antennas. ;)

~~~
rphlx
> the only target audience for that consists of Radar techs walking around in
> front of hugeass antennas

And perhaps US diplomats in Cuba.

------
tzahola
What’s even the point of this spin-bit thing? Of what use is an estimate of my
roundtrip-time to intermediaries?

~~~
nicky0
Network engineers can use it to monitor network performance.

~~~
erpellan
Which they could presumably also do by injecting their own packets, not
snooping on everyone else's?

~~~
nicky0
I don't know, how would you inject packets into an encrypted end to end
connection?

Passive observation of traffic is a useful network performance monitoring
technique.

------
randomerr
Just to make sure I understand this:

\- Google has been spearheading a protocol called 'QUIC'

\- It bypasses tradition TCP by using UDP to create it's own version of TCP
but encrypts the data it sends

\- However it could be (is?) sending a bit and header that can make tracking
the message path easier.

I would guess if that is true then and would not effect the performance the
bit and header should be removed. I don't see what the fuss is about. Can't
two different implementations exist along side each other and the public will
choose the better one?

Correction: I misread the article and thought though spin bit was for an
encryption key. Let this be a lesson to everyone: if you are going to comment
at 2 am in the morning reread your article.

~~~
JoshTriplett
It doesn't make decrypting the message easier. It just exposes a bit of
unencrypted information, that makes it possible to infer information like how
the connection's round-trip time. And the argument is over whether exposing
that information has value, and whether routers and other boxes in the middle
will abuse that information.

------
giovannicarruba
IMO the article completely fails to describe the actual issue.

\- "Then there is the NAT function, where the 5-tuple of protocol, source and
destination addresses and the source and destination port numbers is used as a
lookup vector into a translation table, and both the IP and the inner
transport packet headers are altered by the NAT before passing the packet
onward."

Nice try, but if the outer layer is UDP, then NAT alters the UDP packet
headers. The QUIC payload is never touched.

\- "Many network operators use the IP and transport packet headers to perform
traffic engineering functions, packet interception and forced proxy caching.
Various forms of middleware may reach into the TCP control fields and
manipulate these values to modify session flow rates. All of these activities
are commonplace, and some network operators see this as an essential part of
their service."

I don't see why the people tasked with standardizing Internet protocols now
have to make provisions for the jerks who literally break the Internet.

\- "This bit, the “spin bit” is intended to be used by passive observers on
the network path to expose the round trip time of the connection. The
management of the bit’s value is simple: the server simply echoes the last
seen value of the bit in all packets sent in this connection. The client
echoes the complement of the last seen bit when sending packets to the server.
The result is that when there is a continuous sequence of packets in each
direction this spin bit is flipped between 0 and 1 in time intervals of one
Round Trip Time (RTT). Not only is this RTT time signature visible at each
end, but it is visible to any on-path observer as well."

I have literally no idea why a protocol should actively leak unnecessary
information to a passive observer, and I have no idea what passive observers
would do with the information they can deduce from this "spin bit". The
payload is still encrypted, so you still can't do all the "traffic
engineering, packet interception and forced proxy caching". As a passive
observer you sit somewhere in the middle of the whole stream, and if you can
even fish out the necessary packets (they might take different routes in each
direction), you still don't know how far you are from the other end. Not even
talking about the random processing delays at the server end. You're measuring
garbage. How is this bit even useful for anything?

~~~
nicky0
The point is that QUIC is supposed to be fully encrypted below the UDP layer.
The question is why add bits in the clear on top of this to water it down and
open an avenue for potential abuse.

~~~
IshKebab
Yes but it doesn't really say why anyone wants this spin bit. What are they
going to do with it? Measure RTTs? Why? What for?

~~~
majewsky
The currently top-voted comment in this thread explains it:
[https://news.ycombinator.com/item?id=16696159](https://news.ycombinator.com/item?id=16696159)

