
Google is working on creating a new protocol named QUIC to replace UDP - patrickaljord
https://plus.google.com/u/0/100132233764003563318/posts/b36wVornPtD
======
latitude
I'll tell you a dirty little secret of the protocol design.

Say, you want to design a protocol with reliable delivery and/or loss
detection. You will then have ACKs, send window and retransmissions. Guess
what? If you don't follow windowing semantics of TCP, then one of two things
will happen on saturated links - either TCP will end up with all the bandwidth
or you will.

So - surprise! - you have no choice but to design a TCP clone.

[ EDIT ]

That said, there _is_ a fundamental problem with TCP, when it's used for
carrying secure connections. Since TCP acts as a pure transport protocol, it
has no per-packet authentication and so any connection can be trivially DoS'd
with a single fake FIN or RST packet. There are ways to solve this, e.g. by
reversing the TCP and security layers and running TCP over ESP-over-UDP or
TLS-over-UDP (OpenVPN protocol). This requires either writing a user-space TCP
library or doing some nasty tunneling at the kernel level, but even as
cumbersome as this is, it's still not a reason enough to re-invent the wheel.
Also, if you want compression, it's readily available in TLS or as a part of
IPsec stack (IPcomp). If you want FEC, same thing - just add a custom
transform to TLS and let the client and server negotiate if to use it or not.

I mean, every network programmer invents a protocol or two in his lifetime.
It's like a right of passage and it's really not a big deal. Unless it ends up
attached to a brand name, in which case it starts attracting snarky remarks
like this one :)

~~~
exDM69
> Say, you want to design a protocol with reliable delivery and/or loss
> detection. You will then have ACKs... > So - surprise! - you have no choice
> but to design a TCP clone.

Some use cases don't really want TCP or a TCP clone, for example multi-player
games might only use a semi-reliable loss detection system which has ACK's but
does not attempt retransmission on failure to deliver. This works a lot better
than TCP for that use case, and is not really a "TCP clone".

~~~
xyzzy123
The really sad thing about this though is that "the internet" no longer works
for stuff which isn't TCP. I guess it does, but not for your whole target
market.

The lowest common denominator is users behind an NTLM proxy. Mass market
products don't generally work unless they have a fallback which is "I can make
it tunnel through proxies". Which is HTTP or CONNECT :( Which means no teams
will do something which isn't TCP anymore, since it's not a thing they can
tunnel :( Interested in counterexamples though.

This is the same reason why a UDP replacement is a great idea but a waste of
time for most people developing a product. This makes me sad.

~~~
ajross
As mentioned upthread, most multiplayer games are still UDP. And indeed, they
generally don't work through proxies.

------
advm
This is more than merely UDP with encryption. Rather, this seems to be a
reliable stream protocol that aimed at dethroning TCP. I don't think it
intends to replace UDP. If anything, I would describe it as a user space
replacement for TCP. It uses UDP, yes, but that's because UDP is about as
close as you can get to IP at the user level.

It's about time, too. Certainly user-level substitutes for TCP are not a new
idea (I've worked a good deal with UDT), but this the first one to my
knowledge that has the weight of something like Google behind it.

There are a lot of advantages to doing this as the user level with UDP as
well. Not needing to install a kernel module, for one. But also P2P rendezvous
connections through NAT with UDP (we wouldn't even need to wait for router
manufacturers to adopt anything), better control over protocol options without
going through the kernel, custom congestion control algorithms, finer control
over buffers (mmap'd files as buffer for file transfers anyone?), etc.

I'd much rather see this at the kernel level, but TCP as it's implemented on
most systems has too many issues on modern networks, and most acceptable user-
level solutions (ugh, parallel streams) are really terrible hacks. If Google
does it right, maybe it'll force OS developers to wake up and rethink the
increasingly clunky and antiquated BSD sockets API that is the source of many
of TCP's woes and finally modernize the network stack for the fiber era.

~~~
X-Istence
What exactly about the BSD sockets API makes it a source of TCP's many woes?
What woes are you talking about?

Also, why do we want to re-implement TCP in user space, when it still needs to
traverse the kernel as a UDP packet, this doesn't have any of the benefits
that netmap or PF_RING for example bring to the table where the software is
doing the full stack and thus there is less latency involved.

QUIC as it currently stands will be sitting on top of UDP, so all packets
traverse the kernel, get dumped to user space, user space parses it, and sends
packets back, as UDP. All that extra copying, when with TCP in the kernel the
kernel would be responsible for reassembling packets, would be responsible for
ACK'ing packets, and stuff like that.

Also, mmap'd files as buffer for file transfers ... you are going to have to
explain that one. There are already various zero copy mechanisms for sending
files, such as sendfile().

As for new congestion control algorithms, great, sounds fantastic, but how
much more of a nightmare is it going to be to get the parties to agree on a
congestion control mechanism? We already have that problem now ...

Sorry, but I see a whole lot of complaining without any real data or
information to back it up. [[citation needed]].

~~~
advm
Maybe TCP's issues aren't apparent when you're using it to download page
assets from AWS over your home Internet connection, but they become apparent
when you're doing large file transfers between systems whose bandwidth-delay
products (BDPs) greatly exceed the upper limit of the TCP buffers on the end
systems.

This may not be an issue for users of consumer grade Internet service, but it
is an issue to organizations who have private, dedicated, high-bandwidth links
and need to move a lot of data over large distances (equating to high latency)
very quickly and often; CDNs, data centers, research institutions, or, I
dunno, maybe someone like Google.

The BDP and the TCP send buffer size impose an upper limit on the window size
for the connection. Ideally, in a file transfer scenario, the BDP and the
socket's send buffer size should be equal. If your send buffer size is lower
than the BDP, you cannot ever transfer at a greater throughput than
buffer_size / link_latency, and thus you cannot ever attain maximum bandwidth.
I can explain in more detail why that's true if you want, but otherwise here's
this: <http://www.psc.edu/index.php/networking/641-tcp-tune>

Unfortunately for end systems with a high BDP between them, most of the time
the maximum send buffer size for a socket is capped by the system to something
much lower than the BDP. This is a result of the socket implementation of
these systems, not an inherent limitation of TCP.

An accepted user-level solution to this issue is to use multiple sockets in
parallel, but that has its own issues, such as breaking fairness and not
working well with the stream model. I can explain this more if you want, too,
just let me know.

As for zero-copy with sendfile, well, even when you do zero-copy, the above
issues still apply, because the socket buffer is still used. Admittedly my
research into zero-copy is very cursory, but from what I understand, even when
you use sendfile, it still copies from files into the TCP send buffer, so
zero-copy is actually less zero than it seems. It just doesn't require the
whole user space read-buffer-write-buffer loop, which does yield a noticable
performance increase, but that doesn't mean the buffer magically goes away and
the BDP issue is solved.

What I was suggesting, by using an mmap'd file as the send buffer, would allow
the TCP send buffer issue to be circumvented completely. In a user space
protocol specialized for file transfers (not saying QUIC is, but I'm not
talking about QUIC), you wouldn't need to use a send buffer, and the window
size would become limited only by the BDP; the file contents are always
available without needing to buffer them like the socket API does, and the
network will never have to be idle just because the send buffer is capped at
10 MB or something.

You mentioned QUIC needing to do the whole userland dance to let the
application parse packets and all, and that is a valid point, but, in the case
of reading, it's entirely possible to only read in the header data you need
from the datagram and let the kernel take care of as much of the actual
copying as it can for the rest of the packet. In the case of writing, you can
"plug" the socket, write headers in user space, then zero-copy the file
segment for added performance.

Certainly there is a performance hit, since it's still going into user space,
but there wouldn't be if this was all part of the kernel. Like I was saying,
I'm not so much excited over the fact that this is user space or anything, but
more so that it has the potential to exert pressure on the Linux community and
such to make finally make some broad and well-needed changes to the network
stack. Even if QUIC isn't being designed to address the issues I'm mentioning,
maybe a user space-configurable socket library with the backing of Google will
make experimentation with internals normally obscured by the socket API more
accessible to people, and something good will come out of that eventually.

Anyway, all of these issues stem from the fact that BSD sockets are meant to
be a general purpose communications protocol. However, some applications (such
as file transfers) don't need all of the amenities general purpose sockets
offer, such as separate kernel-allocated buffers so applications don't need to
keep buffers after calling write(), etc.

There are other problems with TCP, such as slow start being, well, slow to
converge on high-BDP networks, bad performance in the face of random packet
loss (e.g., TCP over Wi-Fi), congestion control algorithms being too
conservative (IMO, not everyone needs to agree on the same congestion control
protocol for it to work well, it just needs to converge to network conditions
faster, better differentiate types of loss, and yield to fairness more
quickly), TCP features such as selective ACKs not being widely used, default
TCP socket settings sucking and requiring a lot of tuning to get right, crap
with NAT that can't be circumvented at the user level (UDP-based stream
protocols can do rendezvous connections to get around NAT), and more. People
write whole papers on all these things, though, and I don't want to make this
even more of a wall of stupid text than it already is.

Okay, you can call me out for not having citations or backing up my claims
again now. Problem is most of the public research exists as a shitty academic
papers you wouldn't probably bother reading anyway, and most of the people
actually studying this stuff in-depth and coming up with solutions are private
researchers and engineers working for companies like Google.

~~~
brugidou
how would you differentiate types of loss? The main issue with TCP over large
BDP at the moment is not the limited buffer size IMHO. You can increase is
dramatically on modern Linux. You can maybe attempt to work around congestion
mechanisms and slow start by using very aggressive algorithms. But I wonder
how researchers work with packet loss?

------
jsnell
The title seems wrong, at least based on a quick glance this doesn't look like
a UDP replacement. It could be a TCP replacement that's encapsulated in UDP.

There would potentially be huge benefits to be had from replacing TCP, but of
course that's something the people have been trying to do for a long time and
failed. Google has a huge advantage in achieving this thanks to controlling a
significant proportion of clients and servers.

~~~
reidrac
I agree. Besides they are doing it "secretly" in a public repository.

------
exDM69
These use cases spring to my mind: real-time data and multi-player action
games, perhaps even streaming media.

TCP and HTTP are less than ideal transports when it comes to real time data.
In case of a network error, TCP retransmits the lost data but in a real-time
environment, the data is already old and therefore next to useless. Real time
action games (like quake) favor udp over tcp.

Some questions: Why replace UDP? Why not work on top of UDP? There must be a
reason for this. What about multicast?

~~~
rumcajz
Why not use DCCP then?

~~~
morsch
Is DCCP widely used by anyone?

~~~
rthomas6
Is QUIC widely used by anyone?

------
DanWaterworth
I'm all for innovation, but I become wary when a company that controls a
sizeable proportion of the browser market-share and also serves a not
insignificant proportion of all web traffic makes extensions like this.

SPDY, Dart and now this.

~~~
InclinedPlane
Google is the least scary company in this regard. They're putting their
extensions out into public hands instead of using an "embrace/extend" model.
SPDY is open, soon to become an official web standard, and implemented by
popular web server software and browsers (nginx, apache, firefox). The same is
true of Chromium as well, the core of Chrome.

I'd go so far as to say that in these areas Google is serving as a model
corporate citizen, and I wish others would imitate them.

~~~
taligent

       >I'd go so far as to say that in these areas Google is serving as a model corporate citizen.
    

Well that is a curious way to look at it.

Instead of working with other companies to define a common standard they force
it through attrition. That's the exact opposite of how this sort of thing
should work.

And given Google's disgraceful behaviour in abusing the FRAND process they are
hardly a model citizen by any measure.

~~~
dietrichepp
Okay, can you name some standards which were created by multiple companies
working together? (I'm not talking about maintenance -- HTML was invented by
Tim Berners-Lee, SGML came out of IBM, JavaScript was Netscape, etc.)

~~~
taligent
The Java Community Process back in the day was a good example of how lots of
large companies working together could define a standard and yet still leave
room for innovation.

H.264 and WiFi standards are recent examples of a lot of companies working
together.

~~~
jbooth
Yeah, but 90% of those standards were TERRIBLE. Like, GOD AWFUL. EJB? SOAP?

It's the hangover from those standards that leaves kids today thinking that
Python or Ruby are simpler than Java.

Eventually the JCP turned into "take popular open source ideas like hibernate
and spring, and bless them with a JCP number". Which worked a lot better, and
each of those individual projects was usually launched by an individual or
small group initially.

~~~
walls
> It's the hangover from those standards that leaves kids today thinking that
> Python or Ruby are simpler than Java.

I would really love to hear how you could possibly believe otherwise.

~~~
jbooth
The language Java is simpler than the language Python or Ruby. Fewer keywords,
those keywords do fewer things.

Ruby and Python seem simpler for most web work due to better frameworks and
dynamic typing, which is less simple than static typing, but is easier to work
with in most web situations.

I guess I'm drawing a distinction between 'simple' and 'convenient'. Monkey
patching in Ruby is not simple, but can be convenient and make rails possible.

~~~
Ao7bei3s
The keyword count thing is easily refutable: Python has 38 keywords, Ruby has
41, Java has 50.

<http://en.wikipedia.org/wiki/List_of_Java_keywords>
[http://docs.python.org/3/reference/lexical_analysis.html#ide...](http://docs.python.org/3/reference/lexical_analysis.html#identifiers)
<https://github.com/ruby/ruby/blob/trunk/parse.y> (grep for reswords)

I'm undecided whether Python/Ruby or Java are simpler.

At least to me dynamic typing seems much simpler than static typing: from a
very very pragmatic view that just means no compile-time checks (?), and I'm
not that good with Ruby, but I'd consider Pythons type system much simpler
than Javas (Python has none of: primitive types, null, arrays, interfaces,
visibility (well, there is name mangling for __...), enums, final/abstract,
the checked/unchecked mess (look up what the _correct_ way to catch exactly:
all checked exceptions is!), casting, generics and everything that comes with
it: have a look at the index at the left side of
[http://docs.oracle.com/javase/tutorial/java/generics/index.h...](http://docs.oracle.com/javase/tutorial/java/generics/index.html)
\-- but to be fair Pythons type system has a couple of nontrivial features
(metaclasses, multiple inheritance, ...) too)

Python/Ruby have a lot of very useful language features which make the
languages much less of a pain to use than Java (the language), but for any
meaningful comparison of language complexity you'd surely also have to
consider the intricacies of the underlying VMs. The Java VM _is_ quite complex
and also quite powerful. For simple programs that don't require high
performance or efficient memory management or whatnot that might not matter
much though.

[Now, after having written this, I don't think its possible that any of the
three languages is strictly simpler than any of the other ones.]

~~~
jbooth
[Now, after having written this, I don't think its possible that any of the
three languages is strictly simpler than any of the other ones.]

Me too, great comment, thanks.

------
chetanahuja
As other's have pointed out, it's not a replacement for UDP or "improved
version of UDP with some extra features such as encryption" as mentioned in
the original article but a reliable, encrypted protocol using UDP as transport
layer. That makes sense because most NAT boxes etc. have to let UDP packets
get through because they have to support DNS, VOIP protocols et el. I suspect
the encryption part has more (or at least just as much) to do with keeping the
middle-boxen's dirty hands off of the the protocol innards than MITM security.

------
ck2
Note how SPDY has proven to have limited benefits in the realworld vs a lab.

Not that I want to stop their brilliant minds from looking at these problems
and trying to solve them, especially on their dime.

~~~
youngtaff
If SPDY has limited benefits in the real-world why have both Facebook and
Twitter deployed it?

~~~
andrewflnr
That's how one finds out about the real-world benefits. I'm not an ops person,
but once it's already deployed, how hard is it to just leave it there for the
limited benefit it does give?

------
zx2c4
Prior: <http://news.ycombinator.com/item?id=4854839>

------
pyvpx
why does google need to come up with protocols? does the IETF and other
processes in place for over a decade not work for them? I know nothing of the
protocol (the link only mentions the 'cheeky' naming scheme and the source in
the chrome tree); I'm merely commenting on googles motives...

~~~
avar
How do you think things get proposed to organizations like the IETF? Someone
comes up with a working implementation, proves it to work & to be better than
the existing stuff, and then they propose it for inclusion.

That's what Google is doing, and they're using Google Chrome as their test bed
(along presumably, with a bunch of non-Chrome stuff internally).

This is the same thing they did with SPDY. See
<http://en.wikipedia.org/wiki/SPDY#Relation_to_HTTP> for details.

If what you're hinting at is that they should first start some wider
discussion about what a next-gen UDP should look like before they start doing
anything, then I suggest you look up "bikeshedding". Such a process would
never get anywhere.

~~~
pyvpx
I appreciate their initiative and their wealth of engineering talent. I
completely get what you're saying -- someone has to propose it; and a usable
reference implementation is all the better. Unfortunately it seems, given
Googles mammoth size, that many folks who might otherwise chime in simply
shrug and go "oh, Google has already solved/implemented/popularized a
solution" and move on.

~~~
wmf
That's not happening with SPDY, WebRTC, Dart, or WebP. Google's competitors
are going over these proposals with the proverbial fine-toothed comb. I get
the impression that people are extra skeptical of Google standards because of
the potential to get steamrolled.

------
beagle3
Hopefully they'd take a lot from djb NaCl ( <http://nacl.cr.yp.to/> , which
predates Google's NaCl and is unrelated).

Transport security, endpoint multiplexing and endpoint mobility done right.

~~~
pjscott
Good news! Their crypto stuff looks pretty reasonable. Key exchange uses ECDH
with either the P-256 or curve25519 polynomials. Once the session key is
established, it's encrypted with AES-128 and authenticated with either GCM or
HMAC-SHA256. None of this is implemented yet, but it's at least cause for
hope.

------
rednukleus
Can any boffins please explain what impact this might have in practice?

Is this just going to mean faster and more secure data transfer, or are there
wider implications?

~~~
sageikosa
Low uptake as routers will not have built-in support, and admins will be slow
to enable it when and if it becomes available (unless its tunneled over
another protocol).

Why not just push IPv6 adoption?

~~~
chetanahuja
IPV6 vs IPV4 question is orthogonal to those addressed by the QUIC protocol.
IP layer is for addressing nodes on the internet and routing packets to them
in a stateless manner (more or less... I'm not counting routing table caches
and such as "state"). TCP (and this QUIC protocol) are built on top of the IP
layer to provide reliable end-to-end data transfers in some sort of session
based mechanism.

~~~
sageikosa
I was thinking primarily of the security aspects, which is one of the few
things called out. I'm all for secure and discrete transfers of data between
endpoints, since TCP programming APIs often require maintaining process state
for each connection and can impact service scaling.

If this is really more or a session layer protocol on top of UDP instead of
its own transport protocol, then all the more power to them. It will be more
of a programming model than a network configuration quagmire, and I'm sure
that eventually intelligent optimizations can be performed via packet
inspection if the protocol state details are not encapsulated deep within the
security (i.e., trust/privacy) bubble of the protocol.

------
leothekim
Hm I suppose it's not secret anymore. :-)

The file listing shows reliable_quic_stream.cc and .h files as well, so maybe
it goes a step past UDP?

------
benth
Curious how it compares with SCTP over DTLS over UDP (which WebRTC uses).

------
jacob019
UDP is connectionless. How can we have encryption without a connection? Using
a block cypher would be less secure than stream cyphers and the controversy
could halt adoption.

~~~
popee
It's possible: DTLS. Also CISCO uses UDP for their VPN solutions so go figure.

~~~
zobzu
Not Invented Here.

Google wants control, IMO.

I'd be interested in their reasons for not using DTLS, or improving DTLS. They
actually use it for WebRTC.

------
popee
Hey, what happened to SCTP?

~~~
matthiasl
SCTP is widely used for carrying signalling in mobile and fixed telephone
networks. It is _the_ way of carrying SS7 over IP.

(Another angle on the same thing: it didn't get used much outside the thing it
was originally designed for.)

------
simplexion
Is it connectionless?

