
UDP vs TCP - colinprince
http://gafferongames.com/networking-for-game-programmers/udp-vs-tcp/
======
mjevans
The main difference between TCP and UDP, as this programmer discovered,
relates to quality of realtime service.

Times to use UDP over TCP

    
    
      * When you need the lowest latency
      * When LATE data is worse than GAPS (loss of) in data.
      * When you want to implement your own form of error correction to handle late/missing/mangled data.
    

TCP is best when

    
    
      * You need all of the data to arrive, period.
      * You want to automatically make a rough best estimate use of the available connection for /rate/ of transfer.
    

For a videogame, having a general control channel to the co-ordination server
in TCP is fine. Having interactive asset downloads (level setup) over TCP is
fine. Interactive player movements /probably/ should be UDP. Very likely with
a mix of forward error correction and major snapshot syncs for critical data
(moving entity absolute location, etc).

~~~
lobster_johnson
I always wondered why nobody used UDP for bulk data transfer: Ship all the
chunks over with a sequential number identifying each, then when you've
reached the end, have the client request any lost packets, then repeat the
cycle until all are transferred. After all, the client doesn't need everything
in sequential order if it knows the size, and it can deal with holes as long
as they're eventually repaired.

This gives you (in theory, I've never tried it) better throughout, and you get
the equivalent of TCP's retransmission without the window semantics, since you
move all the retransmissions to the end. On the other hand, it's probably bad
for everyone else, since there's no congestion control.

~~~
scurvy
NFS on Linux defaulted to UDP for what seems like a really long time. There
were perf issues with TCP (can't remember what they were).

~~~
mullen
NFS used to default to UDP for the same reasons that games still would use
UDP, it just needed the data as quickly as possible since it was blocking IO.
Now a days, we don't need UDP for NFS because networks are a lot faster and
the advantages of TCP start to over rule UDP.

~~~
fulafel
The original reason was that using UDP was much faster CPU-wise back in the
80's. There was a requirement to go fast because it was used for transferring
data in bulk, not events like games.

When Linux came around it just followed suit. TCP support came to NFS only
later with other bells and whistles in later NFS versions, and Linux stuck
with basic NFSv2 for a good while.

NFS was designed for local-area networks, so it didn't need to worry about
working over the Internet.

In addition to the network being much faster relative to CPUs back then, the
TCP stacks hadn't seen the tuning they have now. Making TCP fast came later.

------
peterburkimsher
Do you want to hear a joke about UDP?

You might not get it, but I don't really care.

~~~
colejohnson66
Here's a joke about TCP.

Here's a joke about TCP.

Here's a joke about TCP.

Here's a joke about TCP.

Here's a joke about TCP.

> ACK

------
mikepurvis
The recommendation to avoid TCP altogether is surprising to me. Having
encountered a number of video-conferencing systems which are in a similar
space, it seems pretty standard to have separate real-time and control sockets
on UDP and TCP respectively. I skimmed the linked paper and didn't find it
conclusive; can someone summarize how it is that having a TCP socket can
affect UDP traffic on the same interface?

All that said, I certainly see the argument for an all-UDP protocol in terms
of defining your own retransmission approach, or attempting to avoid it
altogether with forward error correction or whatever.

~~~
blackflame7000
I too was wondering how TCP and UDP can affect one another on the same
interface and the explanation I read is that at the lowest level both UDP and
TCP segment data into packets and place them onto the same network adapter's
queue which can affect they way they are transmitted under congestion
depending on the device/drivers.

~~~
user5994461
This makes no sense.

UDP will drop data if there is congestion. Basically, if you care about
congestion and data loss, you should not use UDP.

\---

If you do UDP and TCP. The UDP transfer will loose data during congestion,
while the TCP transfer will slow down and try harder.

This has some usage. For instance, videoconferences software will do TCP and
UDP. The TCP is used for control data -low volume data that needs to work-
(authentication/joining room/leaving/adjusting quality), the UDP is used for
audio/video (some pieces may be lost during congestion, it's okay).

If you use UDP for everything, you'll randomly loose control AND audio/video.
It's the worst of both worlds.

Not to be dismissive but I think the guy who gave you the advise doesn't
really understand networking (or doesn't explain very well or was talking
about some weird usage we don't know about).

~~~
blackflame7000
Error checking can easily be added to udp for control data. Infact in IPv6
it's mandatory. At one moment you say udp will lose out, but we know tcp will
back off on its reattempts exponentially based on the congestion window. Thus
udp would win out. The argument wasn't that it was a significant impact, the
argument is that they can cause interference.

~~~
user5994461
> the argument is that they can cause interference.

UDP ignores everything else and always expect resources to be available for
itself.

We could say that UDP interferes with everything, we could say that everything
interferes with UDP, it's both ways.

~~~
blackflame7000
Who chooses when a UDP packet is dropped? The freaking network adapter device
driver which is what I said in the first place!

~~~
user5994461
There is a lot more in the transmission chain than just _that_ driver.

~~~
blackflame7000
But it's the driver of the NIC that manages its network packet queue! Then it
uses a an interrupt to signal the OS that data is available or that the window
is open for transmit. I'm totally aware there's a lot more to it, I do socket
programming everyday.

------
rstuart4133
The entire article makes me shudder in disbelief.

It's aimed at people who don't know the difference between UDP and TCP (and
possibly wet string). Yet he recommends they implement their own reliably
protocol over UDP, and they avoid TCP because it's better to implement your
own QoS?

Why not add obtaining PhD in quantum mechanics just to round it out? It
wouldn't alter the odds of pulling it off overly.

~~~
user5994461
Agree.

For people who are not familiar with TCP and UDP, the answer should be to
ALWAYS use TCP.

The only exception is some subset of games (FPS, RTS), that this article is
exclusively intended for.

Even there, for the average user who just wanna make a test game project, TCP
may be easier to use and program around. For simple games (e.g. Connect 4),
TCP is a better option either way.

~~~
theWatcher37
In the article he clearly states this is for things like syncing physics in
BF4 or halo.

------
techman9
I appreciate the first user comment on this article:

"I didn’t care for this article 7 months ago. I regret it. My whole game is
useless now. I should have go with udp."

------
geokon
What are some alternative protocols?

It's been a while since my networking class, but if I remember correctly with
UDP you have some serious issues where you can end up clobbering your network,
filling up buffers in the middle and dropping tons of packets. The lack of
congestion control is a huge no-no.

For instance in the example he gives, sure you can tolerate dropped packets
for player-position data, but how do you know if you can tolerate sending at
10Hz 100Hz 1000Hz? Even with TCP you can't (I think....) programmatically
adapt to the size of your pipe. That's kinda abstracted away for you so that
you just say "send file A to B" and it does it for you

Are you supposed to write your own congestion control in userland???? Seems
like this should be a solved problem

~~~
daurnimator
DCCP and SCTP are the two main alternative transport protocols on top of IP

    
    
      - UDP: unordered unreliable datagrams
      - DCCP: unordered unreliable datagrams with congestion control
      - SCTP: optionally-ordered reliable datagrams with congestion control
      - TCP: ordered reliable stream with congestion control
    

SCTP is the most widely used after TCP and UDP: it's often the transport used
in phone networks (very popular with Ericsson); but also now with WebRTC
(which uses SCTP over DTLS over UDP). The main issue being that consumer
firewalls usually only support NAT for TCP, and limitedly for UDP (this is one
of the 'other' reasons to look forward to IPv6)

------
milansuk
> they may arrive out of order, be duplicated, or not arrive at all!

This is first time what I read that datagram can be duplicated. Is it true?
It's duplicated by network or does it mean that peer send it again?

~~~
noway421
Packets can be duplicated by faulty routers and loops in the topology. Happens
all the time to me when I'm lost with iptables.

~~~
ra1n85
Observed this recently with a bug in Cisco's VSS. Some packets were being
duplicated, which destroyed TCP throughput.

------
bsaul
Anyone here knows what protocol should be used for mobile multiplayer games ,
provided they are real time and not turn by turn ? (think people playing on a
decent 3g connexion).

I think you can't send a udp paquet to a phone because the carrier will block
it, but i'm not sure.

~~~
jfries
Yes an update on this topic for the mobile world would be very interesting.

------
zubspace
I tried to build an UDP library once in C# with different methods
(BeginReceiveFrom, ReceiveFromAsync, ReceiveFrom). You learn a ton and it's
quite interesting. My goal was to recreate something similar to .NET remoting
based on UDP.

But be aware that it's a daunting task, because there are so many things you
need to handle all together: lost packets, reordered packets, duplicate
packets, connection handshakes, session handling, reliable/unreliable
channels, packet resends, random disconnects, reconnects, network congestion,
spoofing, protocol hacking attempts, dos-attacks, banning, encryption, etc...

If you're writing an UDP library, you also need to think of performance,
object pooling, connection buffers, threading/async issues and on top of that
you also want to provide a nice API to the outside world for the client and
server... Well, it gets messy...

If you're into this thing, I can advice you to look at haxe libraries. Learned
a lot of them. There are very simple, idiomatic server/client-side
implementations which are easy to follow, even if you don't know haxe [1][2].

[1]:
[https://github.com/svn2github/haxe/blob/master/std/neko/net/...](https://github.com/svn2github/haxe/blob/master/std/neko/net/ServerLoop.hx)

[2]:
[https://github.com/svn2github/haxe/blob/master/std/neko/net/...](https://github.com/svn2github/haxe/blob/master/std/neko/net/ThreadServer.hx)

------
sydd
Ah I love Glen's articles. Ive learned so much from them: game physics,
integrators, game loops, game networking,..

~~~
goodells
Yes! His writeups on networked physics were extremely lucid and easy to
follow. The way that he focuses on minimizing the bandwidth with bitpacking
and goes from something absurdly large to equally absurdly small was a
mesmerizing read for me as a programmer just getting started because of an
interest in games.

------
josephg
Does anyone know if SCTP is suitable / in use by any games? It supports
streams to work around the head-of-the-line blocking problem TCP runs into and
it also supports opt-in unreliable delivery for game data. On the surface
seems ideal for games, though I don't know if its getting much actual use.

~~~
wtallis
Nobody who's in it for the money wants to be the first one to prove that SCTP
can be deployed at scale. Application developers don't want to have to sort
out getting a working SCTP implementation installed and configured on the
proprietary operating systems they target, and past the firewalls of all the
bottom of the barrel consumer routers.

~~~
fulafel
You wouldn't use SCTP on top of IP, you would use on top of UDP and
implemented in userspace. Browsers do it this way (in WebRTC data channels).

------
ozy
"TCP has an option you can set that fixes this behavior called TCP_NODELAY"

That fixes nothing. Now you are sending too many small packets using too many
syscalls. Just like UDP, buffer in user space, send in one go. If you do that,
TCP_NODELAY makes no difference. (The exception is user input, if you want to
send those as they happen, use TCP_NODELAY, but think about the why ... it has
little to do with what this article is talking about.)

Games likely send data only around 25 times per second, and ping is likely <
50ms. Waiting on a dropped packet and the delay it causes is unnoticeable.
Added that clients will need some kind of latency compensation and prediction,
independent of the TCP/UDP choice. Delays and then bursts of 100ms or such are
doable.

The problem starts when the connection stalls for more than 100ms, especially
in high bandwidth games. During the stall both behave the same. After the
stall, TCP will be playing catchup and wasting more time receiving outdated
data, and handing it to user space in order. UDP just passes on what is
received, with a lot less catching up, and maybe some dropping of packets.

But gameplay has been degraded in both cases. UDP just has a higher chance of
masking and shortening degradation more.

Anything more than that is basically cargo-culting, like this article.

~~~
deburo
I will preface this with the fact that the author worked on at least one such
FPS game (Titanfall 2) and probably know what he is talking about.

He offered one solution to fix one behavior of TCP, the behavior being that if
the data written to the buffer is not big enough, TCP might hold it. Setting
TCP_NODELAY forces the protocol to avoid holding data. But this is only
written as a note to coders who might think TCP_NODELAY will fix TCP for
action games, but it doesn't actually, because the protocol has other
characteristics that are undesirable in that type of games.

Moreover, you write this: waiting on a dropped packet and the delay it causes
is unnoticeable. Unless you have a FPS game with TCP as its network protocol
to validate this claim, I call bull. Many network programmers recommend
against TCP. This is probably not simple cargo-culting.

~~~
ozy
"TCP_NODELAY forces the protocol to avoid holding data"

That is the misunderstanding. It will send one non full packet, and only then
"hold on to data" until the next packet is a full packet. But that condition
will reset when all data has been acknowledged. If you buffer in userspace and
write data in one go, at 25 network fps, you basically never trigger the
condition where TCP_NODELAY makes a difference.

It is not nagle's algorithm, but a variant of minshall-nagle, what modern
systems use.

There is a reason why FPS games use UDP, but that discussion should not start
with TCP_NODELAY. That one is misused enough ... often when it fixes anything,
it is because it masks a real underlying problem, had you fixed that problem,
your system would react much better under stress or bad networks.

------
j_s
This feels like a nice introduction.

Anyone here have any experience using QUIC in any application of their own?

The custom congestion control makes me wonder if it only works alongside TCP
traffic - once everything goes QUIC, then what happens? I looked for a bit
about the story in ancient history of some blazing fast server OS TCP
implementation that broke the rules so it fell over when more than one server
was on the network, but couldn't find it.

~~~
CobrastanJorji
QUIC seems like it'd be kind of neat for videogame networking. Does anyone
know whether one can get a QUIC channel to deliver datagram-style, as in
unreliable and no retransmitting? It'd be really cool if you could set up one
QUIC connection that could work across IP address changes and then use it for
N reliable TCP-like streams and M datagram-style game update packet channels.
If you could set up that sort of thing from JavaScript in a web browser,
that'd be straight up amazing.

------
dpeck
reminds me of this old article we had as assigned reading during a networking
class in school.

The Internet Sucks: Or, What I Learned Coding X-Wing vs. TIE Fighter
([http://www.gamasutra.com/view/feature/131781/the_internet_su...](http://www.gamasutra.com/view/feature/131781/the_internet_sucks_or_what_i_.php?print=1)).

------
jokoon
In short, TCP will work hard to deliver 100% of the packets. So when a packet
is lost, TCP asks to re-send the packet. This is fine to display a webpage or
send a file, but it can't be tolerated in games where time continuity matters.
I think it's the same issue in VOIP and video conferencing too.

------
cpncrunch
The reality is that these days there generally isn't any packet loss, so UDP
vs TCP isn't such an issue as it might have been in the past. In fact TCP has
a number of advantages these days such as easier firewall traversal,
WebSockets, etc.

~~~
ra1n85
>The reality is that these days there generally isn't any packet loss

Do you have any sources/data on that? Genuinely interested where you drew that
conclusion from. From my vantage point (anecdotal), ISPs and carriers
routinely under-provision, congest peerings, and don't find a problem well
until after a number of customers complain.

~~~
wtallis
Bufferbloat means that at many potential bottlenecks, you don't get any packet
loss until well after the point at which your latency-sensitive application
(be it TCP or UDP) has given up hope. Major peering points are pretty much the
only place where you _won 't_ see >1second queues when congested, simply
because they can't afford buffers that deep for the rates they operate at. But
for ordinary everyday occurrences, your congestion will be somewhere around
the last mile, where you cannot expect packets to be dropped within a
reasonable timeframe.

~~~
DonHopkins
[https://www.bufferbloat.net/projects/bloat/wiki/Introduction](https://www.bufferbloat.net/projects/bloat/wiki/Introduction)

Introduction

Bufferbloat is the undesirable latency that comes from a router or other
network equipment buffering too much data. It is a huge drag on Internet
performance created, ironically, by previous attempts to make it work better.
The one-sentence summary is “Bloated buffers lead to network-crippling latency
spikes.”

The bad news is that bufferbloat is everywhere, in more devices and programs
than you can shake a stick at. The good news is, bufferbloat is now, after 4
years of research, development and deployment, relatively easy to fix. See:
fq_codel: wiki. The even better news is that fixing it may solve a lot of the
service problems now addressed by bandwidth caps and metering, making the
Internet faster and less expensive for both users and providers.

The introduction below to the problem is extremely old, and we recommend
learning about bufferbloat via van jacobson’s fountain model instead. Although
the traffic analogy is close to what actually happens… in the real world, you
can’t evaporate the excessive cars on the road, which is what we actually do
with systems like fq_codel wiki.

Still, onward to the blow.

[...]

------
johnbellone
If you want to get familiar with TCP/UDP and are not gun-shy with C I would
suggest the Pocket Socket Guide (now Practical Guide to TCP sockets)[1]. I
have a really old edition, but it ages extremely well, and its one of the
books I always use to refresh myself on network programming basics.

[1]:
[https://www.amazon.com/gp/product/0123745403/ref=ox_sc_act_t...](https://www.amazon.com/gp/product/0123745403/ref=ox_sc_act_title_1?ie=UTF8&psc=1&smid=ATVPDKIKX0DER)

------
fivesigma
A huge problem with TCP wrt gaming is the default ACK frequency on Windows
which is set to 2. This effectively almost doubles the latency of game
connections (sending/receiving a lot of time-sensitive small packets).

It can be changed with a registry setting (TcpAckFrequency) but you can't
expect even a significant fraction of your users to do that. Why this isn't a
per-connection option sort of like TCP_NODELAY is beyond me.

------
dozzie
I thought that it was somebody who has recently discovered the existence of
UDP protocol and brags about that to the world, but from skimming the article,
it actually has some non-trivial remarks about UDP and TCP.

BTW, the article's view angle is multiplayer game programming.

~~~
exDM69
This guy (Glenn Gaffer) has been writing network protocols for games for years
and shipped popular titles.

On his site you can also find an irate rant about people with no experience
dismissing his claims as "reimplementing TCP". It has good points but it's a
frustrated rant (reader beware). But it addresses a lot of the commonly
believed fallacies.

His latest project is an open source library for game networking over UDP:
[https://github.com/networkprotocol/libyojimbo](https://github.com/networkprotocol/libyojimbo)

And he is correct, for "real time" games you must use UDP or you will be in
trouble in real world networking conditions. When everything runs smoothly,
TCP and UDP work almost identically but when packet loss occurs, TCP will make
things _worse_.

~~~
jay_kyburz
Glenn _Fiedler_ - Gaffer is his online name / handle.

Worked with Glenn on Freedom Force and Tribes. He knows his stuff!

------
the_arun
There was a similar discussion in Hackernews earlier -
[https://news.ycombinator.com/item?id=7507377](https://news.ycombinator.com/item?id=7507377)

------
JohnStrange
What about UDT? I never tried it but always wondered about it's
benefits/disadvantages.

------
aakilfernandes
Another important difference: UDP can't be used in the browser =(

~~~
rdw
WebRTC is UDP. It's not easy, but it's there!

~~~
foota
nitpick: It can be either

~~~
arkitectual
It is possible to use only one or the other.

------
dogismycopilot
That's a pretty obnoxious 'donate' button. They can't make it even larger?

~~~
bschwindHN
The guy is providing valuable information for free, he can make his donate
button as large as he wants.

~~~
Kiro
Yeah, sure. He can put the Donate button all over the screen if he wants but
then no-one would donate. I also think it's obnoxious.

------
z3t4
I think tcp have an unfair reputation. Our networks are better now then 30
years ago ... Worst case latency for tcp is like 3 seconds, compared to the
packet never arriving. The trick is to hide the lag with animations. I think
google, facebook, and world of warcraft use tcp for their real time apps !?

~~~
taneq
Sad to see you being downvoted when you're absolutely right. Yes, UDP is
better for a twitch game, at the expense of a lot more work. No, unless you're
building a AAA mouse-and-keyboard shooter, you probably don't need more than a
TCP connection sending data packets back and forth using a somewhat well-
thought-out protocol.

~~~
z3t4
I think the biggest mistake is _only testing the app on localhost_. While real
world users can experience up to 150ms latency and fluctuations. It doesn't
matter if you use TCP or UDP you still need a strategy for handling different
levels of latency. For testing I use netem to add delay ...

The second biggest mistake is _trusting the client_. People never cheat in
online games, right !? _smile_ Network latency is often less then it takes to
render, so naive solutions with simple broadcast servers work well until you
need to deal with rouge clients using teleport hacks etc.

There are no set rules when making real time distributed and scalable
applications like MMO Games. In my experience it's best to start with a naive
demo/prototype to see where the bottlenecks are. I would suggest starting with
TCP and then only switching to UDP, or more likely making your own protocol
(TCP ontop of UDP _smile_ ), when you know the requirements.

