
An Empirical Evaluation of TCP Performance in Online Games - cpeterso
http://www.iis.sinica.edu.tw/~swc/pub/tcp_in_games.html
======
e12e
Reminds me about the last few paragraphs of:

[http://250bpm.com/blog:22](http://250bpm.com/blog:22)

Especially:

"Obviously, the right solution would be implement a new transport protocol
directly on top of IP protocol, one that would provide desired functionality —
failure detection and/or multiplexing — directly, without duplicating the
features.

And, as a matter of fact, the above was already done. The protocol built
directly on top of IP with both heartbeating and multiplexing is called SCTP
and is available out of the box in most operating systems.

And here comes the problem: SCTP is not used, even for projects where those
features are needed. Instead, they are lousily re-implemented on top of TCP
over and over again."

(Not necessarily saying that SCTP would be the best fit for games, more the
general point -- that there's room for more than just TCP and UDP on top of IP
-- and also that that might not work, due to firewalls and routers refusing to
route anything that's not TCP or UDP).

~~~
tinco
It's not that SCPT is less known by developers, I think most developers
actively looking for a solution to the downsides of TCP will come across SCTP.

There are two real reasons, the first and most important is that Microsoft
never implemented SCTP. Apparently there was not enough customer demand, but
that's a lousy excuse, I'm not sure what the real reason is. Perhaps they just
have no one thinking about networked applications besides the Web at the
moment. Sadly lack of forward thinking is something that bugs the Windows team
a lot I feel. The second reason can be partly blamed on Microsoft as well, but
most network devices operating at the transport layer have difficulty with
SCTP. Why implement it when no one uses it right? just like IPv6 support.
Anyway, apparently the difficulty lies with checksumming packets, which
requires some more memory than is necessary for TCP/UDP.

It's a shame really. I think you can safely say that SCTP is the perfect
gaming protocol. It supports almost all features gaming networking libraries
usually implement on top of UDP just by configuring your streams.

~~~
throwawaykf
_> Perhaps they just have no one thinking about networked applications besides
the Web at the moment. Sadly lack of forward thinking is something that bugs
the Windows team a lot I feel. _

If you found out about the wide range of networking protocols and services
they provide in Windows, you wouldn't feel that way. For example, P2P. A few
years ago, while working on P2P applications, we discovered Microsoft has a
lot of technologies in that area. Things like Teredo (IPv6 transition and NAT
traversal) and a complete P2P networking API -- which gives you encryption,
self-organizing overlays and DHTs out of the box -- have been around since
around 2001, and have been built into Windows since Vista. They also had lower
level protocols for things like link-layer topology discovery and automated
device (printers, etc) configuration.

Almost all proprietary stuff, of course - except Teredo, which has an RFC and
an open source implementation - but quite interesting nonetheless. We talked
to the team working on this, and they claimed it was even being used by some
customers in 1000+ machine deployments.

For some reason, these things are not widely marketed. Maybe because it's a
niche market. However, I wouldn't say they're not forward-thinking, at least
when it comes to networking.

------
codexon
It is also important to consider DDOS when evaluating protocols.

TCP has an advantage over UDP because data cannot be spoofed. There are many
ddos protection services that will protect against spoofed TCP connections
(SYN floods).

Applications must implement their own 3 way handshake in UDP in order to avoid
these problems. Many applications did not do this which is why COD servers
ended up being a source of attack traffic themselves.

------
dedalus
The problems raised here can be solved with parametrizing TCP itself. A few
notes here:

(a) Packet reordering is highly dependent on the timescale that you use. Its
far worse in micro/smaller timescales than to be seen as attached in the graph

(b) By using TCP URGent byte effectively you can hack some realtime control
stuff onto a data stream

(c) The fast retransmit not kicking in can be resolved by lowering the number
of dupacks from 3 to 1 and add a timer that kicks in the fast retransmit after
the timer expiry. This isnt a new ide. BTW 50% is the standard number for the
whole internet so your stats does not show anything different from the
standard internet

(d) TCP slow start after idle can be disabled on server side

(e) Agreed all of these require some co-operation from the client side which
might be hard to get (especially on windows) but can easily point to a http
proxy for the MMORPG in question and usually the speed benefit will drive the
adoption of such proxies before actually having someone sell a Gamer TCP Stack
:-)

Long story short all the problems can be mitigated to some degree within the
norms if you use the right proxy :-)

------
cclogg
This is a really interesting article, I love reading about game networking.
Hard to find many resources on this type of stuff... the 'Age of Empires'
Gamasutra one is awesome too.

We used Apple's Game Center API for the online portion of Stratosphere:
Multiplayer Defense (iPad), and it was pretty nice that Apple did all the work
as far as network packets go... but you do have a choice between reliable and
non-reliable sending. It wasn't explicitly said, but we assumed one was TCP
and the other UDP. We went with the reliable option for all of our packets and
it didn't really prove to be a problem, even simulating under 3G. Granted,
Stratosphere is a strategy game. The bigger problem ended up being just having
enough people online at one time to even play lol.

~~~
plasma
Check out the Tribes netcode model,
[http://library.theexiled.pwnageservers.com/file.php?id=2694](http://library.theexiled.pwnageservers.com/file.php?id=2694)

------
ericz
If you're a web game developer reading this you should also know that
WebSockets, XHR, and anything built on top of those technologies (socket.io
etc.) are TCP based. Only WebRTC DataChannels and flash are able to do UDP
based communication in the browser.

------
scurvy
Perhaps the developers communicated with their netops teams before designing
the game protocols? Maybe they came to the conclusion that it's better to run
TCP from a security/DDoS perspective than UDP? In other words, the ability to
shut off inbound UDP outweighed any theoretical gains from using it in the
game?

TCP SYN floods are handled with ease today. If you're actually using UDP in
your product, you might as well paint "DDoS me" on your homepage.

------
salmonellaeater
HN ought to parse
"[http://www.iis.sinica.edu.tw"](http://www.iis.sinica.edu.tw") as
"sinica.edu.tw" rather than "edu.tw".

~~~
saraid216
Or more broadly, \w+.(com|gov|net|org}.countrycode should really happen.
Probably a couple other standard ones.

------
Liongadev
I call that thing bullshit. "Our analysis indicates that TCP is unwieldy and
inappropriate for MMORPGs" yet World of Warcraft, Lineage I/II, Guild Wars,
Ragnarok Online, Anarchy Online, Mabinogi all use TCP. Extremly successful. I
using UDP would have help making World of Warcraft better they would have done
it years ago. But they all probably have no idea right?

~~~
eropple
That TCP is ill-suited for this does not mean it cannot be used. It also does
not mean that it is World of Warcraft's particular bottleneck and that it
would be a good investment of development time to change it once it is in
production.

You should probably resist the urge to call bullshit on something when your
best examples rely on legacy concerns.

~~~
derefr
Yes, they all manage it--but this is why "MMORPG network middleware engineer"
has always been an extremely well-paid senior-level position. If it was as
easy as "SCTP.connect(host: 'example.com', port: 3044)" to get something
_sensible_ going, I think a lot more games would do the kind of smooth
network-latency-compensation-via-action-prediction that you only tend to see
right now from the big players.

~~~
eropple
Fully agreed. I've implemented a bad equivalent on top of UDP myself. It
should be easier given how common a use case it is.

