

Hack initcwnd for faster page loading - sajal83
http://www.cdnplanet.com/blog/tune-tcp-initcwnd-for-optimum-performance/

======
necro
Great writeup. Have been hacking initcwd to way back when it was just
hardcoded in the kernel. Other parameters not mentioned that control
congestion:

1\. net.ipv4.tcp_slow_start_after_idle By default this is set to 1, which
means, in a longer keep-alive connection, your connection will revert back to
slow start (whatever you ended up setting it as per article) after "idle"
(idle is 2 seconds last time i checked). Of course in a keep-alive connection
you may have opened up your window from slow start to max, and you probably
dont want it do default back to initcwd on subsequent requests. Especially
applies on static file/cdn type servers where you probably are running longer
keep-alives. You can just set this in sysctl.

2\. the last time i checked (years ago) if you run conntrack and turn off
tcp_no_metrics_save, you get the benefit of the window opening up to the last
remembered window size from your previous connection.

The concepts on increasing initcwd are particularly advantageous in the mobile
arena where the rtt is usually much higher.

~~~
sajal83
I am somewhat skeptical about the benefits to mobile. the RTT is higher, so
lower round-trips is good, but isint the risk of congestion higher in mobile?
Say someone with a poor signal?

~~~
saurik
Good direction, but bad conclusion: "poor signal" is not the same thing as
"congestion". This fact is what causes TCP over any form of wireless
(including high-speed WiFi) to have been a really bad decision (and one that
academic researchers the world over love trying to solve ;P).

The reason is that TCP makes an assumption (almost entirely true in the case
of physical cable) that "packet loss" is always caused by "congestion" (due to
the router simply throwing away packets it cannot handle), and so if you are
losing packets you need to slow down your rate of sending, to help more of
them get through.

However, if the reason you are losing packets is "poor signal", then you are
going to get a pretty much constant rate of packet loss: if 10% of your
airwaves are "damaged", 10% of your packets are going to be lost whether you
are sending them at 100k/s or 1b/s (as your choice of "when to send" is
entirely random from the perspective of the interference: your device is not
modeling the underlying issue so it can try to "dodge the raindrops"; people
work on this, though ;P).

The result is that when your signal is bad, even if you are currently getting
90k/s out after 10% loss, your standard TCP stack goes "oh shit" and starts
dialing down its transmit window to "virtually nothing", leaving you unable to
use that connection to get any data through anymore; even just opening a new
connection when this happens (which may require restarting the app) tends to
help tremendously.

(The solutions, for the record, all involve some sort of mechanism that allows
explicit bookkeeping of either packet loss or congestion. However, this
explicit bookkeeping requires various players that are not the device to be
"in on it", and therefore makes it really hard to find the right balance of
"who needs to install better routers to solve the problem", given the wide
deployment of existing TCP... I mean, even getting IPv6 out there has been a
tough sell, replacing IP entirely is a pipe-dream right now.)

------
perssontm
This is really interesting, but it seems like its the same information
floating around on different blogs. If I didnt know better I'd assume they
copy each other regarding this.

All lack some real and more thorough testing, they include that simple
example, perhaps its enough, but its a bit light on the facts to get me to try
it.

Would be interesting to know how this works in virtual environments, can I set
this inside a VPS, or would the main kernel in the dom0(xen-host-stuff)
override this?

Are there any sideeffects?

~~~
sajal83
id be posting some stats soon about the RWIN of traffic to our site. i.e. what
% of users have a high enough RWIN to benefit from it. The setting works as
expected on EC2 instances, i don't see why this would not work on dom0 as
well. It doesn't need a kernel recompile, its just settings for your
networking stack.

------
wazoox
What I'm wondering is if that could have some influence on other protocols,
too, like NFS. I'm going to do some testing.

~~~
sajal83
I think NFS usually uses UDP by default. Even when using TCP, I think
increasing the initcwnd/initrwnd will only benefit the first few transfers
after mounting. Would be curious to what you find out.

~~~
wazoox
I did some initial TCP tests running netperf on a single GigE line, and there
are some tenuous gains. I'll do NFS over TCP tests with 10 GigE or IB soon to
get some meaningful numbers.

------
ck2
We went through this last week on HN but this article is far more detailed.
Thanks.

