

Clark's Curious Corollary of Concurrency - cgaebel
http://clarkgaebel.com/clarks-curious-corallary-of-concurrency/

======
ArbitraryLimits
[http://rescomp.stanford.edu/~cheshire/rants/Latency.html](http://rescomp.stanford.edu/~cheshire/rants/Latency.html)

> Years ago David Cheriton at Stanford taught me something that seemed very
> obvious at the time -- that if you have a network link with low bandwidth
> then it's an easy matter of putting several in parallel to make a combined
> link with higher bandwidth, but if you have a network link with bad latency
> then no amount of money can turn any number of them into a link with good
> latency.

You can tell it's dated from this little tidbit:

> The Cable TV industry is hyping "cable modems" right now

~~~
finnh
Theoretically, if your bandwidth is high enough, you can transfer the entire
computational state of the distant resource to a local substrate, and then run
the computation locally for a low-latency conversation.

So, if you are annoyed by the slow comms of our alpha centauri - earth
channel, just transfer _your entire brain_ to a local avatar and I'll converse
with that. Then run "git merge" to bring the remote history back to the master
repo.

------
NathanKP
@cgaebel The code for your blog seems to have a race condition in it where the
content is all there on the page, its just invisible, and only appears
properly about 1 in 10 times. Either that or maybe your code is just hanging
because I'm not allowing it to load Google Analytics like it wants.

[http://i.imgur.com/w1ZB7Qy.png](http://i.imgur.com/w1ZB7Qy.png)

I'm using Google Chrome 33.0.1750.149 on Mac OS X. I had to open dev tools and
tweak your CSS to get the invisible text to appear so I could read it.

~~~
overdrivetg
It's probably this (known) Chrome bug:
[https://code.google.com/p/chromium/issues/detail?id=336476](https://code.google.com/p/chromium/issues/detail?id=336476)

I've been hitting it a bunch lately too... Workarounds listed there seem to
improve but not totally fix the issue unfortunately.

------
tbrownaw
This only works for "embarrassingly parallel" tasks, where you have lots of
completely independent things to do. Please remember that the whole world is
not a web server.

This is not trading latency for thruput. This is spending _hardware_ for
thruput, with some overhead in latency.

The relevant generic performance number is CPU-seconds (for CPU-bound work),
or I/O consumption (for disk-bound work), or in general how much of your
bottleneck resource is consumed. Once you know your bottleneck, you can either
improve your code to use less of that resource, or buy more of that resource.

As tasks become less embarrassingly parallel, throwing (non-serial) hardware
at a problem increases communication overhead, and gives lower speedups.

------
neilc
Another factor is that technology improvements tend to favor increases in
bandwidth over increases in latency (and there are hard limitations on
latency, like the speed of light in distributed systems). This short paper by
David Patterson is a great read on the subject:

[http://www.cis.gvsu.edu/~kurmasz/Teaching/OldCourses/CS451/G...](http://www.cis.gvsu.edu/~kurmasz/Teaching/OldCourses/CS451/General/WebPage/ReferenceDocuments/Patterson-
CACM.pdf)

~~~
marcosdumay
Well, in practice the latency that normaly matters is between "start doing
activity" and "finish doing activity", and the time actualy working (that
depends on bandwidth) is normaly orders of magnitude bigger than the time
waiting for data (that depends on latency). There are exceptions, but not
many.

And now that I really thought about it, looks like the article's law isn't
that relevant in practice. Yes, you can always trade latency for bandwidth if
you throw some money at it. But money is finite.

~~~
neilc
_Well, in practice the latency that normaly matters is between "start doing
activity" and "finish doing activity", and the time actualy working (that
depends on bandwidth) is normaly orders of magnitude bigger than the time
waiting for data (that depends on latency)._

I don't think that's true; moreover, it is likely to become even less true in
the future. Once the data needed for a computation has arrived at the CPU, for
most applications the required computation is pretty cheap -- the time spent
waiting for the data often dominates the time spent computing with the data.
Much of modern CPU design has been devoted to trying to hide the high latency
of memory accesses.

------
finnh
"Tell me the 99th percentile latency. Track it. Improve it. When needed,
throughput comes easy."

Actually, if you're worried about tail latency, it can be improved via
throughput. Expensively:

Double your entire infrastructure. Send each request twice, once to each half.
Take the results of the faster one.

I assume that Google does something like this for fast search results.

~~~
packetslave
Yep, and you don't even (necessarily) need to double your infrastructure to do
it. Jeff Dean talks about some strategies in "The Tail at Scale"

[http://research.google.com/pubs/pub40801.html](http://research.google.com/pubs/pub40801.html)
(paper)

[http://www.youtube.com/watch?v=C_PxVdQmfpk](http://www.youtube.com/watch?v=C_PxVdQmfpk)
(talk)

------
ccurtsinger
Little's law certainly applies here. The article's author does have a good
point though: if there is no queue and requests are processed one by one, your
only option is good old serial program optimization.

Still, I think most large systems where latency is a major concern have some
level of concurrency and/or queueing that can be manipulated to reduce
latency.

------
scott_s
_Performance is hard, but we should all be working together towards the things
that matter the most. Latency is more important to optimize than throughput.
Let 's just focus on that._

I believe this is naive. First, it has no context. You optimize for what you
need; there is no absolute best thing. If you have fantastic latency, but your
throughput is not good enough to meet your needs, then, no, latency is not
more important.

Second, while we tend to trade latency for throughput, it's not an even trade.
That is, we tend to trade a _small_ increase in latency for a _large_ increase
in throughput.

The insight the author had is a good one (although as others have pointed out,
we've realized it before), but I think he oversold the conclusions.

~~~
twic
> You optimize for what you need; there is no absolute best thing. If you have
> fantastic latency, but your throughput is not good enough to meet your
> needs, then, no, latency is not more important.

The _whole point_ of this post is that if you have low latency, there are easy
ways to trade it away for better throughput. Whereas there are not easy ways
to trade high throughput for better latency. And, therefore, latency is
fundamentally more important.

~~~
scott_s
And my point is that absent knowing what your needs are, it's silly to talk
about what is "more important". Because of the lopsided nature of the tradeoff
(small latency harm for big throughput gain), it's dangerous to keep around
"latency is more important" as a mantra.

------
sp332
It's hard to write latency benchmarks for desktop apps. I remember this
discussion happening when the "completely fair" scheduler was introduced to
the Linux kernel a few years ago.

------
porlw
Have you heard of Amdahl's Law?

You should take a look - I think it says something similar.

~~~
reitzensteinm
Amdahl's law is only related in the sense that they are both dealing with
concurrency. It deals with diminishing returns due to parts of programs that
can't take advantage of parallel processing.

The OP is saying that there are techniques that hurt latency and increase
throughput, but not necessarily vice versa (except unwinding techniques going
the other way), so it makes sense to focus on latency first.

It's an interesting theory, and I've filed it away as something to ponder, but
I'm not sure in what use cases it's valid for.

If you've got a stream of independent non trivial requests, then the best
throughput strategy is to process each request on one thread; almost all
computing resources are focused on the problem at hand.

The best latency strategy is to devote all threads to each request in turn
(assuming this gives a speedup). The communication overhead will increase the
CPU cost of dealing with each request, hurting overall throughput (if you
process two requests simultaneously, your latency will explode).

In this scenario, it's a simple trade off and thus doesn't seem to fit. Of
course, nothing real world is this idealized, and the devil in the details is
probably where the OP's law fits in.

------
lucian1900
Blank page :(

~~~
krallja
using Chrome?

