
Visualization of L1, L2, Ram and Disk latencies [gif] - Sam_Odio
http://imgur.com/X1Hi1.gif
======
patio11
Anyone like physical analogies more? If you convert them into relative masses:

L1 cache: a squirrel (1kg)

L2 cache: a mid-sized cat (~5 kg)

RAM: a tall, well-muscled man (~80kg)

Hard disk: one hundred blue whales (100 * ~130 metric tonnes)

This is what I mean when I say "It doesn't matter how fast your language is,
you're just racing to get to wait on I/O faster."

P.S. Let's extend the analogy to include two other common factors:

Typical round trip to database: the combined mass of every ship, plane, and
person in the USS Nimitz' air group... with room for another two fleets or so
after you're done (150 ms ~ 1.5 million metric tonnes)

Time for user's computer to render a web page of medium complexity: worldwide
demand for cement in 2009 (2 seconds = 20 million metric tonnes)

But please, spend time optimizing your string concatenation... because that is
going to help ;)

[Edited: Revised and extended because I introduced a conversion error or two
and then compounded them. Word to the wise: mental conversion to fractions of
blue whales not advisable before morning coffee.]

~~~
lsb
Okay, so, if your DB all gets paged into RAM, what tends to be the next mole
to whack in the optimization whack-a-mole?

~~~
patio11
What sort of app are we talking about? If it is a web app, and you're
optimizing the DB... are you sure you're not already working on the wrong
mole?

Take a look at the above scale: page rendering times are going to dominate
everything else. There are simple, repeatable, effective ways to reduce them.
See the presentations from the YSlow guys.

~~~
lsc
key. I mean, all my experience is with squid, but there are many, many caching
proxies. Using caching proxies is a ridiculously easy way for a SysAdmin to
take a slow webapp and make it fast without screwing with the application
code.

Unfortunately most webapp developers don't use reasonable cache control
headers. Most php apps, if your proxy does a HEAD to see if it should re-pull
content, render a full page and throw it away except for the headers. (this
may be false now, but when I did this, they were using php3. In php3, to
handle HEAD requests properly you'd have to actually write code to handle it,
which most programmers did not.)

Still, my experience has been that using something like squid gives you a
pretty massive performance advantage, even when your webapp is uncooperative.

------
randomwalker
I've been HDD-free for a few months now; I can't imagine going back. It's not
just latency that's an issue for me, it is also reliability (I've had 6 HDD
failures in the last two years in various devices around the house.) I also
worry less about damaging something if I drop my laptop.

For my latest work project (I do scientific computing), I realized it's easier
to do it on my laptop instead of the workstation. Since my laptop has an SSD,
I can just use the filesystem as my database. This means that I can have have
millions of files (literally, millions) lying around and process them using
the good old Unix shell. It greatly reduces the development time compared to
using a database. Just for giggles I tried doing this on a machine with a hard
drive, and it was more than one hundred times slower.

~~~
lsc
Have you tried it on a box with a HDD and an adequate amount of ram? I would
think that on modern file systems with ordered metadata writes (or journaling)
like ext3 or ffs with softupdates, so long as you had enough ram, disk speed
wouldn't matter all that much so long as you can keep everything in cache.

SSD is great, the problem is that the good SSD costs something like $15 per
gigabyte, and good registered ecc ddr2 ram costs just over $20 per gigabyte.
Sure, in applications where consistency across power-loss events is a huge
deal, ssd is the right answer, but for most applications, buying a whole lot
of ram is often faster and not that much more expensive.

~~~
randomwalker
RAM was not the issue. The reason for the slowness, as I understand it, is
rather that different files are spread out in different areas of disk, even if
they are in the same directory. This is considered a feature, and I guess it
makes sense under normal access patterns. So accessing a million files (even
to load them into memory for the first time) would require the same order of
disk seeks, and takes forever. I might be simplifying a little bit, but this
is my understanding.

Could I have re-written the code by messing around with inodes and other low-
level details so that it accessed the files in physical order? Probably. Was
it worth my time, rather than using an SSD? Hell no.

I agree that SSDs are still a tad expensive for the average Joe. For most
hackers, considering that we spend most of our work hours in front of a
computer, I feel that the added productivity from an SSD is easily worth the
investment.

~~~
lsc
<http://en.wikipedia.org/wiki/Page_cache>

the idea is that if you have enough ram, you only need to read the files from
disk once. after that, the files are in ram cache. Once the files are in
cache, at least for reads, it doesn't matter how spread out on disk they were.

And yeah, you do need to read the files from disk once, and that is slow; thus
you often see a 'warm up' effect on servers. hitting a new page is often
slower than it is for the second person who hits that same page.

~~~
gnaritas
Ram might not hold all the information, and reading the files isn't the
problem, finding them is. SSD's have virtually no seek time, it is RAM.
Tossing in a 64gig SSD essentially puts the entire file system in RAM, at
least that's what it feels like, they don't need warming up. It feels like
everything's in the page cache all the time.

~~~
lsc
if the file is in cache, you can 'find' it in cache, without hitting disk.
seek time, (which I assume is what you mean by 'finding it') is only a problem
if the file isn't cached in ram.

Yes, running on a SSD takes the entire filesystem much closer to ram speeds.
However, you are doing so at almost ram prices. (I'm speaking of good SSDs,
like the X-25E; which comes to something like $15 per gigabyte; the not so
good SSDs have problems of their own. I have a SSD in my laptop right now that
is branded by one of the gamer ram companies, i forget which one. It was
pretty cheap, under $2 per gigabyte. It's pretty nice for reads, for writes,
sometimes it is good, but often writes are worse than spinning disk.) The
advantage of just buying the ram is that a good virtual memory management
system can automatically optimize to keep the data you access most often in
ram.

Like I said, i use a SSD in my laptop, a cheap brand and it's small, my laptop
doesn't need a lot of storage, so the cost is reasonable, and I use a
journaling file system, so writes are cached and the slow sub-cell size write
speeds of the cheap SSD aren't a huge problem. I'm just explaining why in my
servers, I prefer to go with a whole lot of ram, and then slow, cheap, and
large SATA, rather than less ram and expensive SSD.

~~~
gnaritas
I don't disagree about stuffing ram in servers, that's obviously the best
approach, but it isn't always an option. I'm talking about the desktop
experience. Many of us are limited by time and circumstance and are still
using 32bit OS's on the desktop. I can't stuff it with RAM, but my Intel X-25M
SSD makes my desktop smoke like no other hardware upgrade ever has.

My of us are also stuck with mission critical legacy 32 bit servers that we
can't just take down and upgrade so easily and licenses for _enterprise_
versions of some db's that can handle assloads of ram don't come cheap. SSD's
are much cheaper and a no brainer upgrade for that aging db server that just
needs to be faster. The X-25E smokes here letting you get that speed without
needing that expensive license.

------
kazuya
In case you are too lazy to enter the URL at the top:

[http://duartes.org/gustavo/blog/post/what-your-computer-
does...](http://duartes.org/gustavo/blog/post/what-your-computer-does-while-
you-wait)

There you can find more nice figures.

~~~
gourneau
The post links to a 114 page paper titled 'What Every Programmer Should Know
About Memory" take can be found here
<http://people.redhat.com/drepper/cpumemory.pdf>

------
sho
I'm sure there's an excellent reason, but the first thing I think whenever I
see something like this is .. why doesn't intel (or whoeever) load up the CPUs
with more L1 and L2 cache? Are there really diminishing returns so quickly
after 6MB, or would the size increase make the expense not worth it? And is it
impossible to make it modular and expandable?

It would be interesting to see some account of the cache size / die size cost
/ performance trade-offs.

~~~
wmf
It's commonly accepted that the cost of a chip is super-linear in its area but
performance improves sub-linearly with cache size, so there is definitely a
point of diminishing returns. Processor vendors analyze cost/performance
tradeoffs in detail, but they generally don't publish anything.

~~~
sho
Yeah, that's what I figured. Be nice to see some numbers though.

Not like I have a hope in hell of getting anything I write even into 6MB ..
sigh.

------
malkia
All I see is a red-bar strip. Is this an animation or something (I've tried
Mozilla, Safari)

~~~
yan
Zoom in all the way and look on top

------
lsc
Nice. it makes it real clear why caching disk to ram (as all modern *NIX
variants do) is such a huge win, and why you should always load up your
servers with as much ram as you can afford.

Sure, CPU contention can slow things down, but it's usually not the 'fall off
a cliff' performance degradation that hitting disk (rather than hitting ram
cache) is.

------
jws
Don't try to view this on an original iPhone. It crashes safari.

~~~
Poiesis
iPhone 3g crashes too. :(

------
grinich
I feel like I need a SSD just to deal with this image. How about like 3 lines
of CSS?

~~~
sho
Look, it is 54 KB. That is not large. Perhaps with some effort it could have
been slightly smaller but the author is probably optimising for a different
metric.

~~~
kinetik
The image dimensions are 1120x13800, so it's 59MB uncompressed in memory.

------
mcav
It'll be nice when SSDs come down in price/GB. Should help make day-to-day
work seem just a tad quicker.

~~~
Retric
I can't find good numbers, but it looks like SSD's are still around 0.1
millisecond's which is still 100,000 nanoseconds. And 1/1000th the access time
of RAM if if they are 100 times faster than HDD's. Most people don't really
notice that big of a jump from HDD to SSD's and, I don't see it being as much
of an issue for a while due to the increasing ram and cache sizes.

~~~
leif
The major improvement coming from SSDs is that seek time no longer kills you.
People will notice a difference going from random access on a rotational drive
to random access on an SSD; sequential access, not so much.

~~~
Retric
My point was while SSD seek time is 1/100th HDD seek time you don't get
anywhere near that big a jump. Because, while a HDD might take 100x as long to
get you the first bit, HDD and SSD take about the same amount of time to read
the rest of 4kb the sector.

------
vannevar
How about adding ethernet and wireless latencies?

------
TheSOB88
Why doesn't this take into account the size of the block you're getting back?
1 pixel from the L1 gets you less data than 8 from the L2, etc.

