

Behind the Windows 7 memory usage scaremongering - chmike
http://arstechnica.com/microsoft/news/2010/02/behind-the-windows-7-memory-usage-scaremongering.ars

======
ghshephard
Kind of reminds me the first time I started freaking out over the "Memory Bug"
on my Solaris 2.6 install in 1998. So many people got worried by that data,
that they modified several of the system binaries to report "filesystem cache"
as "free" in later versions of Solaris.

What's old is always new, again.

Linux makes it (reasonably) clear:

    
    
       shephard@u804:~$ free
                    total       used       free     shared    buffers     cached
       Mem:        515440      39976     475464          0       2684      16452
       -/+ buffers/cache:      20840     494600
       Swap:       192740          0     192740
    
       shephard@u804:~$ dd if=/dev/zero of=/tmp/475MB bs=1048576 count=475
    
       475+0 records in
       475+0 records out
       498073600 bytes (498 MB) copied, 73.7244 s, 6.8 MB/s
    
       shephard@u804:~$ free
    
                    total       used       free     shared       buffers     cached
       Mem:        515440     508312       7128          0       1760     475756
       -/+ buffers/cache:      30796     484644
       Swap:       192740          0     192740
       shephard@u804:~$

~~~
FooBarWidget
Many people don't know how kernel caches and buffers work. Even many
developers get confused by this and think that a system is almost out of
memory. Luckily someone created <http://www.linuxatemyram.com/> \- that saves
me time to explain it over and over to people who panic about the issue.

------
Skriticos
I'm by no means fan of windows, but the article was an obvious troll from the
beginning, as it was quite evident that they were mixing up cache with real
memory usage.

What really surprised me was that such a pointless 'story' generated so much
fuss. Waste of time.

~~~
rbanffy
> mixing up cache with real memory usage.

And how real is the use of memory for cache?

All this reminds me of a discussion I had with a mainframe admin in the mid-
to-late 80's:

Me: Look at this numbers. We are using more than 90% of the processor time and
about 100% of the memory. Is it normal?

Sysadmin: Good. We paid them in full. It's better if we use the whole
computer, not just part of it.

~~~
sparky
In some cases, the sysadmin is right. Say you are concerned with performance,
not energy-efficiency. If you have an I/O-bound workload that is only using
50% of your CPU cycles, wouldn't it be nice to automatically use the other 50%
(otherwise wasted, mind you) to figure out some other things you might need in
the future and prefetch them? On the memory side, what good is that 1GB or
unused RAM doing you? If you need more for your application, the OS can drop
its cache on the floor so quickly you'll hardly notice the latency.

Granted, this prefetching is not without downsides. DRAM banks have low-power
states you can put them into if they're not being used, so if you can shove
all of your used memory into a contiguous region of your physical address
space and free up a DIMM's worth, you might be able to save power. This is
unlikely, however, because the virtual-to-physical address mapping function is
usually designed to spread the load very evenly across all available DRAM
banks. One more likely source of power savings is from avoiding the CPU and
disk activity of prefetching, especially if prefetch accuracy is low. Most
people tend not to care about this though, and want the OS to make their app
run faster by using otherwise-idle resources to do prefetching.

~~~
rbanffy
It was an 80's mainframe. I doubt power saving was one concern. We had folks
typing forms in data-entry terminals 24x7 generating the tapes that were then
fed to the bigger machine.

But you have an interesting idea. I wonder if there is any OS that changes
memory distribution and allocation strategy according to desired power
consumption on, say, portable computers.

The Linux kernel supports, IIRC, plug-and-play memory and processors and it
could, conceivably (it it doesn't already) power up and down unused parts of
the machine in order to reduce power consumption and heat dissipation. Even
reads and writes to and from memory could be grouped in bursts if that could
save power.

For instance, right now my running applications are within half of the memory,
both cores are running slowly and the ethernet interface is disconnected. One
memory module, one core and a network interface could be powered off without
perceivable performance degradation. Also, as much data could be cached in the
remaining memory so the disk could be powered down too or, and that would be
interesting, spun at a lower speed.

Lots of interesting ideas in one lazy saturday afternoon. That's what I like
in HN. :-)

------
apower
It's surprising that the CTO of a company specialized on system performance
doesn't know how OS cache memory work.

~~~
jrockway
It's not like you can read the Windows source code to see what it's doing, so
I am going to give him a pass.

~~~
InclinedPlane
He could read the available documentation and literature. This is not some
grand secret. This feature has been around in Vista for years, there's no
excuse for this level of ignorance. It's just laziness.

[http://en.wikipedia.org/wiki/Technical_features_new_to_Windo...](http://en.wikipedia.org/wiki/Technical_features_new_to_Windows_Vista#System_performance)

Better yet, the same article circa Feb. 2007:
[http://en.wikipedia.org/w/index.php?title=Technical_features...](http://en.wikipedia.org/w/index.php?title=Technical_features_new_to_Windows_Vista&oldid=110770917#System_performance)

~~~
jrockway
Documentation is never accurate. No source code, no knowing-for-sure.

~~~
Locke1689
Any operating system developer who knows anything about paging would know
exactly what was going on. Especially since you can immediately tell using the
Windows task manager that it was doing caching.

------
dirtbox
I think several people nailed this when the story broke, it really was a
pretty superficial observation for such an apparently broad analysis.

Slightly off topic, but it just struck me that I'd like to see app icons in
the task manager and windowed programs highlighted in some way. The unabridged
list of processes is a little too lengthy if you're looking to quickly nip
something in the bud. I know there are task manager replacements, but to me
they're not as intuitive.

One for the Win8 suggestion box maybe.

------
spudlyo
I wonder if Windows 7 behaves like Linux in that by default, when faced with
memory pressure, it will much prefer to swap out application pages rather than
abandon buffer cache.

This is really annoying when this memory pressure is due to trying to fit more
(and potentially useless) things into the buffer cache. Let's say you're
performing a backup of a 100+ GB directory to another machine using rsync.
Linux will want to jam that entire directory into buffer cache, which of
course won't fit. Instead of immediately pitching out the existing disk cache,
it will aggressively (depending on the value of swappiness) swap out
application pages, (potentially owned by Oracle or MySQL) rather than lose
existing buffer cache.

This is a real problem, and one that's a pain to fix.

~~~
nkurz
Could you clarify how the problem is hard to fix under Linux? If you set
swappiness to 0, does it still swap out application pages?

~~~
spudlyo
Yes. Even with swappiness at zero, Linux will still swap when you don't want
it to.

------
zokier
Why do people even care about what computer uses its memory for. What I care
is performance, as measured by wall clock, and energy consumption.

