
Anatomy of a Program in Memory - signa11
http://duartes.org/gustavo/blog/post/anatomy-of-a-program-in-memory
======
brl
Submit more articles like this one and less Web 2.0 gossip please.

------
yan
According to the comments on that article, the people _still_ don't understand
the basics of virtual memory.

"Great article! / Quite shocked to know that windows takes double the kernel
memory compared to Linux."

"I’m wondering, though, why does the kernel space consume 1gb? That seems like
a lot.."

~~~
a-priori
As narag says, it's only recently that this has become an issue. When physical
memory size was orders of magnitude smaller than than the address space, it
didn't matter that the kernel carved out 1GB of address space for itself. User
applications still had the other 3GB, which was far more than any system had
physical memory.

It's only during this limbo period when 4GB of RAM is relatively cheap, yet
there are still 32-bit systems kicking around, that it becomes an issue. On a
64-bit system with a 16 exabyte address space, 1GB is a drop in the bucket.
Once again, it won't matter.

(I suspect Linux allocates more than this for itself in 64-bit systems, but I
don't know the details. At any rate, out of 16EB, even 100GB would hardly be
noticed.)

~~~
yan
From the way I read those comments, I gathered that the comments implied that
Windows "consumed" two gigs of ram, where as Linux took 1 gig. I thought there
was a dissonance between how people understood "virtual memory," which is
essentially a mapping function, to "memory," which people assume to be the
physical ram.

~~~
a-priori
It's easy to make that mistake at first, but it is a very important
distinction. Most of the time, for most processes, most of that space is
unallocated. However, since it's basically impossible to change the size of
the kernel area without relinking the process binaries, and since that area is
shared between all processes, the size of that area has to remain fixed.

It is interesting that Linux and NT differ. Perhaps this is due to the fact
that NT needs more kernel memory to house, e.g. the windowing system, whereas
Linux systems put X in userspace? I'm speculating here.

Something I never considered: that design difference probably makes Wine's job
easier, because between the 2GB and 3GB marks is a region of memory the
Windows process assumes is inaccessible, yet in Linux is in userspace. They
could (and probably do) load the Wine libraries here without interfering with
the Windows process. It also means that it would be very difficult to
implement Wine's complement, a way of running Linux binaries on Windows.

~~~
kragen
Linux used to be 2GiB/2GiB too, just like NT. It changed because people with
big processes were unhappy; for a while there was an unofficial kernel patch
for the 3GiB/1GiB split. Maybe NT didn't change because not as many people
were running supercomputers on NT, or because changing NT is harder.

You don't have to relink your userland binaries to do this because they don't
contain virtual addresses of kernel data structures, only their own data
structures. Expanding the space available for _mmap_ won't force those data
structures to move.

You can pretty much load Wine libraries wherever you want; it's pretty rare
for a program to break if memory it assumed was unmapped gets used for a
library, and programs that break that way will have a lot of trouble when
Windows versions change too. Generally programs are only linked with knowledge
of where they themselves will be loaded in virtual memory. (Linux ELF shared
libraries don't even know that.) All the other addresses are given to the
program at startup. It is unusual for Linux binaries to believe that they will
be loaded above 2GiB; the usual start address is 0x08048000.

Here's the memory map of a process running under Wine:
<http://gist.github.com/53853> \--- note that there's a lot of Wine and X11
stuff loaded below 2GiB (0x80000000).

------
lbrandy
Totally worth the price of admission. Every linux dev, especially c/c++
programmers, should be required to read this post. Once a year.

------
acangiano
That blog is a gold mine. Subscribed.

------
moxy
I'd like to understand what's being described in this article, but
unfortunately I lack the necessary educational background. Does anyone have
any links to other resources that will allow me to better understand what is
being discussed in this post?

~~~
wmf
Virtual memory is part of the fields of computer architecture and operating
systems; popular textbooks are _Computer Architecture: A Quantitative
Approach_ and _Modern Operating Systems_.

<http://books.google.com/books?id=pqYl3SWkA64C>
<http://books.google.com/books?id=tqtZGQAACAAJ>

------
jrockway
> This helps prevent pointer bugs, though not as effectively as avoiding C in
> the first place.

I hate to sound like the average Reddit poster and write something like
"quoted for truth", but... quoted for truth.

~~~
jwilliams
I found this the strangest and most out-of-place statement in the whole
article.

------
there
if you're interested in the security aspects of this stuff, there are a number
of presentations by openbsd developers on the openbsd site
(<http://openbsd.rt.fm/papers/>):

<http://openbsd.rt.fm/papers/nycbsdcon08-pie/>

<http://openbsd.rt.fm/papers/ven05-deraadt/index.html>

