
This is why memory fragmentation is bad - saulrh
https://bugzilla.mozilla.org/show_bug.cgi?id=666058#c31
======
spitfire
The sad state of affairs is that papercuts are the death of us all. We've come
to the point where we have "unlimited" resources that we just throw away
memory like it's going out of style.

I remember a profiling of a gnome startup. It touched something like insane
like 40K files. Another profile I saw removed a few bytes (unneeded, it wasn't
some complex optimization) from some commonly used datas-tructure for an 50%+
speed improvement.

Simply we're being cut to death by bloat and pure, sheer lazyness. a web
browser has absolutely no excuse for using 500megs of ram. None. Whatsoever.

I expect this comment to fall on deaf ears.

~~~
wheels
Your comment is wrong on so many levels:

• It's not even on topic. This post is not about general memory usage; it's
about fragmentation in the javascript VM – i.e. that a system object existing
in a general memory page precludes that page from being garbage collected /
deallocated even though the lifetime of system objects is very different. This
is not a trivial matter for which the only explanation is laziness.

• It's spoken like someone who's never seriously worked on a large open source
project. Memory profiling and leak tracking is done widely and often (using
valgrind, mostly).

• You speak as if you know how much memory a web browser should use. How do
you know? Have you worked on modern web browsers? Or are you just, as I
assume, by fiat deciding that 500 MB is too much?

• There's very often a speed / memory usage tradeoff. At present, especially
for web browsers, users tend to prefer speed. (i.e. how many rendered pages do
you cache per tab so that clicking the back button is next to instant?)

• The right time to profile and optimize is usually later in the development
process. We all know the famous quote. Optimizing early tends to lead to ugly
code that, amusingly, is harder to optimize later.

~~~
z92
I remember sometime back probably around 2004. There was a lot of noise
regarding memory leak in FF. And the expert sounding ones started to claim
that there was no memory leak and the very large memory usage in FF was
because of browser history and cache storage. Then one year later they all
accepted that FF was actually leaking memory.

Since then I stopped accepting expert opinion blindly.

~~~
clobber
I could be wrong, but wasn't a large part of this due to Flash?

~~~
starwed
I think it turned out that a large part of it was due to, like, 100 different
small things. There was never _a_ memory leak.

------
saulrh
Summary: Firefox has a bunch of long-living system javascript objects; these
get fragmented in the javascript engine's heap and end up keeping alive a
bunch of mostly-empty chunks of memory. Tests show that removing this
fragmentation reduces Firefox's overall memory use by up to 35%.

------
fomojola
Remarkable: "No we don't free chunks right away (except with the new SHRINK GC
call). They survive 3 GCs because of allocation heavy workloads. Freeing and
allocating right away hurts benchmark performance (bug 541140)."

So they're basically saying one of the reasons a copy of Firefox with no open
tabs is currently consuming 1GB of my computer's memory is because they didn't
want to suck on benchmarks. Is there a flag I can set to tell the damn browser
"I AM A REAL USER AND NOT A BENCHMARK, PLEASE DON'T SIT ON 1GB OF MEMORY, FREE
IT IMMEDIATELY"?

~~~
jrockway
I have 6G of core and 4G of swap. If using more memory makes browsing faster,
I'm all for it. Memory is cheap. CPU time is expensive.

For reference: 24G of DDR3 costs about $220 these days.

~~~
rwg
All of the physical RAM in the world doesn't mean a damned thing if you're a
Win32 process limited to 2 GiB of user address space. Heap fragmentation is a
very real problem there.

~~~
JoshTriplett
So don't run win32. :) We've had 64-bit processors for a while now, run a
64-bit OS and you have all the address space you might want.

~~~
silon
LOL. Yesterday, I restarted firefox because it ate 13GB of virtual memory and
started to get really unusable.

------
iam
What's actually surprising is that it took them this many years to get it
fixed. Memory fragmentation is just as bad as memory leaks. To the user, the
RAM being used just keeps going up and up until they run out of memory and
have to restart FF.

~~~
ootachi
See my comment below; Mozilla has been well aware of memory fragmentation for
years and has been constantly working to decrease it.

~~~
iam
I remember that blog post like it was yesterday. I guess the Mozilla source
code is such a gargantuan beast at this point that it really does take years
to get significant improvements into it?

Could explain their recent flurries of rewriting the javascript jits.

~~~
ootachi
Yes, Mozilla source code is huge (5.2 million lines [1]). But keep in mind
that all browsers are roughly that big (compare Chrome, at 4.8 million lines
[2]).

[1]: <https://www.ohloh.net/p/firefox/analyses/latest> [2]:
<http://www.ohloh.net/p/chrome/analyses/latest>

~~~
nikcub
this is the #1 reason why I sought out a Firefox alternative a long time ago.
I like to have a lot of tabs open, and to almost never close or restart the
browser. Firefox was, and to an extent still is, unusable with that type of
usage pattern.

I hope they get it sorted out because it is the main blocker to getting a lot
of the early adopter crowd who have switched to chrome back onto Firefox.

~~~
saulrh
Maybe you have a different use case than I do; my Chrome instance usually uses
about 30% more memory than Firefox despite having a quarter as many tabs open.

~~~
nikcub
OS X? I have 50+ tabs open atm and it is taking up 690MB in total. Chrome is
also good with swaping/paging tabs that are in the background.

Also, I have flash disabled, and use 3 separate builds of chrome on OS X to
separate web accounts, dev instances and anonymous browsing

~~~
saulrh
Win7. Between eight and ten tabs open, one open to Grooveshark. Everything
gets refreshed every eight hours or so. Firefox usually has anywhere between
five and a hundred tabs open. Both get rebooted about once every three days
for updates. Chrome is at 400MB, FF is currently at 300.

~~~
nikcub
my own experience was that chrome on windows is the best/fastest browser
experience available atm, but I don't use it often. have you tried chrome
canary or webkit/safari? you seem to have a weird problem - it might be down
to a plugin or extension, so I would disable them all and re-enable one-by-
one.

~~~
saulrh
I'm running Canary. I think it's mostly thanks to Flash, really; I've never
had good luck with Adobe software and Flash is the worst of the worst.

~~~
nikcub
chrome://plugins/ - kill flash, enable html5 on youtube, and you will never
miss it. been 18 months for me, and I very very rarely find that I need to
open a flash-enabled browser

------
Impossible
Glad to see they are finally doing something to address memory fragmentation
besides suggesting that users "occasionally restart Firefox". Seems like this
has been a major problem in Firefox forever.

~~~
ootachi
Mozilla has been well aware of this problem for years. See this blog post from
2008, 3 years ago, detailing the changes that went into optimizing memory use
for Firefox 3, including switching to jemalloc to decrease fragmentation:
<http://blog.pavlov.net/2008/03/11/firefox-3-memory-usage/>

------
KobyB
We measure fragmentation level of disks is a unique way, that allows a central
fragmentation control instead of periodical defrag. The idea can be converted
to memory fragmentation - if anybody wants to join us for developing a similar
tool for cases described here.. Koby <http://www.disklace.com>

------
wakemup
'No we don't free chunks right away (except with the new SHRINK GC call). They
survive 3 GCs because of allocation heavy workloads. Freeing and allocating
right away hurts benchmark performance'

This is the real problem then - benchmark score vs real-world useabilty.

------
clobber
He mentioned wanting to get that patch into FF7. Why not FF6?

~~~
saulrh
<https://wiki.mozilla.org/Releases#Firefox_6>

FF6 has already been frozen and released to the Aurora channel; it goes to
Beta tomorrow. FF7's code gets frozen and released to Aurora tomorrow.

~~~
linker3000
By the time we get to FF412 in a month or so I expect it will be awesome and
actually ordering me free RAM from Crucial

