Back then, I thought the conclusion was that there is nothing broken about OS X memory management, and that with every 'fix' you come up with, you will just introduce another degenerate corner case. The same holds for any OS, trade-offs are made that may have some negative effect in some cases, to the benefit of the general cases.
I don't recognize any of his symptoms anyway, and my OS X computers get pretty RAM-heavy use, with almost always a linux VM open, XCode, Safari with ~10 tabs, iTunes with a few thousand songs, etc.
Just to be sure I read through some of the links he provides that are supposed to explain what is going on and why the fix would be of any help, but nowhere do I see any hard facts that demonstrate what is going on. Only that he 'saw in vm_stat that OS X was swapping out used memory for unused memory'. I'd like to see some actual evidence supporting this statement.
I have two Macs, one brand new, one migrated from SL, and 10.7 Safari was almost unusable on both until I installed SSDs. If that isn't a negative effect in every possible use case, then I don't know one. I actually guessed it was just that Lion inofficially dropped support for HDDs (by removing all caches or so).
Given that Apple has fixed none of my reported bugs in 10.7, but I can't reproduce many of them in 10.8, I wonder if it even makes sense to analyze 10.7 anymore - seems it's a done deal for Apple.
It's like the system discards pages of programs just because the app has been inactive for an hour or so. So when I come back and start the same app the f*cking rotating HD I have sounds like a birds nest for too long periods.
Edit: Disabled the pager and the system now seems much more quiet regarding to disk seek noise when I start apps. Feels like a new machine! :-D
From what I've read, Windows memory manager does the same thing - after a while, it swaps out unused pages, even if plenty of free memory is available.
I wonder what's the logic behind this - did the engineers assume that the speedup coming from more free memory being available for disk cache is worth the hassle of waiting for the swapped out page (when it's actually needed)?
In Lion I get the impression they are just swapped out and thrown away / reused for something else despite there are no real pressure on the VM.
And yes, the logic is sound, its better to use a bit of swap for an infrequent daemon and let 4-5 megs of memory be at the ready if needed than leave it in place all the time. The "speedup" is not a speedup for your use, its to allow for better memory management. Which is what the VM subsystem is there for. Second guessing it all the time just makes its job harder.
Swap use when there is free memory isn't a bad thing. This fetish people have with their OS using swap at times seems to border on the ridiculous side. My iMac at home has 16g of memory and 400g of swap used right now (8g active, lots of file cache that'll get purged). Most of the swapped files belong to things like my ruby+pry repl, a clojure repl I haven't touched for 2 days, and other random things I don't use often enough to warrant they stay in active ram. Why SHOULDN'T that memory be reclaimed and at the ready for a new program or some other request? Its just going to page it out then and likely take longer to do. The only time its "wrong" is when I start using those processes again, which takes all of 1-2 seconds.
Its a hard problem, and both OSX/Windows choose the best possible solution you can heuristically.
It is, actually. I find such things unacceptable, be they on desktop or server use cases. I put as much or far more RAM in my systems than they will need, and I expect nothing to be swapped until it's actually full. Many other people do as well, which is why the Linux kernel devs finally started fixing the stupidity several years ago. Time for OS X to catch up.
Windows XP does that. It was a common source of grief. I remember it being mentioned as early as 2004. Since Windows Vista the memory manager doesn't have that problem.
YES. I have a MacBook Pro Core i7 from a little while back with an old style spinning rust drive and a 11" MacBook Air Core 2 Duo.
For purely CPU bound things, sure, the Core i7 kicks the pants out of the Core 2. Same for videogames. For day to day use, though, switching between Eclipse, Xcode, Chrome, etc. the Air provides a much more uniform experience. At its best it's far slower than the Pro at its best, but at its worst it's much faster and more responsive. I rarely see beachballs on the Air. I used to see them all the time on the Pro (the Pro has been sitting on a shelf for the past eight months as I switched to working exclusively on my Air, partially for this reason).
So my experience is that something may not be broken, but something definitely isn't set up optimally for users with poor disk performance and high memory/CPU performance.
Left some big files on the old HD, and symlinked them. The disk stays idle in the CD bay until I need it, then spins up.
If I were to do it over again I'd just get a larger SSD and leave the optical drive alone.
It also turns out I rather like working on an 11" screen. Keeps me focussed.
I haven't tried ML yet. My MBP is brand new (bought in January 2012), factory configuration (4GB RAM, Lion)
If you use it 'lightly' (that is, only Safari open) it's a breeze. But of course, it's never only that if you want to do any work.
Frankly, 4 GB should be enough! My past machine (with 3GB - and Linux) would rarely swap (in fact I could keep swap off and use a Windows 7 VM) But you can go only so far with an aging CPU
Some of the slowness can be attributed to Safari/Firefox, sure
But it really seems to have something wrong. Maybe they really neglected people with spinning disks.
(Yes, I considered buying a MacBook Air but 128Gb was not enough for me and the other options were above my budget)
I see this sentiment a lot, but I disagree with it. What are "iTunes, Xcode, Safari/Chrome, and a mail client" doing now that they weren't doing four years ago? Is it enough to justify their latest versions feeling less responsive than their versions from four years ago?
iTunes: layers of software for dealing with Wi-Fi sync, Ping social network, iTunes Match, and so on
Mail Client: totally agree with you there...
But I still pretty much agree with you overall...
Things change. My old 2006 iMac core 2 duo feels a bit clunky sometimes these days, but it runs a lot of stuff fine and is actually just as good a machine as it ever was.
Lot's of things. XCode was rewritten and does live AST syntax completion, background compiles, etc.
Safari/Chrome have several more features --did Chrome even exist 4 years ago?
Expectations are certainly a big part of the perceptual speed equation. But with OS X, don't underestimate the benefits of keeping your disk less than 90% full. With all the caches, iPhone and iPad backups (over 40 GB in my case), Xcode, sleepimages and swapfiles, installers (Adobe!), SyncServices, etc., a 160GB SSD fills up in no time. When things get slow, getting back below 90% works wonders.
Currently at 145 pages and growing https://discussions.apple.com/thread/3191630?start=2160&...
Yeah, the 2009 iMac started working the day I put Snow Leopard back on it. I then sold it and warned the owner that upgrading to Lion was at his own risk. I replaced my router with an Airport Extreme; useful excuse to buy a new toy; at one point to see if that resolved it. I even moved the iMac NEXT to the router one day.
We have been through this problem again and again and again, in different OSes, at different times and with different things triggering the various problems.
It usually ends with a "neck-beard" saying with enough authority "look, really, they are doing it right even if it seems totally illogical to you and any brokenness is just your configuration, little man". Which is to say "you might not like senseless disk-thrashing but would you rather have your machine randomly freeze when it got out of memory?" And scratching a little more, it comes down to admitting that memory-allocation is a hard problem in its full generality and they don't teach you that in application-programmer-school, and further that the solutions to it that any of these OSes have are tuned-black-magic-split-the-difference-haphazard affairs.
Consider. Either the machine keeps all your information in memory it or keeps on-disk and in-memory and either way, the machine hasn't a clue what information is important to you, my friend. It's just data to it. It's not like the computer is intelligent or anything. Why do you think they call it "random-access-memory"? The problem of dividing up chunks of memory for application programs to use is as hard as dividing up that hard disk for large and small files to live in, EXCEPT that application programs expect to be handed a chunk of contiguous memory when they call malloc. Hard problem even with the powerful tools that have evolved for solving it over the years. So when a given memory management scheme works, it isn't really "fixed", it just has been tuned for the corner-cases that are shouting loudest on the help lines.
And yes indeed, it is "funny" how just getting the "simple stuff" to work is a hard problem. IE you can find lots of simple examples where the standard solution seems fail terribly.
Angry how your 100 GB memory machine isn't faster? Look under the hood and you'll find Scotty from Star Trek shouting "Captain, I'm allocating your memory as fast as I can Sir..."
I think you missed a key ingredient to this problem, which is heavy disk reads caused by either spotlight or time machine.
Reading a massive amount of data (that you'll probably not use again anytime soon) has the unfortunate side effect of polluting the disk cache with junk. Now if OSX is anything like Linux in this regard, it is loathe to toss out old disk cache (in response to all the incoming junk it's being asked to cache) and will instead start swapping to free up more memory for disk cache.
Linux has /proc/sys/vm/swappiness to control how aggressive it will be in swapping stuff out to preserve precious buffer cache, but I don't think OSX has any such mechanism.
Oh boy I wish I could say the same. Admittedly I don't shut down on a daily basis but this didn't used to be a problem in SL. FWIW I've an old tank of a tower that has a video card on its last leg.. shutting down invariably leads to ~30 minutes of downtime while the card heats up and reconnects whatever needs reconnecting for both monitors to work.
Since Lion, I've noticed frequent hangs and beach-balls when doing even menial tasks. Transmit, terminal, texmate, a few tabs in chrome. If time machine starts backing up I can forget about a smooth Preview open or switching to a largish open textmate file without beach-ball'ing. If I want to use a Win7 Parallels VM--I can't do anything else. Even now as I type this I have a ubuntu vm running at the login screent and it causes the machine to shake off the cobwebs between almost everything.
It's certainly not a bad machine--there's 8gb ram, and tons of diskspace--good processor. In fact, I would go multiple months without a reboot and heavy use when on SL without hardly any problems at all.
Then there is the new i5 MBPro. Cool trick you can do: hook up an external monitor via thunderbolt and watch as the [left side] dock becomes a mangled mess with icons miss-positioned and wrongly triggering apps--it's like playing a game of whack-a-mole trying to open terminal to kill -KILL dock :)
The i5 has also been less than stellar compared to the older MBPro I sold to buy it in terms of performance.
Though I did improve matters drastically by telling mds not to index my (Linux) MP3 server and my Time Machine drive. That made it go from nearly unusable to just frequently annoying.
The other crazy thing I've been seeing is it routinely takes Chrome minutes to shutdown -- in fact, pretty much every time I try to reboot my MBP without shutting down Chrome first, the shutdown process times out trying to exit Chrome.
I too have experienced the Chrome issue enough that I don't even try to close it normally anymore. Force quit is the only way I exit Chrome. Thankfully, the restore tabs functionality works well.
That would be the job of the memory manager to decide when to do that. Memory could be kept in the non-free state for longer that it actually is needed, but still be marked internally to be available when needed.
OTOH, if you have paging, as you say, then something is wrong, true.
But I don't think that the screenshot shows something wrong.
I read through the comment of a few weeks ago, and I did not see anything conclusive or even anything that would outweigh my subjective impressions that something about Lion on my (stock) 2011 Mac mini is causing unnecessary lack of responsiveness.
You might be uncomfortable with subjective impressions and many pieces of weak evidence, but given the popularity of OS X on this site, not all of us want to wait for what you refer to as "hard facts" before engaging in a discussion of the issue.
For example: Let's take a large Chrome session (~150 tabs spread over several windows), an IDE open somewhere, Spotify, Steam and some background apps, and a small Windows VM.
Generally, Activity Monitor would show that Chrome in this instance would be eating 2-3GB of RAM, the VM would be eating 1GB + change of unpageable ('wired') memory, the random utils & spotify & steam & ide du jour & crap would eat another 1.5GB or so, and, long story short, there's very little 'free' memory left (think <100MB) on this 8GB system but a good >1GB of 'Inactive' memory.
Everyone agrees that Inactive memory should be freed when more memory is required by the system and we're out of Free. My own limited testing shows instead that opening more tabs during normal use to the point where the Free memory is consumed instead results in massive delays across the system as memory is paged out to disk, and the Inactive memory doesn't seem to noticeably change in size.
I don't really know what it's doing, but it will consistently make the system very unpleasant to use for a good 30 seconds until the hard disk stops clicking. (It's unpleasant enough that I try to limit my browser session sizes now, and only run the VM when I need to.)
You don't have to fill up every last bit of RAM before the OS starts swapping, as there could be pinned memory pages or processes that want to do larger allocations that are only held for a short time. If you have less than a few hundred MB of unused RAM with all the stuff you mentioned going on, it only takes some kind of scheduled OS background job to push the OS over the line where it decides it needs to swap in/out.
That said, from the comments some people posted here, it does appear that at least in some situations there seems to be something going on in some versions of OS X Lion. If Snow Leopard and the Mountain Lion preview are unaffected with the exact same usage pattern, maybe there actually is some kind of bug in the OS X memory management. But I'd still like to see some kind of evidence, facts or statistics, as I have never experienced anything like it myself, not even on my MacBook when it still had only 2 GB of RAM.
150 tabs? A VM? An IDE?
That's just A LOT. Of course things will go south, what did you expect a magic machine that can run everything and whistle away hapilly with 0% load?
edit: Just for the record. I'm not doing some kind of heavy processing work. Most of the time I've got one Chrome window open with some tabs, email client, macvim and iTerm2. It's not like I'm doing some heavy work. I'm not even running a VM.
I thought it was pretty clear that this isn't a "fix", but there's definitely something wrong here.
People say "Get and SSD", well, I've had 2 SSDs and 4 SSD failures (one drive failed three times, the other once and its replacement is still going.)
So, I'm all spinning rust here. 1.5 Terabytes of rust in my Macbook Pro and the only time I have a beach ball is trying to launch Team Fortress (but I blame valve for that).
I have massive, MASSIVE Final Cut and Aperture libraries. I leave the machine up for weeks. I leave Time Machine running all the time- there isn't even a slowdown when time machine is backing up.
My hard drives are encrypted with full disk encryption which means not only am I running spinning rust but its encrypted rust which means every read has to be decrypted.
No slowdowns or beach balls. Sure the occasional poorly written program will have a beach ball, and rendering video takes awhile, but that's to be expected.
Yet people constantly say that Lion sucks? Really? And they have these more beefy machines with more RAM?
Something doesn't add up here.
Came with Leopard, upgraded to Snow Leopard (not a fresh install) and them app store upgrade to Lion
machine is snappy as, unlike my coworker who has a brand new quad core i7 w/ 256GB SSD that runs Lion like a dog. no idea why, but my humble old macbook is faster in every way than his shiny new mac mini