For a few months now I saw a huge improvement on Linux regarding the memory management of Firefox. Previously I had to run Firefox in a separate cgroup to limit its memory usage, because it could easily deplete my whole RAM. And if I did close most of my tabs it did not release the memory back to the system. Now I never actually get to the limit I've set before and also with Auto Tab Discard extension it is well managed. So kudos to the team for such improvements.
In a nutshell we're directing the OOM killer towards less interesting processes within Firefox so that when the system is low on memory they'll get killed first. This not only makes it more stable overall but it plays nicer with other applications too. Web pages that leak memory in the background are particularly likely to be killed by this mechanism and that alone is a huge improvement in overall stability.
I was very excited about this improvement (it was covered on phoronix IIRC), but my systems still ends up thrashing when Firefox grows too big, unfortunately.
I usually keep an eye on the memory usage I have displayed in the taskbar, and have sysrq activated in case. I tried multiple things, including avoiding swapping to disk (only using zram), and Zen kernels. I'll have to see if I use MGLRU.
I haven't set that up yet, though I've watched the recemt developments with interest. Another option would be to launch everything (or just Firefox) in cgroups.
Does this OOM killer direction respect cgroup memory limits? Back when I was using Firefox on a low memory system, I ran it in a limited cgroup, but features like `browser.tabs.unloadOnLowMemory` wouldn't unload tabs on low cgroup memory, but only on low total system memory.
Yes it does, we've disabled tab unloading on Linux because low-memory detection just didn't work. We will re-enabled it sometimes down the line but only to avoid swap, not crashes. The OOM killer does already a good job with this change and we feel no need to do anything further (besides optimizing memory usage, but that's system-agnostic).
I had abandoned Firefox entirely on MacOS until sometime I decided to try again and it was no longer attempting to claim an entire 32GB memory for itself.
So, I am back as a happy user :)
> and also with Auto Tab Discard extension it is well managed
Have you compared it with the default Hibernation (Tab Unloading) recently? I don't use any extensions apart from UBlock Origin since I lost trust with extensions after seeing a recommended FF extension promoting scam.
When compared to Vivaldi's default hiberation(Where I have to manually trigger bg hibernation every now and then), FF's hibernation seems to do its thing (about:unload) quite well.
Firefox stability is funny ... I was at Mozilla for 10+ years and used Nightly as my daily driver on my work Mac. I don't think I got more than one or two dozen crashes in total. A crash was a special occasion and I would walk to someone's desk to show an explore. It barely every happened. On Nightly. So much love for all the stability work that is happening.
In personal use Chrome and Firefox are both pretty stable for me. But I also write and maintain intranet web sites, and the fleet of devices that use them. In that fleet without a doubt Chrome's auto updates have broken more things more regularly than Firefox. Actually, I can't recall Mozilla releasing a version of Firefox stable with a bug that effected us. Chrome has had many buggy releases that has sent us scrambling.
I've also never seen the memory issues described here. But then I don't leave a lot of tabs open: it's rare my tabs don't fit in the title bar. I think that pattern is true for most people: almost all our fleet had very few tabs open. So perhaps the fix, while welcome, doesn't effect most people.
Personal experiences are funny :) I've been using Firefox on Windows, Linux and Mac for as long as I can remember (first version was maybe Firefox 3 I think? The version that introduced being able to resume downloads I think), and it's the Mac version has always been the most unstable and worst performing one for me, while Linux one being the most stable and performant. I can't even remember the last time the Linux version crashed, while I had the Mac version crash last week. Windows one seems stable, but not as performant as the Linux version.
The restart message is due to the interaction of Linux package updaters and Firefox's process spawning. You can avoid it by using Mozilla's tar.gz, the Snap or Flatpack builds.
Comparing to games isn't particularly great since games are ultimately low stakes and companies have learnt that gamers will tolerate pretty much any bug that doesn't literally stop the game from working. See: Bethesda games.
I still remember a few years ago, I had a buggy network driver, that on rare occasion, entered in a rapid memorky leak, eating all my ram, most programs crashed, except firefox, GG.
macOS has awesome memory compression and overcommit -- it's no wonder Firefox never crashed for you. I can't speak for my own experience because I rarely ever used Firefox on my Mac, but the TL;DR is that even if an application has a pretty egregious memory leak, you will probably never even notice, unless that memory actually gets filled up with data.
I can't remember the last time Firefox crashed and I've used it daily on Windows since ... the beginning. Are most issues related to stability more common to Linux/MacOS?
My current instability isn't crashes but with firefox is that it will just randomly "lock up". It will still respond to a click, but will take 10's to 20's seconds to respond. I have to kill and restart firefox to get it working again, but on startup it will usually take on the order of a minute or two to start actually working.
Also, more sites just don't care about having their stuff work in firefox. I use chromium for roll20, because there is just a lot of things that are just broken on firefox.
Disclaimer: I've basically never used roll20 because it's way too heavyweight for my games; I use Owlbear Rodeo instead. But I have tech-savvy DM friends who use and like Foundry
I use Foundry VTT regularly (and develop a plugin for it). While I've never had functionality problems with Firefox, in practice, it's the main thing I keep a Chromium install around for, since on heavy scenes even Chromium barely holds up the frame rate on my laptop, and Firefox is notably worse. This is running on integrated graphics though, it's possible Firefox is fine with a more gaming-friendly setup.
Both of those look quite interesting for my usecases (In person/occasional hybrid games with a vtt/online only - depends on my groups current location and schedule)
I see the exact same thing. It’s frustrating because CPU, disk, memory usage, etc are all super low. The machine is idle and there I sit waiting for the first web page to load.
I use it on a mac. The only time I restart the browser are to install a browser update (I'm no the Beta channel) or when my laptop restarts. Crashes are very rare for me.
Here's the thing: they're not! The reason those users where crashing was because something, somewhere (possibly in the graphics stack) was reserving ton of space without using it. We had crashes on file with 20+ GiB of free physical memory which was the reason why I started looking into why a machine with so much free memory could suffer a crash.
I hope I described how this whole things work because Windows memory management is not well known and some things about it are counter-intuitive; especially if you're coming from Linux.
I can confirm this is the case. Firefox reserves tens of gigabytes and then only actually uses a few. But since those tens of unused gigabytes are still committed, nothing else on the system can use it, and Firefox (+ tons of other apps) will OOM when the total committed memory (of all applications) reaches the total amount of RAM installed.
I have 17 crashes on file (in about:crashes) that are OOMs caused by Firefox committing too much memory and not using it. My solution is to restart the browser when committed memory reaches 38GB or so, but that clears all private windows which annoys me. I shouldn't have to restart the browser so often.
Could you open a bug on our tracker, and point to those crash reports? I'm very interested in analyzing real-world scenarios where this happens to figure out where those committed areas are coming from. Our best guess is that they come from the graphics stack because we account all the memory we commit in Firefox and there's nothing missing, so wherever this is coming from it's outside of our direct control.
From the looks of it you have a minuscule page file (16MiB), that's why you're running out of commit space. I suggest setting it to auto-resize (the Windows default) which should provide a much better experience even with other applications, unless you need to keep it small because of storage concerns. Generally speaking Windows behaves very poorly with small swap files because of its inability to overcommit memory.
Firefox should not be relying on overcommit in the first place. The issue isn't that I "only" have 40GB of commit space - it's that Firefox keeps taking more and more to the point of crashing (itself and everything else I have running).
And yes, my page file had grown to 64GiB before, when I stupidly left it on automatic. There's a reason it's at 16MiB now - it's because completely turning it off disables memory compression. Not like that helps much due to the lack of overcommit.
You're assuming that Firefox is eating all your commit space, but it's not, far from it, I can see it from the crash reports. You've got other applications running and they are also fighting for it, leaving large chunks unused. Without some swap space available Windows is leaving 10+ GiB of physical memory free which it cannot use for anything else.
This is how Windows behaves by design, there's nothing we can do about it. Windows without a swap file sucks.
Funny because I just switched to Chromium and left it on for around 36 hours and my commit space is not being eaten up. Coincidence? ;)
Before, when I killed Firefox (before it crashes naturally, of course), my committed memory would drop from 38GiB to around 18GiB, so uhh, unless some other app is intentionally playing with me and syncing up its memory leaks with Firefox's existence...
(Plus, Process Hacker confirmed that all of Firefox's "Private bytes" added up to about how much committed memory got freed when I killed it. Explain how that is caused by memory fragmentation!)
This is my third conversation with a Mozilla engineer about this! Always looking forward to be proven wrong, but it looks like you just can't seem to find the problem yet. I hope one day you can. :)
It seems unlikely given this exchange, as you seem to have decided to be as unhelpful as possible.
> Before, when I killed Firefox (before it crashes naturally, of course), my committed memory would drop from 38GiB to around 18GiB, so uhh, unless some other app is intentionally playing with me and syncing up its memory leaks with Firefox's existence...
From the article:
> However, we have no control over Windows system libraries and in particular graphics drivers. One thing we noticed is that graphics drivers commit memory to make room for textures in system memory.
Guess what's going to create textures which might trigger such an issue? The browser's hardware acceleration. Guess what happens to browser's textures when the browser is killed? They're freed.
Might be a bug in Firefox, might be a bug in the crash reporter, might be a bug in the driver, might be that chromium is not using hardware acceleration.
But here you have a mozilla engineer specifically working on crashes, and instead of trying to work with them (possibly off-board) and with the data they have to diagnose the issue — because I'd assume when gsvelto talks about commit space contents that's what they see in the crash reports - you're just being a snarky asshole.
> It seems unlikely given this exchange, as you seem to have decided to be as unhelpful as possible.
I'm not being unhelpful on purpose. They are just giving me answers that do not solve my problem. My problem is "Firefox requires overcommit to run for long periods of time ... because it has a memory leak". The solution to this problem is to figure out where the memory leak is coming from, and fix it, right? Not just to add more memory, even if it is only virtual memory.
> Guess what's going to create textures which might trigger such an issue? The browser's hardware acceleration. Guess what happens to browser's textures when the browser is killed? They're freed.
You just described a memory leak, yes, the exact issue that I am experiencing.
> Might be a bug in Firefox, might be a bug in the crash reporter, might be a bug in the driver, might be that chromium is not using hardware acceleration.
Well, I don't think the crash reporter or Chromium are at fault. And I know Chromium is using hardware acceleration because if I choose the d3d11 backend then Netflix goes black in screenshots due to DRM. So, if it is a graphics issue, it's something that Firefox does and Chromium does not. Which leaves Firefox bugs, or bugs in the driver that only Firefox triggers.
> But here you have a mozilla engineer specifically working on crashes, and instead of trying to work with them (possibly off-board) and with the data they have to diagnose the issue — because I'd assume when gsvelto talks about commit space contents that's what they see in the crash reports - you're just being a snarky asshole.
I'm sorry that I came off as a snarky asshole and that wasn't really my intention. I've been dealing with this issue for months, and as a result of that there is a lack of effort from me and that probably shows. I am sorry.
I would like to perform some more experiments to figure out what the issue is, but the problem is that when Firefox runs out of memory it also crashes half the other things on my system, which makes it dangerous.
For me, Firefox slows down dramatically as it uses more memory (e.g. more tabs and windows opened), and reaches a less-than-usable state well before it has exhausted the available memory. Tab discarding - automatic or by literally closing the tabs - does seem to recover the memory used, but does not recover much of the performance (if any). My AMD 5800 and tens of gigs of memory runs into the same crushing performance blockers as my old 8GB FX-8300 machine, with virtually the same "workload" and usage profile.
Kinda the opposite of what I'd want; I usually have over 16GB more that Firefox could use if it needed, and that's once it has reached critical mass with maybe hundreds of tabs.
Its memory measurements usually look sane, so I feel like there's some data structure or algorithm that is doing something insane in the background - which is already confirmed to be the case with the History menu, particularly if you select and delete thousands of items at a time.
You don't need a lot to eventually get a crash on Firefox. All you really need is to hibernate/sleep instead of restarting, and so never restart your Firefox, and have some Twitch stream opened - the memory leaks will eventually use up all your memory, in my experience at least.
Firefox has been solid for me since the dawn of time, and I have or currently run it on Windows, Mac, Linux, and FreeBSD.
For a while it used to be kind of slow on Mac and Linux, but I think that was slow graphics calls, which points to a possible issue with the graphics driver I was using. But the last time I checked (many months ago), it was much better.
I recall a particularly strange Firefox bug on Windows where the browser would die with an out of memory error even though there were upwards of 16 GB free to use. As it turns out something was consuming large amounts of swap on the system, and this was where the out of memory condition was happening.
I had to switch from FF to Brave on macOS because FF constantly kept over utilizing the CPU, leading to bad battery life and warmer MBP. This happened on Intel and M1 chips.
Over the last year FF has gotten a lot better about not keeping my MacBook awake because of content on some random background tab that the browser thinks is multimedia content.
Circa 2021 if I wanted my MacBook to go to sleep I had 5o shut down FF first.
I never had it crash on Linux. The only issue is some sites redirect you to a page saying “you must use chrome to proceed,” which is ridiculously lazy of them
This is also a good example of the benefit of telemetry: that they have crash numbers coming back from the field lets them tell that this really did work in practice and get a sense of how much of the problem they've solved.
opt-in telemetry is effectively the same as no telemetry.
if you have (or anyone has) a problem with crash statistics being tracked via telemetry then I have absolutely zero idea how to convince you that it's a good idea that this blog post doesn't already clearly state.
it's the same with OS updates; people (generally) simply will not perform system or security updates unless they are forced, because everyone thinks they are smarter than the "script kiddies" who would use an attack against them. the user thinks they would see an attack coming and avoid it. in short, they don't. viruses spread, the US Congress calls Microsoft in and asks why the systems weren't patched, and Microsoft says "the users are responsible for patching" and Congress doesn't like it.
so now we are where we are. OS updates are forced after a time, and telemetry is not only the norm, but a very good idea for applications in use by millions of people, like Firefox.
> everyone thinks they are smarter than the "script kiddies"
No. People do not perform system updates because a) it's a chore and b) it get's in the way or even breaks things. To do an update I need to agree to give up control over my device for some time (often undetermined) and then risk that updated code causes issues (it's not uncommon). We need to design apps and operating systems with seamless and reliable updates in mind, not force people to suffer.
Or maybe Microsoft finds it's a cheaper and easier to leverage their monopoly position if they operate in "perpetual beta" by rushing out new features before competition can get a foothold, then using their (paying) users as their testers. Rather than, say, testing and hardening their products sufficiently before launch.
> The tone you employ - "users" having "a problem", needing to be "convinced" and "forced" - doesn't help.
My tone here is borne out of users shooting themselves in the foot and then complaining about the pain and inability to walk. At every opportunity that I have taken to give computer users the choice to do the thing that is good for them, the overwhelming majority have failed to make that choice. The people who visit this site are mostly not that kind of person, though there are plenty here who are.
We have shown Microsoft that we simply will not update our operating system or even reboot unless forced. Many, many times we have made this clear, even though patching is overwhelmingly a net positive for both MS and its users, generally speaking, users simply won't do it. They just won't. History bears this out.
Updating is a short-term inconvenience in exchange for long-term security and stability, and people do not think about those things logically. The importance of the immediate future is amplified by a large factor, and the importance of the future is attenuated by a large factor, in most people, especially when it comes to people who view their computer as a tool. Sitting in front of a computer is indicative of a user wanting to complete a task, and manual updates impede the ability to perform that task. That makes installing updates and stopping work to reboot a non-starter for those people. They just won't do it.
I don't know how else to say it. It's not a matter of tone so much as it is a matter of fact.
Thanks for your thoughtful reply, I hope my additional remarks are
taken as sincere and not in any way personal, except in a good way.
> My tone here is borne out of users shooting themselves in the foot
Frustration. I hear you. Sure, it's frustrating that they exercise
choice (however misguided you see that) and then "complain". It's very
nice if you've written code, and even nicer that you care for your
customers. But, as developers, they aren't our children. I've been
there and it's galling, and feels like a rejection, but to accept what
is unrequited is sometimes harder than giving it.
> the choice to do the thing that is good for them,
This is the elitists' dilemma. Please don't be insulted by that word,
I'm using it literally and appropriately without value judgement (I am
foremost an elitist, and secondarily a peoples' champion, and it is a
position that can only ride on a measure of arrogance - which must be
tempered)
The fact is, it's not your computer. And that really is the long and
short. One must respect that if "users" do not wish to take advantage
of bug fixes more speedily available through telemetry, then it's
their choice to have suboptimal, buggy programs.
In other news, our children will listen to shit music and get into
drugs and relationships we disapprove of etc.
> We have shown Microsoft that we simply will not update our operating
system or even reboot unless forced.
For very good reasons. Microsoft have shown themselves to be utterly
untrustworthy. I really don't think that's even debatable. And it's a
shitshow because I do not believe trust can ever be repaired. It
leaves the reality that one of the biggest vendors on the planet is in
the position of forcing users because it has squandered the
reputation necessary to do good-faith business, to propagate its
updates. That's tragic because they probably see no way out except
doubling down on abuse, authoritarianism and beating users to their
will - and ultimately that confrontation will be the end of so much we
have built.
What makes this worse is that security is about more than personal
choice (think vaccinations). In other words the damage that Microsoft
(and other big-tech abusers) have done goes far beyond simply
destroying the individual trust relations with their
customers. They've corroded the social fabric of trust in computer
security at a more general level - a cost that is incalculable.
> people do not think about those things logically.
You are right. And we should not assume that they should. Emotion is a
powerful reasoning tool, and only a fool ignores that force of
psychology. Once burned twice shy - and we as developers have been
burning a lot of peoples' fingers these past 30 years.
> I don't know how else to say it. It's not a matter of tone so much
as it is a matter of fact.
I see it means a lot to you. That is a good thing in itself. You care,
which is x10 above the norm.
But we cannot force them to be what we wish them to be. Especially not
"for their own good" which is where all tyranny begins. We cannot
force people to adopt products, customs, behaviours, sing the party
line, or any of that hegemonic nonsense without invoking an age of
"consumer communism".
It saddens me that in 2022 we still need to address the patrician
attitude. It's not the way forward. It's sad to see such a
deterioration. But so long as companies like Microsoft persist a
culture of smug superiority, cavalier conceit and intransigent
disrespect to the dignity of their users we will have to accept "fuck
you" decision making. And frankly, more power to those courageous
enough to say it.
> it's the same with OS updates; people (generally) simply will not perform system or security updates unless they are forced, because everyone thinks they are smarter than the "script kiddies
Updates were not the default. And when they became almost mandatory Microsoft started bundling "features" with security updates.
That's when people started to disable this "feature".
> so now we are where we are. OS updates are forced after a time, and telemetry is not only the norm, but a very good idea for applications in use by millions of people, like Firefox.
And this doesn't change anything. Ransomware attacks are still the norm.
> That's an indication that people don't want this.
It's an indication that optional steps which do not immediately benefit the users of the software will not be taken.
The benefits to telemetry are longer-term, and because opting in is not required for the software to function, the vast majority of users simply will not do it.
The thought to turn it on will likely not even enter their mind. Why would it? The software works fine.
Opt-in telemetry was tried by just about everyone that collects telemetry today. Lots of people say they will turn it on, and then never do. Telemetry is used to make better software. I'm sure there are companies that use it for [insert activity that any person might perceive as bad] and I would argue that those companies would likely not allow you to opt out.
If people understood the kinds of things that are collected, at least the things I collect in the software I write for work, I can't imagine anyone having a problem with it, but there's a lot of things that people do which make no sense to me at all, so I'm not really in a position to be authoritative.
I do know what happened in the late 1990s and the early 2000s though, and I know those things are a large part of why telemetry and forced updates are things which exist today.
Opt-out telemetry opens the door to abuse and is invasive. We can see this happening today already, and that the current trend for opt-out telemetry + forced update is not a reaction to "opt-in is useless".
Regardless of how strongly you feel about telemetry and its perceived "benefits", the choice should always rest in the hands of the individuals using the software first and foremost. Your customers should always be informed of their choice and if they feel like they are willing to participate, they can opt-in.
How would you feel if building architects decided to install a camera in your bathroom in order to analyze how you use your toilet and the shower to assist in improving future constructions?
> Lots of people say they will turn it on, and then never do.
Because most of the time, there is no benefit. The software is already made, and further development is rarely informed by telemetry data. Furthermore, customers are rarely informed exactly what data is being collected - and not given the opportunity or benefit to inspect the contents of any telemetry or crash logs that need to be sent.
> If people understood the kinds of things that are collected, at least the things I collect in the software I write for work
But they don't. And no one takes the time to educate or inform them or give them the choice to opt-in with a detailed disclosure of what is being shared, rather than having to opt-out. As people become more aware of these telemetry practices, you're going to see a wider backlash at the kind of unnecessary data that are being collected. Maybe you're not doing it - but others are.
I believe that absolute vast majority of telemetry is simply ignored. And I also believe it is very common that there are several individuals in most companies that couldn't care less about what is included in the telemetry.
A lot has changed since the late 90s, it really isn't good argument for telemetry nor forced updates that things were bad then. Things would have been absolutely awful in the late 90s even if you had perfect telemetry and instant updates that somehow didn't even need internet.
I don't see any real arguments for either in your posts.
Agreed. There is a pop-up asking if you want to help at install(FF & TB). Big data has earned a reputation for just cause. If the Mozillas want to differentiate themselves to get that trust, aka opt-in, I encourage them. So far, "No, thanks".
There might be a correlation between people who push their memory consumption to the limit and people who never opt into anything they don't directly benefit from, thinking that such optional stuff would hinder them from using all of their memory for personal benefit. OTOH, those people might realize that telemetry will eventually increase efficiency, allowing them to do more with their available memory...
The only telemetry you get will be from users that are experiencing a severe problem who also have the technical know-how to turn it on. You will have no idea how prevalent the problem is amongst the general user base. There would be no way to prioritize debugging the issues.
I was under the impression that Firefox was written in Rust. Doesn't this eliminate crashes? Rust is a safe language after all. There should not be any crash logs with Rust.
Is this satire? Rust eliminates memory errors such as use-after-free. Unlike C and C++, `myVec[usize::MAX]` will `panic!` if `myVec` isn't actually that large.
Rust has a `panic!()` macro which will hard-terminate the program and log a stack trace in what is functionally equivalent to a crash. It can be called in various scenarios... including out-of-memory situations (like the ones being addressed in the fine article and this thread).
And even if Firefox was written in 100% Rust, Rust can still panic for logic errors. And system libraries have their own bugs and crashes that can bring down Firefox.
I would like Linux distributions to ship a system wide telemetry service that can be enabled / disabled at the installation time or anytime later on.
This service would be guaranteed to be unidirectional, would store data publicly on non-profit-run servers and domains and fully comply with GDPR (by not storing any PII and ano/pseudonymising everything).
Developers would connect to this service over dbus and consume the uploaded data in daily batches.
Hosting and hardware fees would come from donations by distributions and other organizations distributing money to the FLOSS ecosystem.
> I would like Linux distributions to ship a system wide telemetry service that can be enabled / disabled at the installation time or anytime later on.
There's nothing stopping a person from creating that. You'd package it up and get it added to the Debian, Ubuntu, RedHat, etc. repos and people would be able to install and use it. That's about as close as you'll get to having it generally available for all Linux distros.
Personally I don't see the value, and think it's invasive, so I would never install it, but people who wanted it would be able to use it.
I think what they meant is to configure proxying all possible telemetry via this service, and enforce anonymisation. Sounds good imho. I'm trying to disable telemetry but it's always a losing game, each version adds something new to each app.
The telemetry proxy service would need packages for each distro, including scripts to work with systemd and init, and maybe a "libtelemetry" package to make using the service easier.
The way I see it working is that if the system service isn't installed, isn't running, or has remote telemetry turned off then the commands for sending telemetry will succeed but send the data to /dev/null. Otherwise the data gets anonymized and uploaded to the user's configured telemetry host.
It could almost be built on syslog, now that I think about it more, but that would be terrible.
The thing I like about popcon is it's opt-in, and disabled by default.
I enable it on my personal production systems and disable on everything else, both on privacy grounds (work), and not providing wrong data (disposable VMs).
I do not think a centralized "system wide telemetry" service is a good idea. This has a huge potential for abuse and can be extended to collect other things.
Privacy should be privacy by default, and if you want users to send you crash/usage logs then you need to show them all of the dirty details, let them review it and chose whether or not to send.
Data also needs to not be shipped to a third-party (e.g. Google) to be correlated with other activity outside the app sending the telemetry. There's likely lots of data going to Google Analytics that the software/service owner never looks at, but Google uses for their own purposes.
Couldn't crash reports be separated from other telemetry data, possibly with a dialog letting the user whether to send a crash report or not? IIRC, the dialog used to actually exist in older Firefox versions. I find the amount of data they collect[1] to be borderline creepy.
The crash reports at https://crash-stats.mozilla.org are a separate opt in bit of telemetry which is a dialog that is shown when Firefox crashes. You can opt into automatically sending them by setting browser.crashReports.unsubmittedCheck.autoSubmit2 to true. It can be true if you opted into a dialog about submitting unsubmitted crash reports.
Usage times, usage intensity, list of all extensions, country of origin. I don't understand why they'd need those to improve Firefox.
Next thing you know they might try to increase engagement time like they're some sort of social network. "Unlock the new exclusive colorway by logging in 30 days in a row." seems like something that could be implemented, seeing how they're time limited already.
(I work for Mozilla, though far from where decisions about telemetry would be made.)
Usage times and intensity are of high value when trying to improve market share. People who barely use the browser are at high risk of stopping use altogether. (For example, they might use multiple browsers, but most of their activity occurs on another and if they figure out multiple profiles or something, they'll leave altogether.) You can't do an A/B test to see what improves usage intensity if you don't measure usage intensity. Also, it's far from PII. And making it opt-in would make the stats useless; people who explicitly choose to allow telemetry are going to have vastly different usage patterns than the bulk of people who do not so choose.
Extensions are very important for crash reports. Far less than they used to be; many crashes could only happen when an extension did something specific. Extensions are now sandboxed enough that this isn't nearly as common, but if a crash signature has a high correlation with a particular extension, it can easily turn a non-actionable bug into something actionable.
Extensions for general telemetry are iffier. The info is fairly high value for things like understanding how people are using the browser and what features are popular or missing. But rare extensions also provide a lot of fingerprinting info. It's important to keep those metrics away from PII, and recorded independently so they can't be correlated.
Country of origin is pretty clearly useful. Mozilla has to allocate resources across countries, including marketing resources, but I would think it's really product management where it matters most. Users gain a lot of benefit from the browser adapting to different markets. (Screenshots have a wildly different importance in countries with Asian writing systems; Europe and especially Germany take privacy much more seriously.)
> Next thing you know they might try to increase engagement time like they're some sort of social network. "Unlock the new exclusive colorway by logging in 30 days in a row." seems like something that could be implemented, seeing how they're time limited already.
Heh. I do not want to predict what our marketing people will or won't do. I have mixed feelings about quite a few things. I'm not happy about ads appearing anywhere in the interface. But I'm also not happy about being dependent on Google ad money.
> And making it opt-in would make the stats useless; people who explicitly choose to allow telemetry are going to have vastly different usage patterns than the bulk of people who do not so choose.
First, it's a small percentage. Or believed to be a small percentage. There are some larger buckets (eg distributions that disable telemetry), but it's possible to do very rough estimates of the sizes of those through other means. And anyway, the vast majority of users are on Windows.
Second, it's fair: if you disable telemetry, you're choosing to not be considered in any telemetry-backed decision making. If you want to still be considered, then it's up to you to make your opinions heard in some other way. (Filing bugs or https://connect.mozilla.org or discussions in places like here, though note that the latter is mostly useless. Not many Mozilla people read this forum or take what is said here very seriously. And even if they do people will be vigorously arguing both sides so it's easy to pick the side you already agree with.)
There's nothing wrong with disabling telemetry. I respect the decision, and I'd certainly rather have people using Firefox with telemetry disabled than have those people not use Firefox. But it's your browser, and even the social contract by which you're using it doesn't say you owe us telemetry data.
I get the saying. But in this case, Mozilla gets most of its money from search royalties, primarily from Google. We are Google's product, not Firefox's.
It's both. Firefox sell you to Google, Google sell you to advertisers.
It seems Firefox executives take a large chunk of money from Google; presumably to make sure Firefox doesn't do anything too wild that would reduce Googles income.
People use it purely for that reason. If they don't, their usage will drop, which will reduce their revenue. The fact that this revenue comes from Google is largely a byproduct.
Its not a perfect system, but it should somewhat work.
I'm not sure - I guess the same motivation that supports the open source movement. I am satisfied that the actions Firefox has taken so far does tend to support my interests.
But many of us who used it from the start think it was a much better browser before. For a long so much was sacrificed for next to no improvement.
For me it was more or less rock solid at >800 tabs and with a lot less memory and more exciting extensions than I have now.
I admit this wasn't everyones experience, but as a superuser tool it has degraded a lot over the years.
That said it is still the best browser for me: I don't think anyone else except Orion (which is Mac only) has actual tree style tabs (not to be confused with vertical, non indented tabs as seen in Opera derivatives).
> presumably to make sure Firefox doesn't do anything too wild
I wonder how much of what the Tor Browser version of Firefox does, would be upstreamed to Firefox proper, if not for that deal?
(I wonder how much the Firefox team considers Tor Browser to be "the real Firefox" / "Firefox the way we intended", with Firefox itself just being "the sell-out version of Firefox"?)
Purely technical telemetry like this is indeed useful. The problem comes when telemetry is used to justify deleting useful features such as compact mode.
The "make it hard to find" to "nobody uses it" to "let's delete it" pipeline is very real. Reminds me of the "defund it" -> "it does not work" -> "let's privatize it" pipeline in right-wing governments.
My personal objective with most situations is to discourage other people from enabling telemetry and then enabling it myself.
As a larger piece of the visible audience, I then hope that more attention is given me. This is especially important for open source projects. And I don't care that much about what the company is getting from me.
But listen, they collect all sorts of stuff and you should disable it unless you understand it. Ideally, privacy laws expand to the point where you need to email a signature saying you understand before you opt in to telemetry. Informed consent is required for any reasonable study.
Web browsers are definitely way measurably better and more stable than they've been basically since the beginning. We went from buffer overflows in the URL parser, dog slow JS interpreters, and unclear specs that are implemented completely differently in every engine, to today, where browsers are some of the finest engineering in the whole field, featuring state of the art JITs, extremely precise specifications, and bug bounties blowing past six figures once you get to code execution.
Of course you can do something ridiculous and compare today's Firefox with NCSA Mosaic or something. And yeah, old-school HTML would still be perfectly usable if people wanted to do that, so it's not like this is entirely pointless. But if you even make the comparison slightly more reasonable, like comparing Chrome and Firefox today to their versions from 10 years ago, I mean it's really no contest. From the standpoint of the core functionality, these are better browsers that are faster and generally more efficient. It's not just because computers got faster; that enabled more complicated things of course, but I have several older machines still new enough to run some modern software (Pentium 2-4) and generally, the newer browsers just run significantly better when doing equivalent things. They handle load better, they generally load pages better, scrolling generally feels better... Even on crappy single core processors. Even crappy sites like YouTube can work pretty decently on a Pentium 4.
This is not really that surprising because it's not like they haven't been optimized for this. Even if phones are starting to get very fast, there's still plenty of lower end phones on the market where it's important to be able to deal with slow processors with few cores.
Chromium is often my go-to codebase when I want to see how to do something correctly, especially with weirder OS APIs. Perhaps somewhat fittingly, I used to often use Qt for this, in a different world.
> Web browsers are definitely way measurably better
Just recently Firefox did not let me view a website even though the server was up because the certificate was expired and the site used HSTS previously. No override provided to me as a user. Better? No.
So an older browser that doesn't support HSTS is good, because it means you can browse a page even when your browser has valid reason to believe a downgrade attack could be occurring? The nuances of secure connections can be pretty awful, but a vast majority of the time, HSTS is good. The fact that HSTS is ubiquitous makes it less likely that attackers will even try attacks like these.
If a website admin messes up TLS when using HSTS, that's unfortunate. But: they opted to use it on purpose. It's hardly the browser's fault for trusting the website that it's not OK to browse in this circumstance.
Or they didn't - e.g. the entire .dev TLD is HSTS preloaded or it could have been a previous domain owner. Or they were blindly following up a guide.
But even if they mean to have HSTS, mistakes do happen and the browser should not prevent me (the user) from working around them.
Security without any other considerations is not reasonable - and if you want that you might as well prevent all connections to avoid all 0days.
Ultimately the browser is not in a position to fully judge the threat model and thus should allow the user to override its guess - always. For example when I want to look at a blog of funny pictures without any login info I don't care if someone tries to MITM that connection. And unless you are in a country with shitty consumer protections, an MITM is already so unlikely to be a conspiracy-theory level concern.
> So an older browser that doesn't support HSTS is good, because it means you can browse a page even when your browser has valid reason to believe a downgrade attack could be occurring?
Well I mean false positives are bad because they are false. That much doesn't require further justification or someone to embrace false negatives instead or whatever. This policy of "treat everyone as stupid and gullible and ditch them if they won't upgrade" makes sense for giant tech companies, but not necessarily for everyone. Some of us have to be able to work with old technology.
It's not a false positive because it's broken, though. It's a false positive because it's working as intended and the host is simply violating the rules. It's weird that a site would opt-in to a feature like this, use it incorrectly, and then when the browser correctly rejects it, you would get mad at the browser. Nobody was actually forced to use HSTS here, and there's also no good reason for a TLS certificate to be expired either; in production, this is an incident no matter what.
The browser really isn't treating you as stupid, it's telling you "this is a serious security issue, if you really want to bypass this, you're on your own." You absolutely can, using flags in chromium or config in Firefox, or sometimes by clearing the HSTS cache in either. The benefit of this is that it ensures users who don't know better, the majority, don't stumble into an attack in the most critical situations, and it as well makes it significantly harder for developers and malicious attackers alike to try to convince end users to wrongly bypass security features, a problem that plagued early web browsers which had much worse UX around TLS. Even though it can be annoying, it's helpful to all of us, because the security posture of those around you naturally impact your own security posture, too.
This is all especially reasonable because HSTS is opt-in from the host's perspective. You're supposed to use it when you'd absolutely rather have false positives than not catch an attack.
This particular point doesn't have much to do with old technology, but I honestly don't think most developers set out to just break old tech. I agree that it is a shame the degree of churn we go through, but even if you have a super valid reason to absolutely need to use old technology, it's still not a good argument for the rest of the world to hold off on improving security, privacy and performance by holding back TLS upgrades or continuing to include and debug polyfills for all of eternity. If you really absolutely can't make TLS work for you, nothing is stopping you from running an SSL stripping proxy in the middle. Works pretty well for me.
Hopefully in the future the churn of technology will slow down and computers will last longer, but we're literally still near the beginning of the computing revolution, and the computers from 20 years ago are probably a much more enormous delta from today than the computers 20 years from today will be. (And even if a breakthrough proves this untrue, it still seems unlikely that today's boxes will become useless, with how much compute they pack.) And yet despite that, Linux is still dutifully supporting processors as old as 486, even though it's not really that important to be running the latest kernel on a machine that old. That's pretty good, and even if browser updates are difficult on machines that old, I have little doubt that some people will be maintaining them all the way to the 2038 problem where it will get much harder.
Chrome will do the same thing and so will any other browser that honors the HSTS standard in RFC 6797. You probably want to direct your ire at the owner of the website, setting the HSTS header is a positive affirmation by the server that it does not accept insecure connections.
Special attention should be paid to section 12.1 of that RFC:
>If a web application issues an HSTS Policy, then it is
implicitly opting into the "no user recourse" approach, whereby all
certificate errors or warnings cause a connection termination, with
no chance to "fool" users into making the wrong decision and
compromising themselves.
This reminds me of Microsoft having a strong incentive to make sure hardware manufacturers write their drivers correctly, because if a bad driver causes problems the average user will blame Microsoft, as they don't know that MS don't write the drivers.
In this situation Firefox are being blamed for correctly implementing the standard and preventing access to the site, when the blame should fall on the site owner for not setting the security up correctly.
The html spec also defines that audio can be set to autoplay and the JS spec allows websites to freely open popups. Yet browsers are able to ignore that in the interest of the user - because they are (supposed to be) user agents not website agents that have to blindly follow what the website says. Browsers are free to not implement user-hostile specs or only implement parts of them and they commonly chose to do so.
Ultimately, protocol specs have no purview over user interaction and can at best recommend the expected behavior. There is nothing wrong with the browser telling the user that this website is expected to be accessed trough TLS with a valid certificate and isn't and to redirect HTTP requests to HTTPS but the spec is no excuse for the browser not letting the user overwrite that policy.
So no, the ire should be directed at the browser because the broser is there to make sure my interest as the user are followed not those of some website owner (or someone making decisions for them like Google for *.dev).
Neither the HTML spec or the JavaScript spec require that audio be auto played or that pop-ups be opened without restriction. HSTS is different in this respect.
I'm sure if you want to hack your browser so that it ignores that header you can, but the idea is that any server sending that header is telling you to go away if the certificate is invalid.
HSTS can be annoying when such things happen but from what I understand the browser is acting properly there. Also I've been able to clear that in Firefox in the past, I think I had to clear all data for the site. I do think that an easier mechanism could be a good thing.
By selling that software to user once, then only improving it a year or two later based on any quirks that got enough people mad enough to actually send an email to complain about it. If it’s too critical, then there goes a quarter of your user base.
> If it's too critical, wouldn't they send an email about it?
Usually no. I very rarely email software providers to complain about their software. I'll usually just uninstall.
Some software has more of a relationship with users than others. Doing support for B2B SaaS, I knew many of the users by name, and they knew me. Even then, though, some people would just live with terrible bugs for a long time.
We once had an issue where the initial page post login would take 30+ seconds to load because it synchronously calculated a bunch of their stats. I don't recall anyone writing in to complain, and when I fixed it, I don't think they said anything. (The slowdown was a boiled frog situation, since it got slower as they grew.)
Because it was a website, we could see the load time, and I saw it and fixed it, but if it were desktop software, the only way we'd have known would have been to load up a test account with hundreds of thousands of fake users and orders and fake shipments, and gone through the process of using the fake data in a real way.
Load testing is obviously good, but it will never beat being able to see how it performs for real users, and just hoping those real users will report back to you is completely unrealistic.
Don't get me wrong: I handled thousands of bug reports and feature requests during my time in support, and we got a lot of useful feedback. But insight into the actual performance is critical. Major bugs will go unreported for a long time. People just expect software to be buggy these days.
People have a high threshold for sending emails, and most users will never send them. So if a change only makes things a bit slower / crashier or makes things much worse but mostly just for users who don't complain, you don't know. In this case "Firefox crashes sometimes" remains true even after their fix reduced the number of crashes by 70%.
You wouldn’t know if a product had crashes like this unless you experienced it yourself, or if users sent you messages about it.
Even so, you’d have little way of knowing if the report you’re looking at is widespread or just a tiny anomaly. So you could easily dismiss a report that affected many people, or spent a tonne of time digging into a report that barely ever occurs.
Using telemetry data to enhance software is an incredibly useful and powerful tool. In direct response to your sarcasm: it is a superior method than how it “used to be”.
well they kind of didn't. There was significantly less software in the olden days and you usually had to call someone personally and if you were lucky they fixed your bug half a year down the line when enough stuff had accrued to deliver a new version.
Yes software is significantly better in the sense that nowadays catching bugs through telemetry or other quick feedback and shipping an update in 24 hours before most people notice it is the norm. I don't know why people romanticize the past.
Telemetry just means any data about operations that are sent back to the mothership. Crash logs are just as much telemetry as a click event log. Equating any telemetry=spying is a knee jerk reaction to tech companies abusing telemetry.
> Telemetry is the in situ collection of measurements or other data at remote points and their automatic transmission to receiving equipment (telecommunication) for monitoring
It feels like your response fails to address the point colejohnson66 is making. They are saying "some spying is done via telemetry, but not all telemetry is spyware." The automatic nature of it is orthogonal to its spy-ness or user hostility.
Basically, "automatic" here is an antonym to "manual" e.g. user emailing a bug report.
Personally, I consider the following sorts of telemetry "not spyware":
- coarse grained crash report (build version, arch, etc). this is usually a manual prompt on crash, so "semi-automatic telemetry" is how I'd define it
- anonymized metrics/spans. Basically "foo_bar() took 20ms". These are "automatic" in that the collection and transmission happen without user input, but that's orthogonal to whether the user opted in/out.
Fine-grained usage information is a lot more spyware-esque.
> This is also a good example of the benefit of telemetry:
The benefit can be claimed only if the user consented into their private information being shared with the browser vendor in the first place. With most browser telemetry that is not the case and browser is simply not respecting users' privacy. The right to privacy, as a human right, trumps the 'right' to have the product 'improved'.
Otherwise we can find "benefit" in everything. One of the benefits of hell, for example, is that it is never cold.
If Firefox was selling a physical product in a retail store, they would be able to watch you walk around the store on CCTV, see you avoided an aisle because there is a polar bear lurking, and then remove the polar bear.
But since the product is digital they just have to give it away blind? Never knowing if people even use the features or not?
>they would be able to watch you walk around the store on CCTV
That seems like an unfitting comparison.
The problem doesn't arise in the store, but when using the product at home.
The equivalent of store cctv in this comparison would rather be a server log on the Mozilla website (where people get the product). It's fine to do telemetrics there without me consenting (as long as it's only used by first party) if you ask me.
But after I leave their premises it's none of their business how I use the product.
Sounds like you want it to be ok that your newly bought pack of condoms sends out a message to the factory once you open one.
Software is often described as "tools" and so an analogy to a drill or a magnifying glass is as apt as any. In fact a car is a good analogy to a browser because we use browsers as sort of a "second home" in our computer and it allows us to "visit websites" and a browser is a whole ecosystem unto itself.
So if software is a tool and my drill is monitoring the holes I make in stuff and its efficacy in doing that, that seems fine, but if the drill is sitting in my toolbox being a busybody and sending back everything it can find about me from within the toolbox, that drill is made by assholes, don't you agree?
It does, but I think that's only for whether to send a detailed crash report (which could contain private data). I think (but haven't checked) that the
"number of crashes" telemetry includes cases were you don't choose to allow it to send the full report.
Every telemetry will at least transmit users' IP address (by the nature of how requests are made), which is legally considered private information.
Even if that was not the case, a privacy respecting (any?) browser has no business whatsoever sending any information from my machine, including "I crashed because of OoM" or "I clicked button X" unless I consent with it doing so.
Therefore zero telemetry by default is the only acceptable standard for any browser that claims to be privacy respecting.
The author left off the part where it was invented by a mother and how dentists hate it! /s
I suppose it technically improves stability, but the cause seems like a flaw in the Windows operating system, if I'm understanding correctly.
> Stalling the main process led to a smaller increase in tab crashes – which are also unpleasant for the user even if not nearly as annoying as a full browser crash – so we’re cutting those down too.
I wonder if this has anything to do with my recent experience with Firefox Nightly on macOS. In the last few weeks I started to experience spontaneous tab crashes that couldn't be explained by anything from what I could tell. I could open a new tab with the exact same page and it would manage to not crash. Then I noticed Firefox would sometimes prevent me from viewing tabs until I restarted to update the software. Haven't seen it in the last few days, but it was incredibly frustrating. IMO, Firefox should make a best effort to render a page and not have its core functionality stop working entirely until updating.
> I suppose it technically improves stability, but the cause seems like a flaw in the Windows operating system, if I'm understanding correctly.
It's not a flaw at all, when you understand what is going on. Part of the issue is that in 2022 so many developers come from Linux backgrounds that they assume that the Linux way of doing things is the "normal" or "correct" way.
The NT kernel does not overcommit, and thus does not have an OOM killer. If the kernel cannot commit pages, the system call fails. That's it. No process terminations.
Firefox would crash because (even on Windows) it uses a customized version of jemalloc that is configured to be infallible by default. If jemalloc requests pages and those pages cannot be committed, the heap allocation will fail, and thus Firefox will self-terminate. That's simply a policy decision on Firefox's part.
Going back to Windows: suppose that the commit request failed because the swap file was full. Assuming that Windows was configured to automatically resize the swap file, the OS will then grow the swap file. That's why pausing for a bit and then retrying the VM allocation works: the swap file was grown, and now the pages can be committed.
The Linux design to allow for overcommit is weird to me. Why would you not want to be up-front about the fact that you don't have enough memory available to hand over to a process? The process requested a certain amount of memory to be allocated so surely it expects to use it, no?
> Why would you not want to be up-front about the fact that you don't have enough memory available to hand over to a process?
Just because a process allocates a certain amount of address space doesn't mean it will use all of it.
> The process requested a certain amount of memory to be allocated so surely it expects to use it, no?
No, not really. I think you are confusing memory usage with address space allocation.
You may wish to allocate a huge amount of address space for convenience purposes, as it could allow you to use much simpler data structures (such as sparse arrays) which can be a lot more performant than more complicated data structures and use a lot less memory (by orders of magnitude) than the address space that you allocated for them.
An example of this is AddressSanitizer's shadow memory, which is sort of a giant sparse bitmap whose address space is allocated up front, but only a small part of it is actually used at any one time.
From the blog post, it sounds like Windows has a separate system call that is required to be used in-between allocating address space and actually using it. I think this is a good design and I personally prefer it over Linux's overcommit design.
However, I think it has a disadvantage in that it could require you to do a lot more system calls (and therefore have slower performance) than would otherwise be needed in Linux, presumably (especially if the access patterns are unpredictable in advance?).
Also, if some process misbehaves and starts using more memory than there actually exists in the system, it can cause significant problems in other processes/applications that have nothing to do with it (as their allocations would start failing). Linux handles this by killing the misbehaving process, which allows the system to recover transparently / without intervention (at the risk of possibly killing innocent processes before actually killing the misbehaving one), hopefully allowing other processes to continue working as if nothing happened.
I think a reasonable approach could be to use Windows' system by default and guarantee that processes are never killed by the OS (which would benefit well-designed applications), but allow an application to optionally use overcommit if it wants the added performance (at the risk of being killed by the OOM killer if the system runs out of memory).
Unfortunately, I suspect that applications which actually handle out-of-memory conditions gracefully are very rare and that most of them would just crash or kill themselves if this happens, which on average is probably a worse outcome than letting the OS kill the misbehaving application.
> Just because a process allocates a certain amount of address space doesn't mean it will use all of it.
Indeed.
> From the blog post, it sounds like Windows has a separate system call that is required to be used in-between allocating address space and actually using it.
Precisely. You can reserve pages in the address space[1] without committing physical memory to back those pages. You can then later commit pages[2] when you want to use parts of that address space, or decommit[3] pages when you're done using them, without changing the reserved address space.
Coming from a Windows background I much prefer this system. I understand why Linux, with its reliance on forking, has gone with the overcommit route, but I consider it a sub-par solution. At least for non-server systems.
> Coming from a Windows background I much prefer this system.
I come from a Linux background and I much prefer that system too.
Well, at least in theory, because in practice I don't usually run into these kinds of problems nowadays due to my hardware having significantly more memory than my software needs to run. But I still think Windows is designed more elegantly in that respect.
> I understand why Linux, with its reliance on forking, has gone with the overcommit route, but I consider it a sub-par solution.
Well, if Linux actually supported committing pages like Windows does, then there would be significantly less need for Linux to overcommit memory, because when forking you'd only have to reserve swap space for the committed memory of the process.
This means Linux wouldn't have to choose between reserving no swap space or reserving swap space for the entire address space (or some constant fraction of it, which Linux supports but doesn't make much sense either)!
> You may wish to allocate a huge amount of address space for convenience purposes, as it could allow you to use much simpler data structures (such as sparse arrays) which can be a lot more performant than more complicated data structures and use a lot less memory (by orders of magnitude) than the address space that you allocated for them.
Windows does allow you to reserve address space without committing it, but it then requires you to commit the chunks before you use them (see VirtualAlloc + MEM_RESERVE)
> Windows does allow you to reserve address space without committing it, but it then requires you to commit the chunks before you use them (see VirtualAlloc + MEM_RESERVE)
Personally, I think this is the right approach.
Because you shouldn't have to decide between memory overcommit (which can fail badly in some reasonable scenarios) or "having to reserve all allocated address space of all processes in swap", which is way too pessimistic and would start causing unnecessary memory allocation failures in many reasonable scenarios.
And I also think that even if you decide to support memory overcommit, it should be done per-process (at least), not at the whole system level like Linux is doing.
I'm not convinced that the fork()/exec() issue is the correct justification, at least not entirely. As an example, Solaris supports fork()/exec() yet it doesn't do overcommit (at least not by default, I think) and has no OOM killer. And Solaris has a long history of being a very robust OS.
Also, in most cases I don't see what is the problem with reserving more disk space for swap (disk space which would almost never be used anyway), especially if the system becomes more robust as a result of that.
I think I recall Linus justifying memory overcommit by observing that the vast majority of applications don't handle out-of-memory conditions gracefully, so it's better for the OS to kill a misbehaving process (that is allocating too much memory) and let the others continue working normally than have every single process in the system have to handle out-of-memory failures, which they don't usually handle gracefully. But I'm not sure if I'm recalling this correctly.
I don't think either the overcommit design or the naive "all allocated address space must be reserved" design is the correct approach. I know almost nothing about Windows memory management, but it sounds to me like having "commit/uncommit memory" system calls separate from the "allocate/deallocate address space" system calls to be a better approach.
But I think the OS should try to reserve more space for the swap file before it starts returning ENOMEM to the "commit memory" system calls like Windows seems to be doing (judging from the blog post)...
> Fortunately, this is tunable. You can also turn off overcommit entirely.
I know, but as soon as I tried doing that, my systems started to experience unavoidable failures (although I don't remember exactly why, but I certainly don't think it was simply due to lack of swap space).
Again, I don't remember exactly, but I suspect I experienced these failures because there are applications that expect Linux to be doing overcommit, or at least, they couldn't work without overcommit unless Linux added new features.
I could be wrong (I'm not exactly a Linux memory management expert), but I think the root issue is that Linux is handling committed memory badly, because when you disable overcommit, Linux tries to reserve swap space for the entire allocated address space of all processes. But actually, this is not needed -- it's way too pessimistic and is extremely likely to lead to unnecessary memory allocation failures.
What is needed is a way for applications to communicate with the kernel about which parts of the address space they might actually use (or not use) at any point in time, which may be significantly less (by orders of magnitude) than the amount of address space that they have allocated.
Then you wouldn't need to reserve swap space for all the address space that processes have allocated, you'd only need to reserve swap space for the amount that the processes have declared to be (possibly) using.
This could have a performance cost, but there are ways to reduce it, for example, by allowing this information to be declared in mmap() as a flag (which avoids doing 2 separate system calls in the typical case), by batching several syscalls into just one (similar to readv()/writev()), by using io_uring(), etc.
I also think that, even if you want to use memory overcommit, then enabling overcommit should be done with (at least) process granularity, not for the whole system at once, which again, greatly limits the usefulness of disabling overcommit.
I know about mmap and mlock. Can you be more specific as to how they are useful in the scenario I mentioned above?
Specifically, when memory overcommit is disabled, I can use mmap() to allocate address space but this causes Linux to unnecessarily reserve swap space for the entire amount of the address space allocation.
This means that if the address space allocation is big enough, it would almost certainly fail even though I would only need to use a very tiny fraction of it.
Do you understand what I mean? Just because I allocate address space, it doesn't mean I will use all of it.
As far as I know, Linux can't handle this problem, because there's no way to communicate to the kernel which chunks of the address space allocation I wish to use.
Which means that disabling overcommit in Linux can be completely useless, because many mmap() calls start failing unnecessarily.
I don't think mlock() has anything to do with this problem.
They ensure that regions of memory are backed by actual memory. This sounds like how to tell the kernel that you're actually need and will use this memory.
mlock() forces pages which have already been allocated and are already being used within an address range to stay resident in memory, so that they are not swapped out to disk. These pages were already being accounted for in swap reservations.
When disabling overcommit, what you'd need to tell Linux is actually quite different: it's that you are about to start to use a new address range, so please make sure that there is enough free swap space for this address range (and reserve it!).
Only after the latter is completed would the kernel be allowed to allocate new pages for this range and the program would be allowed to use them. The kernel would also be free to swap them out to disk like normal, unless mlock() would be called (but only after those pages are already being used, not before!).
So as you can see, mlock() accomplishes something very different and is orthogonal to the functionality I'm discussing, which means it can be used (or not) independently of this new feature to reserve swap space.
This new functionality (to notify Linux that you are about to use a new address range) would also implicitly allow Linux not to reserve swap space for any address range for which it hasn't been notified, which would allow Linux to use swap space much more efficiently and would allow users to disable memory overcommit on Linux without causing a bunch of unnecessary program failures / crashes.
mmap(), on the other hand, normally does two things:
1. It allocates a range of address space.
2. It reserves swap space for this address range (when memory overcommit is disabled).
Notably (and people get confused by this a lot), mmap() doesn't actually allocate memory (i.e. memory pages), it only assigns a range of address space for the program to use, exactly as large as the program requested, and then reserves swap space for it, if required.
What I'm proposing is to separate those two things, i.e. allow programs to allocate address ranges separately from doing the swap space reservation.
If you don't separate these two things then programs allocating a huge range of address space will simply fail due to lack of available swap space (again, when memory overcommit is disabled, which is the goal here).
> But I think the OS should try to reserve more space for the swap file before it starts returning ENOMEM to the "commit memory" system calls like Windows seems to be doing (judging from the blog post)...
One could just as easily argue similarly with respect to Unix and needing to loop over write watching for EINTR et al.
> One could just as easily argue similarly with respect to Unix and needing to loop over write watching for EINTR et al.
I'm sorry, but I fail to see how that is related.
EINTR is useful so that you can handle a signal in case it is received in the middle of a very long system call. For example, if an application is doing a write() on an NFS filesystem and the NFS server is not reachable (due to some network outage), the write() syscall could take minutes or hours before it completes.
So it's good, for example, than you can Ctrl-C the process or send it a SIGTERM signal and abort the syscall in the middle of it, letting the application handle the signal gracefully.
What I'm talking about is not related because allocating disk space on local disk (where the swap file would be located) is generally a very quick process. Mind you, only the disk space allocation is needed for reserving swap space -- it's not necessary to write anything to the swap file. In fact, it's not even necessary to actually allocate disk space, but I digress.
And even if reserving swap space would take a long time, allowing the syscall to fail with EINTR would also work fine.
What is not fine is letting the application believe that the system has completely run out of memory when in fact a lot of disk space can still be used for swap reservation.
That Stack Exchange answer is kind of weird in the sense that it conflates different things. You can have demand paging without overcommitting (as NT does); the kernel simply needs to assure that there is somewhere to commit a particular page, even if that page isn't committed yet.
The answer may not be the best-argued one, but it links to other useful discussions, and it is correct that the forking mechanism is an important factor.
Committing is limited by RAM plus swap. You’d have to reserve much more swap than is typically ever actually used by processes, at any given time.
> You’d have to reserve much more swap than is typically ever actually used by processes, at any given time.
But if the swap wouldn't actually be typically used, then what's the problem with that?
Especially considering that the amount of disk space required for that is cheap.
And why not let the user decide for himself if he prefers to guarantee that their applications never get killed by the OS at the cost of reserving some disk space that he probably wouldn't ever use anyway (as most filesystems' performance nosedives after >90% disk usage anyway).
>most filesystems' performance nosedives after >90% disk usage anyway
With modern filesystems using delayed allocation to reduce fragmentation, and SSDs reducing the cost of fragmentation, you can often get good performance at higher occupancy nowadays.
Sure, I know, but if you're talking about really modern filesystems (which do copy-on-write/CoW), then the fragmentation caused by CoW is even worse than that avoided by delayed allocation.
SSDs certainly alleviate this problem, but even in SSDs, sequential I/O can be much faster than random I/O.
Anyway, I guess my point is that the vast majority of systems don't run with >90% disk space usage, so reserving up to 10% of the filesystem for swap space is not unreasonable.
Note that this would just be a space reservation. You wouldn't need to actually allocate specific disk blocks or write anything to the swap file, unless the system starts running out of memory.
In reality, you'd need much less than 10% (in the vast majority of cases), especially if you have a Windows-like API where you can allocate address space separately from committing memory (which means uncommitted address space doesn't need swap space reservation).
Operating in a low memory environment is inherently about tradeoffs.
Where is the blame?
Maybe Firefox is too bloated or memory-inefficient. Maybe Mozilla didn't understand Windows's memory management strategy until now.
Or maybe Windows is too
bloated and memory-inefficient. Or maybe the memory management tradeoffs were suboptimal.
Or maybe nobody is to blame, and they are taking advantage of something in a novel way that allows them to squeeze more juice out of the same fruit than others.
> Maybe Mozilla didn't understand Windows's memory management strategy until now.
That is part of it. A lot of FLOSS engineers come from a Linux background and tend to assume that the Linux way of doing things is the "normal" way. While I was there I had to explain to more than one developer that Windows doesn't have an OOM killer because the NT kernel doesn't overcommit.
Not sure, but sound stopped working in VLC about one year ago and was restored this month at the same time that dropping files stopped working. Every time after a Windows update and with the same VLC version.
I don't think Windows is too bloated or memory-inefficient, but I do believe that they're abusing the AV and "monitoring" stuff. It's a cat & mouse game, with me trying to disable crap and they re-enabling it with every update or simply making impossible to disable annoyances.
Also I suspect they're trying to "fix" drivers that worked before the fixes.
> I don't think Windows is too bloated or memory-inefficient.
You know I used to be on that boat up until very very recently. I have a 2019 HP Stream 10" with 32GB eMMC and soldered 4GB DDR4. Windows got it bricked on a botched update, so I installed Linux Mint to get it back up quickly.
Not only is Linux using a lot less storage (which was always and issue with Windows Update), but the RAM usage is about 700MB without any applications running, and well under 4GB for most common uses. Sure, Chrome(ium) will eat a hefty chunk of it when you run it, between zram-config and some extra swap space, it handles things a lot better than Windows ever did.
Antoher point of anecdata: shared libraries are actually shared a lot more between processes on Linux than on Windows. That tends to mean a lot less RAM usage on certain process. I know it's down to how they handle sharing things and that Windows takes the "safe" approach (pretty much what the article talks about!) but it ends up hitting memory a lot more than Linux and macOS.
The safe approach to me is exactly their answer to the tradeoff question.
Forcing the application developers to be cautious is just changing who needs to be safe, because reducing problems for your app while increasing overall system instability is still bumping up against the issue at hand.
It is sort of like how someone might be upset with speed limits, but the people who set the speed limits aren't thinking about your particular desire to get to work today, but the overall flow of traffic and public safety needs.
I realize linux still has out-of-memory handling and various safeguards in their memory management, but it is sort of like arguing whether to set the speed limit at 50 or 60. Nobody is actually right, it is just different preferences.
If the swap were allocated to the whole HD that wasn't used for actual files, then this hack wouldn't work.
If the swap were leaner that it already is, this hack would be necessary in every program.
If I had to point a finger at who is to blame, it's the Windows swap allocation team. Some combination of predictive analytics or even just a saner ratio of swap to free HD for an incoming file would fix this problem for most users most of the time.
But computers are hard and people want to keep them running for days on end. I get that memory just slowly, slowly gets eaten up by all the zombie procs out there.
Right. What I'm saying is that swap vs available HD is a tradeoff the OS should be making and it's probably going too far in the "have avail HD" side of the spectrum or at least not employing enough predictive analytics to figuring out the right balance for any given time.
That update behavior where ff is still running and existing tabs mostly keep working, but new tabs don't, has been a thing for years and everyone hates it, not just you or me.
I thought it was possibly related to ff on ubuntu switching to being a snap by default (even though I thought I had forced my system to have no snaps and no snapd and added a special ppa for ff) and said something in a comment on hn, and several people clued me in it's way older than that and I'm not the only one who hates it.
It's like ff devs don't actually use browsers, which is crazy of course. But, they really are ok with always having to blow away everything ypu have going at any random time middle of the day? (it's always someone's middle of their day or stretch of involved work)
They never have tabs open with partially filled forms or search results or web apps that "restore tabs" won't restore the way they were? Or this just doesn't bother them?
It feels like a case of "you're holding it wrong", as in the user should shape their usage pattern around ff's update strategy, like, always do and apt upgrade before sitting down, and never after starting to work, and if you leave tabs and work open over night, well I guess just don't do that?
> But, they really are ok with always having to blow away everything ypu have going at any random time middle of the day?
Y'all don't get your tabs restored when you restart your browser?
For me, the restart experience pre-snap was very easy - close, re-open, and you're right back. Most 'serious' webapps will happily save your drafts if, for whatever reason, you don't want to finish and send that half-composed slack message before restarting.
>Y'all don't get your tabs restored when you restart your browser?
Ironically, the tab restore breaks when you open a new tab and it hits the "Restart required" page.
On restart, the browser found a clever way to maximize user frustration. It simply attempts to restore the "Restart required" page again (and fails), leaving the new tabs you tried to open blank after the restart.
I still have updates enabled, despite the "Restart required" page providing a strong push to disable them. But at the current rate I might give in eventually.
The weird update behaviour is because file got replaced while Firefox is running. On Windows and macOS the updates happen on next start (whenever you choose that to be) so it's not an issue; on Linux updates are handled by your system package manager, so they couldn't line it up as nicely.
Of course that also means you could end up having queued but not applied updates for a long time on Windows and macOS…
You can avoid this issue (which pretty much only ever happens on Linux, mind you) by installing the package directly from their website instead of from your distro's package manager. If you'd like to help improve stability or use try features before they're fully stable, try the beta, dev, or nightly channels.
I run Arch and upgrade ~weekly, but I am very rarely inconvenienced by this restart behavior. Restore Tabs works pretty well on the modern web, and since old tabs still work, you can always complete whatever outstanding forms you have first.
Problem is when you click a link or enter a URL, restarting your browser restores the previous page from before you had triggered the restart prompt, discarding the page you were trying to open.
Overcommit exists mostly because of fork(). fork() exists because it was the simplest possible process creation API the Unix designers could get away with, which was an important consideration on a PDP-7. Now that computers are far more capable, we no longer need to sacrifice the ability for applications to handle low memory conditions in a more useful way than crashing.
Microsoft put out an interesting PDF paper about fork():
I suspect a reputation for robustness is at least partly a self-fulfilling prophecy. People who care about robustness will be more likely to write software for an OS they believe to be robust, and that software is also more likely to be robust.
fork() + no overcommit + no OOM killer can work if you're very careful with allocations in processes that fork, but it would be a disaster on Linux. Willingness to put up with the drawbacks of Solaris is a good signal that you value robustness/stability very highly. IMO, most people developing for Linux have different priorities.
I think I agree with you, but would like to add that from reading the blog post, it sounds like Windows is actually doing this better than both Linux and Solaris (but my knowledge is a bit limited in this area).
I don't quite like the overcommit approach nor the "all address space must be reserved in swap" approach, because both can fail badly in some reasonable scenarios.
I think having a separate system call to commit/uncommit memory like Windows seems to have is probably a better approach than just having mmap()/munmap() system calls (without a way of communicating with the kernel about which parts of the allocated space you are using), because then you can have the advantages of sparse address space usage while not having the drawbacks of having the OS kill innocent processes.
This would also have the advantage that in fork(), the kernel would just need to reserve swap space for the amount of committed memory, not the amount of allocated address space, the latter of which could be much larger (by orders of magnitude).
I've been a linux/osx user for 20+ years so I'm not familiar with Windows memory management. It'd be interesting to know why MS chose this approach and if it has any benefits? Why not just let userspace request whatever it wants?
Well, over-committing is an arguable choice. "Here's the memory you requested! I don't know if I have it available, but anyway. Here it is!"
It turns out it probably works well/better in many cases, because apps don't actually use all the memory they request (up front), and for other reasons, but it's not the obvious choice to make. I would intuitively expect my OS to fail malloc if it does not have any enough memory available if I didn't know better.
I would expect an OS capable of expending its swap file to try doing it before failing my malloc call though.
Yeah, I was pretty incredulous when I first discovered over-commit in Linux - I asked for memory and you gave it to me without an error, and now that I am deep in the guts of processing and can't easily recover, now you decide to tell me you don't really have it!
But once you know about over-commit there are workarounds in languages where you control memory allocation, like touching every page right after you allocate it, but before using it. And in a garbage collected language you don't have any control or insight into when OOM exceptions will occur in either approach. So the ability for the OS to not trust lazy and greedy software that asks for memory but doesn't use it seems like a reasonable trade-off.
I think one of the main reason overcommit is a thing on Linux (certainly one of the things one runs into if one turns it off) is dealing with the weirdness of fork()/exec(). For a moment there in between the two calls you _technically_ have double the RSS allocated - so spawning processes with fork()/exec() from a process with very large RSS is dicey if you don't want to technically overcommit the memory. Since 99.9% of the time very little of that memory is touched/COW'd before the exec() letting it overcommit rather than dying when a 4GB process tries to spawn a new child just because you don't have another spare 4GB of ram sitting around "just in case" is seen as a reasonable tradeoff.
(Modulo vfork() and spawn() of course which are different and arguably better solutions to this issue.)
But 99.99999% of the time, when a program calls malloc, it needs it to succeed. So if there is a callback or something to notify that a wait is required or whatever, then 99.99999% of programs are going to do it. That means the high cost of expanding the swap file will be incurred basically every single time...so why not make that the default?
In the rare case where a program wants to handle a failed allocation differently, then they should use a native system call that provides a more detailed interface than standard malloc. It doesn't matter if it's not portable since this is really a Windows-only thing.
Crashing is not good for anyone. A temporary freeze sucks, sure, but that's what you get for not having enough memory.
...plus, it's not like random freezing is a foreign concept to Windows users.
The Windows approach let you fail at a predictable place failing at the time of memory allocation. The overcommit approach causes OOM crashing at random places depending on how your program touches memory.
But then we end up with the OOM Killer, which is awful - randomly kill the biggest process because "reasons".... It would be better if the OS could say no.
In this case, the OOM Killer would be better, because a parent process is allowed to sacrifice one of its children, so the browser could kill the least-recently-used tab instead of itself.
If allocation predictably fails, you don't need an OS-level OOM killer to kill least-recently-used - you could just do said killing manually on failed allocation yourself. And you'd be able to do so in a much more controlled manner too while at it, instead of hoping the OS does what you want. (and if an OS/stdlib wanted to, such behavior could be made the default operation on allocation failure)
Right, it wouldn't help when one process wants more memory but you want an unrelated one to get killed, but the question here was about a browser killing one of its own tabs instead of the main browser process dying. (though, for what its worth, in the case where processes themselves can't decide how to free memory, I, as the user, would much prefer to be given the option of what to kill anyway; linux completely fails to do that, and given that overcommitting affects DEs too, it'd be pretty complicated to allow for such a thing)
not dynamically chosen though, at least in the case of earlyoom; whether to prefer killing the browser, or a random long-running process that has built up a couple gigabytes of RAM usage (or even just a bunch of small processes I don't need) will entirely depend on the intent (or lack thereof) behind the process, and what's currently happening in the browser.
Without over-committing, you could be preventing something that would work anyway. With it, the OOM could (often does) pick the biggest process that could very be using its memory legitimately AND be the most important process of this machine too.
One thing it is worth noting is that the user experience on Windows of running out of memory is a lot better than on a Linux desktop environment. While things usually slow down due to a lot of swapping, the main UI continues to be functional enough to allow you to carry on using it and close things.
Work is being done these days to improve the situation on Linux, but the default experience can be pretty painful. I was using Android Studio on a Fedora machine with on 8Gb of RAM, and sometimes the whole system would completely freeze for 10s of seconds at a time. This is not fun.
OTOH the mere existence of committed memory makes it hellish as a user when *something* leaks committed memory. You start getting out of memory errors while half of your RAM is empty, just because some half-assed driver somewhere leaks. To add insult to injury, task manager / resource monitor is always unable to show the exact process/driver that leaks; I had to randomly kill things to find the culprit.
I'll take the linux behavior any time when dealing with poorly written software (which is most software).
If you're over-committing memory then you don't find out your program ran out of memory until you try to access memory you've previously "allocated". If you're not over-committing, then you'll find out your program ran out of memory on an allocation where you might be better prepared to handle it.
Windows is strictly better here because 1) it will never pretend that it has more memory than it actually does, and yet 2) it allows processes to reserve as much address space as they need without using them (which is the sole justification for overcommit), by providing the APIs to control this in a fine grained way.
The Windows approach is better from a "perfect system" point of view: in theory an application knows all the code that it is loading and has a grasp on its own memory usage. You still have virtual address space because it is used for other things too (like memory mapped files), but you "commit" your current upper limit. You can be sure that if needed you'll actually be able to malloc (well HeapAlloc) that much memory. It might be slow due to swapping but it won't fail.
The Unix approach is better from a "realistic" point of view: most processes have a lot of library code they don't control and don't have time to audit. Usage patterns vary. And most processes end up reserving more memory than they ever actually touch. Note what they mentioned in the article - 3rd party graphics drivers run code in every process on Windows and that code allocates whatever it wants that counts against your commit limit. That isn't under your control at all and worse most of the time the memory is never touched.
Having lived under both systems I think I prefer the Unix view. In practice almost no Windows software does anything useful with commit limits so it just creates extra complexity and failure modes for little benefit.
> And most processes end up reserving more memory than they ever actually touch.
I find this assertion dubious. Explicitly pre-allocating large arenas is common when micro-optimizing, which isn't something that happens for most apps out there - they just do the usual malloc/free (or equivalent) dance as needed. So for your average app, it boils down to the quality of implementation of its heap allocator. And there aren't that many of them to get right.
For me, there's been some kind of weird instability or resource leak that resulted in Windows Firefox getting slower all around and less stable the longer the application is running. It's been around and 100% reproducible (over an active session of a few days) for a couple of years.
The general problem used to feature some sort of bug where some window processes would completely fail to render/paint UI components - instead, rendering them as pure black. The rendering problem is gone, same with a correlated memory leak, but the complete performance slowdown that accompanied it is still there.
One day I'll submit a bug report or profiler trace or something, but I find it odd every time I see a post about stability or performance fix, it never happens to be the big one that I run into, regardless of the window device or extensions.
It makes me wonder if some users just have browsing habits that most others don't, so they hit obscure bugs more frequently. But since everyone has their own obscure habits, and thus bugs, there's a theoretical endless deluge of problems with no critical mass to justify prioritization or investigation.
One thing I can say is that it is extremely rare that Firefox actually crashes on me. The instabilities are in the behaviors of the browser, its UI and the tabs/pages themselves. It can slow to a crawl, and even hourglass on me in an extreme case (I usually get fed up and just restart the browser and all the tabs at once to fix the issue before it gets that bad) but it manages to keep itself some, somehow.
That said, I'll poke around in there next time anyways and see if anything stands out. Thanks!
Has Firefox stopped allocating so much RAM that it fills up all my 40GB? Overcommit isn't always enabled, because I'm not making a 64GB page file just to keep Firefox running for more than a week. (That is how large Windows tried to make it before I disabled the page file.)
On my computer, Firefox running out of memory is its own damn fault. Retrying memory allocation won't fix anything! And running out of commit space crashes everything else too, so Firefox letting itself crash isn't going to help, because it still brings down half the system. (And it still, already crashes when it runs out of memory.)
Afaik memory-mapped files don't count towards the commit limit. Couldn't they roll a custom file-backed allocator (essentially manual swap) to get around the windows limit?
It’s good to somewhat limit the amount of memory that gets paged to disk, as it is incredibly slow memory. There should be backpressure on memory allocation. Getting around the OS on this matter would be silly, IMO. We’re talking about an interactive UI environment where massive page files would be doubly bad, swallowing up surprising amounts of disk space in order to provide incredibly slow memory.
However, the type of solution you’re talking about would apply well to a caching server, and Varnish (on FreeBSD I believe) uses it quite successfully.
Reminds me how some folk (uTorrent team, AFAIR) thought they are smarter than those guys in Redmond and rolled out their own caching algorithm. Except they weren't in fact smarter and fucked up the memory usage by allocating all the availble memory for a disk cache... pushing everything else including OS to the swap.
Don't give them ideas about how to use even more memory. For someone who's first computer had 64KiB RAM, it's incomprehensible why rendering a single web page should consume 100MiB.
I use firefox mobile and it has very strange behavior quite often. It's the only browser I would ever use because of the ad block. But sometimes tabs become blacked out and you can't refresh them, you have to close them and start new ones. And sometimes I have two different (consecutive?) youtube tabs, and the second tab basically fuses with the first tab, to where both tabs show the same content and the same video. I wonder if in both of these cases a tab process was killed. It's pretty common
Haha I put sleep-retry to every explicit memory allocation in every delphi program I wrote since windows xp and not one person acknowledged it as a good practice. This is just too good.
Every couple of weeks my firefox was crashing tabs one after another.
And I always had to restart firefox.
This weird trick fixed it.
I made the entire firefox directory readonly.
Yeah I'm very happy with Firefox desktop no matter the OS. The Android version of Firefox is not nearly as polished or capable at the moment. It's a shame because I really do still use it a ton. It has great potential.
Ah, very few crashes on Android (using it right now), but my biggest gripe is that the first page load can take a while to even start, after opening a new tab or typing something in the address bar. Tapping on stuff lyke the reply/comment button is pretty fast though.
Thank you for providing more stable experience. Would be great if something could be done about some Unity web games, that sometimes crash whole browser. It is a bit peculiar, while each tab seemingly runs in separate process, but one website crash takes down whole browser.
All these hacks blog posts are really interesting insights by Firefox engineers. Browsers in general are such a complicated system, so it's pretty remarkable what they come up with and what they learn when attempting to update Firefox.
Only somewhat related but what I'd really like to see is an investigation as to why Firefox takes so long to acquire a connection on Linux.
I suppose it could be something particular to my setup but it is very slow in comparison to 'ungoogled-chromium' which I keep around as a second browser.
On the network tab I routinely find Firefox in a 'blocked' state for 100ms. On chrome this rarely exceeds 1ms. I've tried messing around with various configuration options and haven't found an answer. The end result is Chrome feeling much snappier.
> The first computer I owned shipped with 128 KiB of RAM and to this day I’m still jarred by the idea that applications can run out of memory given that even 15-year-old machines often shipped with 4 GiB of memory.
I think a safe number for the modern web is like 64 Gb. At least if you want to have anything productive open besides the browser...
Why do people keep so many tabs open? I usually top out around 5. Bookmarks are a thing, I wonder if just leaving pages open is the new bookmark? But the URL is gone when a tab is closed.
IMHO people using so many tabs seems to suggest a shortcoming somewhere else.
At work I start off with 14 tabs, they're tools and references that I use every day.
At home I have several projects, I use TST (tree-style tabs), each project has a subtree. Sometimes I'll close off a tree, sometimes I'll save a tree to bookmark. An example tree was where I was investigating a problem with pdf files across some public resources (related to work), I had the principle sources for the PDFs, the PDFs themselves open, 6 tabs, then some searches looking for bug reports and each of those having child tabs of actual reports, then reference material raised through the bug report pages (gs and convert documentation). We're at about 20 tabs now -- meanwhile my console has at least 4 tabs open with gs and convert commands and man pages, I upload finding to gitlab (for reference when I'm at work) -- gitlab pages added to tab count. I test some pdf conversion commands, open a file manager (with its own tabs!) and open the results in the browser too. Midway one of the kids asks me about buying something, I switch to my Amazon tree of tabs, open a couple of searches and my basket, stack up some comparison prices in other tabs; one shop has several options where Amazon only has the one so I open them all and control-tab to do visual comparison; roughly +30 tabs, but I close off the tab trees that are poor prices or out-of-stock, down to about 10 tabs left until I make the actual purchase in ~10 days time.
At work I have five tabs open just for the story I’m working on. Five more for our telemetry monitoring. Researching an API issue can easily be another five, and when I find the solution I may discover I’m wrong and then have to remember which pages in my history are involved, or I can leave them open until I’m deployed cleanly in preprod. It can be hours or days before I notice the extra tabs and close them.
And then a couple times a day I’ll open a bunch of threads in HN to read over the next few hours between other things.
Anything researching for a hobby can reach 10 tabs easily, though that usually happens “at home” so replaces rather than supplements the work tab count. And now we’re already at a few dozen tabs.
Having three monitors and other apps running you can “lose” a window and thus end up with the same site open in two or three windows. 50 tabs is usually about when I start garbage collecting, and I always manage to close something I still needed.
I use it like a program uses its stack. That's why I prefer to use extensions like TreeTabs, and/or Tab Groups (of which I was a regular user).
For every subtopic, I open a few tabs, then browse them, compare them, reorder them, then close them. This is a recursive operation, hence why TreeTabs works well, esp. with the abimity to collapse.
If a new topic comes up, I'll open a new tab.
I often leave tabs for if I have time to go back to them later, and that's where my thousands of tabs come from :/
Ideally I'd like to represent my tabs (and other OS windows, why not?) as a mind map. Open and close some topics. Export the topic to my bookmarks, share it... pearltrees come close.
For people who don't spend time sorting and managing their bookmarks, tabs can be an equivalent alternative. Modern browsers will search and suggest both bookmarks and open tabs, so the UX is pretty much the same. Just that unsorted bookmarks are hidden somewhere in a separate menu/window, a bookmark bar would shrink the available space for web content. The tab bar is always there and a huge list of tabs tells me at a glance that I might want to get rid of old stuff.
> But the URL is gone when a tab is closed.
Accidentally closed tabs can be easily restored for as long as you don't delete the history. And when I reinstall a system I just restore the profile directory from the backup and Firefox will start with all plugins, settings and tabs from before.
Because tabs and bookmarks are not the same thing. It's harder to find something in a bookmark, easier to just go through the opened tabs quickly if you know it's there but don't remember the name
Look at them open few more and it's already easy to land on 10-20 tabs. Add some stuff always open (chats, webmail etc) and it's easy to get into 30s
I start every day with zero tabs but getting to 40-50 (then mass-closing if I found solution) is common. Vertical tabs addon is necessity (hell it's even nice with just 10 tabs)
weather, two work chats, gmail, hnews, personal mail, 2 fediverse accounts, current work tabs (currently at a low 8 for that), then the ones that change hourly, social media, youtube, wikipedia, searches, etc.
Tabs are really starting to bug me, i've tried a few tree tab managers and the ones i tried just don't work. I don't have a better solution. multiple windows is potentially bad news, as a crash may lose tabs you had open, even with a tab session manager. That is if you load several tabs in a new window and the computer or firefox crashes / reboots, you could lose history, cookies, possibly your old window layouts, etc. Tab session managers help a bit, by keeping consistently open tabs saved somewhere, but it doesn't help with new tabs and new windows if there's a crash.
Also it kinda solves the new window problem, as long as the new window is set to one of the tab groups (closing the window doesn't delete the tab group).
Also afaik when firefox crashes and is set to recover tabs, it should recover all the windows as well as the tabs.
You haven't shared what kind of pages you have loaded in those tabs. If they're 100 simple text pages without javascript bloat or anything, you can blame firefox. If they're 100 pages which ought to be simple but actually have tens of megabytes of complex javascript with memory leaks, you can blame the pages you have loaded. If they're 100 complex desktop-style applications or games, you can blame yourself for trying to do more with your PC than it's specced for.
> If they're 100 pages which ought to be simple but actually have tens of megabytes of complex javascript with memory leaks, you can blame the pages you have loaded.
... but every opinion piece that would fit in 1.5 kb of plain text loads 250 Mb of javascript ...
Yeah, which means the right answer is probably to blame web pages for being bloated, maybe with a little bit of blame for the web technologies for allowing pages to be so bloated.
True Firefox, forks of Firefox before the dropping of XUL and before the switch to multi-process, can do 300 tabs in less than 2GB of ram. It's just the chrome-a-like modern firefox that, like all multi-process browsers, is a ram hog.
Dramatically improved application security and performance at the cost of cheap hardware, which for most people gets significant upgrades every few years on desktop and every couple years on mobile, seems like a good trade to me.
Yeah, for most people new Firefox or Chrome is a no brainer. They don't need a browser that can surf the web. They need a fast javascript engine to run a few tabs of applications that just happen to be delivered over HTTPS.
People like me that just want to web surf can disable JS by default and then single process mode -JS is infinitely more secure than a multi-process +JS. Just different use cases.
Click bait title. On Windows, if they get an error on memory allocation, they wait a bit and try again. This gives the OS time to resize the swap file which means that the second try often succeeds.
I think the click bait is assumed. The use of "this one weird trick" is humorous at this point (edit: and I found the article actually interesting/worth reading, the title not really deceptive, which is usually the main problem with click bait. and... yes, it is actually kind of a weird trick?).
To expend on your summary, this trick is necessary on Windows (only), because it does not allow over-committing and also does not have an OOM killer which you can instruct to kill content processes instead of the main process.
> The use of "this one weird trick" is humorous at this point
Only when it's used ironically. If the article uses that as a headline and forces you to read the whole thing before giving so much as a hint what the "one weird trick" is, then it's legitimate clickbait.
Don't get me wrong, it's an interesting article. I just think if an article is attempting to humorously use a clickbait headline, then it owes it to the reader to at least add a subheading.
Why? By what measure is this not simple clickbait? Because it's a well-known pattern that has been used to bait people for years already? Because it's targeted at a tech audience? Because it's written by someone from Mozilla? None of these seem like reasons not to go with a more descriptive title.
Yes, exactly, because it's a well known trope by now, it's no longer deceptive and has a completely different meaning now than originally. Originally it was click bait, now it is a shared culture joke.
It's only clickbait if you claim that you have never seen it before. Do you wish to try to claim that?
The title is exactly the same both with and without the trailing extra few words, except one is dry and one attempts to be humorous.
You may for some reason find the attempted humor intolerable, but that's more about you than the article.
There is actually still a clickbaity element, but it's not anything you complained about, it's leaving out a few words to say "on Windows by avoiding overcommittting memory" or similar. Beginning a sentence and withholding the gist so the only way to know the end is to read the article, is indeed click bait.
I generally agree, though it occurred to me that the fact it is now a cultural joke may make you want to know what's behind it, making it a second-order click bait.
I agree but then that makes it "dual use" clickbait, since there's always less-savvy new people coming along.
Someday it will be possible to effectively filter all info that is presented on my devices, and along with anyone/anything mentioning Trump and Musk, overly-clever articles that sound like clickbait will be gone along with the real thing.
It is click bait. It's not really deceptive. It could have been more descriptive indeed. It's hard to summarize in a title though. Probably not impossible. What would you have written?
Anyway, everything fully worked as intended for me: I had some fun reading this title, expected something interesting coming from hacks.mozilla.org and got it.
Remember, this is Mozilla, the same folks who still think, after 20 years, it's the funniest thing in the world to give a cryptic "Zarro boogs found" message to users who are desperately searching the bug tracker to resolve a major software frustration, many of whom aren't native English speakers and won't get the ultra-hip reference.
I understand that some people might have found this cryptic, but they added an explanation, so it's not that cryptic anymore. This problem is fixed. It was not even that cryptic. The first time I encountered this, I was a child and not a native English speaker and I still got it. I find it nice that people try to keep a bit of history/fun.
Now, its not even the same people that are involved in this issue and this article. So no, its not "the same folks".
Anyway, if it's the worst thing you have to complain about Mozilla, we are good I think. People really have weird battles to fight.
Maybe they don't like it the same way I don't like the "aw snap!" error message I've seen more than one place. That one really, really, bugs me. Something just crashed, and you expect me to be ok with you treating it like a joke? Not to mention the assumption that everyone else even finds "aw snap" a natural kind of thing to say instead of about 11 black metro millenials. I think I could never even say the words with a straight face and certainly not be taken seriously if I ever did. I'm failing to articulate just how utterly wrong that error message is. And I am not one of that crowd trying to remove all the life from say the linux source comnents and logs. It's not about any deviation from absolute sterility, there is something very out of touch about that particular example.
>I understand that some people might have found this cryptic, but they added an explanation, so it's not that cryptic anymore.
Right. After about 15 years, they finally relented. So, your position is, "it's not a problem, but they deserve credit for fixing that problem that shouldn't have been fixed"?
>Anyway, if it's the worst thing you have to complain about Mozilla, we are good I think.
Where are you getting that? When Mozilla did something stupid, I pointed to a related, similar stupid decision, originating from the same cultural practices. From that, you twist my words into meaning that's the only thing there is to criticize? That's the mind of an ideologue, not an honest evaluation for truth.
But, if you want to stay in the mentality of automatically trivialzing every unforced error on the part of Mozilla, then, by all means, apply that same reflexive defense to these cases as well!
- Looking Glass: Forcing a cryptic extension on users that accustoms them to ignoring changes that look like software compromise.[3]
- The fact that, post-2016-update, we still don't have the same add-on functionality as before, including the ability to customize controls.
- Not allowing side-loading of unsigned add-ons "because security" even though Chome has long allowed this with no issue.
- Then neglecting to keep the signing key up-to-date. [1]
- Then making previously-signed add-ons stop working as a result of that, compromising user privacy, possibly causing users in hostile countries to be killed. [1]
- Then assuring users that, no, it's okay because we can remotely force updates via an opt-out feature buried in "studies".[2]
All of those are worse, so no, the shitty, confusing in-joke at the expense of frustrated bug victims isn't actually the worst part about Mozilla, it's just the most relevant to the OP's comment. But sure, it provides a convenient way for you to imply that nothing else is wrong with Firefox.
>People really have weird battles to fight.
I'm sorry, what? Wanting a universal, everyday-utility software product to be accessible outside of a small clique ... is a weird battle to fight? That mentality is exactly Mozilla's problem!
> Right. After about 15 years, they finally relented. So, your position is, "it's not a problem, but they deserve credit for fixing that problem that shouldn't have been fixed"?
I didn't say this. At all. My position is "this minor issue is fixed now and does not really deserve any attention whatsoever anymore, didn't much at the time neither [, let's move on to actual problems]". I'm not saying Mozilla is perfect and don't have flaws. They have many. Some of which you have listed. And including their main source of cash. But this "Zaroo bogs found" thing? I appreciate that you don't agree with me, but sorry, it seems so irrelevant! By the way, how many regular, non-technical users face Bugzilla? None that I know of, and I'm surrounded by Firefox users. Most users don't write bug reports. And Firefox's bug reporting system is rather nice anyway, compared to other systems. It's localized, clear to follow, etc.
> post-2016-update, we still don't have the same add-on functionality as before
For the better and the worse, we are never getting this back. I hope you are not holding your breath over it. It's not happening. There are good maintainability and security reasons for this. We have the right to not agree with this but we are not the ones who maintain the browser. I work for a company that allows its software to be highly user customizable, it's our strength but that's not free and it comes with its own issues. We are stuck with old tech forever and cannot move very fast. I personally believe the restrictions are for the best.
> Wanting a universal, everyday-utility software product to be accessible outside of a small clique ... is a weird battle to fight
I didn't say this. I want this too. But Firefox is definitely already accessible outside of a small clique.
> From that, you twist my words into meaning that's the only thing there is to criticize?
I didn't say this. I said "IF". My sloppy phrasing really meant "that does not seem a big deal, of all the issues you could have found around Firefox and Mozilla".
> That mentality is exactly Mozilla's problem!
First, I have nothing to do with Mozilla, and second, I believe Mozilla is actually successful at providing software everyone can use.
I'm a native English speaker and had no idea it was meant to be particularly funny or hip; I just assumed it was done kind of in-joke, common enough in FOSS. But it didn't seem very confusing since the next line explains: "We couldn't find any bugs matching your search terms. You could try searching with fewer or different terms."
In-Jokes are only ever going to be funny for those within that group; that's pretty much what defines them. Since I'm not in any Mozilla hacking group I wouldn't know whether it's so, or just a bit of silliness, but either way it doesn't add more than a one-time bump.
But is it? I wanted it to be tongue-in-cheek but the title is actually a rather accurate description of the contents of the article. This is a ~20 lines weird hack that we threw at the wall to see if it would stick... and it massively improved Firefox stability. I can't stress it hard enough, it's the largest, most visible stability improvement in the past decade.
More specifically - on the memory front - we spent lots of engineer/years working on improvements which sometimes would barely register, only for some weird trick make OOM crashes fall off the radar entirely.
Also I'd like to point out that we have no way to tell if this is working because we give Windows time to resize the swap file, if it's because other processes in Firefox die, or it's because other unrelated processes die, or a mix of all the above. It's pure speculation on our part.
Why don't more applications use this "weird trick" of just checking to see if memory allocation succeeded and waiting and retrying if it didn't, instead of unceremoniously crashing?
It's not like this hasn't been possible to do for many decades already.
Even better question is, if it is this simple, why didn't Microsoft implement their malloc or whatever like this in the first case. If the OS needs to resize the page file, why don't they just delay the return of the function until that is done?
I'm guessing the answer is something along the lines of "it makes applications freeze for <amount of time> and users don't like that".
Sure, they prefer their applications to just crash :-)
Bu yes, the problem is probably not totally simple. adding delay in malloc while application might expect an immediate return could cause bugs. But swapping is a thing anyway and introduces delays so I don't know why they would not do this.
> Also Windows should step in and say that this application is consuming almos all of the remaining memory
This is much more complicated that it appears, because the user could be intentionally doing that - if you have little memory (8 GB), you are sort of always almost out of it, if you have a lot but you are using professional apps like Photoshop, video editing, or doing machine learning with large datasets.
I'm sure that the reason Windows doesn't do this is because they couldn't figure out a good way to warn which is also not annoying.
There are some Windows warnings, I've seen them, but they only appear when the situation is so dire the computer is about to blue screen.
I disabled the swap in Windows, because I have 32G RAM (and why put unnecessary wear on the SSD). And now it runs out of memory while half the RAM is empty.
A marvelous operating system.
But yeah, at least I saw the warning fast, with the swap file it first came to a sudden halt, and after a lot of time it might have shown the alert.
Turning off swap will lead to running out of memory earlier than you’d think because different tasks can allocate a lot instantaneously - you just need a few of them to line up and try allocating a lot at once.
I don’t see how you can realistically operate a computer this way unless you control all the software on it, or you enjoy buying way more RAM than you need.
(Also swap isn’t the only alternative to allocating memory - there’s also purgeable allocations and memory compression.)
> but what if we could recover from this situation instead? Windows automatically resizes the swap file when it’s almost full, increasing the amount of commit space available. Could we use this to our advantage?
I expect followup by some annoyed blogger that notices swap file ate his system partition and he spent whole day debugging the reason of that...
Doesn't Windows still limit swap by default to like maximum 20% of the disk ? And that issue of running out of disk space because swap ate it isn't new, it really came into prominence when we switched from slow but huge HDDs to fast but small SSDs...
It's less boring this way and based on the article I think it would be difficult to include more information in the title anyway - the solution is simple but it follows multiple paragraphs describing the problem.
Sure, it could also just be "Improving Firefox stability", but that doesn't tell me the nature or scale of the update.
