Enabling overcommit machine-wide is a puerile, broken approach that not only converts your server to an unreliable toy, but encourages other idiots to rely on the same broken behavior in their libraries, language implementations, and so forth, basically leading the current plethora of collection libraries that don't even bother to monitor their own memory use or check malloc's return. It is software engineering plague, a rot on the underbelly of allegedly-solid code. oomkiller's unpredictability causes any number of problems in actual production environments, usually by killing the wrong process, and secretly ripping the stability out of programs whose code does check malloc's return. The answer is:
Which restores classical semantics and allow processes to identify memory allocation failures and respond to them responsibly in a number of ways (garbage collect being an obvious one, clean, safe exits after logging being another).
Now, if we could say that a specific process was allowed to overcommit because we could guarantee it would use the bogus memory allocation, then we'd have something vaguely useful.
And after following this advice, you end up with a system that can fail to fork() even when half of the computer's memory is free. This can also turn your once-working server into an unreliable toy.
(Also, see the comments in the original article that talk about vm.overcommit_memory=2 not actually doing what it claims to do...)
I was under the impression that fork() basically set up copy-on-write pages for the child process, and thus if it exec()'ed it would lose all those pages anyways and start fresh. Are those VM settings changing that behavior?
It wouldn't affect the copy-on-write optimization or exec(), it just causes fork() to return an error earlier. 'vm.overcommit_memory = 2' attempts to guarantee that after a fork, all of the copy-on-write-pages can still be dirtied (i.e. causing a copy) without the system running out of RAM.
Good luck making sure that all your running programs use vfork(). And why would they? If you read the man page, it is hardly encouraging:
It is rather unfortunate that Linux revived this specter from the past. The BSD man page states: "This system call will be eliminated when proper system sharing mechanisms are implemented. Users should not depend on the memory sharing semantics of vfork() as it will, in that case, be made synonymous to fork(2)."
some apps also use fork and the copy on write behaviour deliberately to implement features. redis uses it to create its backup file (http://redis.io/topics/faq)
>> Redis background saving schema relies on the copy-on-write semantic of fork in modern operating systems: Redis forks (creates a child process) that is an exact copy of the parent. The child process dumps the DB on disk and finally exits.
It's possible to use safely, and the benefits are worth it --- no commit charge problems and no time spent copying page tables. (Even in the memory itself is copy-on-write, you still have to set up the child's address space.)
I'm sick of people cargo-culting ideas like "vfork is bad" without really understanding the issues.
If you run a mission critical server on Linux, then you need to engage your brain and understand the requirements of your workload. That's what system administrators are paid to do.
The question here is what are suitable defaults for non-critical desktop uses of Linux, given that there will always be limits on the amount of RAM we can put into a machine and/or badly behaving processes written by CADTs.
Enabling overcommit machine-wide is a puerile, broken approach that not only converts your server to an unreliable toy
Why should the reliability of your server matter (beyond a certain point)? For years Erlang developers have been following the "let it crash and a supervisor will restart it" model. They seem to have the uptime numbers to back them up.
Because erlang is also built around that concept. Often when a server fails it needs to be brought up by hand and then wait until it can be brought back into rotation (depending on how it's used &c).
Additionally, some applications, like a database often only run on a single host (sure you have hot spares, but fail over is often manual and recovering is defiantly manual).
So while I get your point, we're not going to throw out everything we have simply because it wasn't built around "let it fail".
{ echo 'vm.overcommit_memory = 2' ; echo 'vm.overcommit_ratio = 100' ; } >/etc/sysctl.d/10-no-overcommit.conf
Which restores classical semantics and allow processes to identify memory allocation failures and respond to them responsibly in a number of ways (garbage collect being an obvious one, clean, safe exits after logging being another).
Now, if we could say that a specific process was allowed to overcommit because we could guarantee it would use the bogus memory allocation, then we'd have something vaguely useful.