
When did we stop caring about memory management? - matthewwarren
http://www.hanselman.com/blog/WhenDidWeStopCaringAboutMemoryManagement.aspx
======
jasonjei
I think this quote summarizes hackers today: ``[T]ime spent sweating memory is
time you're not writing your app. The hard part is developing the experience
is that you need to know when you need to care."

Not having to worry about memory is now a luxury we can afford in the age of
plenty. While writing an efficient spellchecker would have been a great feat
in the 80s, there's no point of that when you can load the whole dictionary
file in memory.

In other words, memory management is a #highclassproblem for many developers--
you worry about efficiency once you have a lot of adoption. (Within reason; if
your app is unbearably slow that isn't good.)

~~~
PavlovsCat
We also live in the age of plenty in regards to how many things we have
running at the same time, or how many tabs there are open in a browser. Things
can add up pretty quickly.

And why settle for "without optimizing it's slightly faster than things used
to be when people had to optimize", when you can have "optimized, it's orders
of magnitude faster than things used to be"?

> you worry about efficiency once you have a lot of adoption.

Sometimes things can be tricky to refactor if you didn't move carefully at
every step. Initially not worrying about efficiency at all might mean having
to rewrite from scratch once you do.

Of course, this can't be generalized. Sometimes it really, really doesn't
matter, unless you care about craftsmanship as a value in and of itself --
taking 10-20% more time to write code that makes you happy and proud might
very well make you _more_ effective in the long run.

~~~
fixermark
If you can sell it as faster than the alternatives, it makes sense to
optimize.

But in the web-focused marketplace, adding 6 months to your launch calendar to
get something fast to market may mean someone else's good-enough solution
becomes the entrenched, de-facto standard that you are now fighting to prove
you're superior _enough_ to that it justifies people's time cost to move their
data to your solution.

That's the risk tradeoff.

~~~
PavlovsCat
It really depends, avoiding some pitfalls isn't always that much more work,
but merely requires knowing (and caring) about them in the first place.

~~~
kylequest
It's easy to point out the places where you could've done a bit more
optimization after the fact. And, yes, sometimes those optimizations don't
cost an arm and a leg, but it's very easy to fall into the premature
optimization mode that kills the product and the company. Not everybody has
tons of funding in the bank. For some startups shipping a day later can put
you in the death spiral trajectory. Timing is a huge factor in a startup
success.

It's easy to think why not just do it right in the first place. Aside from the
timing you, as a startup, don't really know what "it" is or "it" is constantly
moving. Committing to perfection at that stage can be a sure way to kill the
company. Even if "it" is known and it's not changing you still need to keep in
mind the Gall's Law.

------
lordnacho
To answer the headline, we stopped caring when a bunch of off-the-shelf
solutions became available.

For a lot of problems, once you have this off-the-shelf solution, you're just
not going to have a memory issue. It's like cooking without having to manage
where the pots, pans, and plates are. Say you hire some guy to clean up for
you, and the guy just implements some simple strategy. It's a lot more
pleasant, and if you have a large enough kitchen, it won't make a difference
to the quality (and timeliness) of the food. In some cases you may have to
consider the guy running around, but most of the time you net win anyway.

For things that are truly performance sensitive, you still need to think about
cache issues and such, and you definitely still care about memory management.

~~~
mmsmatt
Complete agreement with the last sentence.

I can think of two projects in my past five years that really showed
performance sensitivity. I knew that at design time so I chose to care about
where data sat in memory and how it would behave in multilevel caches.

All other projects, couldn't care less where the kitchen dude stored my pots,
and he never lost one.

None of those projects jumped from performance agnostic to performance
sensitive overnight.

------
allard
If your closet or garage increased in volume at the rate of Moore's law, you'd
pay less attention to how you put stuff away.

~~~
colordrops
Especially if you had a robot that would organize and cleanup your garage for
you.

~~~
zanny
yc2017 anyone?

~~~
bobbles
It'd be a cool idea to sell warehouse storage with drone delivery of
individual items if needed, that way you dont need to care as much about how
much you can fit into your apartment at any one time

------
jordwalke
This article conflates two separate issues: Automatic memory management, and
Virtual Machine runtimes. People often make the mistake of thinking that some
sort of garbage collection strategy (such as reference counting[1] collection
(ARC) or tracing collectors) implies that you must be running in a VM (which
typically employ some kind of JIT to achieve reasonable performance). This is
not true - see Haskell, OCaml, SML, and Swift (for reference counting
collection).

In my experience, many of the issues with debugging performance issues are not
related to memory management, but understanding JIT heuristics which are
moving targets (especially in the dynamically typed languages).

I think we need to collectively separate the idea of some kind of automated
(or semi-automated) collection strategy, from a language level JITed Virtual
Machine. You may have one without the other.

[1][https://www.cs.virginia.edu/~cs415/reading/bacon-
garbage.pdf](https://www.cs.virginia.edu/~cs415/reading/bacon-garbage.pdf)

~~~
aidenn0
I reread the article after seeing your post, and could not find where they
made this conflation.

I've not worked on performance critical code running under a JIT, but have
worked on performance critical code running under a GC, and having an intimate
knowledge of memory can be vital to understanding the performance of your
application.

~~~
jordwalke
The article divides programming languages into two groups: Those that require
you to manage memory manually and those that rely on language level managed
memory. However, the examples of the managed memory languages also rely on
language level Virtual Machines and/or JITs (which are critical to achieving
excellent performance in those respective languages). Then, the article points
out that the people using these "garbage collected languages" (which are also
JITed (in a VM likely)) tend to have problems debugging performance issues. If
he's right, then the examples aren't making the case very convincingly because
the realm of languages with garbage collection isn't divided granularly enough
in his analysis. The result is attributing the problem to a common language
feature that they all share, without acknowledging that they also all share
other very important features.

I'd never claim that understanding memory usage/allocation/freeing isn't
important. It's important to understand how memory is used whether that be
understanding how your collector handles generations of memory in a GC'd
language, or whether it be understanding how your programming patterns cause a
waterfall of destructor chains in a manually managed language. Often,
understanding these things is difficult.

My claim was that, in my experience, the _biggest_ difficulties in debugging
performance were in understanding JIT heuristics in language level Virtual
Machines. This is because (especially in dynamic languages) the heuristics
aren't documented anywhere, they're subject to change, and are likely not
consistent between particular runtimes you're hosted within. (Those
optimizations you made just for V8, could even _hurt_ perf if you run your
library in JSC).

There's also the issue with language level virtual machines in that they
usually impose some system wide tax on many operations (a layer of
indirection), so I could see the people mentioned in the article having a hard
time squeezing out that last portion of performance due to that indirection -
again, that might be the fault of there being a language level Virtual Machine
- and not necessarily as much related to memory (it depends on your
application, and amount you allocate, of course).

In my experience, optimizing collection behavior has been relatively
straightforward in comparison and the techniques you learn for one collector
seem to translate to others. Perhaps my experience is biased towards JIT/VM/GC
in dynamic languages where the JITs are so much more complex in comparison.

I have seen core hot paths in critical client side JS libraries be slowed down
9x because of obscure JIT deopts caused by properties names beginning with a
dollar sign. I'm not just talking about a 9x slowdown of field lookups - our
entire end to end benchmarks tanked (since fixed). Perhaps the VM architects
decided that something with a leading dollar sign would not likely benefit
from being made into a hidden JIT class? I've never had anything so obscure
happen with any kind of garbage collection, whether that be reference counted
ARC, or a tracing collector. Edit: spelling

------
pcwalton
This article has a hint of "C-style memory management is good for
you"—implying that you might want to write in something like JS and then go
back and write in C++ in order to "tighten the screws". I think that's
wrong—by and large, it's been a very good thing that people have stopped
caring about memory management in production. That's because correct C-style
memory management never† scales up to large software projects in a way that
doesn't result in crashes and, worse, vulnerabilities.

So, yes, we've traded performance for automatic memory management. But we've
gained reliability, safety, and security. For the vast majority of server-side
apps, that's a worthwhile trade.

† Unless you use the processes like NASA and the like use, which for most apps
are infeasible from a cost point of view.

~~~
narag
_Unless you use the processes like NASA and the like use, which for most apps
are infeasible from a cost point of view._

LOL. I have to ask: do you "know" this from experience or someone else told
you?

Thousands of applications were made not so long ago with manual memory
management. Many of them are still the most used desktop apps in the world.

Methods to secure them are not rocket science, really. I would gladly concede
it's cheaper to do CRUD apps with GC, but you're going hyperbolic with NASA
;-)

~~~
prodigal_erik
>Thousands of applications were made not so long ago with manual memory
management.

And pretty much all of them are unreliable garbage that we as an industry
should have been too embarrassed to ship.

~~~
narag
_And pretty much all of them are unreliable garbage..._

So you have experienced thousands of unreliable garbage apps. I feel really
sorry for you. My experience has been the opposite: lots of wonderful and
reliable programs. And when I found one that wasn't good, I stopped using it
and replaced it with a better one.

I mean as a user. As a programmer I've seen all kind of programs, from
"critical mission" to "quick hack" and from excelent to unreliable garbage.
And you know what? It's not language features but ludicrous incompetence what
makes an app unreliable garbage. There are programmers out there that can make
unreliable garbage in any language.

The "secret" to make some app reliable is not reserved to NASA and the like.
Follow two simple patterns:

1\. For every user action, validate input, try executing the code, catch the
occasional exception and show an error message.

2\. For every resource, allocate, try{ using it } finally { release it }

Put each of these skeletons in a function or method, so the resource is a
local variable (no colateral effects) and it's easy to debug.

That's all. Well, also: don't be too "creative" unless you know what you're
doing (most times you don't).

I'm fifty-something, spent a lot of painful time mechanically applying that
recipes to rotten codebases, and when I read the kind of criticisms you made I
assume they come from people too young to have real world experience on what
they're criticizing. The alternative is worse: to be the kind of people that
created that rot and then blamed the language.

Just in case: I don't believe that manual memory languages are "better". I
love Python and (most of the time) JavaScript.

~~~
pcwalton
> I read the kind of criticisms you made I assume they come from people too
> young to have real world experience on what they're criticizing.

Actually, every single year I've worked on security-critical software I've
understood more and more how impossible it is to secure large C and C++
codebases worked on by large teams with just "careful coding". The picture
changes a lot when you have people actively searching for vulnerabilities.

~~~
narag
Please, reread carefully your initial comment and the one by prodigal_erik
(the two comments I've answered to in this thread) and you will see that
you've made a gross generalization, while now you're narrowing the field you
meant to apply your words.

Do you believe that security software or browsers are better writen in other
languages than C/C++? I won't dispute that, even if I'd have some questions
about relative merits. Please think of some well known Perl and PHP
applications and their security record.

Anyway the unqualified:

 _That 's because correct C-style memory management never† scales up to large
software projects in a way that doesn't result in crashes and, worse,
vulnerabilities._

... and talking about manual memory management in general...

 _And pretty much all of them are unreliable garbage that we as an industry
should have been too embarrassed to ship._

... are simply not true. Worse, they're insulting for everyone who has worked
in this kind of application.

~~~
pcwalton
> Do you believe that security software or browsers are better writen in other
> languages than C/C++?

Yes. And lots and lots of apps (most public network facing things) are
security sensitive.

> Please think of some well known Perl and PHP applications and their security
> record.

Nobody is saying that "memory safe == no vulnerabilities". I do think it's
true that being memory unsafe results in _many more_ vulnerabilities. I've
measured this precise statistic in our bug tracker.

> That's because correct C-style memory management never† scales up to large
> software projects in a way that doesn't result in crashes and, worse,
> vulnerabilities.

It's true. And I work in native code myself!

It's only an insult if you presuppose that introducing vulnerabilities
reflects badly on the programmer who introduced them. I don't believe that;
most major vulnerabilities I've seen were introduced by very good programmers.
It's a problem with the tools they're working with.

------
rdtsc
Needing to think memory management turned out be a bit un-expected you might
say. There were a few things that went well and few that were a bit
unexpected.

Things that went well:

1) RAM got ridiculously cheap: 2x 8GB is now what $60-70? So it would seem, in
that regard, yeah you shouldn't care about memory.

2) GC languages and runtimes. Believe or not there was heavy skepticism that
GC runtime could be ever be a serious strategy in practice. It was an exotic
technology that many thought would left for academia and fringe research area.
But most popular frameworks and languages today are GC based.

Some things were a bit un-expected:

1) Memory access speed is fast but it didn't keep up with CPU speed. A cache
miss will costs hundreds of nanoseconds. CPU speeds may have been doubling
every 2 years or so, but memory access speeds have only been increasing by
10-20% or so.

In the 80s and 90s you just didn't care as much about cache hits and misses.
But those are the forefront of performance optimization strategies. "Align
your struct members","use structs of arrays instead of arrays of
structs","think about cache lines and strides when you loop through your data
arrays..." etc.

And because CPU speeds go so much faster, memory access also stands out on
performance charts. So GC languages and runtimes can still seems slow and
sluggish for applications that don't pay attention to memory.

2) Applications got more concurrent and servers are expected to handle a
larger number of clients / connections / requests. That means each one needs a
context, so it needs memory. So memory is again at the forefront. Accessing it
and its latency, GC pause times are important.

3) Mobile happened. Mobile devices have proliferated so the needs for fast,
small footprint applications never went away. Even as there are serves witih
terabytes of RAM around, there are even more little arm cores and then
everyone talks about IoT. That means langauges/runtimes which compile ahead of
time will never be 100% displaced. Heck even on Android, ART (AoT copiling) is
replacing JIT.

~~~
userbinator
_Align your struct members_

On modern CPUs, cache misses take so long and unaligned access is nearly free
in comparison, so that padding can become detrimental. Of course ordering
members so that they're all aligned is the best option, but if that can't be
done, it's better to have misaligned and pack more in the same space than
leave wasted padding.

~~~
firethief
The results of unaligned access can vary greatly, even among the x86 family.
You can hit a large performance penalty if one of your fields straddles a
cache line boundary, and $deity forbid you cross a page boundary on a Core2.

Under the right circumstances packed structs can improve performance, but they
come with a lot of technical debt since they can cause sporadic extreme
slowdowns on certain common CPUs, and performance problems, aborts, or even
data corruption on non-x86 platforms.

------
andmarios
Memory management still matters but by how much differs based on your
specifications. Are you programming for a microcontroller? For a low end VPS?
For a low-volume high-turnover web app? Do you have a deadline? How many
people should be able to come up to speed and contribute to your codebase?

The author's example is interesting. He talks about a webserver that after
optimizations can reach 1.2 million requests per second. In reality though,
your gigabit ethernet interface can't handle that much anyway.

------
georgefrick
As I sit here on Sunday afternoon, trying to get a Phaser game to run on an
iPad 2; I read this and think "We haven't stopped caring, we care when it's a
specified business requirement".

This bites us in the ass, because I can't make Wistia (for example) stop
eating all the memory on this thing with my app running next to it.

So, personally, I stopped caring when it was not a priority and didn't, on
average, affect user experience. But; it was a mistake and I am watching
memory/etc more and more these days. This is for various reasons (old
hardware, more than one app on a page, growing amount of bundled media
content, etc).

~~~
jasonjei
You've just reminded me of how much I hate iTunes and the CPU/memory hog it
is... And the other app I'm forced to use, Xcode...

~~~
georgefrick
No warning, no error, mobile safari just kills the tab. It's not fun. Optimize
images, optimize audio, stream audio, stream images, reduce text, pack text,
pack everything. Repeat, repeat, repeat. Still crashes. How do people work
with Apple stuff? I got a macbook and xcode, it lets me view logs/etc, but
nothing helpful.

------
72deluxe
C-style memory management is important to know about but in C++ land, RAII and
C++11 features are here to help. Move semantics, use of containers, references
etc. all mean many of these bugs are a thing of the past.

In addition, extra tools and profiling / static code analysis has helped me to
write better code.

~~~
0xcde4c3db
The language has gotten to a pretty nice place, but the ecosystem is nowhere
near caught up. A lot of APIs still haven't even incorporated the idiom of
passing const references, almost exclusively taking pointers to non-const and
defining their own idiosyncratic ownership schemes. Qt4 is a good example:
RAII of widgets is hazardous because windows/containers will delete their
"children" (i.e. things that are under them in the widget hierarchy, not
members), even if they're on the stack. This means that simply declaring local
variables in the wrong order can cause a double delete. I encountered this in
tutorial code on the Qt site, which kind of put me off choosing Qt for new
projects.

~~~
72deluxe
This is very true. What did you use instead?

wxWidgets is the same - when a wxWindow gets deleted, it will remove its
children in the view hierarchy, which are likely not children of it as data
members. I think a lot of this comes from the fact that underneath native
pointers are being used (eg handles on Windows) and widgets/controls are not
necessarily going to be data members of a parent. It also means you can
reparent widgets/controls to other GUI items at runtime. I do this (for
example) when switching notebook pages to rehouse a GUI element in order to
save RAM; I just assign it new data to display and it means less RAM usage and
creation time for each notebook page.

For your own data controls, we can still ensure that they clean themselves up
alright and safely, just use pointers for the GUI items. In some sense, the
GUI toolkit takes care of deleting its own members, eg. wxWidgets will delete
all of a parent wxWindow children at teardown, so it isn't a leak situation.

~~~
0xcde4c3db
> This is very true. What did you use instead?

Since I was looking at it for hobby/educational projects, I basically decided
to not do something with a GUI. I think I had also looked at wxWidgets and
FLTK, hoping that one of them was cleaner and more modern; FLTK managed the
"cleaner" part, at least. The last time I looked at C++ GUI libraries it
seemed like JUCE was the most promising, but it also had a bit of weirdness
(GPL license without linking exception, heavy bias toward audio applications).

------
zoul
I just don’t understand why there aren’t more languages using automatic
reference counting as featured in modern Objective-C and Swift. ARC has very
low overhead and predictable performance profile (no GC waking up to choke
your app for hundreds of ms). I think it makes almost no sense to consider
going back to manual memory management because of performance, not even in
games. Unless you are writing for some quite limited embedded hardware.

~~~
tamana
Is ARC actually safe from memory leaks?

CMS GC doesn't choke your all for 100ms

~~~
setpatchaddress
No, you can still have memory leaks with ARC.

In practice, the GC Apple was using prior to ARC could collect cycles, but it
consumed a nontrivial number of CPU cycles compared to ARC.

Apple's tools people seem to be unconvinced that GC is efficient enough for
production use on slow CPUs.

------
pslam
I never stopped caring. It just turns out it never really mattered for a large
fraction of cases.

There's still plenty of cases where you absolutely do care: low level
firmware, kernels, absolute performance, and high reliability. It's more a
case of not worrying about it when there's much bigger risks in a project than
having loose guarantees about allocation performance or fragmentation.

------
bsaul
Developping on smartphone and tablet makes you still caring a lot about memory
management if you don't want your app to be killed by the os... I guess the
question was asked to server side web developpers...

------
PavlovsCat
> How useful is it to know about C-style memory management when you're a
> front-end JavaScript Developer?

Try making your own particle engine and a simple game running at 60fps. I
guess "avoiding GC" is only a very small subset of memory management, but if
you're a noob like me, it surely will open your eyes about some things you
would never have guessed could cause problems.

~~~
fixermark
JavaScript is a brutally poor target for that application space though. It'd
be better to target Unity and their web plugin; that's designed to be a game
engine.

~~~
PavlovsCat
That's why you learn so much about more efficient JavaScript when you do it
regardless.

------
shmerl
Some didn't stop caring about it. It's all about how much you care about
performance, and whether you think no garbage collection is preferable (or
not).

 _> How low should we go? How useful is it to know about C-style memory
management when you're a front-end JavaScript Developer?_

There are things in between as well (such as RAII). That's about using it.
Usefulness of knowing it is another thing. Someone will have to implement
things on the lower level either way. I.e. garbage collector itself is going
to manage memory after all.

------
chewyfruitloop
Enterprise applications moved to Java on purpose to lose the issues that
memory management causes. In my last job I first came in contact with people
who had never had to do any type of memory management, that was only 8 years
ago (that guy probably has his PhD by now and will be Dr Coole...seriously).
If you can stand the speed of GC languages then, there's a whole bunch of
savings you get in development time and especially in support.

~~~
xxs
As a matter of fact GC is godsend when writing concurrent code - there is no
ABA issue.

------
frik
So many were wrong that by Moore's law we will have plentiful ram and
inexpensive CPU to run bloated Java or C# programs. In 2016, desktop
applications are still 99% C/C++ and fast mobile apps in C/C++/Object-C.
Languages like Rust and leaner new languages with JIT that care more about
resources like Go, Julia, Lua, Swift, PHP7, Hack or even newer iterations like
C++14 are certainly good way to go.

~~~
pcwalton
1\. I don't see any reason why Lua (presumably you mean LuaJIT?) and PHP 7
would be faster than Java or C#. The compilation techniques are the same,
except that Java and C# have more mature JITs.

2\. Java and C# JITs perform many more optimizations than Go 6g/8g do.

3\. Objective-C message passing is a lot slower than method calls in Java and
C#.

~~~
chc4
Not defending the grandparent's post, but LuaJIT _is_ actually faster than
Java or C#. Hell, it's faster than raw C in certain cases because it can
optimize better due to being a simpler language. Mike Pall is a god amongst
man.

I think it's due to LuaJIT being one of the very few tracing JITs that are
still in development, since they are notoriously difficult to write and
maintain.

~~~
xxs
Tracing compiler might add a bit of performance in the naive cases where the
developers don't know how the JIT optimizes.

If I care about performance I'd check the generated assembly and modify
accordingly till I get decent result.

Baring that: do you have any reputable source claiming LuaJIT outperforms
C#/java/C?

~~~
chc4
Unfortunately the Benchmarks Game seems to have removed LuaJIT in favor of Lua
5.3, but [https://stackoverflow.com/questions/2588071/is-luajit-
really...](https://stackoverflow.com/questions/2588071/is-luajit-really-
faster-than-every-other-jit-ed-dynamic-languages) mentions it as being an
order of magnitude faster. I've seen it said multiple times that LuaJIT is the
gold standard of JITing, but can't find very many concrete benchmarks anymore.

~~~
xxs
Anything saying 'order of magnitude' can't be taken seriously.

That would imply a horrid baseline in order to get it that fast. Merely I
don't believe a tracing JIT would have a great impact in a real world
scenario.

------
forbin_meet_hal
Having worked in the memory chip business for 10 years: It's cheap!! Hell, for
a while, companies were dumping memories below cost.

I remember a Fry's newspaper ad that said something like: "$50 1GB DIMM, $0
after in-store discount. (Limit 2 per customer.)"

Kind of similar to the old (and possibly apocryphal) Sun Microsystems legend:
Frustrated that too many domestic programmers were spoiled with ample CPU
speeds, they hired a bunch of Russian programmers who dealt with the export
restrictions on computing gear and thus had learned to write tight, elegant
code.

------
eva1984
Since it is a web server we are talking about here, let's assume it is the
backend space :)

I guess the answer is obvious: it is nice, but might not worth the effort. 1.2
million requests/s is pretty good, but in reality, you probably would still
need 2-3 other hosts as a failsafe backups, let alone there are other factors,
like memory constraint, in which case, less optimized solution might be just
acceptable because it is sufficient. So under common usage cases, the need for
scalability outweighs the urge to squash the maximum performance from a single
machine.

~~~
kasey_junk
Just to be clear kestrel benchmarks at 1.2 million requests per second. You'd
need 10 of your servers to match 1 of theirs.

That kind of scalability difference does still matter.

~~~
eva1984
Thanks for pointing that out. It did seem like that could make a difference by
this metric alone in some scenario.

------
mgrennan
Software growth is not keeping up with hardware growth.

My parents told me to "Clear your plate." That was because they lived through
the depression. I remember writing code in assembler and 64k and feel the say
way now. Programs I wrote in that space now take 64m and I think "What a
waist".

------
nroose
It seems to me that we have been doing this more and more as memory has gotten
bigger and bigger, we care more and more about performance, and more and more
people are writing more and more code that will be used for shorter and
shorter times under tighter and tighter deadlines.

------
Tiquor
When we got the wonderful blend of plentiful ram, inexpensive CPU and storage
along with enough variety in languages and separation of architecture concerns
that not everything you built had to care about memory.

------
wonkaWonka
First with Smalltalk, then with Java, and eventually Objective-C as well.

------
redwood
Premature optimization is. ..

~~~
ansgri
.. is 50% chance to die by thousand paper cuts, 25% simply wasted time, AND
25% chance to avoid death by thousand paper cuts.

At least in image processing and robotics, from my experience.

It's very annoying when you have optimized all obvious hotspots 10x only to
get 3x speedup because the remaining 30% code consistently abuses « simple
ways », e.g. return big objects by value, use std::vector instead of proper
Eigen arrays, constantly serialize objects everywhere, etc.

