
Tmpfs considered harmful - rwmj
https://rwmj.wordpress.com/2012/09/12/tmpfs-considered-harmful/#content
======
praptak
Minor disagreement about the following: _"Everyone must worry about whether
their files need to survive a reboot"_

I believe there was never such expectation of /tmp. On many systems it's being
wiped periodically or even when the user's last session ends. The example
app(s) that use /tmp for drafts need(s) to be fixed.

~~~
rwmj
Sure, but if the power goes down, there's at least a _chance_ that I may be
able to find that long email I was writing. With /tmp on tmpfs, it's gone.

~~~
colanderman
That's why the GP considers mutt broken. Drafts should be stored in ~/.mutt or
/var, not /tmp. Files in /tmp should not be expected to survive a process.

~~~
rwmj
This is exactly the point of the posting. Every program has to be examined for
this problem, and it forces a complex decision about the storage hierarchy
onto every programmer.

~~~
saurik
That's like saying "RAM considered harmful", because every time a developer
has to think "do I just store this in memory, or do I sync it to disk" "it
forces a complex decision about the storage hierarchy": /tmp is simply RAM
that has the API of a file; it used to be implemented using a horrible kludge
with files on disk that were periodically deleted, but with tmpfs the dream
became a reality.

~~~
prodigal_erik
RAM really is a harmful kludge. Neither Turing machines nor lambda calculus
call for some of an algorithm's state to suddenly be obliterated. The only
reason we put up with it is that disk and SSD are so much slower.

~~~
rwallace
Speed is not the only benefit of RAM. The internal data structures of most
programs are gnarly messes of intertwined pointers, fragile, easily corrupted,
hard to upgrade in place, almost impossible to fix when they break. The
division between RAM and disk forces everything to be serialized at intervals
to clean storage formats that can be reread for a fresh start. As a practical
matter, that's a big enough benefit to justify enforcing the division at an
API level even if disk were as fast as RAM.

~~~
prodigal_erik
The industry might produce less worthless crap without the crutch of "it's not
quite unusable if you start over often enough".

------
raverbashing
"Everyone must now be careful never to store a file in /tmp that might grow
large"

If you're going to trash my /tmp with gigantic files please don't. Really

"it’s better to fix the filesystem to make it faster"

Yes, let's spend adding more complexity to the file system to work out every
/tmp abuser out there. But his original point was _avoiding adding complexity_
in the first place

So basically, he contradicts himself.

Mounting tmp as a memory backed fs is great (to which there may be better
options than tmpfs), avoids (potentially) spinning up disks covered in rust
for minor tasks.

~~~
RobAley
>If you're going to trash my /tmp with gigantic files please don't. Really

I write statistical programs that often generate temporary intermediate files
of multiple GB size, which need to persist at the end of runs for trouble-
shooting, partial re-runs etc. i.e. the software shouldn't delete them when
it's finished, but they don't need to be kept long term. I kept forgetting to
delete them when I finished a session and my disks kept filling up, so I
decided to use /tmp and let the OS manage the space. Is there a better way to
manage large tmp files than this that I'm missing?

~~~
mpyne
Given that they should persist, the better way would be /var/tmp I would
think, at least if persistence means "should survive reboot".

Even before shifting to tmpfs I'm pretty sure Fedora was erasing the contents
of /tmp at startup (and I know Gentoo/OpenRC is).

There might be an even more appropriate FHS-compliant directory (/var/run
perhaps?)

~~~
RobAley
Apologies, by persist I meant just beyond the end of that run of the software
(i.e. the software itself couldn't manage/delete the files), to see what went
"wrong". Deleting at reboot is perfectly fine.

/var/run "This directory contains system information data describing the
system since it was booted" which doesn't seem to fit my use case that well.
Not to say there isn't another directory that does fit, but I haven't found
it.

------
forgotusername
9 times out of 10, tmpfs is explicitly _not_ for use mounted as /tmp. The only
obvious exception I can think of are cluster nodes where the workload is
specialized and the node has no local storage whatsoever.

I guess it only takes one silly person to connect "tmp" in the name to "/tmp",
and the rest is history.

~~~
justincormack
sysfs is mounted on /sys

debugfs is mounted on /sys/kernel/debug

securityfs is mounted on /sys/kernel/security

So tmpfs is mounted on /tmp? Can see that...

------
gizzlon
Linkbait headline. Thought it was going to be about a gotcha with tmpfs in
general, but it's only about _one_ specific use of tmps in _one_ specific -not
even release- version of _one_ linux distribution.

I get that the guy want's attention for this life-and-death issue, but
linkbaiting is just mean :(

------
notatoad
Why on earth would anybody ever use tmpfs for /tmp? If I'm writing a program
that deals with a big file, and I want to temporarily write something to the
filesystem, it is because I _dont_ want it in memory. Why would you ever write
to /tmp for any reason other than to get something out of ram for a while?

~~~
rogerbinns
I have been using tmpfs for /tmp for many years (since 2005 IIRC). Note that I
do have large amounts of swap space (typically 64GB).

One simple benefit is that files there do not get caught in backups - they are
temporary detritus so I don't want backup space wasted by them.

The major benefit is that fsync is really slow on Linux. Typically it turns
into a sync, and a lot of programs like to ensure you don't lose data by being
fsync happy. For temporary stuff that is even more painful - you don't care
about data loss - that is why it is in tmp in the first place.

TLDR: I prefer filesystem operations on transient/temporary files and data to
run at the speed of RAM, and not end up in backups or waiting for syncs.

------
buster
Oh, i hope they revert this change!

The big trouble for me would be "Everyone must now be careful never to store a
file in /tmp that might grow large". I often use /tmp for all kinds of
temporary storage. Be it a small text file or a GB big tar file.

~~~
ps2000
Rare use case...

I remember CD burning softwares and torrent clients asking for special temp
directories to be used for that.

Allowing every day applications to create GBs of tmp files would mean you
would need to worry once your partition has less than 100 GB free.

However, nobody stops you from mounting /tmp anywhere else or even disable
tmpfs.

~~~
buster
Very common for me, not so common for you. ;) And: when is use /tmp on my
laptop, it gets cleared after reboot, so that's perfect for _temporary_
files..

------
dmpk2k
_One of the hardest parts of writing a compiler is register allocation._

Not quite. There are many much more difficult problems in a competitive
optimizing compiler before hitting that step.

You can get reasonable results from even a linear scan allocator, which is
simple.

~~~
aidenn0
It really depends on the architecture. There are some architectures with only
1 register, and others with many thousands of registers.

Current x86 implementations do a good job of being fast even when registers
spill.

I've seen dual-issue Power cores (32ish registers) that only have 16k of 4-way
dcache, which means register spills can easily kill performance.

------
alexchamberlain
It makes sense in certain cases to use tmpfs for /tmp. However, I'm not sure
this is on desktop machines, which (generally) have lower RAM than servers
thrashing their /tmp directories.

~~~
noselasd
Even in that case, using tmpfs on /tmp sounds like band aid. If there are
applications that suffer performance because stuff it puts in /tmp , then
change those applications - possibly by using an application specific tmpfs ,
or a new system wide tmpfs that doesn't overload something as ubiquitous /tmp

~~~
rwmj
The systemwide tmpfs already exists: /dev/shm.

------
snorkel
Bad title. tmpfs is good. Fedora maintainers assuming that all of /tmp should
be in tmpfs is bad.

Apps are already in the habit of writing a lot of expendable garbage to /tmp
that doesn't require the fast I/O of tmpfs. Clogging RAM with this junk is bad
idea.

------
donavanm
Here, let me man 7 hier that for you:

/tmp This directory contains temporary files which may be deleted with no
notice, such as by a regular job or at system boot up.

------
keeperofdakeys
> "Everyone must now be careful never to store a file in /tmp that might grow
> large"

Be warned that when he says this, he doesn't mean it will eat all your memory,
it will only grow upto a maximum size (by default half of your ram). When this
is reached, it acts like any other filesystem that is full: you get write
errors. There is actually another implementation called ramfs, which doesn't
have such a size restriction. However, tmpfs is what is actually being
implemented here. The ramfs itself is more of a 'toy' filesystem, also being
quite limited, including not being able to use swap space (unlike tmpfs).

------
tsahyt
Running out of swap and memory is one thing, although unlikely considering how
much of that stuff we've got nowadays. I for instance have 8gb of main memory
and 10gb of swap space. 300mb are used after system boot (still too much).
That leaves me with plenty of space to "waste" on temporary files in a tmpfs.

The Linux FHS explicitly states "Programs must not assume that any files or
directories in /tmp are preserved between invocations of the program." That's
not only reboots. It's _not meant_ to be used to store drafts or anything that
you might want to reopen at a later time. That's what we've got /var/tmp for.
The FHS specifies as follows: "The /var/tmp directory is made available for
programs that require temporary files or directories that are preserved
between system reboots. Therefore, data stored in /var/tmp is more persistent
than data in /tmp".

Tmpfs isn't harmful because of this, it's the default mutt configuration which
is broken. If developers would have adhered to the standard the problem
wouldn't exist. Since programs _should_ adhere to the standard, because
decisions like the one in question are made upon it, they _should_ be reviewed
and their default configurations _should_ be changed in order to fix this
issue.

------
armored_mammal
My theory on this is that it's intended as a step to avoid SSDs writes.

~~~
chimeracoder
Exactly - this is a major benefit of using tmpfs on /tmp

------
peterwwillis
Example of exactly when to use hard disk and when to use tmpfs:

User browsing the web for a month without rebooting. Flash stores temporary
files in /tmp/. For a month straight, Pandora is pumping out songs to /tmp/ as
a cache. 43829 minutes in a month, imagine an average of 3 minutes per song
that's 14609 songs, at an average 1.5 megabytes per song that's 7304 megabytes
that's 7.13 gigabytes. Probably shouldn't be stored in RAM.

On the other hand, you have ganglia clusters of about 2,000 hosts. You have
multiple gmetad's collecting data from different data sources and pooling it
into your web interface box. It's updating ~10,000 tiny files every 30
seconds. Your i/o wait and system time is through the roof. You move the place
gmetad writes to a tmpfs mount, and rsync to the disk every minute. Suddenly
the i/o and system times are at 0.01%.

Tmpfs _for /tmp/_ is considered harmful, but extremely helpful in other cases.

------
dsr_
The fact is, is that not every app that uses /tmp does so in a safe manner.

As long as that is true, it's not a good idea to use tmpfs for /tmp.

It's perfectly reasonable, however, to use tmpfs for, say, /var/local/tmp and
let individual users point $TMP at that. Want to live fast and dangerously?
Set $TMP.

~~~
alexchamberlain
This assumes processes use $TMP.

------
spartango
I'd disagree with the more global point (in the title) that tmpfs is harmful:
it's actually quite handy for all sorts of things.

As an amusing example, the SHM (shared memory) system in linux is actually
implemented on top of tmpfs (/dev/shm is a tmpfs mount).

Other uses include CoW shadow mounting (when unioned to an immutable
filesystem).

That said, the point that /tmp should not be a tmpfs mount is actually a
decent one, just because we're used to the idea that /tmp is neither space-
constrained (it's scratch space) nor extra performant.

------
krakensden
I'm surprised no one is chanting "CADT" and linking to "What killed Linux on
the desktop". This sort of subtle breakage over time is absolutely goddamn
maddening.

------
binarycrusader
Solaris has done this for many years; it's not the end of the world as the
author believes.

------
asdfaoeu
What's wrong with swapfiles?

~~~
keeperofdakeys
It should be mentioned that swapfiles are accessed directly, going around the
filesystem. They can suffer from fragmentation if you don't create them when
the filesystem is fresh, however.

------
scotty79
tmpfs is awesome to run rails from (i.e. for development).

IO bound becomes CPU bound so you're getting as good speed as you can get.

