
Rob Pike: the origin of dotfiles - keyist
https://plus.google.com/101960720994009339267/posts/R58WgWwN9jp
======
keyist
Instead of putting a dotfile or dotdir in the user's home directory, do follow
the XDG Base Directory specification:
[http://standards.freedesktop.org/basedir-spec/basedir-
spec-l...](http://standards.freedesktop.org/basedir-spec/basedir-spec-
latest.html) .

It's easy to understand and requires only a marginal increase in effort/code.

~~~
moe
I see no benefit.

It's not at all easy to understand, nor easy to implement, nor does it have
any tangible advantages.

It's one of these superfluous pseudo-standards that do nothing but add
needless clutter. But gladly nobody seems to be using it anyway, I see only
one directory in my ~/.local: vlc.

~~~
gioele
> I see no benefit.

The first benefit is that it removes clutters from your $HOME.

The second benefit is that you can now manage and backup your settings in a
sane way.

* ~/.config contains config files (should not be lost, but if lost you can recreate them);

* ~/.local contains user data files (save them often, never lose them for they are not replaceable);

* ~/.cache contains cached information (can be tmpfs mounted, you can delete it any time you want, no loss in functionality, you just lose some optimization);

* ~/.run is for temporary system files (must be tmpfs mounted or it must be cleaned during at shutdown or power on).

Luckily most of the apps used on Linux systems now use it, you are probably
using Mac OS X.

~~~
onli
> The first benefit is that it removes clutters from your $HOME.

Invisible clutter? That's a strange concept.

But the rest of your point indeed makes sense. It still is easier and probably
comment to backup the whole $HOME. But those points you can see as a benefit,
though not obvious.

~~~
lubutu
As Rob mentions in his post, the more dotfiles there are in $HOME the slower
path resolution for any subfiles becomes. How do we navigate to ./src? We open
the directory and read all the entries until we find the one called "src".
What happens if we encounter a morass of dotfiles beforehand? src takes a
while to find. The clutter may be invisible to _you_ , but it does gum up the
works.

~~~
kbolino
For what it's worth, most modern file systems (JFS, XFS, ext4, reiserfs,
btrfs, ...) have logarithmic (or better) directory lookup times. This is
achieved using hashes and b-trees (or even hash tables).

~~~
lubutu
Fair point. Though anything using the standard POSIX dirent API would still
get the performance hit (even if path resolution doesn't).

------
acabal
I hate having dotfiles in ~/ for the same reason why I hate "My Documents" in
Windows: because it's supposed to be _my_ space that _I_ organize, not a
generic dumping ground for your config files, brand-named folders, or other
nonessential garbage.

I want my space to be _mine_. Keep your app's stuff out of there!

~~~
chris_j
Where would you choose to put application config instead?

~~~
acabal
Literally anywhere but my personal space. Why not have /etc/<username>/? I
could even live with a single ~/.config/ but very few apps use that.

~~~
moe
It's actually quite useful to have everything that belongs to a user under
_one_ directory.

The canonical example is the backup. There's a strong case to be made for "tar
czf /tmp/backup.tgz ~". Do you really want your backups to become as
complicated as they are on OSX and Windows?

Likewise, being able to mount home-directories from remote servers, and being
able to _easily_ delete/move/quota users are highly desirable features in
multi-user systems.

------
rehack
I think the case of hidden dot files is a good example of 'convention over
configuration'[1].

Point is accepted, that it came into being due to a lazy programmer. But
surely early people might have just liked the unintended consequence of some
files (dot files) being hidden. Just like most of us, whenever we learnt unix
thought that it is by design.

If early users, had found the consequence a handicap, it would have been fixed
long back.

Its similar to the use of hash-tags on twitter or the @for addressing which
got adopted by users first and so became features (although the paths to them
being considered features are different).

[1] <http://en.wikipedia.org/wiki/Convention_over_configuration>

Edit: Grammar

------
caf
The fact that "." and ".." were allocated actual directory entries and were
returned when reading the directory, rather than just being handled by the
kernel when parsing pathnames seems like the original sin of expediency here.

~~~
jlgreco
Why? Seems like a brilliantly simple quick solution that would be rather easy
to roll back in the future and lacks any real downsides besides being a tad
weird.

~~~
caf
Because it was creating real directory entries that caused "." and ".." to be
visible to userspace programs reading a directory, which then led to the hack
in "ls" to hide them, which is where the article picks up.

It also isn't that easy to roll back once userspace programs start to rely on
it - for example, the assumption that the number of files in a directory is
equal to st_nlink - 2 is now so widespread that it's part of the UNIX API.

~~~
jlgreco
_shrug_ , it seems to have all worked out pretty well.

~~~
strictfp
But all programs must make exceptions for these entries. Imagine trying to add
a third magic filename, '...', for the grandparent for example. You would have
a lot of coding to do. Not really possible. It would have been better to mark
the dirs as special somehow rather than having each program have the
convention programmed in.

~~~
caf
...and of course even the kernel had to have exceptions for them anyway - it
knew they were special because it had to stop userspace from unlink()ing or
rename()ing them.

------
mseepgood
He also writes why the dd command is so horrible: "dd is horrible on purpose.
It's a joke about OS/360 JCL. But today it's an internationally standardized
joke. I guess that says it all.﻿"

<http://en.wikipedia.org/wiki/Job_Control_Language>

------
antirez
dotfiles are not perfect, but to have this very negative vision on a feature
that also helped is a bit a revisionist attempt IMHO.

Dotfiles provided a poor, but _at least simple_ way to store program-specific-
user-specific configuration, since another standard was missing. After all
it's a simple and decentralized system that worked very well with the concept
of unix user and ACL: you write something inside your home directory, and this
changes the behavior of your program.

Consider that this was invented many decades ago. Now it seems a lot better to
have directories with sub directories. Maybe back then it was considered to be
a waste of resources, inodes, and so forth.

We can improve it, create a new standard, and have something better than dot
files, but dot files are better than many other over-engineered solutions that
I can imagine coming out of some kind of design commission to substitute them.

Every time to passed your vim configuration to a friend you just copied a text
file, sending it via email: you enjoyed one of the good points about dot
files. Every time you did something like _cat dotfile | grep option_ you
enjoyed the positive effects of single-file plaintext configuration.

Also it's worth saying that dot files are not just the concept of an hidden
file with config inside. A lot of dot files also have a common simple format
of multiple lines "<option> <value>", that's better than some XML or other
hard to type format (IMHO JSON itself is not good for humans).

~~~
ajanuary
How does any of those advantages apply only to dotfiles and not just plain
files?

~~~
FuzzyDunlop
I think there's an aspect of convenience, too. Personally, I prefer that to,
say, configuration stored in some unspecified location in
~/Library/Application Support/Application/* (on OS X).

------
seanalltogether
_For those who object that dot files serve a purpose, I don't dispute that but
counter that it's the files that serve the purpose, not the convention for
their names._

I would like to hear a good argument for why hidden files and folders are a
good thing.

~~~
frossie
They are a natural segregator of novice and advanced users. If you have a
limited level of interaction with the unix shell (remember than in the old
days everyone in a science academic department used a Unix machine, even the
dusty professors you kept in the back of the supplies cabinet), setting up
their account and then making sure that they couldnt get into any trouble
because they didn't know about 'ls -a' was very much a feature, trust me.

Moreover I personally much prefer them to global configuration directories
because they are always local to the thing being configured and you can always
override a global configuration by using them.

In fact I think they are a very elegant way to handle "hidden options" - stuff
you want to expose to the power users but not bother newbies with.

tl;dr: I am not convinced they are a misfeature.

~~~
seanalltogether
I wonder if that is a good solution for separating novices and pros though?
For me, hidden files and folders only help to reinforce the confusion people
have about computers. It's one thing to provide a folder to a user with a lock
or shield on it, it's another to completely hide it from them.

~~~
brianpan
Hiding is absolutely good design at every level. Not seeing UI controls,
preferences, or _anything_ that is not relevant to the task at hand can make
that task simpler, easier to understand, and more productive. If you need to
see dotfiles, well, you might need to know about man pages or at least that
commandline programs take flags. It's not like they're invisible.

There are plenty of examples: video games slowly reveal more skills as you
learn and encounter progressively harder enemies. A good app should be usable
at first launch (or only require minimal setup). Configuration and advanced
features can come later.

Another great filesystem level example are OS X app bundles- an entire
directory hierarchy appears as a single file/application. If you need to look
inside (not likely), you have to know about right-click or the action widget,
but for 99.9% of the time, you see only what you need. OS X and Windows also
both completely hide "system" folders in Finder/Explorer as well.

Yes, hiding things can be confusing if it's not done right, but the
alternative of showing everything always is definitely not the way to go.

~~~
Dylan16807
What games reveal skills you have always had, as opposed to unlocking better
powers? I would be really annoyed by players-guide-only abilities.

~~~
jlgreco
In the Quakes, various movement techniques are only available to those who
know they exist. Worked very well I think, lend the games a much longer
learning curve.

------
ilaksh
The main takeaway I think is that the reason we are doing things the way we do
them is because that is how we have been doing them, not because of some kind
of genius design. And actually the whole thing can start out as a mistake, and
then later on basically becomes a religion. Its very funny actually.

~~~
mturmon
Yes, very organic. The original programming shortcut accidentally created a
sort of ecological niche, in which all sorts of things started to take root.
Now, nobody can bulldoze the lot because there's too much stuff growing there.

------
skeletonjelly
If not dot files lazy programmers would have just found another way. Just look
at the results of lazy programmers on any Android SD card.

~~~
voltagex_
Is there a preferred standard for using the SD/external storage on Android?

~~~
skeletonjelly
[http://www.reddit.com/r/Android/comments/t3x0p/randroid_lets...](http://www.reddit.com/r/Android/comments/t3x0p/randroid_lets_start_project_clean_sd/c4jc41o)

> I do find it odd that so many apps have this problem. It's trivially easy to
> get the proper location. Just call getExternalFilesDir() . Deletion at
> uninstall happens automatically. In fact, it's the ONLY way to make sure
> those files are deleted when you uninstall, because you can't run code at
> uninstallation.

~~~
barrkel
Probably because it came in fairly late (Froyo, I think).

------
mcgwiz
The author's gripe seems to be that the hiding of dotfiles was unintended,
ergo dotfiles are Bad. Whether they were intended is irrelevant; their wide
usage vindicates the practice. After all, traction = value. The problem of
program state/configuration/metadata storage is adequately met by dotfiles.

There are, no doubt, numerous unintended behaviors of programs. Most of these
are simply ignored and certainly not leveraged the way the dot behavior is.

People don't go out of their way to abuse an unintended system behavior; they
simply leverage all capabilities of a system ("intended" or not) to meet their
needs. Had dotfiles not gained traction, some other solution would have been
designed (or "engineered") to meet the needs of program
state/configuration/metadata storage.

[Tangent: All of this reminds me of grammar freaks that harp on "correct"
usage, completely oblivious to the fact that grammar changes, and "correct" is
merely a lightweight pointer to the current norm.]

------
eliben
I really like it that G+ is becoming a "lightweight blogging" platform. There
are too many of these around, and folding them into a "social network" seems
like a good idea. I wish they would add more features that would make this
easier, though, since in general I think it's in everyone's interest and will
pull more traffic to the site.

~~~
saljam
A bit off topic, but is it just me or is the quality of comments on public G+
posts is absolutely terrible? Within a few hours there has been more than 100
replies. Most are inane, rude, of the useless “great post!” type, or just
plain old spam. I could only manage to glean about 8 or 9 insightful replies.

Is this something intrinsic to G+? Is it a function of the author's
popularity? The discussions here are normally much more sensible, but I would
expect HN's readership to have a fairly similar demographic to that of Rob
Pike's G+ subscribers.

~~~
dredmorbius
G+ is fair to middlin' at a bunch of stuff. Comments lack threading. There's
poor noise control. Content filtering is limited to "+1" or "flag". There's no
"-1" button (though various Chrome extensions have offered this at various
points.

The main advantage is that it's a large community (10m+ users presently)
initially seeded by Googlers (e.g.: tech-savvy people), and including a few
notable luminaries such as Rob Pike. And if you've got the right circles,
eventually good content finds its way into your stream. Sometimes (content
discover/surfacing is something G+ does surprisingly poorly, and is an area at
whch HN, Reddit, and StackExchange win hugely).

------
dsrguru
That's really interesting. I always assumed hiding dotfiles was a deliberate
convention, but to semi-quote one of the commenters, Rob's got a point. Or
two.

~~~
adambratt
Heh, I remember when I first found out about dotfiles. I thought it was a
genius idea...

Little did I know.

------
emperorcezar
If someone thinks hidden files are a misstep, then I don't want them designing
a OS. Grandma really doesn't care about some conf file, or anything like that.
She cares about the pictures of her grandkids.

The opinion that there shouldn't be hidden files comes from a perspective of
someone who is a "power user" and who can't step back and see that most users
really don't care for some .config file. To them it's clutter that gets in the
way of finding what they really care about.

That said, dot files maybe the wrong way to do it. I like ~/Library in OSX.
That's one good way.

Edit: Note that I'm talking about a general trend in the post's comments and
on here. Not the author's opinion.

~~~
UK-AL
Did you read his post? He said exactly that. Put them in a config directory of
some sort.

~~~
emperorcezar
Yes. I'm not talking about his post. I'm talking about a few other comments
here and on G+.

------
tyrmored
This is a problem that libetc is supposed to help solve:
<http://ordiluc.net/fs/libetc/>

~~~
malkia
It also begs the question, why it was named "etc" :)

~~~
napum
I'm guessing it's because /etc contains configuration files and dot-files are
just configuration files. Or do you mean, why was /etc named etc?

~~~
deno
I think they meant the latter. Why is it /etc instead of, perhaps more
obvious, /config or /settings?

~~~
malkia
If it was meant for configuration - probably it would've been - /cfg /ini /set
/opt /flg /arg /prm (params)

As someone said - naming things is the hardest!

------
leothekim
"How many bugs and wasted CPU cycles and instances of human frustration (not
to mention bad design) have resulted from that one small shortcut about 40
years ago?"

Sigh, if only most of us had worked on a software system that has lasted as
long as that.

------
specialist
Tangent: I'm grumpy that Eclipse IDE uses the file names .project and
.classpath. So they're hidden by default. Requiring special treatment.

Their content is XML. What's wrong with project.xml and classpath.xml?

------
garfee
omg!

------
89a
Can't read without signing into Google…

oh well nothing of value was lost.

~~~
mproud
I'm with you.

