
In Unix Everything Is a File - mutor
https://ph7spot.com/musings/in-unix-everything-is-a-file
======
pjmlp
Any UNIX old timer knows that actually:

In UNIX everything is a file, except when it is not.

~~~
majewsky
> In _Plan 9_ , everything is a file

Fixed that.

~~~
rdtsc
> > In Plan 9, everything is a file > Fixed that.

It was a short snarky comment. Also the article mentions that too at the
bottom.

Yeah they "fixed it" but by that time not enough people cared. The abstraction
purity was nice and developers and OS fans loved it, but it just didn't matter
for most application. Another way to put it is that Linux was already fairly
decent and it wasn't worth switching to a whole new OS just to have everything
be a file.

Another obstacle was that Plan 9 wasn't marketed or pushed enough. Imagine if
Google backed Plan 9 and pushed it as the Android OS, put it on Chromebooks,
updated it regularly etc. That would be enough for people to switch to it.

~~~
yepguy
I understand why it didn't take off, but I don't think it's true that Plan 9
was just a tiny incremental improvement over UNIX's architecture. Modern
networking and distributed systems would be a LOT nicer today if Plan 9 had
taken off in the 90s. I think the economic benefit over today's state of
affairs would have been massive, not just something that appealed to computer
geeks' sense of elegance.

~~~
rdtsc
> Modern networking and distributed systems would be a LOT nicer today if Plan
> 9 had taken off in the 90s.

But it didn't take off. That's already decided. It wasn't enough. One can say
well if it happened 10 years earlier maybe, it would have taken off. Or if
Linux never happened. But it's speculative.

Heck, Plan 9 source is there. If so much economic benefit can be derived, why
hasn't it become a dominant player. It had been around enough time, and there
are enough developers around to have flocked to it and make it overtake Linux
by now.

> I think the economic benefit over today's state of affairs would have been
> massive,

I think we already reaping enormous benefits open source licensing model in
general. We live in a world where you can download production grade operating
systems, databases, compilers, libraries for free and also inspect their
source. That is often taken for granted and is incredible compared to who it
used to be.

~~~
yepguy
I can't really disagree with most of your post.

The reason Plan 9 or its ideas can't be resurrected very easily _now_ is
because every machine on the internet has to pretend to be UNIX and speak a
thousand ad-hoc protocols. UNIX's many flaws have been papered over, one by
one, by incredible brute force (instead of in one fell swoop). I'm only
lamenting the fact that the 90s were a crucial period in the development of
the commercial internet, and in another timeline it could have developed in a
very different direction.

------
Pete_D
"Everything is a file" is a nice idea not taken far enough. The execution is
full of frustrating omissions and holes at the edges, e.g.:

\- unless you are using Plan 9 or gpm, reading mouse events is almost
certainly going to be a lot more involved than opening /dev/mouse

\- you can't portably use select() to wait on pthread conditions

\- lots of useful things (like POSIX shared memory objects) are file
descriptors, but aren't filesystem namespaced

Sometimes I wonder what the world would look like if Plan 9 had won.

~~~
pjmlp
That is easy, microservices über all.

I never had such appreciation for Plan 9, as I consider Inferno the best
design of what they were trying to achieve.

The only caveat was that they never came around to add an ahead of time
compilation support to DisVM.

------
rzzzt
Which was the last Unix release/flavor where you could still read(2) a
directory? All manpages that I've found say that you get EISDIR if you try it.

~~~
Someone
That depends on the file system (and, possibly, the OS, but I think the OS
should just forward the call to the relevant driver and return its result)

FreeBSD’s man page says
([https://www.freebsd.org/cgi/man.cgi?sektion=2&query=read](https://www.freebsd.org/cgi/man.cgi?sektion=2&query=read)):

    
    
      [EISDIR] The file descriptor is associated with a directory
               residing on a file system that does not allow 
               regular read operations on directories (e.g. NFS)
    

NetBSD says something similar, adding _”The readdir() function should be used
instead.”_

I also am not convinced that, for example, the Linux man page
([http://man7.org/linux/man-pages/man2/read.2.html](http://man7.org/linux/man-
pages/man2/read.2.html)) guarantees you get _EISDIR_ when reading from a
directory. It says that if you get _EISDIR_ , _fd_ refers to a directory, but
isn’t explicit about the converse.

~~~
rzzzt
You are right. The default implementation in Linux returns -EISDIR, but
filesystems are free to change it to something else:

[https://github.com/torvalds/linux/blob/2a56bb596b2c1fb612f99...](https://github.com/torvalds/linux/blob/2a56bb596b2c1fb612f9988afda9655c8c872a6e/fs/libfs.c#L211-L214)

[https://github.com/torvalds/linux/search?p=1&q=generic_read_...](https://github.com/torvalds/linux/search?p=1&q=generic_read_dir&unscoped_q=generic_read_dir)

------
lucb1e
Wordpress strikes again.

Page is from 2014. Archive link:
[https://web.archive.org/web/20091119184152/https://ph7spot.c...](https://web.archive.org/web/20091119184152/https://ph7spot.com/musings/in-
unix-everything-is-a-file)

Edit: archive of 6 years ago with some styling:
[http://archive.is/B9He](http://archive.is/B9He) (The page actually loaded for
me on the second try, and the design looks a lot more modern now.)

~~~
ChrisSD
Your link says the page is from 2009, not 2014 (2009-11-19 to be exact).

~~~
lucb1e
Oh right, my bad. I just saw a first archive.org index at 2014 but that is of
course not necessarily the time of writing/publication.

------
epx
I like the Linus Torvalds' interpretation: that (almost) everything is
actually a file descriptor. The few things that are not, like fork() returns,
are subject to criticism (DJB wrote about how fork() should return an fd, I
just couldn't find the URL right now.)

~~~
severine
This? [http://cr.yp.to/docs/selfpipe.html](http://cr.yp.to/docs/selfpipe.html)

> _Richard Stevens 's 1992 book ``Advanced programming in the UNIX
> environment'' says that you can't safely mix select() or poll() with SIGCHLD
> (or other signals). The SIGCHLD might go off while select() is starting, too
> early to interrupt it, too late to change its timeout.

Solution: the self-pipe trick. Maintain a pipe and select for readability on
the pipe input. Inside the SIGCHLD handler, write a byte (non-blocking, just
in case) to the pipe output. Done.

Of course, the Right Thing would be to have fork() return a file descriptor,
not a process ID._

~~~
epx
I have a faint recall of a DJB text with more in-depth discussion around the
topic, thanks anyway for providing a true reference!

~~~
severine
You're welcome! I found the link here:
[http://www.greenend.org.uk/rjk/tech/fork.html](http://www.greenend.org.uk/rjk/tech/fork.html)

------
ArtWomb
Is the _sendfile_ system call still the fastest way to transfer data locally
on Linux systems?

For communicating between processes running locally on system. Or, for
example, a server running locally on an android device. Unix domain sockets
typically beat TCP connections for most large file transfers. But is there
anything faster?

And what about transfers over the public net from machine to machine? I
usually just use netcat, ssh, or scp.

~~~
Pete_D
Consider trying rsync instead of scp - I've found it to be faster for multiple
small files.

~~~
arendtio
I love rsync, but being exceptionally fast is most certainly not one of it's
properties.

------
lvh
And it looks like the server may have run out of files (well, descriptors) :-)

google cache link:
[https://webcache.googleusercontent.com/search?q=cache:L6aopS...](https://webcache.googleusercontent.com/search?q=cache:L6aopSQWczwJ:https://ph7spot.com/musings/in-
unix-everything-is-a-file+&cd=1&hl=en&ct=clnk&gl=us&client=firefox-b-1-ab)

------
zmix
And if you use XML, all you need to do, is to add another forward slash after
the filename. And now, the network, the host and the data have become one:
Everything is a node (or an atomic value).

But hey, that's stupid. Let's not do that...!

~~~
techbio
By way of XPath. I don't disagree but--that's stupid because of... using XML
at all? Complications of altering/documenting ad hoc taxonomies? Mixing
paradigms under a unified syntax? Something else?

~~~
zmix
Would you mind giving me the one or other example? Or references to some
documents about this (in regard to XML), that I could reach?

------
zokier
I think the author might have confused Plan9 and UNIX.

~~~
DonHopkins
He also confused "The Art of UNIX Programming" with a "fantastic book".

~~~
Lordarminius
What's wrong with the book ? I have read it and I think it provides a great
introduction to Unix.

~~~
DonHopkins
One obvious problem with the book which this article demonstrates is that it
leaves the reader with the mistaken impression that "In Unix Everything Is a
File".

I assumed this was common knowledge, but apparently not, so here goes: Another
reason not to buy the book is that the author, Eric S Raymond, is a racist
sexist Islamophobic conspiracy theory spreading global warming denying gun
nut, who threatened to kill Bruce Perens.

In summary, and just because you asked, Eric S Raymond is a mediocre
programmer who has made a career of trying to pose as a hacker, define what a
hacker is, tell people how to become a hacker, all while viciously,
personally, professionally, and unfairly attacking actual successful
influential hackers like Richard Stallman, including his philosophy and his
life's work, by hijacking and distorting the definition of Free Software and
the very definitions of words in the hacker's dictionary, to reflect his own
extremist narcissistic racist political ideology, which most certainly does
not align with the non-bigoted ideology in the Hacker's Manifesto: "We exist
without skin color, without nationality, without religious bias."

[https://en.wikiquote.org/wiki/Eric_S._Raymond](https://en.wikiquote.org/wiki/Eric_S._Raymond)

"In the U.S., blacks are 12% of the population but commit 50% of violent
crimes; can anyone honestly think this is unconnected to the fact that they
average 15 points of IQ lower than the general population? That stupid people
are more violent is a fact independent of skin color." -Eric S Raymond

"A clash of civilizations driven by the failure of Islamic/Arab culture
(though I would stress the problem of the Islamic commandment to jihad more
than he does). I think he [Steven den Beste] is also right to say that our
long-term objective must be to break, crush and eventually destroy this
culture, because we can't live on the same planet with people who both carry
those memes and have access to weapons of mass destruction. They will hate us
and seek to destroy us not for what we've done but for what we are." -Eric S
Raymond

"And for any agents or proxy of the regime interested in asking me questions
face to face, I’ve got some bullets slathered in pork fat to make you feel
extra special welcome." -Eric S Raymond

"When I hear the words "social responsibility", I want to reach for my gun."
-Eric S Raymond

As much as he thinks he and everyone else should own lots guns, Eric S Raymond
has recently stated (2017) that he believes blacks are too unintelligent to
own or to be trained to use guns:

[https://twitter.com/tqbf/status/816449724127608833](https://twitter.com/tqbf/status/816449724127608833)

>Unfortunately, this doesn't cover the BLM crowd, which would have an average
IQ of about 85 if it's statistically representative of American blacks as a
whole. I've never tried to train anyone that dim and wouldn't want to.

[https://rationalwiki.org/wiki/Eric_S._Raymond](https://rationalwiki.org/wiki/Eric_S._Raymond)

Eric S. Raymond (born 1957) is a computer programmer, advocate of open source
software, and author or editor of several written works, including The
Cathedral and the Bazaar and the Jargon File. Coincidentally, "Yonder Racism"
is an anagram of his name.

Unfortunately, his work and expertise in computer technology[1][2] has been
all but overshadowed by his batshit insane wingnut tendencies in the wake of
9/11, his misogynistic claims,[3] and his increasingly wanky ego-gazing,[4][5]
as he acknowledges that people aren't so crazy about him anymore.

[https://en.wikipedia.org/wiki/Talk:Russ_Nelson#Blacks_are_la...](https://en.wikipedia.org/wiki/Talk:Russ_Nelson#Blacks_are_lazy)

Eric S Raymond threw down the gauntlet and defended Russ Nelson's infamous
"Blacks are Lazy" article by calling everyone who wanted him to resign his
position as president of OSI "fools and thugs", and demanded OSI spend their
precious time and resources fighting critics of racism and defending Russ
Nelson's racist beliefs which ESR shares, instead of promoting open source
software.

[http://www.eweek.com/servers/new-osi-president-steps-
down](http://www.eweek.com/servers/new-osi-president-steps-down)

>“Abetting somebody elses witch hunt is no less disgusting than starting your
own.” “Personally, I wanted to fight this on principle,” Raymond said. “Russ
resigned the presidency rather than get OSI into that fight, and the board
quite properly respected his wishes in the matter. That sacrifice makes me
angrier at the fools and thugs who pulled him down.”

This would be funny if it weren't exactly like Trump's recent "'nt" flip-flop
on Putin:

>Nelson subsequently sought to explain himself in an essay titled "Blacks are
Not Lazy."

[https://encyclopediadramatica.rs/Eric_S._Raymond](https://encyclopediadramatica.rs/Eric_S._Raymond)

In 1999, Debian developer Bruce Perens published an "email threat" that he
allegedly received from Raymond. Raymond then "clarified" that he only meant
to defame Perens. From this we can assume that he is batshit insane and will
fucking kill and or write to anyone that says anything about him or his
software. If you are lucky you might get an O'Rielly book about you.

[https://lists.debian.org/debian-
user/1999/04/msg00623.html](https://lists.debian.org/debian-
user/1999/04/msg00623.html)

    
    
        To: debian-user@lists.debian.org
        Subject: email threat
        From: bruce@perens.com
        Date: 5 Apr 1999 22:48:42 -0000
        Message-id: <19990405224842.2386.qmail@perens.com>
    
        Today I received the following threat in e-mail from Eric Raymond. The message
        was copied to the Silicon Valley Linux User's Group officers, who you may
        consult regarding its authenticity. The police have been notified.
    
        Because I know that Eric is a firearms enthusiast, for my own protection,
        I feel the best strategy is for me to publicize the threat widely.
    
                Thanks
    
                Bruce Perens
    
        > Damn straight I took it personally.  And if you ever again behave like
        > that kind of disruptive asshole in public, insult me, and jeopardize
        > the interests of our entire tribe, I'll take it just as personally --
        > and I will find a way to make you regret it.  Watch your step.
    

The irony of Eric S Raymond threatening someone else for behaving "like that
kind of disruptive asshole in public" is rich -- very rich.

[https://geekz.co.uk/lovesraymond/cat/bruce-
perens/page/10](https://geekz.co.uk/lovesraymond/cat/bruce-perens/page/10)

[https://geekz.co.uk/lovesraymond/archive/terrorismistic](https://geekz.co.uk/lovesraymond/archive/terrorismistic)

~~~
ernst_klim
Your reasons are totally ad hominem and dictated by your political views and
have nothing to do with the topic.

~~~
DonHopkins
Sure they do. He asked about what was wrong with the book, which I answered.
Besides the obvious factual error I pointed out (which you chose to ignore,
but is _literally_ the topic of this discussion, therefore relevant), there is
the well known fact that Eric S Raymond is a racist with a long history of
making ad hominem attacks against entire races and religions, as well as his
personal death threat against Bruce Perens and long career of attacking
Richard Stallman and the Free Software community.

I'm sorry that a single quote is just not capable of capturing the vast depth
and breadth of his racism and hatred of Perens and Stallman.

Is there something you find inappropriate or annoying about me expressing my
(apparently controversial) political view that racism is wrong, and quoting
Eric Raymond's own words to prove he is a racist? Why is that more annoying to
you than ESR's racism, threats and ad-hominem attacks themselves?

In the same vein (or bloody car seat), whenever anyone mentions the Reiser
File System, the fact that he murdered his wife often comes up. Do you also
find it inappropriate to mention that fact on Hacker News or Wikipedia too?

[https://en.wikipedia.org/w/index.php?title=Comparison_of_fil...](https://en.wikipedia.org/w/index.php?title=Comparison_of_file_systems&oldid=209063556#Features)

[https://en.wikipedia.org/wiki/ReiserFS](https://en.wikipedia.org/wiki/ReiserFS)

~~~
ernst_klim
I don't see what Islam or irrational wiki or racism has to do with unix
programming and the book. Your political beliefs contradicting with ones of
the author does not make a _programming_ book bad.

~~~
DonHopkins
I was directly responding to a question from somebody who didn't know about
ESR. Were you previously aware of the fact that Eric Raymond is a batshit
crazy racist, or did you just learn something new from me that you didn't
previously know? If so, then mission accomplished! I'm glad to have opened
your eyes even a little bit.

Do my political beliefs about racism being bad and obligatory to call out
contradict your own beliefs, and annoy you so much that you're compelled to
argue and continue to drag this out? Or do you know some actual facts you can
prove about ESR that contradict the proof I posted with links and quotes? Then
by all means share your proof I'm wrong, instead of stifling the discussion!
Why are you so irritated and motivated to suppress those widely known facts
from being mentioned on Hacker News?

If simply stating the facts and quoting ESR's own words sounds to you like an
ad hominem attack, then that's ESR's problem, and maybe he should have kept
his mouth shut instead of publicly saying things like "The average IQ of the
Haitian population is 67... Haiti is, quite literally, a country full of
violent idiots", and he exercises enough personal responsibility that he
doesn't need you to bravely step up and defend his racism.

[http://esr.ibiblio.org/?p=7011](http://esr.ibiblio.org/?p=7011)

Or is your real issue with the valid criticism I raised about the book (and
that you awkwardly ignored and haven't countered) about "In Unix Everything Is
A File" being wrong, which is LITERALLY the topic of this conversation, a
point many other people are discussing, and a major mistake of the book and
the article?

If you really believe that my criticism of the book in response to
Lordarminius's direct question has "nothing to do with the topic", then why
aren't you criticizing Lordarminius for asking me "What's wrong with the book
?"

And why aren't you arguing with all the other people here who have pointed out
that the API for reading directories is different than the API for reading
files, that ioctl is a kitchen sink of non-file-like operations, that many
objects in the Unix kernel are simply not files, etc?

Or is it just my second criticism of ESR's racism what ticks you off so much
that you totally ignored and failed to address my first criticism of his
book's factual errors, and incorrectly claimed my reasons for disliking the
book are "totally ad hominem"? Go re-read the very first sentence of my reply.

I don't see what ESR recasting the Jargon file to reflect his own political
beliefs has to do with hacking, but that's what he did, and it's also valid to
raise that point when discussing the flaws of other books he wrote.

[https://news.slashdot.org/story/03/06/08/1534249/esr-
recasts...](https://news.slashdot.org/story/03/06/08/1534249/esr-recasts-
jargon-file-in-own-image)

[http://www.ntk.net/2003/06/06/](http://www.ntk.net/2003/06/06/)

------
nailer
Are there Unixes where TCP / UDP ports are files? setcap on Linux gets really
old when you just want to give a user access to a porf.

~~~
squirrelicus
I can't say for sure, but i do know ucspi-tcp is a great tool that wraps the
noise for you into FDs. But it's only useful if you can tank the cost of
invoking a process per tcp connection.

[https://cr.yp.to/ucspi-tcp.html](https://cr.yp.to/ucspi-tcp.html)

------
jiveturkey
this needs a 2009 tag.

it should actually be removed as the article is wrong. even in 2009, how could
someone get this wrong? pretty sure 2009 was finally going to be the year of
the linux desktop! IOW unix had reached the masses by then.

