Am I the only one who doesn't get excited about the idea of absolutely everything being a file?
People seem to like the idea of being able to say things like "cat file.wav > /dev/dsp". But a sound card is a complex thing that can be configured in umpteen different ways; what if the file has a different sample rate than the card is currently running at? Even if you gerry-rigged a kernel driver that could handle this case (by, for example, noticing when the file had been open()'d and expecting the next bytes to be a WAV header, which it uses to reconfigure the card), you're left with something that doesn't have even the most basic functionality that you'd expect from a sound player; for example, "pause."
So what has representing /dev/dsp as a file really bought you? It doesn't help you know what devices are available, because there's tons of crap in /dev that doesn't necessarily have a kernel driver loaded for it. It's not that useful as an end-user interface for selecting a sound card, because users don't want to choose between /dev/dsp and /dev/dsp2, they want to see nice names like "Internal Speaker" or "Apogee Duet."
Files give you a simple API for lots of common operations, which is nice, but the fact that all files are not created equal means that operations sometimes don't work like you expect. For example, how would you guess that "tail -f" is implemented? If you're like me, you would have expected that it works by calling poll/select/etc. and waiting for the file to become read-ready. But this approach doesn't actually work for monitoring growing files; the select() will just return immediately with read-ready even if you're at EOF. So what "tail -f" actually does is a loop that alternates between sleep() and fstat(). I'm serious! http://git.savannah.gnu.org/gitweb/?p=coreutils.git;a=blob;f...
Some files are different. So? Do you have the same qualms with stdin/stdout not being seekable because they are often connected to a terminal?
The "everything is a file" abstraction is mostly helpful and rarely gets in the way in practice; I don't know how plan9 exposes sound, but it is probably a directory, with multiple files - one you write to to set sampling rate; another gets data, and the third is stop/go/cancel.
The most useful feature, though, is network transparency that really works. You don't need Vnc/rdp; just mount the remote display in your file system. You don't need special ip forwarding - just mount the remote network in your local file system. You don't need a network aware server for anything - it all is.
You could then search for configs by using find, back them up by using tar, be able to easily replicate settings between computers using rsync etc.
The idea is not to have one unconfigurable binary reciever, but rather to expose the interface as a hierarchy of rw files instead of a number of C functions, because the C functions have no easy shell representation, and most command-line tools fail using them.
> The idea is not to have one unconfigurable binary reciever,
It _is_ configurable, why do you think it is not?
This idea that somehow one can fit any interaction with hardware into a filesystem ends when hardware becomes complex enough to simply NOT fit (like modern GPU).
How do you guarantee atomicity when saving configuration with tar(1)?
Linux recently deprecated textual conntracking data interface in favour of shipping binary data over netlink socket. Reason -- read() speed in real life high load scenarios.
I actually tried to point out that configurability _is_ very possible with such an API.
Your complexity argument does not hold. If I can control something with a C API, why not with a file API?
Take atomicity as an example. C has no inherent concept of atomicity. Yet you can acheive atomicity by using C. So why not with a REST or file-based interface?
You also mention speed. Interesting since complexity and performance worries are the two roots to premature optimization if you ask me. Since this solution is still on a concept level I am of the opinion that one should avoid such discussions at this stage. But I see no _inherent_ speed limitations in a file-based API. Send binary data if you want. Text is only for readability.
> Take atomicity as an example. C has no inherent concept of atomicity. Yet you can acheive atomicity by using C. So why not with a REST or file-based interface?
You achieve atomicity because inside kernel handler of relevant imaginary ioctl(2) which dumps all hardware configuration there is synchronization primitive which provides necessary guarantees, so this imaginary ioctl(HW_GET_STATE) writes coherent state to userspace.
If you merely expose individual hw config into flat files, POSIX directory semantics does not guarantee you atomicity.
So, you have to invent something instead, like additional file which kinda sorta provides locking. But if you invented it, you have implicitly mandated that ALL users must be nice and go through locking API/file, whatever. At this point, nothing prevents rogue (and more importantly) buggy userspace from messing with well-behaving users.
At this point, you've failed to provide something another developer is asking you to provide, namely, a guarantee that hw state will be coherent.
At this stage real world examples show that once scale of the state becomes sufficiently large (large number of conntracks on big firewalls), text interface always lose.
So, play with toy flat text file interface if you want, but you might as well do it right from the very beginning.
Right now, /proc/stat slowness is being discussed in linux-kernel.
Part of the problem:
heuristics determining how many pages needs to be allocated for buffer containing text info sucks because integers written in decimal are variable sized. Kernel first allocate 1 page, only to fill it and see that 1 page is not enough for big enough machines (NR_CPUS * NR_INTERRUPTS)
2 patches are proposed: first is to allocate 2 pages from the beginning, second is (no kidding) to print decimals faster (printf "%u" can be made faster since it's known that "unsigned int" is going to be dumped and C has pathetic support for compile-time evaluation).
But the correct from every angle (except existing /proc/stat users) answer is obvious:
dump BINARY info already into userspace buffer which will be appropriately sized because it's easy to multiply 3 integers). Simply ship information to who is asking, without print bullshit.
Real programming languages and environments should easily eat result (Python's struct.unpack springs to mind).
Excuses for programming languages will not.
Can you transform /proc/bus/pci/00/00.0 content into one-dimensional array of bytes with you favourite PL?
And if you can't, whose problem is it?
And yet another example, Linux USB bus sniffer kernel module (don't remember exact name/config option) gained binary interface deprecating text one.
As for _inherent_ speed limitations, read(2) does memcpy(), so you have to provide mmap(2) for your file (most virtual Linux filesystems doesn't do it for majority of files).
This idea that somehow one can fit any interaction with hardware into a filesystem ends when hardware becomes complex enough to simply NOT fit (like modern GPU).
The idea that one can fit any interaction with hardware into the mere toggling of registers and stuffing of byte buffers...
I don't know what FS you're using, but any good modern filesystem (say, ZFS or something similar) should prove more than adequate for handling that many/that size nodes.
Everything else can probably be fixed in the kernel level--for example, duplicating the hardware directory prior to tar'ing it, and have the kernel/FS note what actions to do to guarantee atomicity. I mean, we do the same thing with sharing /dev/dsp anyways (I believe daemons can be set up to handles this), so what's the deal?
We're discussing anything that can do open(), read, write(), provides files and directories. Do you understand that virtual filesystems do not have problems with inode counts?
Dupping a directory means
a) hw state is saved on open(), in which case you're holding it unnecessary because read() simply may not happen
b) hw state is saved on first read(). OK, what to do with parallel dumpers? How to determine when transaction starts and ends? What to do if dumper process is sigkilled?
ioctl(HW_GET_STATE) doesn't have these problems, kernel knows how to kill process inside itself.
Try to save /sys/class/net/lo/* coherently and write down all assumptions which you did.
Each program sees its own copy of /dev/dsp. Behind the scenes, that copy is backed by some magic on the OS's part that handles state tracking, mixing, etc.
The program only ever sees a dumb bit bucket with some flags on how to handle it--that's why we have OSs, after all: to avoid having to write hardware driver code for all our programs (see the bad old days of DOS game programming).
The program still needs to manage sound mixing in userland to fill that buffer. That's nothing new, however, and you can use PortAudio, FMOD, XACT, whatever to do that mixing for you, or roll your own.
This abstraction would greatly simplify that process.
Thanks to you and others who pointed out that plan9 exposes private copies to each program. I did not know that, and it would solve the issue I wondered about.
But that doesn't solve the underlying issue. If my program uses two libraries, both of which play sounds, then how does each library get its own configuration filesystem? Is there a way to create a new view, so that the two libraries can be disconnected from what the main program needs?
I'm not an expert in plan9 stuff but I think each process sees its own version of some parts of the file system (IIRC mounts are also process specific), so, maybe those commands would only apply to the current process.
The advantage of "everything-is-a-file" is that every process gains thread-safe access to some API without having to pull in the headers, link, and manage a foreign memory management regimen.
But the real boon is that each process can conceivably be linked with any 'file'. I liken this practice to the STL library's iterator pattern: algorithms are the processes, and files are the iterators (although I bet STL was inspired by unix). You can redirect your tty through an ssh tunnel, you can scp a regular file, or you can ssh tunnel your soundcard or mouse. And of course 'ssh tunnel' is only one such kind of glue. There are pipes, devices, fifos, etc. In the end it works out that there are O(n*m) possibilities for only O(n+m) implementation cost.
But remember that with STL, not all algorithms will work with all iterators, and vice versa. And likewise, not all processes can work with all files, and vice versa. The benefit comes from the significant overlapping portion.
You can do cat file.au >/dev/audio on a lot of Unix-y OS's. For wav, the problems you mention are solved with "sox" which converts the wav file to raw data with suitable options.
All of the problems you mention are problems of the interfaces exposed by the files, not problems with the "everything as a file" idea in itself.
Look at /proc on linux. /proc/[pid] represents a running process, but it's a directory, not a file, all the information about that process are inside.
A hypothetical filesystem API for audio designed to be friendly might expose a /proc/sound/[device]/name file containing a user friendly description and a /proc/sound/[device]/output file that a converter gets mounted at that you can just copy files to, and a /proc/sound/[device]/control that accepts commands like "pause" or "play".
You also seem to also be assuming that these interfaces are meant for ordinary end-users. While some of the functionality does make it easier for end-users to interact directly with the lower levels of the system, nothing precludes friendlier interfaces being put on top of it.
What it gives you is an interface that has a bunch of standard tools you can use to operate on them, either because they're simple enough to use directly, or to make building the user friendly interfaces more easily, and that which is at the same time reasonably explorable if it's well designed. /proc is a good example of this - you can poke around in it and learn a lot about your system with just a standard shell without having the faintest idea about how it's structured initially. Meanwhile, the non-file-related system calls require you to write code to test them out - they're far more opaque.
> But this approach doesn't actually work for monitoring growing files; the select() will just return immediately with read-ready even if you're at EOF.
This really shouldn't be surprising. select() specifically tells you if the filedescriptor is read for read. A filedescriptor is ready to read from if there is more data or you're at the end of the stream. The behavior is entirely consistent. Perhaps a way of requesting notification only if there is more data to read, rather than if there's more data or end of the stream, would be useful, but that is not what select() provides. It illustrates some of the flaws in the Unix system calls, not an inherent problem with a filesystem interface.
I'm not sure that really contradicts his point, that's a terribly limited API. Stereo only, no way to specify channel layout, 16bit only, no way to change the device buffer sizes, no way to tell if the device is in use, no way to tell the device latency, seemingly no way to examine what sample rates or bit depths the device supports, seemingly no way to subscribe to any kind of notification when the device has actually changed sample rate (or other properties) or when playback has stopped etc, no ability to lock the device to prevent other processes changing the sample rate or other properties, no ability to get device time or determine if the device is synchronised with system time.
Some of these missing things can be added fairly easily to the existing API of course, but I don't know enough about plan9 to suggest how, for example, notifications for device property changes could be added.
As it stands this interface is almost useless even for casual home users who just want to play back audio: multiple channel support is a basic requirement these days. Serious audio work is not even possible at all.
Were the sound APIs from other operating systems of that era any less limited? Presumably you could reproduce a modern API just as easily with a filesystem, using the same general schema.
The thing to keep in mind is that this was written for a SoundBlaster back in the early 90s. I remember being really impressed that Commander Keen 4 could use the SoundBlaster 16 in my 486 to make decent music at all.
As for notifications of device property changes, you would probably get them by reading from the various control and status files. Although I use Plan 9 at home and at work, I have never really done anything with audio, I'm afraid.
Plan 9's users consider that a feature. Standing boast: "GNU's compiler manual is bigger than our whole system!"
> I don't know enough about plan9 to suggest how, for example, notifications for device property changes could be added.
You'd add a virtual file named "ctl". Processes could open it and call read, which would block until something changed, then return with the notification.
Those processes will need threads that work. Plan 9's do.
> As it stands this interface is almost useless even for casual home users who just want to play back audio: multiple channel support is a basic requirement these days.
Really? I must therefore, be somehow a less than casual music listener - I am perfectly happy with my stereo.
I wasn't suggesting that all users require it, merely that a non-trivial number do; more for watching video with multichannel soundtracks than listening to music.
The HUGE advantage of "everything is a file" is that if you make a tool that works on one "thing" and then want to use it on another "thing" it is quit possible that it will just work. For instance, on OSX, one of the things I sorely miss is the sysfs from Linux. Sure, OSX has the ioreg, but I cannot use the tools I am used to (find, grep, cat, echo, sed, awk, ...).
To put it in terms the web programmers can understand, it is like REST, you have a GET (read), and a POST (write), and I suppose a DELETE (unlink), and with that you can implement all you need. Everything fits into a very few syscalls (open, read, write, close).
> But a sound card is a complex thing that can be configured in umpteen different ways; what if the file has a different sample rate than the card is currently running at?
Sure, you can use an ioctl for this. You could use some combination of ioctl()s to tunnel absolutely any API, just like you can wrap any text string in pointy brackets and call it XML.
The question is whether this is useful. In particular, once your interface requires ioctl(), you lose the benefit of easy interoperability with UNIX command-line tools.
People seem to like the idea of being able to say things like "cat file.wav > /dev/dsp". But a sound card is a complex thing that can be configured in umpteen different ways; what if the file has a different sample rate than the card is currently running at? Even if you gerry-rigged a kernel driver that could handle this case (by, for example, noticing when the file had been open()'d and expecting the next bytes to be a WAV header, which it uses to reconfigure the card), you're left with something that doesn't have even the most basic functionality that you'd expect from a sound player; for example, "pause."
So what has representing /dev/dsp as a file really bought you? It doesn't help you know what devices are available, because there's tons of crap in /dev that doesn't necessarily have a kernel driver loaded for it. It's not that useful as an end-user interface for selecting a sound card, because users don't want to choose between /dev/dsp and /dev/dsp2, they want to see nice names like "Internal Speaker" or "Apogee Duet."
Files give you a simple API for lots of common operations, which is nice, but the fact that all files are not created equal means that operations sometimes don't work like you expect. For example, how would you guess that "tail -f" is implemented? If you're like me, you would have expected that it works by calling poll/select/etc. and waiting for the file to become read-ready. But this approach doesn't actually work for monitoring growing files; the select() will just return immediately with read-ready even if you're at EOF. So what "tail -f" actually does is a loop that alternates between sleep() and fstat(). I'm serious! http://git.savannah.gnu.org/gitweb/?p=coreutils.git;a=blob;f...