

Two pieces of code someone really ought to write - lisper
http://rondam.blogspot.com/2010/08/two-pieces-of-code-someone-really-ought.html

======
_delirium
There are a few attempts to make sockets look like a Unix filesystem, but none
seem to have caught on, perhaps partly because sockets are pretty portable,
while fs-like interfaces, unless one gets widespread support to the extent of
being a POSIX-like "assumed to be on all UNIXes", are inevitably tied to a
particular OS flavor.

I believe Plan9 was the first to have support for the idea, though, and NetBSD
has had it as an option for about 15 years, though I'm not sure if it's
currently maintained. Here's an old paper about NetBSD's implementation:
<http://www.kohala.com/start/portals.ps>

The existence of netcat also makes it a bit less pressing imo. In the cases
where I want a quick shell script that does sockets, I can use netcat as
basically a socket library. It's not the everything-is-a-file style of Unixy
coding, but it is at least the pipe-simple-utilities style of Unixy coding.
And if I want to write a "real app" in some higher-level language, the
language can wrap socket-opening at the library level to look like file-
opening if it wants to, so it doesn't _really_ have to be done by the OS.

~~~
lisper
> while fs-like interfaces, unless one gets widespread support to the extent
> of being a POSIX-like "assumed to be on all UNIXes"

FUSE runs on pretty everything, doesn't it?

~~~
sjs
It runs on most popular Unix-ish systems.

------
adulau
Regarding the first point (socket as file), Plan 9 operating system is not
using any system call for socket but use a pseudo file-system /net.

<http://doc.cat-v.org/plan_9/4th_edition/papers/net/>

Using the same approach in a fuse implementation would make sense.

------
moe
FWIW, Bash has this built in;

    
    
        cat </dev/tcp/www.google.com/80
    

It's unfortunately disabled in the bash that ships with debian, but should
work pretty much everywhere else.

~~~
gommm
Is there anything similar in zsh? That could be quite helpful?

~~~
gnosis
There are two much more powerful tcp interfaces built in to zsh:

The zsh/net/tcp Module:

<http://tinyurl.com/3alarxf>

And zshtcpsys:

<http://tinyurl.com/32z6bwe>

------
DougWebb
If you allow this:

open("/fuse/sockets/www.whatever.com/tcp/80", "r+")

Then what do you do with these?

/fuse/sockets/www.whatever.com/tcp /fuse/sockets/www.whatever.com
/fuse/sockets

They and other variations would each need different semantics, some allowing
rw, some ro, some wo, and some not being allowed at all. Some path segments
would not allow arbitrary names (tcp/udp, port number) while others require a
specific format (host/ip) and others are arbitrary. Some have to be
directories, some have to be files, and some are neither. None of this is very
filesystem-like. I think the current design is right: opening sockets is kind
of special, but you get a filehandle that is very much like an ordinary
filehandle.

~~~
lisper
> Then what do you do with these?

The same things you do with any other incomplete path name. I don't see why
this is an issue at all.

~~~
swolchok
There's no such thing as an "incomplete path name", those have to be
directories.

~~~
lisper
So treat them as directories (or more specifically, as mount points). I still
don't see the problem.

------
jrockway
The first issue is a programming language problem, not an OS problem. His
language of choice exposes the direct low-level API, which is not usually what
application-level programmers want.

A few library functions, and the problem is solved without FUSE or a
performance impact. And it's still portable to everywhere.

~~~
nostrademons
But then you have to implement the solution in every language, while with
FUSE, you implement it once and each language runtime instantly has access to
it.

~~~
jrockway
Knock yourself out, then. Every language I use (including C) already has a
sugary socket API, though.

------
joelhaasnoot
Regarding your first point... You forget one thing: you open a file and get a
file descriptor. If you open a socket (though it takes 3 calls), it also
returns a file descriptor. Sockets can then be written to and read from just
like files. Won't even get started then about the fact that sockets allow you
to do just about any type of networking, and are very abstract by purpose, to
allow a variety of uses.

~~~
glhaynes
Sounds like opening a socket could be reduced to one call, so why not fopen()?
Seems like it'd be an easy thing to get into a kernel, any reason not to?

~~~
joelhaasnoot
The question is if fopen() would then still work with embedded or archaic
architectures. Examples such as radio links on tiny 16-bit hardware come to
mind.

------
nl
NO NO NO!!! For the sake of all that is holy don't do it!!!!

Or rather, read & understand the fallacies of network programming:
<http://www.jezuk.co.uk/cgi-bin/view/jez?id=2650> before you do it.

The failure mode of networks is quite different to that of files, and the
programming model reflects that.

~~~
lisper
> The failure mode of networks is quite different to that of files

Actually, with a remote file system, the failure modes of networks is
necessarily exactly the same as that of files.

~~~
nl
Yes, and you'll note that a huge amount of the remote file driver code
involves dealing with those failures.

If you drop that down a level and turn sockets into files then suddenly
programs that are written to work with files will try to deal with these kind-
of-file-but-not-really things, and fail in unexpected ways.

------
dedward
"But now suppose that I have a single machine with a single IP address hosting
multiple virtual servers, and I want to replicate this setup for each virtual
server, i.e. I want each virtual server to have its own instantiation of the
custom server application. Now I have to _manually_ assign _each_
instantiation of the app to a separate TCP port number. If I have hundreds or
thousands of virtual servers on the same machine (Oh? You think that's not
reasonable? Can you say "multi-core architecture"?) that can become a serious
administrative (to say nothing of security) nightmare."

Not quite sure I'm interpreting you correctly here - but woudln't you be using
localhost for this and/or using loopback interfaces?

Also - if you were designing an application that had to scale out to that many
cores, you'd be dealing with things at a system level and using sockets
anyway. Whether you are opening unix domain sockets or assigning ports, you
still have to track and configure everything - and that can still be
automated. While it would be a neat feature - it doesn't seem like a terribly
necessary one.

Yes, it would be cool if apache could forward things to a local unix domain
socket. Agreed there.

But there's no security nightmare unless you create one - in the instance you
describe you bind applications to loopback interfaces, either on ports or
creating more loopback interfaces. Designing the applications to use TCP
sockets rather than unix domain sockets also leaves you with more flexibility
when it comes to design decisions later on - you don't have to leave them on
the same machine.

ALso - one wouldn't generally use apache for this - one would use something
else as a front end these days that's better suited to forwarding requests and
dealing with timeouts, resources, etc.

~~~
lisper
> Not quite sure I'm interpreting you correctly here - but woudln't you be
> using localhost for this and/or using loopback interfaces?

Sure. But you still have to assign a port number. And you still have to make a
round trip through the network stack. It's also a potential security hole if
you're not careful to configure the server apps to only listen on the loopback
interface.

> Also - if you were designing an application that had to scale out to that
> many cores, you'd be dealing with things at a system level and using sockets
> anyway.

Why? Running N servers on N machines is completely straightforward. Why should
it not be just as straightforward to run N servers on one machine with N
cores?

> you still have to track and configure everything - and that can still be
> automated

Of course it _can_ be automated, but automating the assignment of virtual
servers to TCP/IP ports is not straightforward. You have to coordinate the
port assignment on both the client and the server side. You have potential
synchronization issues if one virtual server goes down and another comes up
and wants to use the same port as the one that went down. Port numbers are a
scarce resource. The file system namespace is essentially infinite. Why make
things harder than they need to be?

------
silentbicycle
About the first point: several people have mentioned plan9, GNU awk also
handles sockets as a virtual filesystem. You seem to think it's better handled
at an OS level (and I agree), but it's a good way to try it out.

------
mleonhard
Nginx can proxy incoming connections to unix sockets

<http://wiki.nginx.org/NginxHttpProxyModule#proxy_pass>

------
brutimus
You can sorta get what he's asking for with a bash feature.
<http://tldp.org/LDP/abs/html/devref1.html#DEVTCP>

~~~
iuytrfghj
Yes but thats a hack built into bash

The point of unix is to do everything in the file system then anything that
can open a file can do everything

------
kd0amg
My network programming background is a bit small. Could someone explain for me
the difference between what OP calls for with sockets and a tun/tap device?

~~~
pmorici
A tun/tap device is a way to write a user space network driver for a piece of
hardware to make it look like a network device to the OS. ie: you could make a
serial serial cable look like a network device.

What this blog is asking for is a different way to access network resources so
instead of calling a function like socket(some_ip, some_port, SOME_PROTOCOL);
(which is a gross over simplification for the typical way it is done) he wants
to be able to say open("/net/some_ip/protocol/some_port", "rw");

