
Webfs: A Filesystem Built on Top of the Web - FisherGuy44
https://github.com/brendoncarroll/webfs
======
tootie
Remember WebDAV? It was a similar concept, but never really found its footing
and most of the implementations were pretty shaky. I always thought it was a
good idea though.

[https://en.wikipedia.org/wiki/WebDAV](https://en.wikipedia.org/wiki/WebDAV)

~~~
Eikon
The rfc is _super_ dumb though. For instance, when handling a PROPFIND request
(more or less listing files / folders), it’s not mandatory for the server to
honor the Depth header (how many levels are returned). There is also no
mechanism for the server to advertise whether or not it’s honoring the Depth
header. That means the Depth header is useless because the client has no way
to know whether there was only one hierarchy depth or if the server did not
honor the Depth header. Therefore, your only option is to always scan the full
hierarchy using PROPFIND at each level.

The RFC is full of that kind of crazy gotchas, not to mention the overuse of
“MAY” or “SHOULD” which will drive you crazy if you try to implement a client
/ server.

If you want to go further in insanity, just look at how crazily over
engineered the locking mechanism is. I have no words for it.

[https://tools.ietf.org/html/rfc4918#section-6](https://tools.ietf.org/html/rfc4918#section-6)

Unfortunately, even if you _think_ you implemented the whole rfc correctly,
your implementation will work with almost nothing as not that much
implementations are any good in the wild. A useful WebDAV implementation must
be full of vendor-specific workarounds.

~~~
tootie
There was an even worse protocol back then called CMIS. It was an attempt to
define a standard API for content management which turned into this absolute
enterprise monstrosity that would make SOAP blush. It was also impossible to
implement.

It's interesting looking back but I think developing a standard has a higher
chance of success coming from some dude's GitHub than it does with $1T of
market cap behind it.

------
austincheney
I am working on a similar idea myself. Here is a video demo from a month ago:

[http://mailmarkup.org/sharefile/demo1.mp4](http://mailmarkup.org/sharefile/demo1.mp4)

~~~
dredmorbius
Watching now.

1\. Thanks for the direct mp4 link.

2\. Consider reducing your desktop resolution for webcasts. I can see that
there are dialogues open. I can't for the life of me see what's presented in
them.

------
brendoncarroll
Hey everyone, I'm the author of WebFS. Happy to answer any questions.

~~~
dredmorbius
I'm going to assume you're familiar with Plan9OS's WebFS and 9P Protcol?

[https://en.m.wikipedia.org/wiki/9P_(protoco)](https://en.m.wikipedia.org/wiki/9P_\(protoco\))

I've been kicking around similar ideas (nowhere near implemented) for a while:

[https://old.reddit.com/r/dredmorbius/comments/6bgowu/what_if...](https://old.reddit.com/r/dredmorbius/comments/6bgowu/what_if_the_web_was_filesystemaccessible/)

~~~
brendoncarroll
I was not aware of Plan 9's WebFS. It looks like it presents websites in the
local file system. IPFS can do something very similar with its content. I have
used that before.

The name here comes from using web resources (referred to by url) as building
blocks for a file system. I guess "Web for FS" rather than "Web as FS".

~~~
dredmorbius
The notion of remote services accessed via local filesystem dynamics is pretty
well established. Among implementations:

\- NFS, particularly with the Solaris-originated concept of automounts over a
/net mountpoint.

\- Various virtual filesystems. Midnight Commander ("mc") offers several of
these, including archive formats (tar, cpio, afio, rpm, deb) and remote (FTP,
SSH).

\- SMB/CFS/Samba

\- Various FUSE filesystems, including again ssh, ftp, and others. These
generally require specifying in advance specific mountpoints.

The notion of on-demand access to remote resources over protocols (e.g.,
http/https, or others), under filesystem dynamics, is interesting -- you can
use any general tool, utility, or application for access, mediated through the
filesystem by way of drivers, rather than a specific application (e.g., Web
browser, FTP client, etc.)

There are numerous issues. In particular, applications tend not to respond
well to remote resources disappearing, changing, or failing to return from
change requests -- NFS's behaviour with nonresponsive remote hosts is ...
notorious.

Consistency, availability, and partition resistance (CAP) are long-standing
concerns, and there's no way to solve for all three. I'd add latency as
another major consideration.

[https://en.wikipedia.org/wiki/CAP_theorem](https://en.wikipedia.org/wiki/CAP_theorem)

The general notion of managing and tracking changes locally, and pushing them
to remote, has merits. I'm _not_ aware of a "gitfs" ... though of course, one
does exist, TIL:
[https://www.presslabs.com/code/gitfs/](https://www.presslabs.com/code/gitfs/)
The notion of using git (or other versioning system) as a mediator for
remote/local revisioned access seems to have merits. Obviously not viable for
very-high-variability systems, but adequate for many occasionally-modified
resources.

I'm not sure if you're looking at using your WebFS itself as a _publishing_
mechanism, though in general I think I'd recommend _against_ doing this. For
small-n peer-to-peer distribution that's probably workable, but for large-
scale provisioning-and-request systems, relying on HTTP or other established
transports is likely more sensible.

One area I've recognised as being particularly fraught is the whole notion of
security and privacy. Providing unfiltered local access to remote resources
which may change arbitrarily is a great way for allowing malware onto local
systems -- your transport layer should probably implement some level of
security and mounts deny direct execution of content. The fact that remote
content could be copied to an executable mountpoint remains, and would make
numerous attacks possible.

Similarly: access, update, write, and/or publishing actions all leak
considerable information which could be of concern to specific users or
organisations. Hash-based indexing (already addressed in this thread) being
only one of several such vectors.

~~~
z3t4
SMB/CFS/Samba are not meant to be used directly over the internet. eg. without
a tunnel. The best alternative right now is Dropbox unless you are a
developer.

~~~
dredmorbius
Neither, really, is NFS.

The point isn't whether or not these are protocols that are utilised on the
naked Internet, but that they offer access to network services via filesystem
semantics.

That is, rather than use a specific client or API to access remote content, or
copying it locally as a separate step, you simply open a file in an existing
application, or, within a program, using fopen() or equivalent operators. The
networking is ... translucently ... handled in the background by the
filesystem interface and/or driver(s).

The _reasons_ SMB is not generally used or advised over the Internet are worth
looking at, as this touches on many of the security / privacy concerns of any
such service.

------
cryptonector
HTTP is basically a filesystem protocol that supports magical files -- not too
unlike sharing named pipes over SMB. Not too unlike what it'd be like if one
could open(2) AF_LOCAL sockets on Unix/POSIX systems instead of having to
connect(2) to them -- if that had been so in 1982 in BSD, it would be true
now, NFS would have supported the same, etc.

