
Willscott/go-NFS: Golang NFSv3 server - pjf
https://github.com/willscott/go-nfs/
======
willscott
Wasn't expecting to see this here!

This turned out easier than i was expecting it to be. It's nice to be able to
mount a VFS without needing privileges on the server side. The main intention
for this code is to eventually use it to replace fuse on mac, since nfs is a
valid mount time for mac clients to consume.

~~~
_prometheus
This is great! SOOO useful to be able to do mounts w/o fuse, w/o kernel
extension, w/o root. This is a HUGE UX thing, specially for macs.

Fuse is a really great idea & project, but sadly today impls require a lot of
install steps in some platforms, and some have painful bugs/UX. Amazing if we
can mount VFSes from Go without those hurdles.

~~~
jchw
Just to lay them out, some notable other options:

\- Under Linux and FreeBSD, bazil fuse offers a nice low dependency path to
supporting FUSE. It is all pure Go code.

\- Under Windows, you can use WinFSP, which is a rock solid usermode
filesystem driver with FUSE support. Despite its name, you can use this with
Cgofuse to get FUSE on Windows without CGo.

Combining the two above gets you pretty far already (though if you want to
actually avoid a CGo dependency you do need to explicitly disable it.)

I think the next level would be to have a general filesystem interface (there
are several options in the ecosystem already) that can be used as a common
ground between different usermode filesystem and network filesystem server
implementations. Then ideally, a magic Go library can exist that gives you the
best possible setup with switchable backends depending on the platform and
build configuration.

Right now Go itself is possibly standardizing a read-only filesystem
interface, an extension of this with write support might serve as a decent
jumping point for getting toward an idealized pluggable filesystem library.

~~~
yencabulator
> I think the next level would be to have a general filesystem interface

That's starting to take shape:
[https://old.reddit.com/r/golang/comments/hv976o/qa_iofs_draf...](https://old.reddit.com/r/golang/comments/hv976o/qa_iofs_draft_design/)

It won't be enough for the likes of FUSE, though. Some extensions might be
enough to bridge that gap, it's too early to tell. FUSE is pretty particular
about things like rename(2) preserving node identity.

------
anderspitman
Considering how many things have been implemented on top of HTTP, I find it
interesting that something like WebDAV (or Solid[0], or remoteStorage[1])
hasn't nearly completely supplanted NFS/SFTP/etc.

Not saying that would be ideal from a technical perspective (WebDAV at least
has some issues), just surprised the ability to access remote filesystems from
the browser hasn't been a bigger driver.

For example, there are a lot of interesting apps built on top of Google Drive
as the storage backend, but overall the concept doesn't seem to have gained
much traction.

[0]: [https://inrupt.com/solid](https://inrupt.com/solid)

[1]: [https://remotestorage.io/](https://remotestorage.io/)

~~~
api
NFS is a lot faster. A custom protocol will almost always beat a kludge on top
of HTTP

~~~
fulafel
But NFS isn't a custom protocol for any specific app either, and apps that
need good fs performance (eg databases, video editing platforms, "big data"
compute platforms etc) avoid it.

Implementation-wise HTTP has the advantage that the apps can tune the client
code because the protocol client implementation isn't in the kernel.

In addition HTTP security (transport & user authentication) is works and
everyone understands it, whereas on NFS it's a huge mess and enemy of
interoperability.

In practical popularity NFS fares extremely poorly eg on AWS (how many apps
use EFS vs S3).

~~~
ralph84
EFS is much more expensive than S3, so there is a strong incentive to make
your app work with S3 unless you absolutely need a filesystem.

~~~
kondro
As with all things AWS, it depends.

Yes EFS is 30¢ per GB and S3 is 2.3¢.

But S3 also charges $5/million object writes (and 40¢ for reads), but EFS is
"free" (factoring burst capacity, per-instance maximum throughput limits,
etc).

Couple that with EFS' lifecycle migration allowing you to get storage costs as
low as 2.5¢/GB for many write-once-read-(almost)never after 7 day
applications, things get a lot more complicated.

------
brianm
This is cool! At a previous company we got a lot of mileage in debugging and
ad-hoc tasks by exposing various things as filesystems. We mostly did webdav,
but NFS is way better from a client transparency perspective. For many folks
find, ls, and grep very much beat curl and jq.

~~~
q3k
A nice protocol to do this with is 9p [1]. It's much easier to implement a
server for it than for NFS, either from scratch or even with a good library,
and it's about just as usable on major operating systems. It has a tiny API
and therefore surface area, also making it great for high security
applications. It’s probably the closest thing we have to a cross-platform,
network transportable FUSE.

Even though it started its life as a component of Plan9, 9p is getting through
a renaissance period right now: qemu, WSL, gVisor, ChromeOS VMs (crostini)...
all use 9p!

[1] -
[https://en.wikipedia.org/wiki/9P_(protocol)](https://en.wikipedia.org/wiki/9P_\(protocol\))

~~~
ComputerGuru
Unfortunately all the default implementations really suck at performance (for
Linux and Windows, at any rate), and it’s very noticeable under IO pressure,
lots of small files, etc. They're stable and seem to be correct though, which
is something.

------
anderspitman
Didn't see it mentioned in the README, so I'll ask here: any particular reason
to go NFSv3 vs NFSv4? I'm not familiar enough with the protocol to venture an
educated guess.

~~~
willscott
V4 has many more things going on (it's stateful, has a much more complicated
locking system integrated, and adds some other points of complexity that get
better performance). More than I was ready to take on for a first pass :)

~~~
anderspitman
You had me at stateful

~~~
yencabulator
For what it's worth, stateful is a good thing; NFSv4 leases are what allows it
to safely do more aggressive caching than NFSv3. Back when it was new,
benchmarks showed a good speed increase all around...

~~~
anderspitman
Stateful isn't inherently good or bad. It's a tradeoff. But the more
programming I do the less I think it's a tradeoff worth making.

~~~
yencabulator
In the context of NFS, it's a trade-off that enabled higher performance (and
some new features too).

------
cyphar
When looking for something like this a year or so ago, I found [1] which
supports both NFSv3 and v4 as well as p9. It worked alright in my experience
though I eventually switched to ZFS which has built in support to auto-
configure NFS shares.

[1]: [https://github.com/nfs-ganesha/nfs-ganesha](https://github.com/nfs-
ganesha/nfs-ganesha)

------
jonathonf
Possibly useful existing implementation:

[https://github.com/unfs3/unfs3](https://github.com/unfs3/unfs3)

This has the benefits of being tested and having a working read-write
implementation (along with still being user-space).

------
toomuchtodo
@willscott: Do you have any experience to share running this over Wireguard or
similar?

Edit: Thank you!

~~~
willscott
Most testing has been on a Lan or on the local machine for local VFS use
cases.

This layer eschews responsibility for multiple concurrent clients for
simplicity. No promises you'll get either decent performance or proper cache
invalidation when using it that way. It tends to be conservative in not
filling in all the opportunistic caches.

There's a hook the backing application can use to tune reads and write sizes
that becomes much more relevant in a network case. How those should be set in
practice isn't something I've spent time on.

------
Zach_the_Lizard
I wonder if this is a good way to support projects like Microsoft's Git VFS
without requiring specific kernel drivers. Mounting a special Git NFS drive +
having all the magic hidden behind NFS seems like it could be interesting.

------
pbnjay
I've always wondered if I could take the Unix "everything is a file" approach
way too far and hook it up to a web service. This looks like exactly the kind
of glue to make that easy...

~~~
nitrogen
My favorite "everything is a filesystem" hack from around 2001 was cdfs for
Linux, which would expose each audio track as a .wav file you could cp, and
each separate data session as a subdirectory, so you could access prior states
of the CD filesystem.

------
rawoke083600
Good work ! PS is NFS still a thing ? I thought Gluster(FS) or S3 sorta
replaced it in most shops ? I could be mistaken of course.

~~~
zuppy
I’m using nfs for Docker on Mac, as its native share system (osxfs) has awful
performance. Not the best setup, but didn’t find a better one yet.

~~~
rawoke083600
Interesting... maybe give Gluster a spin when you bored of have time.

