
FreeBSD gets pNFS support - trasz
https://svnweb.freebsd.org/base?view=revision&revision=335012
======
mprovost
Interesting that this was tested at Framestore (the visual effects company),
they used to be a Lustre shop years ago then moved to commercial NFS servers
and must be back onto commodity hardware now.

~~~
nomadlogic
I could see a use for both - using Lustre as a shared SAN for their
flame/smoke/etc suites and using pNFS for workstation/render-node shared
storage. at least that's how i've deployed things in the past.

i could see this replacing isilon (which is built on freebsd) for the later
use-case :)

------
loeg
Very alpha code, YMMV, buyer beware. I don't want anyone to try this and be
turned off FreeBSD if it doesn't work well.

The author only has 100 Mbit networking, so performance bottlenecks may not be
well-explored.

------
davidmr
This seems pretty cool in principle, but I got as far as a single
nonreplicated MDS before I lost interest in trying it out at this stage.

Admittedly, my background is in HPC where metadata performance and reliability
requirements would preclude using something like this. Does anyone who is a
more familiar with FreeBSD than I am (which is pretty much anyone who’s used
it in the last 10 years) have any more information on long their term plans
for at least replicated block storage failover or cache coherency in an
active/active MDS setup?

~~~
loeg
pNFS in FreeBSD is one developer's toy project. I don't know of any widespread
interest or plans for enterprise features.

------
kev009
It will be fairly amusing to see this in a Billions dollar product line like
NetApp or Dell EMC Isilon.

I can't find any photos of Rick Macklem's setup but it was developed in a
fairly.. steampunk fashion.. on several obsolete i386 laptops with <4GB RAM.

~~~
jim00338811
Panasas, who open sources pNFS, is a serious scale company competitive with
Isilon and NetApp

[https://www.pcmag.com/article2/0,2817,2135264,00.asp](https://www.pcmag.com/article2/0,2817,2135264,00.asp)

[https://www.panasas.com/](https://www.panasas.com/)

~~~
GalacticDomin8r
Since Panasas contributes a fair amount to FreeBSD, perhaps they are happy as
well.

eg
[https://people.freebsd.org/~rpokala/2017-03-24.nvdimm.txt](https://people.freebsd.org/~rpokala/2017-03-24.nvdimm.txt)

~~~
loeg
Ravi never actually got around to committing that NVDIMM work :-(.

------
brian_herman
What is pNFS?

~~~
jdhawk
Parallel NFS, which was introduced in the NFS v4.1 protocol.

~~~
brian_herman
Thanks!

------
kondro
I’m still surprised that NFS doesn’t have a TLS option. Or a way to grant
access without relying on IP whitelists.

~~~
toast0
NFS tends to be living in the kernel, which sounds like a terrible place for
TLS. That said, IPSec lives in the kernel to some extent, you could layer NFS
on top of ipsec....

~~~
justincormack
Linux and FreeBSD have both got kernel TLS support (not sure if freebsd got
merged yet but Netflix have used it fir a while)

~~~
loeg
Netflix's TLS support is an extremely limited hack that suits their
performance needs. It does not support initial session negotiation or
rekeying. They do the former in userspace before handing off a symmetric key
to the kernel, and drop connections in the latter case, relying on the client
to reconnect. There's no chance it will be merged to FreeBSD; it's not a
general solution.

As long as we're talking about TLS and IPSec, though, I'd point to Wireguard
as maybe something viable for kernel use.

~~~
justincormack
The Linux kernel TLS support does the same.

------
philg_jr
Red Hat worked pretty closely with NetApp and their implementation of pNFS.
I've used it and it is very nice for mitigating any downtime while you perform
maintenance on a controller.

~~~
gnufx
How does a parallel filesystem make maintenance easier? Unless it's
specifically redundant, I'd expect the opposite.

------
mozumder
Is NFS viable at all with untrusted clients on a LAN? Seems it requires all
machines on a LAN to be trusted, with proper user IDs.

It's fast though.

~~~
maayank
I've actually implemented NFS servers professionally (for my sins) and used
various competitor implementations. It's possible with the mechanisms others
have mentioned, but ultimately each vendor has its own specific ways to
circumvent the "Kerberos is annoying and often not well configured at the
client" issue[0]/allow various 'hacks' to be deployed, so if you're interested
in a specific vendor's product (e.g. ONTAP) the best option is to look through
their online manual for the specific CLI/Web GUI features, usually around root
squashing and the like.

[0] I actually really like it, but the reality and customer feedback is what
it is...

~~~
mozumder
Is Kerberos even an option in an environment with Mac OS & mobile phone
clients, as well as Windows?

~~~
maayank
It is, but (IMHO):

1\. There aren't many NAS devices that gracefully interact with both Active
Directory and Kerberos (if you care about that, for both SMB and NFS access to
the same storage). The best that I know that is still commercially available
is ONTAP (I was not employed by them, but a competitor). I don't know much
about its internals, but was impressed with the features, team, my own play
with it, etc. It's also the most expensive. If interoperability with AD is not
an issue, then disregard.

2\. Mac OS X support for NFS v4 and its variants is abysmal. To the point that
I could craft packets that would cause a reboot on the latest OS (which to me
yells probable 0-day... but hopefully fixed since then and security analysis
is not my expertise). Their SMB client is really good though, second to
Windows in terms of keeping up with features, using them correctly, etc.

3\. What's the use case for the mobile devices? The NFS/SMB clients I've seen
for mobile are clunky as hell, but it was also never my focus.

While still only my opinion, this is true up to early 2017. I've since moved
to another industry and don't actively research it anymore.

~~~
amorousf00p
I've used krb5 and nis in an osx environment with NFS as file store. My basic
opinion is that osx is not about unix legacy and they move as desired. Running
on osx is agsinst a moving target.

Unsuitable for production.

------
partycoder
Subversion? oh man, what a blast from the past.

