
Versity open-sources ScoutFS - fh973
http://www.versity.com/blog/versity-open-sources-scoutfs
======
jorangreef
"The super block is stored in two 4KB blocks at a known location at the start
of the device."

There should be more than 2 slots for the super block, e.g. at least 64 slots
for the super block, to allow for better wear-leveling for SSDs. AFAIK this
has been a problem for some filesystems, which wear out the device due to the
high frequency of super block updates.

The latest super block should also be replicated 2-4 times on disk to survive
corruption. Otherwise, the loss of a mere two contiguous disk sectors could
result in loss of the entire disk.

A super block of 4KB also looks like they might be relying too much on the
device to guarantee atomic sector writes (this is almost always the case, but
not necessarily), instead of using a cryptographic signature to verify that
the super block was written atomically. Also, 4KB seems small these days for a
super block, especially if the super block needs to include replicated
information relating to RAID sibling devices etc.

------
SOLAR_FIELDS
If anyone from Versity is reading this, there is a typo in the release:

“We are __releaing __the technology under the GPLv2 open source license.”

~~~
meghan-versity
Doh! Thanks for letting us know, will get that fixed. Of course,
patches/contributions are always welcome.... :)

------
a012
What's the difference of archival filesystem vs. distributed filesystem? I
assume other distributed filesystems could be used to archive files as well,
namely
[https://github.com/moosefs/moosefs](https://github.com/moosefs/moosefs)

~~~
notacoward
The focus here seems to be on _very_ high scale, not just capacity but also
number of files and scan performance which would be important for an archival
store and which are often pain points for other distributed filesystems.
Similarly, ensuring data integrity is a high priority for such a system. I
certainly wouldn't compare it to a single-metadata-server pseudo-distributed
filesystem that doesn't even honor O_SYNC writes.

~~~
meghan-versity
Couldn't have said it better myself! For more details on requirements as we
designed the filesystem and why current filesystems didn't meet our needs, see
this blog post: [http://www.versity.com/blog/versity-open-sources-
scoutfs](http://www.versity.com/blog/versity-open-sources-scoutfs)

------
patrickg_zill
What I didn't see in the white paper, is whether the contents of each file are
checksummed?

It seems only the metadata is checksummed.

I have used ZFS and the ability to run a check on the entire set of metadata
and actual files is very reassuring (though it is definitely time consuming).

~~~
meghan-versity
Both metadata and data are checksummed. A unique feature of the filesystem is
that it doesn't trust what the storage tells it, it actually verifies it.
Thanks for pointing this out - we'll get these details in the next version of
the paper.

~~~
patrickg_zill
That is great to hear and I look forward to reading more about it, thanks.

------
notacoward
Looks interesting, and apparently it's _really_ a filesystem unlike so many
thing that just slap "FS" on the name and call it a day. _tears of joy_ White
paper etc. is a bit low on details, though.

------
unixhero
Linux mainline soon?

~~~
meghan-versity
Definitely.

