

Show HN: ObjectiveFS – Shared file system for EC2 and laptop - objectivefs
https://objectivefs.com

======
mieses
How is different from s3ql?
[https://github.com/s3ql/main](https://github.com/s3ql/main)

Edit: Support for concurrent mounts seems to be the main difference
[https://bitbucket.org/nikratio/s3ql/wiki/FAQ#!can-i-
access-a...](https://bitbucket.org/nikratio/s3ql/wiki/FAQ#!can-i-access-
an-s3ql-file-system-on-multiple-computers-simultaneously)

~~~
objectivefs
S3QL has a lot of great features. You are right that support for concurrent
mounts is one of the main differences. ObjectiveFS also keeps the file system
data consistent and stores it in S3 asap, for sharing and to handle EC2
unexpected termination. S3QL uploads all metadata changes at a regular
interval (by default every 24h) or during unmount.

~~~
hedgehog
Rough guess on the design: The main metadata history is in a primary log, then
each client extends that with its own branch of the history for filesystem
operations. Logs are probably some kind of Merkle tree. Bulk data is content
addressed and referenced from the logs, maybe with large objects split using a
rolling checksum like Adler-32 to reduce object sizes and allow for partial
updates to large objects. Someone (you guys?) runs a coordination service
routes notifications between clients and manages leader election. The elected
client reads all of the logs, does conflict resolution locally, and then
updates the primary log and collects garbage.

One drawback to this design would be that many small files in the filesystem
would translate to many small objects in S3 (with associated operations). One
solution would be to put small objects right into the metadata log.
Alternately they (or maybe all objects) could be put into a log-structured
merge tree.

Another problem with this design is that S3 doesn't support append operations
so sync latency would be bounded by client log flush intervals, again creating
lots of small objects. Maybe the coordination service routes some of the data
to manage this?

Anyway, really interesting design problem. Is this close?

~~~
objectivefs
ObjectiveFS does all the coordination among the clients through S3, so there
is no extra coordination service needed. This is why we are really happy that
Amazon recently moved the S3 us-east-1 region to read-after-write consistency
(like all their other regions).

We do write bundling before sending data to S3 so lots of small files would be
packed together and stored in a single S3 object. This also helps reduce the
number of object store operations.

You are absolutely right that frequent sync will necessarily create many small
objects, which is why small objects will be combined into bigger ones (this
compaction is done in the background). Sync latency is of course bounded by
the S3 PUT time, since fsync(2) can only return after your data has been
safely stored in S3.

It is a really interesting design problem. Thanks for sharing your ideas.

------
NeutronBoy
Awesome project!

It might just be a release timing thing, but what's the advantage of this over
Amazon Elastic File System [1]? I see you've got a tagline on your signup _No
need to wait for EFS_ , are you planning a feature comparison or similar?

[1] [https://aws.amazon.com/efs/](https://aws.amazon.com/efs/)

edit: derp. Turns out I should have read further!

 _How is ObjectiveFS different from Amazon EFS?

ObjectiveFS uses highly durable object stores (Amazon S3 and Google Cloud
Storage) to store your files. Your data is accessible from all regions, your
office and your laptops. Performance scales with your object store.

ObjectiveFS supports Linux and OS X and is running in production today.

Amazon EFS is NFS-based, with access to each file system limited to EC2
instances in the same region. It is not available today and will only be in
limited preview for some customers in summer 2015. It will cost $0.30/GB and
performance will scale with the number of GB stored._

[https://objectivefs.com/faq#how-is-objectivefs-different-
fro...](https://objectivefs.com/faq#how-is-objectivefs-different-from-amazon-
efs)

~~~
graceofs
Thanks! We plan to publish a feature comparison table soon. A couple more
differences:

1\. We have strong end-to-end integrity checks and encryption on by default.

2\. Amazon EFS's performance is dependent on the filesystem size: larger
filesystems include more iops.

------
objectivefs
We built a shared file system with an S3 backend. We've been running our own
servers for many years, and wanted something low maintenance, reliable,
scalable and secure. So, we built this POSIX-compliant shared filesystem for
Linux and Mac.

We love to hear your feedback. Feel free to try it out. We will also be here
to answer questions.

------
bio4m
How is this different from say mounting s3 as a volume with fuse ?

~~~
graceofs
It's a complete POSIX filesystem, and we implement a log-structured filesystem
on top of S3. So, you can use it with your regular programs expecting POSIX
semantics.

Mounting S3 lets you view the objects in a bucket, but it doesn't behave like
a regular file system.

~~~
x5n1
[https://github.com/s3fs-fuse/s3fs-fuse](https://github.com/s3fs-fuse/s3fs-
fuse)

"large subset of POSIX including reading/writing files, directories, symlinks,
mode, uid/gid, and extended attributes"

~~~
graceofs
s3fs-fuse is a one-to-one mapping and doesn't do things like atomic rename of
directories, mutual exclusion of open exclusive, append to file requires
rewriting the whole file and no hard links. This breaks the expectations of
many linux programs.

Our implementation has these features and works with linux programs.

~~~
mbreese
It's difficult to tell from your site - does it still use FUSE under the hood
or is this a kernel-level FS? I'd check the RPMs if I could, but I can't
download the RPM without signing up first.

~~~
objectivefs
It uses FUSE under the hood. Thanks for the feedback, we will make it clearer
on the website.

------
FullyFunctional
This would have been nearly perfect for me if it had supported disconnected
operation with later synchronization (~ Dropbox, CODA, ..). Alas, it's a live
fs. (Dropbox is great except it doesn't encrypt locally and has to have a
local copy of everything, unlike a cache).

------
hrez
No compression?

~~~
objectivefs
ObjectiveFS comes with fast compression that is always enabled, it works great
for log files and other compressible data. Let us know what you think if you
give it a try.

~~~
hrez
Thanks. It would be nice to put on a feature list along with some details.

------
benwilber0
website is slow as shit

~~~
graceofs
Thanks for letting me know. We are getting a lot of traffic, and working on
fixing it.

UPDATE: the website should be faster now.

