

Show HN: A versioned filesystem inspired by Git - artagnon
http://github.com/artagnon/phoenixfs

======
tolmasky
OK, first off -- awesome! Secondly, I have a number of questions that probably
have to do with the fact that I'm unfamiliar with a lot of this stuff.

1\. Would it be possible to build this fully on top of git (perhaps using
libgit2)? I ask, because the "holy" grail for me is finding some magic way for
my designers to use git without knowing it. Right now my designers (and I
think most non-programmer people), love to use Dropbox. Dropbox has a number
of issues as a shared project tool in my perspective (not the least of which
being that you are only allowed ONE dropbox account per computer, so you start
having a shared folder mess and your dropbox begins to balloon in size on
every computer with that account). It appears that something like this could
_look_ like Dropbox to everyone disinterested in vcs and act like Git to
everyone else who cares.

2\. The other reason for libgit2 is to ideally also git-push after every
edit/save, to something like github.

3\. Max OS X support please :)

~~~
s_tec
Have you tried SparkleShare? It sounds exactly like what you want. It connects
to a normal Git server and presents the repository as a Dropbox-esque folder.
It has support for Linux, Windows, and Mac, although it's mainly a Linux
project.

~~~
sciurus
More info:

"SparkleShare is a 100% free and open source, cross-platform file sharing
system started by Hylke Bons. The idea for the project came from the 2010
GNOME UX Hackfest. It's very similar to Dropbox or Box.net, but it is backed
by the free and open source version control system git.

Basically, the way it works is that you will have a special folder on your
computer, called 'Sparkleshare', and everything in that folder is also present
in a version-controlled remote git repository. If you create a new file in a
Sparkleshare folder, it will immediately be committed and pushed into the
remote git repository, and everyone else who is using Sparkleshare to connect
to that folder will be able to see the file you just created appear in their
Sparkleshare folders. If someone else opens the file you just created and
modifies it, their Sparkleshare client will automatically commit and push
their changes to the repository, and you'll be able to see them in your
Sparkleshare folder (and you'll get a handy notification!)

Sparkleshare is a great, free & open source way to collaborate on files,
without worrying about manual version control or having to manually upload and
broadcast links to your work."

<http://sparkleshare.org/>

~~~
tolmasky
Wow I will definitely take a look at this.

------
akg
Congrats on a tremendous technical feat!

Any advantage of using a full-scale FS over something like git-annex
(<http://git-annex.branchable.com/>)?

~~~
artagnon
The seamless workflow that only a full-blown filesystem can offer- new
revisions are created automatically when files are edited and saved.

~~~
spartango
You should be able to achieve this with Filesystem events
(inotify/kqueue/filesystemwatcher). Just another way of doing things.

I'm not the biggest fan of FUSE filesystems(until highly-tuned, ala NTFS-3G),
but they certainly do work.

~~~
halayli
No, you cannot achieve atomicity using these calls. By the time you receive a
notification the file might have been edited twice.

~~~
gwern
If the file is changing _that fast_ , do we really care? I'd be happy to have
all my files tracked at only 5s temporal resolution...

~~~
rhizome
Sure, but _which_ 5s? :)

------
gcr
Can this handle large binary files? That's been my main problem with storing
backups of my servers in git, and it's one of the strong-ish points of git-
annex.

There seems to be nothing inherently limiting in Git's repository design.
Interested parties can look into bup, for example, which stores large binary
files in git's packfile format. <https://github.com/apenwarr/bup>

~~~
artagnon
Yes; I don't see why not. I haven't stress-tested this enough to tell for
certain though.

------
tensor
This is relevant:

[http://www.hpl.hp.com/personal/alistair_veitch/papers/elepha...](http://www.hpl.hp.com/personal/alistair_veitch/papers/elephant-
sosp/efs.pdf)

If only half the stuff done by the likes of IBM and HP were to make it into
common usage the world would be a very different place.

------
mosburger
Nice! Reminds me a bit of a better version of VMS's versioned filesystem.

Also: I'm old enough to have used VMS's filesystem. :(

~~~
josephscott
Agreed, VMS filesystem was the first thing that came to mind for me as well.
Seems rather sad that in 2012 we don't have that as a standard feature like we
did back then.

For the record, you don't have to be all that old to have used VMS's
filesystem :-)

------
al3xbio
Nice. Why not use libgit2 (<https://github.com/libgit2/libgit2>)?

~~~
artagnon
It's a bit of an overkill for my purposes- I've only copied out (and modified)
a little of the packfile/ packfile index generation code from git.git. Plus,
dealing with the actual code is a great learning exercise :)

------
adam-a
Sounds a bit like ZFS snapshots[1]. I always wished that would catch on a bit
more, seems like it would be very useful.

[1]: <http://en.wikipedia.org/wiki/ZFS#Snapshots_and_clones>

~~~
justinmk
And btrfs[1] of course.

There's also git bup[2] and git-annex[3].

[1] <http://www.dedoimedo.com/computers/btrfs-snapshots.html>

[2] <https://github.com/apenwarr/bup>

[3] <http://git-annex.branchable.com/>

------
halayli
[https://github.com/artagnon/phoenixfs/blob/master/delta.c#L4...](https://github.com/artagnon/phoenixfs/blob/master/delta.c#L489)

[https://github.com/artagnon/phoenixfs/blob/master/fuse.c#L59...](https://github.com/artagnon/phoenixfs/blob/master/fuse.c#L594)

Missed malloc check.

<https://github.com/artagnon/phoenixfs/blob/master/fuse.c#L82>

system calls like mkdir can fail. Better to check return values of such calls.

~~~
voltagex_
Might want to post that to the issue tracker

------
icebraining
Extremely interesting. I wonder: when does it "commit"/create a new rev?

~~~
artagnon
When you modify a file and save it.

~~~
rix0r
So on close() ?

~~~
artagnon
Yes. The event is called `release`: see phoenixfs_release() in fuse.c:374.

~~~
ferrouswheel
What about when a file is fsynced?

------
nicholassmith
This looks pretty good. I had a look at doing something similar (but wayyyyy
smaller scale than full FS), and gave up as there was many dragons. Nice to
see someone pull it off.

------
jarvuschris
I'm working on a similar concept which provide copy-on-write over HTTP as
well: <http://emr.ge>

------
jeremyjh
Very cool. I would like to have some automatic revisions for example of my
/etc tree.

~~~
RobotCaleb
There's a tool that will commit changes to /etc before every apt-get install.

This might be what I'm thinking of: <http://joeyh.name/code/etckeeper/>

------
jblz
This is tremendously cool!

Any plans for MacOS (or other *nix) support?

~~~
artagnon
There's a FUSE for OS X, so a port should be possible (should be possible for
other *nix too, as long as they have FUSE). I currently don't have access to
an OS X machine, so no immediate plans.

------
prezjordan
Wow, awesome project!

