
BitTorrent file system - matt2000
https://github.com/johang/btfs
======
johansch
Most use cases or apps don't deal well with read latencies in the thousands of
seconds...

From quickly reading the source code; it doesn't do any particularly clever
optimizations. I'd file this in the "IP over pigeons" category.

~~~
rprospero
I agree that the standard torrent use case of torrents containing small
numbers of huge files rather fails with this protocol. However, I think it
could be interesting for systems with large numbers of small files.

For instance, think about a LaTeX install. The user could, in a matter of
seconds, download a torrent file to a multiple gigabyte repository of all of
the fonts an modules that are available. They then run latex on whatever
document that they're trying to render. The filesystem would download the
libraries as needed, so everything the user needs would be available, but she
wouldn't have to wait while downloading a bunch of libraries that she didn't
need. Nor would she need to decide before hand which libraries to install or
ignore, as everything would be on the filesystem. The user experience of
waiting for the executables and base libraries to download whne run for the
first time wouldn't be great, but arguable better than waiting for the entire
distro to install in the standard case.

~~~
cbhl
Having access to a large repository of files without actually downloading the
files to your computer already exists at large corporations, but you want a
more traditional serving hierarchy in order to get acceptable read/write
latencies.

Bittorrent is more useful when 1) local peer transfer speeds exceed the
transfer speed from a master copy, and 2) everybody wants an identical copy of
the data.

For example, if you want to deploy a new version of Facebook.com to a far away
data center, Bittorrent is great (you only have to upload maybe 2-3x the size
of the binary to the datacenter, and the thousands of nodes inside the
datacenter can share the pieces amongst themselves).

~~~
chrissnell
See: [https://blueprints.launchpad.net/glance/+spec/glance-
bittorr...](https://blueprints.launchpad.net/glance/+spec/glance-bittorrent-
delivery)

------
enginn
Reminds me of
[https://en.wikipedia.org/wiki/InterPlanetary_File_System](https://en.wikipedia.org/wiki/InterPlanetary_File_System)

------
jzelinskie
There are actually a couple of implementations of this. I think that I most
recently saw it in Pulsar[0]. I'm kind of glad these have never gotten too
popular because to maintain a healthy swarm using the BitTorrent protocol,
peers should not be prioritizing chunks sequentially.

[0]: [https://github.com/steeve/pulsar](https://github.com/steeve/pulsar)

~~~
rahimnathwani
Pulsar is a bit different. It doesn't present as a filesystem. IIRC it
presents an HTTP stream to XBMC.

Re: swarm health, Pulsar's predecessor project's README says this:

"Doesn't sequential download on bittorrent is bad?

Generally, yes. However, XBMCtorrent respects the same requirements "defined"
by uTorrent 3. Also, XBMCtorrent tries to make it up to the swarm by seeding
while you watch the movie."

~~~
jzelinskie
You are correct. I had made an assumption after seeing a TorrentFS type
previously[0].

[0]:
[https://github.com/steeve/pulsar/blob/fb0ed2d4691c683af781cf...](https://github.com/steeve/pulsar/blob/fb0ed2d4691c683af781cf3ed55202a2c42ca3bc/bittorrent/torrentfs.go#L21)

------
bm1362
IPFS ([https://ipfs.io/](https://ipfs.io/)) is essentially the same thing with
a use case bolted on top of it.

------
nickpsecurity
Seems really impractical. Better go use a distributed filesystem. Here's a
little-known one that's used with supercomputers:

[http://sector.sourceforge.net/](http://sector.sourceforge.net/)

------
ArnoldP
I wonder if this prioritizes chunks in order? Or does one have to wait for an
entire mp4 to be downloaded before knowing it will play all the way through?

~~~
ratsimihah
Right, if streams aren't downloaded in order, that'd be very inefficient, and
if it has to wait for the whole file to be downloaded, that'd defeat the
purpose of streaming.

------
jaimehrubiks
It sounds interesting. Is there any daemon running a torrent client? Or is it
possible to select a custom one?

~~~
omribahumi
Btfs is a "glue" between libfuse and libtorrent.

libfuse is used for the Kernel Fuse interface and libtorrent is used as a
bittorrent client.

Read more about Fuse on Wikipedia:
[https://en.wikipedia.org/wiki/Filesystem_in_Userspace](https://en.wikipedia.org/wiki/Filesystem_in_Userspace)

------
aaronem
There is at least one other project with the same name ("btfs"):
[https://github.com/arvidn/btfs](https://github.com/arvidn/btfs)

Seems a bit more feature-complete than this one.

------
alfanick
is this compatible with BitTorrent Sync? because that what BT Sync does
exactly...

~~~
rahimnathwani
BT Sync is for syncing folders between different computers, with access
control. It's not the same at all.

------
zaporozhets
Would this be easy to port over to osx?

~~~
adrusi
Probably, given the existence of osxfuse
([https://osxfuse.github.io/](https://osxfuse.github.io/))

~~~
zaporozhets
Yeah, trying it now - but I'd be surprised if it builds

update: nope. [http://pastebin.com/Zf8fVh1D](http://pastebin.com/Zf8fVh1D)

~~~
geocar
There's also:

[https://github.com/mafintosh/torrent-
mount](https://github.com/mafintosh/torrent-mount)

which claims to have been used with OSXFuse...

~~~
xnyhps
I've use that on OS X, worked quite well. Ultimately I don't see much use for
it though, maybe if you want to get information about the headers of files
before you download them completely or something like that.

------
godzillabrennus
Is this basically all the benefits of health checks in ZFS through distributed
computing?

~~~
rincebrain
Not exactly; the checksums in BitTorrent are fairly strong, but you could get
that benefit with just a FUSE layer that knew how to associate checksum
metadata with files.

I also am not entirely sure what benefit this brings other than "amusing proof
of concept", currently, so take my above note with a grain of salt.

------
stultus
Does it seed the torrent?

------
runn1ng
Is this read-only filesystem? It seems like it, right?

~~~
kilovoltaire
Yes, the readme says that it is read-only.

