
IPFS is a new peer-to-peer hypermedia protocol - tambourine_man
http://ipfs.io/
======
_prometheus
_(EDIT: we 've a meetup coming up soon. The SV Ethereum Meetup is hosting an
intro to IPFS later this month in San Francisco:
[http://www.meetup.com/EthereumSiliconValley/events/221350594...](http://www.meetup.com/EthereumSiliconValley/events/221350594/)
come hang out!)_

\--------------- Hello Everyone! \---------------

@jbenet here. Thanks for all the attention! Did not expect to wake up to this
post today :)

The best resources for learning about ipfs are:

\- The IPFS Paper:
[http://gateway.ipfs.io/ipfs/QmR7GSQM93Cx5eAg6a6yRzNde1FQv7uL...](http://gateway.ipfs.io/ipfs/QmR7GSQM93Cx5eAg6a6yRzNde1FQv7uL6X1o4k7zrJa3LX/ipfs.draft3.pdf)
(note: DRAFT 3, working on DRAFT 4, lots has evolved.)

\- The Alpha Demo:
[https://www.youtube.com/watch?v=8CMxDNuuAiQ](https://www.youtube.com/watch?v=8CMxDNuuAiQ)

\- The First Tech Talk:
[https://www.youtube.com/watch?v=Fa4pckodM9g](https://www.youtube.com/watch?v=Fa4pckodM9g)

\- The IRC Logs:
[http://botbot.me/freenode/ipfs/](http://botbot.me/freenode/ipfs/) \-- there's
so much knowledge in these, we're working on moving it over.

We're working on a bunch of more expository media:

\- Specs!! [https://github.com/ipfs/specs](https://github.com/ipfs/specs)

\- More Demos

\- More Talks

\- Draft 4 of the paper

But there's only so many hours in a day, and we spend most of our time
developing. All the resources will be way more polished as time goes on.
Please bear with us as we make these available. In the meantime, ask questions
in our new FAQ (below), or in #ipfs on irc.freenode.net -- And we'll be
answering questions here too of course.

I've _just_ created:
[https://github.com/ipfs/faq](https://github.com/ipfs/faq) \-- if you ask
questions there we can build a knowledge base

IPFS already works-- you can install it and try it out. It's not production
ready yet, but it's already very reliable. I use it to move around all sorts
of personal files, and we use it to host many websites.

Some people asked about what applications are there now? There's many static
websites on ipfs, and some simple webapps. We ship our entire ipfs webui with
ipfs itself (turtles all the way down!). In particular, Eris Industries
recently made a decentralized YouTube clone using IPFS and their Decerver:
[https://github.com/eris-ltd/2gather](https://github.com/eris-ltd/2gather) If
you see more-- please let us know at
[https://github.com/ipfs/community](https://github.com/ipfs/community) so we
add it to an index (we just reorged our repos under the github org last
week)..

We need to make some more tooling to make web-publishing with ipfs really,
really nice. We'll be making that over the next few weeks, so if you're
interested in pushing out websites with ipfs-- come talk to us.

Another big use case we're going after is Docker container distribution. We're
preparing a whole bunch of demos right now that we'll show off in the next
week or two. You can peek at: [https://github.com/ipfs/container-
demos](https://github.com/ipfs/container-demos) We can already boot entire VMs
and containers over ipfs in <10 seconds :D which makes me really happy. You
can boot to ipfs with this makefile:
[https://gist.github.com/jbenet/76afa70955910a2ed097](https://gist.github.com/jbenet/76afa70955910a2ed097)

If you want to help, the best ways are:

\- Come hack with us! We're particularly looking for Go hackers

\- Just use it!

\- Write applications on top.

\- Report issues: [https://github.com/jbenet/go-
ipfs/issues](https://github.com/jbenet/go-ipfs/issues) and so on.

\- Help us with docs

\- Write another implementation (this is blocked by
[https://github.com/ipfs/specs/](https://github.com/ipfs/specs/))

Some ipfs community members i've seen posting below are: whyrusleeping,
dylankpowers, inconshreveable

Also, IPFS development is sponsored by our company, "Protocol Labs, Inc" \--
we also make Filecoin. We are hiring, so check out:
[http://ipn.io/join](http://ipn.io/join) (yeah-- we just changed the company
name. still need to update ipn.io / get a new domain.)

See you around the tubes! :) \- @jbenet

~~~
taliesinb
Congratulations on a nice talk! And thanks for all this information.

------
wongarsu
Their protocol github repository [0] is much better at explaining the project
than the website.

"IPFS is a distributed file system that seeks to connect all computing devices
with the same system of files. In some ways, this is similar to the original
aims of the Web, but IPFS is actually more similar to a single bittorrent
swarm exchanging git objects."

[0] [https://github.com/ipfs/ipfs](https://github.com/ipfs/ipfs)

Edit: a more compelling summary from an unfinished paper: "The InterPlanetary
File System (IPFS) is a peer-to-peer distributed file system capable of
sharing the same files with millions of nodes. It features a completely
decentralized architecture, secure and efficient peer-to-peer block
distribution, and a path-based naming system supporting distinguishing mutable
and immutable names. The Web today still uses HTTP as the main data transport.
IPFS is capable of evolving the web to take advantage of versioning, p2p
distribution, cryptographic operations, and decentralized publishing.
Moreover, it presents an opportunity to construct a web whose links do not
rot, whose files are deduplicated globally, and whose websites are no longer
“sites”. IPFS is a step toward The Permanent Web."

~~~
jmcmichael
A PDF of paper quoted, 'IPFS - Toward The Permanent Web (DRAFT 1)', may be
found here: [https://github.com/ipfs/papers/blob/master/ipfs-ttpw/ipfs-
to...](https://github.com/ipfs/papers/blob/master/ipfs-ttpw/ipfs-toward-the-
permanent-web.pdf)

------
inconshreveable
If you're looking for the in-depth technical detail of IPFS, you really want
to read Juan's paper on it:

[https://github.com/ipfs/ipfs/raw/master/papers/ipfs-
cap2pfs/...](https://github.com/ipfs/ipfs/raw/master/papers/ipfs-
cap2pfs/ipfs-p2p-file-system.pdf)

I've explained IPFS to friends in the past:

Imagines IPFS as running Git where all of the objects (files, directories,
commits, etc) are available over a BitTorrent swarm. All files are accessed
via their checksum so they have cryptographic integrity even over insecure
networks. Then imagine you can mount it as a file system or browse it via
HTTP, and run your own node to seed files.

Popular files will be seeded by more people, making them quicker and more
reliable to access. As long as clients are still requesting a file, it will
continue to be seeded even when the original publisher goes offline.

Pitching a developer product is hard. Traditional marketing advice encourages
you to pitch value because the implementation details are usually not
important to your customer. Developer products require a more delicate balance
because ultimately we have to work with the technology and want to understand
what its real limitations are.

------
taliesinb
These ideas, and how they combine, is natural and elegant, even inevitable.

For example, the prototypical idea of swapping out Git's disk-trie content-
addressed object store for an actual DHT to get immutable distributed file
system must have occurred to a lot of people when they first learned about
Git's internals.

I don't know about @jbenet's particular journey to these ideas, but I suspect
Rich Hickey deserves a lot of credit for using Clojure (and his fun, smart
talks) to popularize immutability and name/value separation for a new
generation of programmers. Even just ambiently.

That's certainly how I started realizing the power of that paradigm, even
though I was already using it for my day job! _Especially_ when I then saw it
used building Datomic, which of course leverages immutability to totally
refactor the traditional database architecture from client/server into self-
serve-client/server/transactor in a really interesting way.

Also, with Clojure's HAMTs, you've got hash trees and cheap immutable update,
and that again primes you to read about (or reinvent) Merkle trees. So I doubt
nearly as many people would know about such neat CS things without Clojure.

Actually, Bitcoin is also a really obvious way these ideas would become more
widespread, though I don't know how many people really go through the trouble
of understanding the blockchain mechanism.

And Linus, of course, for Git. Git internal design is really nice and everyone
should dip in at some point in their career!

We should give Ralph Merkle a Turing award -- that single idea unlocks so
much. Except we shouldn't because he patented Merkle Trees, apparently.

~~~
_prometheus
> I don't know about @jbenet's particular journey to these ideas

Git is the single largest influence on me. After that, various other papers
(all the CAN literature, bitcoin/bittorrent, CRDTs, etc), and conversations
with people. in particular, two close friends (jzim, davidad).

> We should give Ralph Merkle a Turing award

I agree! Zooko recently told me that a long while ago they dubbed themselves
"Merkle Tree Huggers". I am too!

> Except we shouldn't because he patented Merkle Trees, apparently.

Sigh. patents... :(

~~~
KMag
Please never ever ever implement a Merkle tree as you did in
[https://github.com/jbenet/random-
ideas/issues/20](https://github.com/jbenet/random-ideas/issues/20) . This is
vulnerable to the same trivial collision attack as the original Gnutella THEX
proposal.

EDIT: also, please implement non-power-of-2 leaf node Merkle trees as power-
of-2 lead node trees with the rightmost leaves containing the hashes of the
zero length array, that way the final block along with its hash chain to the
root provides a cryptographic proof of file length. This prevents denial-of-
service mischief where a malicious server claims a given resource is terabytes
large and a poorly written client starts allocating TB of disk space on a
filesystem that doesn't support sparse files.

~~~
KMag
Or, more efficiently, at each level of the tree where there are an odd number
of nodes, add another leaf node containing the empty byte array on the right
side of the tree to make an even number of nodes.

------
lewisl9029
I just came across Filecoin on their Github readme FAQ and it sounds like a
great idea to me:

"What is the relation to Filecoin?

Filecoin is the sister protocol of IPFS. While neither protocol depends on
each other, they are perfectly complementary. IPFS handles high performance
distribution of objects; Filecoin handles incentivizing the world to
backup/seed objects."

Cryptocurrency backed by real, useful, and valuable commodities (storage and
bandwidth).

~~~
schoen
Didn't Mojonation, from the Evil Geniuses for a Better Tomorrow, originally
have plans kind of like that?

~~~
woah
I believe mojonation's currency was not decentralized

~~~
myers
It wasn't, but you could have had multiple issuers of currency. Not that it
ever got big enough to flex that part of the protocol.

------
pierrec
This blog post, which summarizes a talk on IPFS, provides a better
introduction than the landing page or the Github readme:

[https://sourcegraph.com/blog/ipfs-the-permanent-web-by-
juan-...](https://sourcegraph.com/blog/ipfs-the-permanent-web-by-juan-benet-
talk)

It's annoying that there is no exhaustive comparison to existing systems, such
as Bittorrent Sync, but I think that in this case it's genuinely difficult to
find something that really resembles IPFS. The closest thing is probably Git,
but IPFS maps the entire "global network" to your local file system.

It could be useful for any form of infromation sharing, but the author
provides a bunch of "trivial little JS apps" as examples. However, I'm not
sure if web apps are the best application for this, since the database would
need to be hosted in a traditional way anyways (unless you can exclude it from
being version-controlled?)

Like everyone else, though, I wish there was less hand-waving and more down-
to-earth explanations on the subject (especially on the landing page!)

~~~
natrius
The database is Ethereum (for data that needs consensus), your filesystem (for
data that only matters to you), and Whisper (for ephemeral data that doesn't
need consensus, like messages). You build a JavaScript app that calls into
libraries that implement these technologies, and you host your code on IPFS.

The world is currently figuring out the best ways to utilize this sort of
architecture. Join us.

[https://dappsforbeginners.wordpress.com/](https://dappsforbeginners.wordpress.com/)

------
patcheudor
This looks very cool, but I'm wondering about abuse. I watched the video and
did some followup research using Google and couldn't find anything covering
how to handle abuse within IPFS. While I'm sure there will be a lot of photos
of cats and wildlife videos it seems that this solution would be ripe for
abuse by the seedier regions of the web. If I'm participating in IPFS how do
filter for or stop illegal content from being saved and hosted by my node?

~~~
dylankpowers
The only content saved and hosted from your node is content that you have
requested yourself. In that regard it is no different from BitTorrent. Any
content you request is purged from your node over time unless you have
explicitly pinned it to keep around forever. So as long as you stay away from
the seedier regions of the web, you will be safe.

~~~
minthd
> Any content you request is purged from your node over time unless

Won't this mean that rarely used files aren't really permanent ,which is a
known problem with bittorent ?

~~~
dylankpowers
Yes that's true. Files are only guaranteed to be available indefinitely if
someone has decided the file to be noteworthy enough to pin and their system
is online. Ideally, systems like [http://filecoin.io/](http://filecoin.io/)
will be built upon ipfs to enable those that care enough about their files to
obtain distribution through paying a little money to incentivize others to
host their files.

------
yason
Hint to the developers: summary and overview in plaintext, please. I have no
idea what this is beyond the title, but it looks interesting.

------
matt_kantor
Here's a thread about IPFS from last year:
[https://news.ycombinator.com/item?id=8069836](https://news.ycombinator.com/item?id=8069836)

~~~
dang
And another from this:
[https://news.ycombinator.com/item?id=9083849](https://news.ycombinator.com/item?id=9083849).

------
bascule
Hmmm...

[http://blog.bittorrent.com/2014/12/10/project-maelstrom-
the-...](http://blog.bittorrent.com/2014/12/10/project-maelstrom-the-internet-
we-build-next/)

See also:

[https://www.tahoe-lafs.org/trac/tahoe-lafs](https://www.tahoe-
lafs.org/trac/tahoe-lafs)

~~~
whyrusleeping
We talk with the tahoe-lafs guys a few times, our projects have different
goals, and even quite a bit of opportunity for interoperability.

~~~
_prometheus
i recently had the pleasure of hanging out with both zooko and warner and
they're both excellent. as @whyrusleeping mentioned, there's lots of room for
interop + collaboration on things. (in short, we <3 Tahoe-LAFS)

------
ohazi
Can someone explain why all of the hashes in this demo appear to start with
the same few symbols ("Qm" in this case)? Ordinarily I'd expect this to
suggest that it isn't actually a hash, but perhaps there's some detail I
missed.

~~~
zaroth
Assuming it's anything like Bitcoin, it's a network ID prefix separate from
the hash.

~~~
_prometheus
that's a base58 encoded multihash:
[https://github.com/jbenet/multihash/](https://github.com/jbenet/multihash/)
and [https://github.com/jbenet/go-multihash/](https://github.com/jbenet/go-
multihash/)

------
zackmorris
+∞ for double NAT traversal.

~~~
_prometheus
yeah double NATs are annoying :)

------
randall
So what if I want to dynamically create content? Limited to using this as a
sort of cdn edge?

~~~
notduncansmith
I was wondering this as well. Ultimately, distributed systems are just a very
different animal. The first step is designing your data to be stored in files,
rather than a database. In practice of course, all durable databases persist
to disk, so that's not a big deal.

From there though, you have 2 more problems: keeping all data from being
public (maybe becoming "friends" with another user involves a key exchange?)
and distributing updates in a reasonable way. It'd be kinda strange if Lady
Gaga had to buy a ginormous server to serve all the requests to her Twitter
feed. I'm not sure how IPFS is implemented but if it's similar to BitTorrent
in the sense that data can be seeded from any node, then that problem sort-of
takes care of itself. You definitely can't think about "dynamically creating
content" in the sense that twitter.com/:username may render different things
at different times. The concept of web search also becomes an (even more)
interesting one.

On a somewhat-related note, one also has to wonder how this system would
interact with the European "right to be forgotten". Any legislature ensuring
such a "right" would make this protocol a bit of a non-starter (at least in
those jurisdictions).

Ultimately I think IPFS is a pretty cool concept, one I've kicked around a bit
but never fleshed out. Major kudos to Juan and everyone else for taking it on
- I look forward to seeing where this goes.

~~~
hosh
IPFS has ipfs for immutable data, and ipns for mutable ones. It uses a DHT to
name an ipfs hash.

We're actually in the middle of finishing up a demo for mounting a Docker
registry on top of IPFS, and showing we can freely push/pull from it.
[https://github.com/ipfs/container-
demos/pull/19](https://github.com/ipfs/container-demos/pull/19)

------
krick
I watched the Alpha Demo and briefly read through the comments on this thread
and description on the github, but still am a bit confused.

I mean, there already are multiple file-sharing protocols. There are video-
streaming protocols. There are p2p data-exchange protocols. Some of them need
more friendly end-user tools to become widespread, but still, they do their
job and usually are introduced as having some feature, which "all the previous
stuff" lacked.

So what _exactly_ is IPFS and why I would use it over… whatever else I can use
to share files? After all, it's a protocol, and not an end-user tool yet,
right? So what does this protocol present us? What exactly all that "p2p,
decentralized" stuff means in the case of this very protocol?

Why I'm asking it here and not seeking answer in documentation and source-
code? Well, there's quite a real reason beyond my laziness: from what I've
understood so far, it seems like some pretty ambitious project. And ambitious
projects need one short, clearly expressed idea, to avoid becoming a mess
during the development process. I don't see one.

------
justifier
is there a way to circumstantially throttle ipfs specific traffic?

also, note the docs' command lists anchor to the wrong page:
[http://ipfs.io/#ipfs-config](http://ipfs.io/#ipfs-config) ; stead:
[http://ipfs.io/docs/commands/#ipfs-
config](http://ipfs.io/docs/commands/#ipfs-config) ;

~~~
whyrusleeping
we are going to be able to set bandwidth caps soon, most of the code required
is in place, just need some more dev time and testing

~~~
justifier
should have mentioned in the parent.. looks really interesting and feature
rich, great job

------
marcell
Can anyone explain to me how the p2p network determines which files to
persist, given a limited amount of storage available?

------
ilaksh
This seems to make many of my infrastructure problems go away for a number of
applications I can imagine. One thing I am wondering about though: is there
any type of push or pub/sub so I can be notified of updates to some data or
set of data?

~~~
whyrusleeping
Currently no, but I'm brainstorming a pub/sub system that will operate through
the DHT, its still on the backburner for now as other things are stealing my
attention.

~~~
cjbprime
Just in case you're looking for references: I think of
[https://github.com/mnaamani/otrtalk](https://github.com/mnaamani/otrtalk) as
a pub/sub system over DHT.

------
chrisac
@jbenet Interesting stuff! We worked on something related to your work for our
master's thesis - including location-independent naming, a WebRTC-based Chord
DHT and an ALM/PubSub layer using Scribe. Check out
[https://github.com/boplish/core](https://github.com/boplish/core) and
[http://inet.cpt.haw-hamburg.de/members/christian-vogt](http://inet.cpt.haw-
hamburg.de/members/christian-vogt) for the publications.

------
z3t4
We used to have proxies that cached http requests. And there are already
technology for broadcasting.

I do not see how this will be better then say Torrents. Or if anyone would use
this for more then sharing "copyrighted" content.

I'm not saying this is bad, just that you can do better at marketing/selling
it. You mentioned on the second video that this is targeted to those that do
not know how to set up a LAN connection. Then maybe you shouldn't use the CLI
in the demo.

------
CodeGenie
While other comments about security may be right, it probably isn't a reason
not to pursue this. It's a great idea and very worthwhile pursuing.

My main concern is over using hashes as an address. A hash isn't a unique
address but I haven't seen any mention of protections being built in to
protect against the risks of downloading different files with the same hashes.

~~~
icebraining
Searching for "ipfs collisions" returns this IRC discussion, in which joeyh
makes the same point:
[https://botbot.me/freenode/ipfs/2015-03-05/#33399953](https://botbot.me/freenode/ipfs/2015-03-05/#33399953)
(search for "collision" if it doesn't appear immediately).

------
blake8086
I have a question about IPFS. I don't mean this in ANY kind of negative way at
all, but _what problems does IPFS solve?_

I got really excited about IPFS when I first learned about it, but after to
talking to people for a while, I actually made a document titled "What
problems does IPFS solve?" and thought for a long time. It's still blank.

I would love to hear from anyone involved with the project what applications
this could have.

~~~
corndoge
It solves the hacky kludge that is the state of the web. I suspect this
opinion is unpopular, but the web was never designed for web applications. It
was designed for static text and images. JavaScript as it is currently used
(to drive complex web applications) is an abomination. Browsers and their
monstrous, labyrinthine, all-encompassing codebases and extension ecosystems
and security problems are an abomination. They're also broken in every way in
terms of security. Nobody's responsible; the current state of the web is just
the accumulation of two decades of layering technology upon technology and
making the mistake of continuing to layer where we should have built something
new. The initial discovery that web apps ran everywhere by default kind of
prompted a tidal wave of development just for the web that few ever bothered
to take a second glance at. As a result the browser became an operating
system.

The web was not designed for web applications.

IPFS can fix this. Instead of putting all our shit in the browser, with the
steaming mess of CSS, JS, JS frameworks et al, now you can write native
clients that use IPFS to provide the same kind of functionality the Web does.
The infrastructure is there.

The problem is that nobody wants to write a native client for every device.
That's why the browser was popular to begin with -- it's an application as a
runtime, an application that everyone knows about and anyone can install. That
everyone does install. IPFS doesn't solve the problem of having to write
native clients more than once for different platforms. But it sure as hell
solves the hard part, which is the networking part. At least I think so.

~~~
blake8086
So, what's an example? It's really hard to have dynamic content with IPFS
because it doesn't really have the concept of a "request", only a lookup-by-
key type of thing. For example, consider how difficult it would be to make a
simple messaging app with IPFS.

~~~
grrowl
Frequent-ish polling of a file will keep looking for it in the nearby nodes. I
suppose it would be smart enough to build up a network around a specific file.
If it's close enough to real time, it could be a real solution

~~~
blake8086
It's true that works, but which part of that is an advantage over the web the
way it is now?

------
niche
Finally! Our file sharing efforts are no longer bound to Earth

------
liminal
I don't see a way to install it on Windows.

------
kordless
> Gateway Timeout

~~~
ianlevesque
They should've hosted on IPFS!

~~~
rcthompson
I think they already do: "There is a mirror of this website hosted at
[http://gateway.ipfs.io/ipfs/QmTjgwrstCTTWCe1BvoMm5hmX7GMj1q1...](http://gateway.ipfs.io/ipfs/QmTjgwrstCTTWCe1BvoMm5hmX7GMj1q1wE8GzikGvRtyPd)
We plan to transition to hosting it on IPFS as soon as DNS -> IPNS -> IPFS
naming proves robust, so we can use nice looking URLs."

