
Neocities is implementing IPFS – distributed, permanent web - stesch
https://ipfs.io/ipfs/QmNhFJjGcMPqpuYfxL62VVB9528NXqDNMFXiqN5bgFYiZ1/its-time-for-the-permanent-web.html
======
mortenjorck
Reading this felt kind of like the first time I read a writeup on Bitcoin.
There's the same sense of throwing out some old, formerly immutable rules
here, the excitement of something that's going to test some boundaries and
inevitably clash with some authority (how can you, for instance, comply with
the EU's "right to be forgotten" when that information is scattered into a
redundant mesh?). Interesting times ahead for IPFS.

~~~
viraptor
Right to be forgotten however applies only to search engines right now,
doesn't it? The articles the results used to point to still exist. So a
redundant mesh itself doesn't seem to apply here.

~~~
skissane
The "right to be forgotten" applies to any information service, not just
search engines. However, search engines are often easier legal targets since
they do business in the EU and thus can be made subject to EU law. If I have
no business in the EU, and post my website on a non-EU hosting provider, EU
law can demand I take down a page but has no jurisdiction to force me or force
my provider. But e.g. Google, since they do business in many EU member states
(primarily sell advertising), if they disobey EU law, EU governments/courts
can take legal action against them (e.g. confiscate their EU advertising
revenue).

~~~
tokenizerrr
As far as I'm aware that's simply not true. See
[http://ec.europa.eu/justice/data-
protection/files/factsheets...](http://ec.europa.eu/justice/data-
protection/files/factsheets/factsheet_data_protection_en.pdf)

Furthermore the right to be forgotten only mandates hiding results when
someone searches specifically for a name, for example. The results are still
allowed to come up for unrelated search terms.

~~~
skissane
Directive 95/46/EC (the Data Protection Directive) never specifically mentions
"search engines", it talks about "controllers" of personal data. The ECJ ruled
(Case C‑131/12) that search engines such as Google count as controllers of
personal data, and thus are subject to the "right to be forgotten" under the
Directive. But clearly the Directive is applicable to many other things than
search engines, so the "right to be forgotten" cannot be restricted to search
engines either. (Newspapers should be exempt because article 9 of the
directive has an exemption for "the processing of personal data carried out
solely for journalistic purposes or the purpose of artistic or literary
expression", but search engines don't fall into that category, and other web
sites may or may not either.)

The court judgement -
[http://curia.europa.eu/juris/document/document_print.jsf?doc...](http://curia.europa.eu/juris/document/document_print.jsf?doclang=EN&text&pageIndex=0&part=1&mode=DOC&docid=152065&occ=first&dir&cid=437838)
\- paragraph 26 says "As regards in particular the internet, the Court has
already had occasion to state that the operation of loading personal data on
an internet page must be considered to be such ‘processing’ within the meaning
of Article 2(b) of Directive 95/46" \- so by the Court's reasoning the
Directive potentially applies to any web page, not just search engines.

~~~
Trombone12
Indeed, the Data protection directive is not about search engines: it is about
privacy online in general and any web page published or plausibly controlled
from within the European union is subject to it (and of course also to any
IPFS page, although who controls it is harder to say). It has always applied
to a lot more than search engines.

The "right to be forgotten" is a phrase used in the argument for de-listing,
and later became the name for the ruling itself. This was however not a right
that the court granted, but rather the court points out that the data
protection directive stipulates that personal data shall be: """ (c) adequate,
relevant and not excessive in relation to the purposes for which they are
collected and/or further processed;

(d) accurate and, where necessary, kept up to date; every reasonable step must
be taken to ensure that data which are inaccurate or incomplete, having regard
to the purposes for which they were collected or for which they are further
processed, are erased or rectified; """ and judges (in paragraph 93) that in
particular, data that was once in compliance might not be at a later time.

It also states that the failure of the particular data in this case to be
those things means the person described by them can invoke the right to object
to the processing, in this case having personal data listed on a page as a
response to a "search made on the basis of his name" (in paragraph 94).

It is not completely clear, but I think the query being judged upon includes
only the name of the person, and adding a keyword relating to the particular
year, or bankruptcy actions, would be enough to make the data "adequate,
relevant and not excessive".

------
enkiv2
Just to point out -- hypertext systems prior to the web almost universally
expected permanent stable addresses (and this was, for instance, required by
the various Xanadu specs). Enquire (Tim Berners-Lee's immediate predecessor to
the Web) had permanent stable addresses, too.

The web didn't have permanent stable addresses because the web was a hack,
intended as a way to explain Enquire to people who didn't have a background in
real hypertext systems. (Let the entire existence of the Web, with its myriad
flaws, be a lesson to you: your crappiest demo may well become the thing
you're known for twenty-five years later, so make sure your crappiest demo
doesn't suck as bad as the Web does.)

~~~
oskarth
> (Let the entire existence of the Web, with its myriad flaws, be a lesson to
> you: your crappiest demo may well become the thing you're known for twenty-
> five years later, so make sure your crappiest demo doesn't suck as bad as
> the Web does.)

The web is amazing, and you are using it right now. It has flaws, but so does
anything that achieves success at the level the Web has. How many technologies
have a direct impact on _billions_ of people's lives?

There are subtle reasons why "crappy" technology often wins. One is that worse
is often better ([http://www.jwz.org/doc/worse-is-
better.html](http://www.jwz.org/doc/worse-is-better.html)) and the second is
that people are more prone to complain about popular technologies, which
create the illusion that their alternatives would _obviously_ be so much
better.

EDIT: Note that the web is not just a technological phenomena, it's a social
one. If you care about the real world, it doesn't make sense to look at it as
a technological innovation in isolation. Anyone can dream up The Perfect
Protocol in a research paper, it's an entirely different beast to actually do
it and make it work for billions of people.

~~~
tikhonj
A big (biggest?) reason is that popularity is largely independent of quality.
Things get popular for lots of reasons heavily affected by the shape of social
networks and early choices by a small number of people involved—often based on
what they have and have not heard of more than an analysis of what is better
or worse. Quality plays a small role at best.

This Nautil.us article[1] does a good job of summarizing the relevant research
on the issue and references some supporting studies.

> _My own research has shown that fame has much less to do with intrinsic
> quality than we believe it does, and much more to do with the
> characteristics of the people among whom fame spreads._

Put another way: the web has a direct impact on billions of people less
because it's any good and more because popularity is inherently a power law
and it happened to come out ahead. This doesn't invalidate any criticisms of
quality!

It's perfectly possible if not likely for something popular to be crappy
because most things are crappy and that's not enough to stop them from getting
popular. Then they require tons of engineering effort and hacks to scale and
improve again not because of quality but because of legacy. The web is what's
there already, after all!

That's how things are, but that doesn't mean it's inevitable or a good state
of affairs.

I really wish people wouldn't conflate popularity, quality and success. The
first two operate on different axes and the last, if it doesn't, _should_.

[1]: [http://nautil.us/issue/5/fame/homo-narrativus-and-the-
troubl...](http://nautil.us/issue/5/fame/homo-narrativus-and-the-trouble-with-
fame)

~~~
oskarth
> Quality plays a small role at best.

This is simply not true in the long-term. Good technologies survive because
they are useful.

Of course quality is not _enough_ to ensure success, but it sure as hell
matters for long-term survival. Homer, Shakespeare and the wheel survived
because they are very good / useful. Are there lost Homers? Of course there
are. Will the Web survive 100 years? I have no idea, and neither do you. A
decade? Most likely.

Be careful with conflating what _is_ with your idea of what something _should_
be. If you think something sucks and work on making it better, that's great!
Keep doing it, and maybe someday we will have something that's even better.
But don't diss the popular just because it isn't ticking all of the "it-
should-be-this" boxes.

~~~
enkiv2
Quality is definitely a small factor in success, when we're talking about
network effects. When we add in capital, with all its perverse incentives,
quality matters even less.

Technologies become popular primarily based on how easy or cost-effective they
are to replicate. The fewer concepts you introduce, the lower the bar is for
people to reproduce your intellectual inventions; the cheaper the materials
and manufacturing process, the easier it is for knock-offs to be manufactured.
Popularity means a plurality of knock-offs. Quality often requires subtlety
and attention to detail -- two things that add effort to replication, and thus
work against popularity.

The web became popular not because it was better than the alternatives but
because it was stupider than the alternatives -- it had fewer features, fewer
ideas, and fewer nuances. The web is no longer less featureful or nuanced than
its competitors, because unplanned organic development grows complexity like a
rat with a radium drip grows tumors, so as a result the very simplicity that
made the web grow so quickly in the first place would prevent it from growing
again, had it not become the focal point of an enormous amount of lock-in and
path-dependence.

The web is a huge bureaucratic mess right now, just waiting to be disrupted by
a superior hypertext system.

------
PhaseMage
I absolutely love the concept and the underlying moral reasoning for why we
need IPFS!

One of the first questions that came to me: Is there any way to add a forward
error correcting code, like an erasure code to IPFS? I didn't find any
discussion of this in the IPFS paper.

This seems somewhat critical to be able to compete with modern centralized
storage systems which are likely to use FEC extensively to provide the level
of redundancy that customers expect. Modern FEC codes can provide phenomenal
levels of redundancy with less than 50% overhead. IPFS seems to rely on many
fully-redundant copies?

~~~
crypt0x
Interesting Idea! I think TAHOE-LAFS also uses something like this?

From a practical standpoint, I'd like my local node to have a full copy so
that my friends coming over don't need to reach out to the (maybe offline)
internet but I don't see why we shouldn't support encoding the chunked blocks
in such a way?

------
kang
This sounded like many of the bitcoin projects, specially filecoin and upon
reading further I came to know it is by the same guys.

Similar projects have been in development for past few years such as
[https://github.com/feross/webtorrent](https://github.com/feross/webtorrent)
and zeronet

This has same problems as the bitcoin infrastructure though:

1\. It is unscalable. A page built on IPFS receiving huge inflow of comments
would generate many diffs quickly and as soon as they spread in the network
they get outdated, thereby clogging space on people's disks but more
importantly clogging the bandwidth where nodes compete to download the quickly
changing diffs.

2\. This is not completely decentralised because it uses bittorrent trackers
to identify the nodes in the network. Taking down the the trackers would take
down the system as well.

Webtorrent is an already working, fast, alternative to IPFS but still
centralised. Think of it this way, can you do a pure peer-to-peer webrtc site
without needing a ICE/TURN/STUN server? Peer discovery is the centralised part
of the problem.

~~~
conceit
1\. Don't use it for what's basically messaging

2\. I thought torrents can work decentralized e.g. using magnet links and
DHTs.

~~~
kang
Even DHTs require a server to bootstrap the network, due to a node's unicast
nature on the internet.

~~~
conceit
That's why there's the magnet links, isn't it?

------
dperfect
I wish this article mentioned something about BitSwap; before reading the IPFS
paper, I was doubtful about anyone wanting to participate without incentives.

Still, even after skimming the paper, it sounds like a _lot_ really depends on
how well BitSwap is implemented. At one point, the paper says this:

> While the notion of a barter system implies a virtual currency could be
> created, this would require a global ledger to track ownership and transfer
> of the currency. This can be implemented as a BitSwap Strategy, and will be
> explored in a future paper.

So it kind of sounds like they need something like bitcoin, but don't want it
to be tightly coupled to bitcoin - which is probably smart. But without a
single clear solution for a reliable BitSwap strategy (and I'm not sure how it
could be reliable without a distributed global ledger of some kind), it's hard
to see how/why the necessary resources will be contributed to the IPFS
network.

~~~
_prometheus
Check out another protocol we (the IPFS creators) are working on:
[http://filecoin.io](http://filecoin.io)

~~~
dperfect
Interesting - I'll have to check it out. Thanks for pointing me to this.

Really cool stuff by the way - I'm very excited to see people working on this,
and I hope IPFS (or something like it) is successful.

------
daveidol
Awesome writeup!

Sure, it's easier to setup your own web presence with "the cloud" today than
it used to be, but this only further centralizes control of content to the big
cloud providers.

Not to mention the cost of bandwidth when serving content via HTTP. Sure, you
can distribute your content via BitTorrent, but what kind of user experience
is that? Can my grandma use it? Probably not.

I hope IPFS sees further adoption.

~~~
_prometheus
Thanks! Help us make it :) --
[https://github.com/ipfs/](https://github.com/ipfs/)

and, you'll love what we have coming. Soon, you won't need to install ipfs at
all :D

~~~
giodamelio
Are you talking implementing IPFS on the browser level? Is there public code
yet?

~~~
reit
See [https://github.com/ipfs/ipfs.js](https://github.com/ipfs/ipfs.js)

It's coming along, not fully functional yet but you can watch its progress (or
contribute code :D ) at [https://github.com/ipfs/node-
ipfs](https://github.com/ipfs/node-ipfs)

~~~
antiq
Does this mean that browser-only users will not be able to 'host' content?

------
giancarlostoro
So is this primarily for static websites? I don't see dynamic websites going
too well with this system.

Edit:

If someone like GitHub supported this for GitHub pages it would be a great
step forward for this as well.

~~~
adrusi
With IPNS, which allows the creation of links whose destination can be mutated
by whoever has the private key, you can implement quite a lot of
interactivity. You won't ever be able to implement facebook over this
platform, but you can replicate all of its functionality by using a tumblog-
like architecture. Facebook's design is inherently centralized, and therefore
doesn't really fit into the world of IPFS.

The idea is that every user owns their own content, not some central
organization. Every user maintains a log of all of their actions, like an RSS
feed. Every time they make a status update, that's an item. When they comment
on someone else's status update, that's an item. Even when they just like
something, it's an item. Each update of the action log would also have send
ping to all friends, over some kind of distributed IM like tox. They already
need an IM connection like that for messenger.

Given that information alone, anyone can reconstruct the activity of all their
friends into a news feed, and post new content. You'd need a lot of client-
side leverage, and the ability to upload to the IPFS network from the client,
but all the dynamic information is there.

And there you have a fully distributed facebook clone. More resilient and
privacy-sensitive and monopoly-averse than even tools like diaspora and
statusnet.

~~~
colordrops
> Facebook's design is inherently centralized

Not really. There's no reason you couldn't implement Facebook's core
functionality with personal servers containing a profile at each node and a
pub/sub (or even just pull) system connecting each node with its friends.

~~~
junto
Actually, this is exactly what I'm looking for. I want to be able to control
my photos and other content, and share that with family and friends.

All data is encrypted and my server can be hosted anywhere (just as simple as
setting up a WordPress instance).

My own "news feed" are a combination of all of the other nodes I'm subscribed
to.

My problem with Disapora is that they push the "sign up to someone else's pod"
rather than "install your personal pod".

It just feels like a multi-centralised model. Rather than one Facebook with my
data, now there are 311 "Facebookesque" Diaspora Pods (as of today).

As much as people hate the idea, a WordPress like (PHP) application as a
"personal server" would be successful. There are lots of hosting companies out
there where you can get up and running for peanuts.

My aim would be for a simple install:

\- Download software and upload to hosting company

\- Enter MySQL auth details

\- Choose local filesystem or third party (S3, DropBox, Google Drive, MS
SkyDrive) with OAuth as the backing store (files stored encrypted).

\- Finish

Then I would be able to subscribe to other friends and family personal
servers, just like RSS feeds. Friends can access my own personal server
through OAuth and leave comments on content. Those comments are echoed back in
my private feed which is displayed in their merged news feed on their own app
instance.

The technology is already there. It just needs someone who is damn good with
PHP!

~~~
ianopolous
We're heading towards this with Peergos
([https://peergos.org](https://peergos.org) source at
[https://github.com/ianopolous/Peergos](https://github.com/ianopolous/Peergos)).
Encrypted, self-hostable file storage, sharing and social network. We're
considering using IPFS under the hood for fragment storage/finding. You'll be
able to either use your own disk, or enter credentials for a cloud provider.
It's still very early days yet though.

------
johnmaguire2013
I'm very excited to see what happens with IPFS. The article talks about
replacing HTTP however, and this is definitely a tricky task.

Someone in this thread already asked one of my questions (So is this primarily
for static websites?) but my second question is: So is this primarily for
personal websites?

I'm having a hard time finding a good way for Facebook, for example, to
monetize their website. Targeting ads go out the window with mostly static
content. Even more so though, what about Netflix? How is DRM done? How do you
make sure only the correct users can access your objects?

edit: Also, doesn't a "permanent web" have an inevitably fatal flaw that you
can't free space?

~~~
logn
One monetization strategy in general for distributed networks is that users
pay for bandwidth. They're rewarded for the bandwidth they provide. They
profit if they provide more bandwidth than they use. And app developers are
rewarded when their apps cause bandwidth to be used. I don't know if IPFS
works this way.

As far as paying a premium over raw bandwidth to access premium content
(Netflix), you could pay them to unencrypt it for you to download. DRM though
isn't secure in the first place and decentralization won't make it better.

~~~
johnmaguire2013
Oh, of course, I'm not trying to argue that DRM is good or that
decentralization would make it better or even work as well. The problem of
course is that without it working as well as it currently does, premium
content providers have no incentive (and in fact, are decentivized) from
making the switch.

The general idea of decrypting premium content at a cost would work fairly
well, however I'm wondering if it's something that can be done easily and
dynamically via IPFS. For example, can I, without burdening the network with a
billion copies, encrypted with each the client's key, hand that data off to
them? If I simply decrypt it and hand it over, it's on a distributed network
and anyone can grab it.

~~~
TuringTest
Delivery of the unencrypted premium content would likely happen through a
direct link with a different protocol. As you say, the content providers don't
want that content to be accessible to everyone in the network.

The website used to navigate the catalog of premium content could be built as
a IPFS page though.

------
rdancer
When a massively distributed file system[1] was introduced into the Linux
kernel circa Y2K, there was what I remember as a lot of hype.

Nobody, but _nobody_ wanted it then (or since, it's still part of the kernel),
and I would be surprised if anybody wanted something similar today.

[1]
[https://en.wikipedia.org/wiki/Andrew_File_System](https://en.wikipedia.org/wiki/Andrew_File_System)

~~~
njloof
All undergraduate accounts at Carnegie Mellon were on AFS in the 1990s. It may
have been ahead of its time -- all I know is (a) it was much less reliable
than NFS; (b) its security policy was different enough that people (admins
included) left lots of security holes; and (c) *.struck

~~~
mileswu
CERN still uses AFS extensively for all users' files. I believe they use a
forked version of OpenAFS. My experience with it at CERN has been okay. It's
been fine to use locally, but transatlantic over the Internet can be very slow
(it does work eventually though).

I'm not sure if there are any alternatives that work any better over long
distance. I suppose CERN also has CVMFS [1], which is a read-only caching
filesystem that retrieves the files from upstream via HTTP. This works much
better, but it is read-only so only satisfies certain use cases.

[1]
[http://cernvm.cern.ch/portal/filesystem](http://cernvm.cern.ch/portal/filesystem)

------
viraptor
I was running IPFS as "my own pastebin for files" for a while (it's great!),
but was wondering what can they do to improve adoption / popularity. This move
is amazing! Useful, interesting for people who care, and visible for others.

Next, I'd really like someone to implement git backend in IPFS / IPNS. Right
now there's [https://github.com/whyrusleeping/git-ipfs-
rehost](https://github.com/whyrusleeping/git-ipfs-rehost) but that's just
simple hosting.

~~~
_prometheus
see also: [https://github.com/cryptix/git-remote-
ipfs](https://github.com/cryptix/git-remote-ipfs)

~~~
viraptor
I love the person who made it. It's exactly what I needed.

~~~
crypt0x
thanks :) if you find any issue - let me know

~~~
viraptor
I'm assuming that once ipns starts to work reliably you'll add the support for
it? Or is there something that would prevent it? (I mean for single-user repo
anyway - multi-user would have to share the key... somehow)

~~~
crypt0x
Right, that's the idea.

Nothing is technically stopping me from adding ipns except finding the time to
do it.

------
wodenokoto
This reminds me of a feature I sorely miss on my blog (and almost every other
blog I visit!) which is for the blogging engine to automatically take a snap-
shop of webpages you link to and let users read the cached version at will.

If I link to a page and it goes down, or the content changes, it in a way
changes the content of my blog too - in an unwanted way that is!

~~~
crypt0x
This! I already thought about starting a project for a local browser proxy
that builds an index of <$url,now()> => $snapshotHash as I go. Maybe mixin
readability and metadata like referrals.

I find it quite belittling how browser vendors treat their history features in
terms of archival.

~~~
wodenokoto
Zotoro saves snapshots, although it is for scientific references (citation
manager). Due to ads and what not, it will balloon in size surprisingly fast.

~~~
crypt0x
Interesting. The ads problem is what i meant with 'mixing in readability' by
using techniques to reducing a page to its core content. Only text paragraphs
from blogs, images from flickr/imgur albums, audio/video from media sites,
etc.

------
gtrubetskoy
My first thought is how does it handle SHA hash collisions? Googling for it
took me back to HN:
[https://news.ycombinator.com/item?id=9322489](https://news.ycombinator.com/item?id=9322489)
AFAICT it's not yet resolved.

~~~
hardwaresofton
In the (relatively rare) case of a SHA hash collision, couldn't you try and
resolve a collision by using (the results) of other hashing functions?

While two things may produce the same SHA hash, they may differ with other
hashing functions.

~~~
TheDong
All you're doing then is fiddling with probabilities. You can do that just as
well by changing the length of the sha unless you have a reason to think sha
is broken.

However, the probability of a hash collision is already incredibly low. As a
demonstration of this, here's a list of a tiny subset of bitcoin private keys:
[http://directory.io/](http://directory.io/)

The human mind just isn't capable of understanding how large of a number the
number of sha hashes is.

~~~
hardwaresofton
Right, but fiddling with probabilities is exactly what you want to do in that
case right? I don't know if a hashing algorithm that has no collisions exists
(I suspect the answer is no, maybe other than the identity function). If
collisions start to become a problem, you can move that problem further out
(much much farther out) by mixing more hashing functions in, which was my
point.

It's well documented (of course also by the link you posted), the super low
likelihood of hash collisions, but I think (hope) the original poster knew
that -- was trying to answer assuming that extraordinary case actually
happened.

~~~
walrus
> _I don 't know if a hashing algorithm that has no collisions exists (I
> suspect the answer is no, maybe other than the identity function)._

There is no hash function without collisions. The set of inputs is infinite
but the set of outputs is finite. (The identity function isn't a hash function
--its output isn't a fixed length.)

~~~
hardwaresofton
Thanks for the correction! Somehow forgot the fixed length output requirement
of hashing functions.

------
mindcrime
Well, I'm not 100% convinced this is going to take off, but I'm intrigued. I'm
going to make an effort to get Fogbeam Labs' website up and running on IPFS
shortly as well. I'm curious to see where this goes.

------
rgawdzik
While IPFS can certainly complement HTTP (for example in preventing censorship
in states such as China), it will not replace HTTP. Decentralization does not
work with centralized services (99% of the web) such as Twitter, Google, etc.
You'll need HTTP to run server code. I for one will not be hosting a node
where arbitrary code can be run. Besides the obvious fact that a centralized
service wants to control what serves from their backend, and the funny
scenario where a 51% consensus can decide what Google's logo is.

~~~
reit
>You'll need HTTP to run server code

It's better if your application is redesigned to be distributed on a
fundamental level, but you can still send custom data through IPFS in a
standard client/server relationship if you want to, see
[http://gateway.ipfs.io/ipfs/QmTkzDwWqPbnAh5YiV5VwcTLnGdwSNsN...](http://gateway.ipfs.io/ipfs/QmTkzDwWqPbnAh5YiV5VwcTLnGdwSNsNTn2aDxdXBFca7D/example#/ipfs/QmThrNbvLj7afQZhxH72m5Nn1qiVn3eMKWFYV49Zp2mv9B/api/service/readme.md)
for an example.

>I for one will not be hosting a node where arbitrary code can be run

This isn't a distributed computation platform like Ethereum, it's purely for
transferring files, and the only files you host are those that you explicitly
request. Also, IPFS is not a blockchain with a 51% to take over, it's an
entirely different animal, the only way to redirect a site is with the correct
associated public/private key (unless SHA-256 gets broken at some point in
which case IPFS migrates to a different hashing algorithm).

------
lewisl9029
So let's say I'm building a decentralized chat app built on some DHT and would
like to host it on IPFS.

I imagine getting the static html/js/css for the client-side app onto IPFS
should be simple enough, but how would I handle bootstrapping?

Even in a decentralized service, clients need to connect to a seed server
somewhere to start querying for other peers on the DHT. Would it be possible
to host such a seed server on IPFS? Or would the seed server still need to
live on a traditional server somewhere on the "old web"?

~~~
whyrusleeping
The way our network currently bootstraps is to dial a set of peers that are
hardcoded into the default config file. We also have mdns for finding peers
that are on your LAN without the need for any hard coded addresses.

~~~
lewisl9029
Thanks for the quick reply. I was actually thinking of an app running on a
separate IP-based DHT, but your reply brought to my attention that IPFS
actually already runs on a DHT.

Since IPFS runs on its own DHT, are there any plans for a client-side IPFS DHT
library for apps to use to connect to peers on the IPFS network? Similar to
[https://github.com/feross/bittorrent-
dht](https://github.com/feross/bittorrent-dht), but with native support for
IPFS peers?

Something like this would make building a client-side real-time app based on
IPFS a much more streamlined experience since developers won't have to depend
on an external DHT.

------
emilburzo
But what happens if mobile devices will become the norm instead of PCs?

The number of hosters will be far less than the number of clients.

The same issue is now in bitcoin, lots of lightweight clients (that don't keep
a full ledger), and not so many full nodes (probably even less after mining
rewards will be zero).

~~~
david_ar
On the other side of the coin, I'd say mobile clients offer more opportunities
for local p2p distribution (Bluetooth, etc)

~~~
emilburzo
So even more battery drain?

I mean indeed local p2p distribution sounds nice, but what's the use if the
device is dead in a few hours?

~~~
reit
A neat system might be, when the reciprocation system gets more advanced, to
allow for nodes to build up a reputation/trust with other nodes while they're
at home sitting on charge/wifi. Then, they can 'cash in' these brownie points
when they're out and about, to download stuff without reuploading anything at
the time.

There are a number of ways this reputation could be represented, perhaps one
possible approach might involve exchanging some unit of cryptocurrency
(Filecoin?), or perhaps a web-of-trust style system where nodes publicly
'trust' other nodes based on how much use that node was to them. Personally, I
think cryptocurrency is the way to go here, but there's lots of work out there
on P2P incentivization strategies.

Of course, this could work in tandem with the standard barter system that
bittorrent uses, IPFS is not locked into any one reciprocation algorithm and
could happily use multiple different systems depending on the situation.

------
vancan1ty
I have always wondered if there is some way to make a distributed network
which provides content in such a way that users' freedoms are maximized. I
don't particularly like handing over all control of my data and internet
presence to cloud providers. IPFS looks really, really cool!

As ot asked, I am also curious about how this compares to other efforts to
create decentralized networks, like Freenet and GNUnet. I definitely plan to
pick one in the coming weeks and start using it, hope this catches on.

------
ot
That's interesting, but I see no reference in the website or the paper to
GNUnet or Freenet or other existing DHT-based file distribution networks. How
does this compare?

~~~
david_ar
[https://github.com/ipfs/faq/issues/18](https://github.com/ipfs/faq/issues/18)

------
HeyImAlex
> Organizations like the NSA (and our future robot overlords) now only have to
> intercept our communications at a few sources to spy on us.

If you had the hash of some content, couldn't _anyone_ find _all_ of the ips
currently serving it?

EDIT: Found this

[https://github.com/ipfs/faq/issues/12](https://github.com/ipfs/faq/issues/12)

~~~
typish
"ipfs dht findprovs <hash>" gives the hash of the nodes serving it. I'm not
sure how to get from that to an IP.

~~~
whyrusleeping
"ipfs dht findpeer <peerID>" will search the DHT for connection info on a
given peer.

------
roel_v
\- So I want to download a file over IPFS today, how do I go about it? What
sort of client do I use?

\- Now I want to view a whole web page. Is that possible?

\- Let's say I have a site that I want to publish through this, along with a
bunch of files (images, downloads). I'm serving that site now through Apache.
Do I need to 're-publish' each file (using the command in the article) every
time a file on there changes? Or is there some automated way to have a
distributed version of _the latest version_ of my site? I mean, it's fine and
dandy that people can store cached copies, but what I want is that if there's
an update to my software, that people actually get the latest version when
they click 'download', not that they get an old version because some dude
somewhere made a mistake at one point 'hosting' a static version of my site
and not updating, and visitors just happening to use that version because that
dude is closer to them (in network topology terms) than I am.

~~~
crypt0x
1) you either use one of the public, legacy http gateways and dial
[https://ipfs.io/ipfs/$myContentHash(/andDepending/maybe/Some...](https://ipfs.io/ipfs/$myContentHash\(/andDepending/maybe/SomePath.png\))
or you use the go implementation which is available for download at
[https://ipfs.io](https://ipfs.io) as a compiled binary.

You can use it like a unix command line tool to add and get files but you can
also run it in daemon mode. It then also opens a HTTP server locally which
allows you to use the same URLs as above just with
[http://localhost:8080](http://localhost:8080) as the schema and host.

2) You can use the ipfs cli tool to add a directory with your sites html and
other static content. The client will then give the the root content hash.
Then you open /ipfs/$rootHash/index.html or /site2.html and voila! :)

Btw, this article is serving from ipfs. Try 'ipfs get
QmNhFJjGcMPqpuYfxL62VVB9528NXqDNMFXiqN5bgFYiZ1' and take a look in that
directory on your harddrive. Also: when you now run 'ipfs daemon' your node
will help with requests for this content.

3) To update content with a static name, you can use IPNS, it's also mentioned
in the article. IIRC: To resolve those entries (and get the latest ipfs
content hash) the node with the corresponding key for the ipns entry needs to
be online. If it isn't, you get an error like 'Path Resolve error: could not
resolve name.'. Maybe come to the #ipfs irc channel on freenode for more
specific questions.

------
phodo
This is great news. Dapps [1] are coming, and DHT [2] /storage technologies
like IPFS (although IPFS is more than just a DHT) are the other much-needed
side of the coin - no pun intended - to make that reality happen. Exciting
times ahead. [1] decentralized app [2] distributed hash table

------
nly
A universal distributed file system that looks native on my system (thanks to
FUSE) and caches content I access locally, while serving it securely to
others, is interesting... assuming you can solve key problems like search,
identity when you want it, privacy when you dont, and human memorable
addressing... which afaict IPFS doesn't address.

The "permanent web" on the other hand you can create using MHT archive files
and BitTorrent magnet links. I struggle to get excited about that.

Wake me up when someone figures out how to make apps that require seamless
multi-user collaboration, like say distributed clones of HN, Facebook or
Github.

~~~
kordless
> Wake me up when someone figures out how to make apps that require seamless
> multi-user collaboration, like say distributed clones of HN, Facebook or
> Github.

There are people working on this, so stay awake.

------
SimplyUseless
Distribution is a great idea. Decentralizing is a great idea. Open Source is
great idea.

Permanent.... is a great idea and a very conflicting idea.

While the utility of permanent cannot be denied but the implications for
privacy, piracy are far reaching.

~~~
david_ar
This isn't freenet where you have no control over what you store. If you have
an objection to certain content, you don't have to store it. It's only
permanent so long as at least one person is willing to distribute it.

~~~
kitd
eg. dodgy Snapchat pics your mate's gf sent him?

Transparency can work both ways.

------
xhrpost
I wonder what the legal implications are for hosting a node. I imagine you
would have to comply with DMCA requests in the US, so just hosting a node
would require you to setup a whole process for that.

~~~
zxcvcxz
Unless everyone just ignores DMCA requests and they become unenforceable.

~~~
nightpool
Ignoring DMCA requests opens you up to being sued for copyright infringement
though—that's the entire purpose, is to provide a copyright "safe harbor" for
places that host user generated content, provided they meet certain
restrictions.

------
dingaling
Just reading back though the specification of IPFS, be wary about the claims
of 'permanent Web'.

Distributed content in IPFS is eventually purged from nodes unless one of the
hosting nodes has pinned it to be retained. Therefore, if no-one at Time _x_
views certain content and pins it, then unfortunately at Time _x_ \+ _n_ that
content might disappear just as thoroughly as under HTTP.

Unfortunately I fear that means that 'popular' content persists whilst niche
and unique data might still fade away.

~~~
hosh
The idea is that people who care most about keeping it up will pin it, or pay
to keep it pinned.

~~~
Coding_Cat
If this gets implemented in browsers (native or as an extension) I would hope
that bookmarking would serve that purpose. Seems like to logical choice.

Perhaps it is also possible to tweak the caching algorithm to keep track of
files that are rare on the network.

~~~
david_ar
I believe incentives for seeding rare files are in the future plans for
bitswap

------
spuz
For more details about IPFS and other potential uses I recommend listening to
this Software Engineering Daily episode with the creator of IPFS Juan Benet:
[http://softwareengineeringdaily.com/2015/08/25/interplanetar...](http://softwareengineeringdaily.com/2015/08/25/interplanetary-
file-system-ipfs-with-juan-benet/)

------
zkhalique
We've been thinking about this problem for quite some time. My friends and I
are passionate about re-decentralizing the web. The biggest challenge we've
come across is this:

 _Many users want to do things with other users_

That means there has to be a way to persist state and history (of a group
activity, an evolving resource, etc.) across machines. Mental poker is still
very hard to do for groups. Since most clients will not have amazing uptime,
that means you still need a network of servers. Servers that manage security
behind a firewall.

So, the conclusion is -- for group writable data, the best you can do is have
servers be completely interchangeable, but you will still need servers and
networks of servers.

The good news is, many of the challenges are orthogonal to each other.
Someone's building a Mesh network, for instance. Someone else is building the
protocol layer with IPFS. We're building the social layer for this coming web.

~~~
ZenPsycho
Couldn't you do group editing through using lisp inspired link lists? See a
comment thread? CONS your comment onto the end of the comment thread and push
into the ether, and notify about the new list head.

Take other patterns from clojure immutable data structures for various other
kinds of group edited time travelling sites

~~~
zkhalique
It might be possible to do some sort of self contained data structure that
authenticates everybody and only allows authorized access to it, which solves
the Byzantine Generals problem like BitCoin does, and which resists forking
and instead uses vector clocks and Operational Transformations to consolidate
actions done to the tree. But even so, how would it resolve conflicts? It
could be a long time until a conflict is discovered, and then do edits get
dropped, or what?

~~~
ZenPsycho
Why would you want to resist forking, and incur all that other complexity when
you can just have an "authoritative" source for the entry point? Similarly, if
someone wants to fork the conversation off and become a new "authoritative
source" why would you need to stop them?

as for conflict resolution, merging, etc, it kind of looks like a (mostly)
solved problem. I don't see how conflicts could occur in a comments stream.
It's append-only.

~~~
zkhalique
It's append only until B and C get a copy and each append their own comments,
then D and E get two separate forks and append their own comments. That can
work for trees to which you only append, such as comments. That does NOT work
for any operations that do non-append operations on trees, such as operations
which rearrange the tree into a line.

So yes, for append-only tree structures, this may work. In terms, of say, a
Google Doc that means you can never delete or insert any text, but only append
text to the end of paragraphs etc. Certainly very limiting.

Unless there's something you haven't described, in which case I encourage you
to provide a more in-depth comment describing a solution for actually
inserting or modifying the tree.

~~~
ZenPsycho
actually, google docs ARE append only. They use the vaunted google wave tech
of Operational Transforms. every single timestamped keystroke is saved
separately in the database. when two users are involved, and editing
simultaneously, the OT algorithms resolve the merge conflict on the spot. it
kind of looks like a tree structure, or an append only array. the actual
document you are looking it is actually something like a flattened keyframe in
a movie that's reached the end.

------
phatbyte
This looks great, congrats to everyone involved, this is really exciting
stuff. I got a couple questions I didn't quite understood.

1 - What prevents someone from altering a JS file and serve it to other peers?

2 - Is it possible to obtain all the version of an object and see it's history
of changes (like Git)?

~~~
charriu
To 1: If you alter the contents, it's hash would change, thus it's address
would change, if I understood correctly.

------
crypt0x
You might also want to take note of the browser addons for firefox[1] and
chrome[2].

Those redirect requests to public gateways
([http://(gateway.)ipfs.ip/ipfs/$path](http://\(gateway.\)ipfs.ip/ipfs/$path))
to the http server of the local ipfs daemon. It fetches the content using
bitswap protocol and makes it in turn available to other nodes, helping with
the distribution.

[1] - [https://github.com/lidel/ipfs-firefox-
addon](https://github.com/lidel/ipfs-firefox-addon) [2] -
[https://github.com/dylanPowers/ipfs-chrome-
extension](https://github.com/dylanPowers/ipfs-chrome-extension)

------
espes
Is there any incentive to host nodes?

~~~
shortly
yes. [https://ipfs.io/](https://ipfs.io/) has more info, in particular, the
linked pdf.

~~~
adrusi
Also check out filecoin, which as I understand uses hosting data as a proof-
of-work for cryptocurrency.

------
beyti
I just couldn't digest all, after I read this entence in the description:

\--- When neocitieslogo.svg is added to my IPFS node, it gets a new name:
QmXGTaGWTT1uUtfSb2sBAvArMEVLK4rQEcQg5bv7wwdzwU. That name is actually a
cryptographic hash, which has been computed from the contents of that file.
That hash is guaranteed by cryptography to always only represent the contents
of that file. \---

And opened a reddit/eli5 question, just to understand "if we can use a hash to
identify content uniquely?" here:
[https://redd.it/3k8g51](https://redd.it/3k8g51)

Can anybody elaborate and lighten me up a bit?

~~~
VikingCoder
Okay, I can't explain it like you're 5... But here's as close as I can get.

You've got DNA. Everyone's DNA is unique (shhh, just go with me for a minute).
In fact, let's say that we found about 140 GATC pairs, which were also unique
(we don't need your whole sequence, just those 140 pairs). Meaning, if we take
your 140 GATC pairs, and we take my 140 GATC pairs, we'd be guaranteed to get
different results.

Now, if I want to refer uniquely to a person, I can just use those 140 GATC
pairs, which looks roughly like QmXGTaGWTT1uUtfSb2sBAvArMEVLK4rQEcQg5bv7wwdzwU
(when expressed with upper + lower + numbers).

But, you've been thinking to yourself, WHAT ABOUT TWINS?!?

Well, great question...

...but the cryptographic hash that we use is one where twins are EXCEEDINGLY
UNLIKELY. Like, amazingly, shockingly, oh my god, unlikely. They exist, but in
practice they're so uncommon that it doesn't matter.

Maybe sometime, some unfortunate guy is going to request a picture of Natalie
Portman, and get an MP3 of Ted Nugent. It could happen. It's just super
unlikely to ever happen in practice.

So, there you have it - a cryptographic hash is... excuse me, because this is
a supremely flawed analogy... kind of like the DNA of a file.

~~~
beyti
Thanks for the response, I've just thought of that single example and couldn't
digest it, but it is just me.

It's like [guid]([http://stackoverflow.com/questions/2977593/is-it-safe-to-
ass...](http://stackoverflow.com/questions/2977593/is-it-safe-to-assume-a-
guid-will-always-be-unique)) as far as I can understand and even bigger that
we shouldn't worry.

------
twoodrow
I participated in an REU a couple years back working with Named Data
Networking. Seems pretty similar... The _ahem_ central problem lies with
ourselves: TCP/IP was designed as a communication network, not a distribution
network. NDN seeks to be a ubiquitous data distribution network. Iirc, the
design helps fend off DDoS.

[https://en.wikipedia.org/wiki/Named_data_networking](https://en.wikipedia.org/wiki/Named_data_networking)
[http://named-data.net/](http://named-data.net/)

------
bobajeff
This brings up a important issue with the web and internet we have now. And
that's the difficulty we have in supporting new protocols in clients.

------
JosephRedfern
"especially when bandwidth rates for small players start around $0.12 per
gigabyte" \- who the heck is paying $0.12/gb?!

~~~
Tepix
More like 2€/TB

------
kitd
Video streaming is mentioned, but I don't see how this works for streaming,
unless the metadata is the content. Is this right?

------
Ono-Sendai
Sounds like the DHT lookups could be quite slow, 20 queries to find the
content is going to be a lot of latency. Sounds like a better algorithm than
binary search (which I assume is used) is needed. Having said that I think the
immutable data / DHT stuff is a good idea in theory and I hope something like
this takes off!

------
martin1975
Bears some resemblance to Maidsafe.

------
stuaxo
Will the browser be able to download different parts of the same file from
different servers ?

~~~
reit
Yes, bitswap is the same basic idea as bittorrent.

------
StavrosK
Hmm, wouldn't this be a fantastic way for archive.org to host stuff? Set up
servers around the world (e.g. with volunteers), copy every file you come
across the web there, voila. Permanent.

------
ZenPsycho
how does IPFS compare to bittorrent's project maelstrom?

[http://project-maelstrom.bittorrent.com](http://project-
maelstrom.bittorrent.com)

~~~
Confiks
In my view Maelstrom instantly disqualifies itself because it's kept closed
source. I really don't understand how BitTorrent (the company) can take itself
seriously preaching about "truely neutral" networks.

I've worked with IPFS, and it's really amazing. The development on it is very
active, and the main developers are easily accessible for questions on
Freenode#ipfs. I've been trying to build a tiny consensus protocol on top of
it, but its native support for signing is still a little rough around the
edges (and of course I've been distracted by a thousand other side-projects).

------
amelius
Good idea. But a lot of websites also have a database on the server-side. I
wonder how they deal with that.

------
vinautomatic
honestly don't get the big deal or what the advantages are of this or what it
solves.

bad marketing

------
curiousjorge
I think this is a great trend we are seeing.

------
fizzbatter
I love IPFS, the team has done an amazing job with the implementation. As much
as i love it though, i can't help but feel
[https://morph.is/](https://morph.is/) 's technology of a Targetted Block is
required for the web. I know that is outside of IPFS' specific scope, but is
that something any of the IPFS/IPNS/IPN developers are thinking of?

Being able to push data to a target seems to be a tool that will empower real
change - not just solve a technical problem (HTTP failings).

Thoughts?

~~~
fizzbatter
Not sure why the downvotes without explanation. I'm requesting legitimate
discussion, so meaningless downvotes only harms the community imo.

Please contribute

