
Decentralized Web Primer - xwvvvvwx
https://www.gitbook.com/book/flyingzumwalt/decentralized-web-primer/details
======
blunte
Could be that I'm just really out of touch, but I need at least one paragraph
saying "What is this?" (and ideally also, "Who is the target audience?" \- or
"Why would I be interested in reading this?")

~~~
da02
I'm with you on this one. I had to Google all of this for a few minutes. This
is my best interpretation below. I hope others will come in and correct my
silly mistakes. (Disclaimer: I'm not a programmer and English is my 2nd
language):

"So you know how people use BitTorrent to distribute big-budget Hollywood
movies with occasional mediocre storylines? What if you could distribute your
own data on a similar P2P network, but view it directly in a web browser
(among other HTTP clients)? You suddenly turn your seed box into a "web
server". Plus, you can stream videos on it too. So your home computer is not
just a "web server", it can act as your own "Netflix" server to publish your
home movies. And you could do it in a secure fashion. You would need a few
protocols (eg IPFS), concepts (eg Merkle Trees), and some freely available
software to make it practical, easy, and secure. These tutorials will show you
how to get all of those things working together to publish & stream to the
world."

Watch a demo of this @ 7:00 in this video:
[https://www.youtube.com/watch?v=8CMxDNuuAiQ](https://www.youtube.com/watch?v=8CMxDNuuAiQ)

~~~
mxstbr
While he doesn't touch on IPFS iirc, this talk by Mathias Buus is what got me
excited about decentralization:
[https://www.youtube.com/watch?v=7tf14VpeHlE](https://www.youtube.com/watch?v=7tf14VpeHlE)

He does a great job explaining e.g. Merkle Trees in a practical manner that
made me immediately go "Wait, I could use this for X and Y?!"

Recommended watch!

~~~
bringtheaction
Thanks for the recommendation. Positively great talk and you are right it
inspires a lot of usecases. Speaking as someone who had heard about Merkle
trees but never looked into the details of what they were about, this talk was
absolutely worth watching. Regarding IPFS there was a question at the end
about the difference between IPFS and the stuff he talked about so it's
briefly talked about by the speaker then.

------
throwaway2016a
This looks really interesting and I look forward to reading it but IPFS is not
the only player in the decentralized web game. One only needs to start typing
"IPFS vs" in Google.

Perhaps "An IPFS Primer" would be a better title?

~~~
esfandia
I agree, it seems disingenuous to describe IPFS as the only way to implement a
decentralized web. For example in our group we've explored the idea of
essentially a decentralized Merkle tree where the document id's are resolved
via P2P gossip, like in early versions of Gnutella.

------
staunch
"The Decentralized Web" is the WWW system Tim Berners-Lee invented and that
we're all using right now. It relies on the DNS and HTTP protocols. This new
IPFS-based decentralized web is interesting but it's not even a measurable
percentage of web traffic today. Far from being "The".

What we need more than anything is to actually realize the beautiful dream TBL
had for the WWW. HN is one great example but there should be a million more.

~~~
loup-vaillant
No. The web as it works right now is naturally centralised, for a simple
reason: the bandwidth costs on a server is proportional to the size of its
audience. This is why we need YouTube to begin with, instead of millions of
people posting their videos on their personal page.

There are other factors of course: firewalls, asymmetric bandwidth, security
issues, technical ignorance… But the client-server model does play a
significant role.

If you want a truly decentralised web, you need to modify a couple things: get
rid of HTTP, and use a P2P protocol instead. Generalise IPv6 and get rid of
NAT. Give people symmetric bandwidth. Make secure software that is simple to
implement, simple to use, and hard to misuse (it's not that hard dammit, just
look at qmail).

~~~
jkarneges
> This is why we need YouTube to begin with, instead of millions of people
> posting their videos on their personal page.

People can post videos to their own hosted pages already. We "need" YouTube
because 1) people generally don't want to host anything, and 2) it provides
connectedness (search, related videos, subscribers, etc).

P2P makes the first issue worse. The second issue is a problem of data
federation, not client/server systems.

~~~
loup-vaillant
> _People can post videos to their own hosted pages already._

No they cannot. Here's an example: what if make a nice tutorial, put it on my
web site and submit it here and r/programming? If the thing is well done,
people will download the video, and the sheer amount of requests may be enough
to render my site unresponsive, simply because I don't have the bandwidth.

So If I ever make a video, there's a good chance I host it to YouTube. Despite
my reluctance to feed Alphabet. Even though I already operate a server.

> _1) people generally don 't want to host anything_

People want to publish stuff —that much is obvious. They want to avoid hassles
if possible. And they rarely think about what centralised hosting entails.

That doesn't amount to "don't want to host anything". Granted, hosting stuff
yourself is a major hassle these days. It doesn't have to be, though.

~~~
WJW
You can easily get more bandwidth though, it's not that it is somehow
forbidden. You just don't want to pay for it. That is what youtube gives you:
'unlimited' video hosting bandwidth (and improved discoverability) in return
for the opportunity to play their ads before your video. The bandwidth costs
also don't magically disappear with IPFS, it is just a convenient way to make
other people pay for it.

In order of increasing independence you could choose to: \- Host it on
youtube, with the benefits and disadvantages you already mentioned. \- Host it
on a public S3 bucket and link to it from a HTML5 video player on your web
page. You pay for only the storage and bandwidth used. \- Keep the video on
your own server, but pay a CDN to cache it for you. Actual bandwidth use of
you server is now very low again. \- Host it on a (big) server in a colocation
center. You would typically buy bandwidth in advance though, so you want to be
able to roughly predict how much you need. \- Contract with a fiber laying
company to lay a dedicated fiber line from the nearest backbone to your home.
(Alternatively, a large microwave connection) Buy several 100Gbit NICS to go
with it and negotiate peering agreements with the major networks.

Most people don't want the hassle and cost of the latter options, but they are
very possible and very legal. In your example with the programming video, if a
ton of people want to see it then somewhere along the chain a lot of bandwidth
is going to be used and it is _not_ free. All you do is shift who pays for it.

~~~
loup-vaillant
> _You can easily get more bandwidth though, it 's not that it is somehow
> forbidden. You just don't want to pay for it._

Exactly, I'd like my _viewers_ to pay for it. There are limits to my
generosity, after all.

------
mtgx
How hard to censor is IPFS? It would be quite disappointing if we managed to
switch everyone over to IPFS, and in the end, after all that effort (and the
compromises we'd have to make compared to simply using centralized content),
governments would remain quite effective at censoring IPFS content.

Could ISPs throttle or block IPFS content, for instance? The developers should
start assuming some extremely aggressive environments, like China, and then go
from there, because honestly, we don't know if 20 years from now a lot more
governments would see China's censorship as a role model to follow. We're
already seeing multiple countries in Europe considering it to stop "extremist
content", the IPSs in Canada are now banding together to block pirated
content, and the US is probably not going to be too far behind after the
repeal of the net neutrality rules. And those are just the "democratic"
countries. It's obviously much worse for Middle Eastern or African countries
already.

So I just hope the IPFS developers will always try to develop the platform
from an extreme resilience point of view.

~~~
colemannugent
From what I gathered by reading the whitepaper mass censorship is extremely
hard to accomplish.

> _Could ISPs throttle or block IPFS content, for instance?_

They wouldn't know what was IPFS and what was regular encrypted TCP, the only
way I can tell IPFS traffic apart in wireshark is the port number. Once ISPs
start blockings random encrypted connections people will get mad when their
games or other applications stop working.

If we are talking about bad actors in the IPFS network, I reccommend you read
their whitepaper. They layout a clever karma based system for incentivizing
nodes to help others and punishing nodes that misbehave.

~~~
wanderingbort
> They wouldn't know what was IPFS and what was regular encrypted TCP

The extreme conclusion of state-sponsored censorship is not a black list or
DPI. It is a white list. It is easy to restrict traffic to a small set of
addresses that are rigorously monitored for any ability to proxy non-
conforming traffic. Everything else would get dropped on the floor.

IPFS and the like sit too high in the OSI layers to combat state sponsored
censorship in the long-run.

It makes me wonder what the target market is. If this ever got big, it would
just trigger the next step in the arms race it is incapable of sustaining

~~~
hedora
I’d think it would support grafting in chunks of the merkle dag via usb
drives, etc. Otherwise, I don’t understand the claims about disconnected
operation and breaking the dependency on the internet backbone.

------
gtirloni
I don't have any experience with IPFS so I just downloaded the client and ran
`ipfs daemon`. It's been using 500-1500Kbps down and 200-500Kbps up for the
last 15min or so. It also has 330+ open connections. Is that expected? It
seems a bit aggressive for idling. No message on the console indicating what
exactly it's doing either.

~~~
devttyeu
By default every node acts as DHT server. If you have easily reachable node
this is fairly typical load. You may opt-out from serving DHT requests by
adding --routing=dhtclient parameter to the daemon.

~~~
gtirloni
That did it, thank you! For now that's more manageable to start learning about
it.

------
harperlee
Decentralization is frequently sold as a means against censorship: if we use a
decentralized system such as IPFS, we don't need to have a DNS hierarchy to
serve content, so it is no longer viable to block a particular domain.

But as long as we have ISPs and a common communications architecture, if we
start using a content-addressable system, doesn't that help censorship? As a
censor, I jump from having to censor all domains that may serve one particular
document (which is difficult, as we can see with pirate bay), to just having
to force ISPs to block urls with the hash of the document that I want to
censor.

So we go from having to jump among domains, to having to jump among content
hashes, which seems much less practical, isn't it?

In my mind, until we have some sort of mesh network with efficient cache
systems, the decentralization topic seems (to me) that is providing answers to
the wrong questions.

~~~
Asdfbla
I think by default IPFS also doesn't even replicate content. So you don't even
have to block hash lookups in the whole network but just take down the one
host that currently has the only replica.

~~~
fiatjaf
By default IPFS replicates content. That's it's main feature.

~~~
Asdfbla
Pretty sure it used to be that you have to deliberately pin content that you
want to share on your node. Else nodes that accessed it will throw it out of
their cache if they don't need it anymore.

Maybe they changed the behavior in the meantime though, but IPFS didn't
permanently replicate uploaded content in the past without some deliberate
user action.

~~~
fiatjaf
Yes, you have to pin to keep it, but you download it and begin serving it
automatically every time you load something. You can continue serving that for
a while after -- specially if you're not downloading many other things after.

------
hossbeast
One thing I've never understood is how IPFS would serve dynamic content. Like
imagine building amazon.com on IPFS how would that work? Every single page
served would have Different hash, because the content is always changing.

~~~
marknadal
The next step after DAGs is CRDTs.

They have a research page on this: [https://github.com/ipfs/research-
CRDT](https://github.com/ipfs/research-CRDT)

One of them is us
([https://github.com/amark/gun](https://github.com/amark/gun)), David reached
out the other week, now we're trying to find time to discuss with Juan how to
best integrate.

~~~
xrd
That's exciting that Gun is looking up adopt CRDT. Is that what you are
saying? A real open source competitor to Google Firebase.

~~~
marknadal
Thanks! GUN is already a CRDT (it is a generalizable CRDT on graphs), which is
why IPFS is chatting with us. Our CRDT has an emergent property of
composability though, which lets other specialized CRDTs be implemented on top
in just a few lines of code (take this example, of a counter CRDT:
[https://github.com/amark/gun/wiki/snippets-(v0.3.x)#counter](https://github.com/amark/gun/wiki/snippets-\(v0.3.x\)#counter)
).

------
oskarth
> To build the HTML, PDF, epub and mobi versions of the book with one command,
> run ./build-book.sh

Is it built as a one page anywhere already? Seems weird not to include this by
default.

------
Jommi
Here's a pretty cool talk on the subject as well by Andre Staltz. It's nothing
groundbreaking, but just simple facts he puts on the table to show what the
current situation is.

Highly recommend.

[https://youtu.be/qZDJ1z0apVk?t=7786](https://youtu.be/qZDJ1z0apVk?t=7786)

Disclaimer: I helped organize the event where this talk is from.

~~~
lioeters
Thanks for sharing this talk - it inspired me to study and get into these new
developments more; now I'm ready to read in depth the book Decentralized Web
Primer.

------
bababooey
The biggest advantage of centralization is the ability to delete something
from the normie web and have it deleted.

Granted, nowadays you have things like the internet archive, but I don't see
normal people going along with IPFS for dynamic content like social media as
long as you cannot delete things.

~~~
NoGravitas
Yeah, one of the problems with content-addressable systems like IPFS is that
they don't support the "right to be forgotten". And I've heard a lot of big
supporters of content-addressable systems arguing that's a good thing, because
links never break. But I don't like a world where I can't take down my own
blog post[0], only unlink it from my home page.

[0]: Yes, I'm aware anyone can host a copy of my deleted page. But it will be
served on a different URL, and it takes extra work to do, unlike IPFS or
similar.

------
riston
Not sure but didn't get any information about how the data is stored between
the nodes. Does each node keep full state and synchronize when connected to
the network(similar to blockchain), or is the data split between available
nodes? Could someone clarify the storage part?

------
aqsheehy
Unless your project needs censorship resistance, you can probably ignore this.

~~~
thesagan
Well, my personal project as a voracious reader and critical thinker _does_
needs censorship resistance on behalf of a diversity of authors who wish to
publish their thinking.

~~~
aqsheehy
Who is trying to censor you?

~~~
thesagan
Not me, authors I may read.

~~~
aqsheehy
Which authors? Who is trying to censor them?

------
golemotron
Poor name. The web is already decentralized.

------
distagon
The one who comments below does not live in China.

