
Decentralising the web: Why is it so hard to achieve? - muneeb
https://www.computing.co.uk/ctg/news/3036546/decentralising-the-web-why-is-it-so-hard-to-achieve
======
aphextron
The web is and always has been decentralized. But people only want to use
centralized services. What does that say? Centralized services have benefits
that can make them strictly superior when done _ethically_. We just need to
come up with better ways of funding them that don’t involve building a massive
survellaince apparatus.

What’s easier: building a new internet, or forcing companies like Facebook to
be honest and transparent?

~~~
brokenmachine
_> What’s easier: building a new internet, or forcing companies like Facebook
to be honest and transparent?_

Those companies have proved they can't be trusted.

So I believe the answer to that question is to build better, easy to use,
decentralized platforms. I think that given a real choice, people would be
happy to use open systems. Up until now they haven't been given easily usable
alternatives that do all the things they want to do.

~~~
twblalock
> So I believe the answer to that question is to build better, easy to use,
> decentralized platforms. I think that given a real choice, people would be
> happy to use open systems. Up until now they haven't been given easily
> usable alternatives that do all the things they want to do.

The problem is that decentralized systems are always going to be less usable
than the centralized systems they compete with. Centralization is what makes
better usability possible.

~~~
brokenmachine
_> Centralization is what makes better usability possible._

I disagree with this premise.

There's less opportunity for monetary gain, so less effort has been invested
in decentralized systems.

I don't think it's an actual inherent technical limitation of decentralized
systems. In fact decentralized systems promise many technical benefits. Of
course there are also challenges, but in my opinion, not insurmountable ones.

~~~
TeMPOraL
There's a kernel of truth in that premise, in that centralization enables
better usability by making things _within_ a centralized system consistent,
and thus interoperable.

When you let people run away with their creative freedom, you get the Web.
That is, nothing talks to anything else in any reasonable way.

~~~
heavenlyblue
I don't think so. Standardisation is what makes decentralisation possible. The
only one thing that the likes of Facebook really do well compared to
decentralised is spam detection.

And by the way - spam is a problem that those platforms don't solve themselves
either as they use their communities to filter that out. So it's only a matter
of time before those communities realise they might as well work outside those
platforms.

------
claydavisss
Most users are non-technical so the entire notion is gibberish to them

Otherwise, decentralization imposes performance penalties most deem
unacceptable for common use. Furthermore, in the case of p2p, you can end up
serving illegal content.

~~~
sametmax
Also decentralisation make fighting abuses hard, such as spam, flood, dos,
etc.

~~~
danShumway
On the contrary, this is one of the few areas where decentralized systems are
_much_ better than centralized ones.

The problem is that moderation doesn't scale. Small communities and small
instances will nearly always be better at moderation than a large platform,
because they can rely on a small number of individuals to get the job done -
individuals who are more likely to be consistent, more likely to pay attention
to community reactions to posts, and more likely to care in general about what
they're doing.

It's difficult for me to think of any large platform (Facebook, Github,
Twitter, Reddit, Steam, Amazon, etc...) where moderation and general content
quality is better than their smaller counterparts (Mastadon, Itch.io, Gitlab,
self-hosted forums, etc...).

Large platforms also become large attack vectors. The same benefits of
centralization for users - (content discovery, single sources of truth, and so
on) make it very cost-effective and efficient to spam and harass users. With a
decentralized system, spammers are less likely to care about whatever small
community you're hosting. It's not impossible for them to crawl around the
internet spamming everyone, but the cost of doing so is a lot higher than
targeting a single platform.

So centralization inevitably leads to large platforms because the market for
these platforms is winner-take-all, and it almost a kind of pseudo physical
law that large platforms can not be moderated well. I don't want to be
absolutist about it, but I'm really having a hard time thinking of even a
single exception to that rule.

Maybe Wikipedia, but I'm even kind of doubtful about that - and most of
Wikipedia's quality moderation comes from a group of people who are completely
obsessed with the project. That mindset also doesn't scale to Twitter/Facebook
sizes.

~~~
Kalium
You're right! Gitlab is much better moderated than Facebook. Small, isolated
services can be policed strictly in unscalable ways.

Perhaps there's another comparison to be made? Facebook's moderation, for all
its many and myriad faults, is infinitely better than the stuff your average
SMTP server sees every day.

Federated decentralized systems are a hell that combines the worst of both
centralization and decentralization. It's a great way to offer users the
features they want, but at the cost of a set of systems where there's no
practical way to stem abuse.

~~~
danShumway
I would still disagree with this - federated decentralized services (think
Mastadon, or PeerTube) are made up of smaller nodes. Each of those nodes are
more tightly policed by smaller communities, and you subscribe to a subset of
those communities that you care about.

This is great for moderation - Mastadon instances tend to be very focused and
to ban bad actors quickly. It's bad for content discovery, because not all of
the content is localized in one place.

It's a tradeoff, one that I'm increasingly suspecting is inherent to the
debate over centralization, and one that I think might end up being a deciding
factor for people choosing which direction they support. If you care about
mass indexing and content discovery, maybe you just can't have moderation. If
you care about moderation, maybe you just can't have content discovery or mass
indexing.

Granted, decentralization is worse for censorship because bad actors can go
off and make their own instance, and other people can still subscribe to them.
But, censorship != moderation, those are two entirely different concepts that
just occasionally overlap with each other.

This is assuming that you're talking about moderation that's more in-depth
than just adding a captcha to defeat spammers. With the example you gave of
hosting your own fileshare; Facebook has more resources to deal with a certain
kind of attack - DOS, XSS, etc... but you're really only talking about a very
small subset of attack patterns in that instance. I think federation makes
most of those problems go away, because you can standardize purely technical
defenses and for the most part they scale just fine across decentralized
services.

The moment you start talking about hard problems of moderation like choosing
which files you're going to host in the first place, the private SMTP server
wins.

When I think about scaling moderation and curbing abuses, I'm not really
thinking about the low hanging fruit, I'm thinking about curbing bad actors
who are already actively participating on the platform, or who are coming up
with novel attacks that don't have any one-size-fits-all solutions.

------
arendtio
Actually, the web is decentralized. Many people mess around with the terms
here. Just today I wrote a short post defining decentralized, distributed and
federated in another thread [1].

While I am completely pro decentralization (I run my own Nextcloud, XMPP and
E-Mail server), I am not very fond of all those 'distributed' ideas. Many of
them rely on encryption which might be safe to use today, but encryption has
this anoying property becoming unsafe in the future. So puting my data
encrypted to some distributed network doesn't feel right for me.

The problem with the tech giants is that their products are so easy to start
with that many consumers just use them because they don't see real
alternatives. And quite frankly, often there are no equivalent alternatives,
as the tech giants throw so much money at their products that very few other
products have the same ease of use.

I am not sure about a solution to this dilemma, but I hope that some day we
will have more high quality, easy to use, federated services.

[1]
[https://news.ycombinator.com/item?id=17695808](https://news.ycombinator.com/item?id=17695808)

~~~
mike_hearn
I've been working on decentralised systems of one form or another for a long
time. I think there are several ways forward.

One way is to build high quality platforms. Part of why centralised apps tend
to be more common than decentralised is that we have pretty good or at least
well understood platforms for making centralised apps, but the platforms for
making decentralised apps are obscure at best and largely non-existent.

Another way is to narrow and repoint the focus a bit. People thinking about
decentralisation tend to immediately gravitate to consumer applications. I've
found it easier to get traction in the business-to-business space. Despite all
the noise generated by the press consumers often don't really care about
things like lockin or data privacy - the tech firms protect people's data well
enough, and there isn't usually enough lockin to really worry outside of maybe
smartphone operating systems. However businesses are often much more sensitive
to these issues. Yes they will outsource and use centralised services too, but
they're much more likely to care about things like confidentiality, privacy of
their data/trade secrets, they sometimes have regulatory constraints that push
them in the direction of decentralisation etc.

So that's one reason why I'm working on Corda, which is a platform for p2p
business. It tends to get described as a blockchain system and it's got a lot
of tech inspired by Bitcoin but I see it more as a platform for building
decentralised apps (or more accurately, apps where you can precisely
distribute and control trust points in accordance with your needs).

Better platforms are absolutely the way to go for consumer apps as well.
Unfortunately building such platforms will probably involve sacrificing some
sacred cows. For instance the web is a terribly centralising platform. It's
great at decentralising _information_ and _documents_ but when (ab)used as a
way to build ordinary applications it creates far more centralisation than
we've ever seen before. To get decentralised apps becoming commonplace we
probably need to tackle the dominance of HTML5 and HTTP as a (desktop/laptop)
app platform.

------
cameldrv
It's already here. I'm not sure why Danielle Robinson merited a photo and a
quote but there was no mention of Beaker Browser. Beaker uses Dat as its
storage layer, which is similar to IPFS or Bittorrent, and then layers on data
storage and sharing systems to enable services like Fritter, their Twitter
clone. It's very easy to use -- just download Beaker, and you can be
publishing web pages or using totally decentralized web applications.

~~~
mattlondon
+1 - where IPFS and Gun present a very tech-focused entry point to the
distributed web (I still do not even know how Gun works - it just says it
works and that I dont need to worry etc, without much info that I could find
for actually _how_ it works e.g. how is my data shared? who runs the other
nodes), Beaker presents a very friendly "on ramp" to it all that is easily
understandable in a browser context. I've found it very interesting to play
around with. Highly recommended.

------
marknadal
I spoke at the conference this week, it was mind blowing to meet all these
people. :)

The article seemed well done, the problem (from a information theory / network
graph) though seemed weighted towards blockchain-like assumptions. This is
fine, as more information needs to be seeded on this.

For those interested in the title's question:

Coordination might seem like the hard problem, but it is only hard because the
majority of us in this community made it hard by using immutable data
structures. If we use mutable data structures (you know my bias, but it is
possible with [https://github.com/amark/gun](https://github.com/amark/gun) ,
proof: Mitra at Internet Archive integrated it in 1 week, now decentralized IA
runs on GUN), we get some other problems but they are less than O(log N) in
complexity.

Here is why coordination on immutability is hard:

O(N^2)

Just sit and think about it.

Intuitively it makes sense:

If I have an index to make something fast, and if it is immutable.

When I "update" the index, I have to create a new index.

Now the old index can't find the new index.

Therefore, I need to index the index, etc.

Repeat.

Therefore, as long as the web never changes, a decentralized immutable web
will eventually be fast, but it must be true that if while the decentralized
web changes, it will be hard.

This is not true if you can do decentralized mutable data. Check out our work!
I hope this comment added novel insight to your day. :) If it did, let me buy
you coffee next time you are in town. Cheers!

~~~
adrianN
You don't have to copy the whole datastructure to update it, even if it's
immutable.

~~~
marknadal
In something like an append-only log or DAG, you'd still have to index where
in that log to skip to (or you'd be scanning for it every time), which is
still computational complex, and not nearly as scalable as a mutable index. So
yes, while you are correct, it either way is still better to use the mutable
approach.

------
ardit33
Napster, Limeware, Bittorrent, Kodi, Bitcoin, were/are part of the
'decentralized web'. They were build by necessity to get around copyright,
monetary restrictions, censorship etc.. and not necessary for convenience.

At the end of the day the average user just wants a convenient and reliable
way to reach content and information. Centralization has been the best way to
serve it, and I don't think it will change.

~~~
buboard
Napster was by far the best way to share music, just like how popcorn time is
the easiest way to get a movie right here right now. However nowadays people
ditched their desktop computer/node for an iphone, and ISP upload speeds are
in most places pretty bad. I think this has crippled p2p services and made
centralization more convenient (while it isn't). Unfortunately we 're not
gonna see decentralized web unless the general public gets back control of the
compute and traffic.

~~~
foldr
>However nowadays people ditched their desktop computer/node for an iphone,
and ISP upload speeds are in most places pretty bad. I think this has crippled
p2p services and made centralization more convenient

Most people were on dialup in Napster's heyday, so I don't think that can be
the issue. Music files are small.

~~~
buboard
but phones do not act as always-on nodes like napster or kazaa or torrent
nodes did. battery issues + mobile data limits also

------
rvense
Most people use products, not protocols. We need a more product-oriented
approach. Decentralization and federation aren't the first or second item on
the list of things most people are looking for, unless we get to "exactly like
X, but decentralized."

It's wrong that people don't understand or want to not depend on something
like Facebook. At least where I am, a lot of people understand the issues. But
there's nothing really like it, and life in my city is quite difficult
without.

It's nothing that can't be fixed though! I think a project like Pixelfed
([https://pixelfed.org/](https://pixelfed.org/)) is doing things right. We can
get there.

~~~
TeMPOraL
> _Most people use products, not protocols._

Except e-mail and HTTP.

> _We need a more product-oriented approach._

I think this is how we landed in the current situation. Instead of protocols
(IRC, XMPP), you get products (Slack, Hipchat), which are walled gardens. I
wish there was a way to incentivize companies to stick to shared protocols,
instead of creating their own.

~~~
WJW
The people use products, which then implement some protocol. If you go down to
the nearest supermarket and ask what people use to visit their news website of
choice, I'm willing to bet that 99+% will name a browser or a phone type and a
very very small minority will answer HTTP, even though technically all of them
use HTTP. You'll get similar answers with email and SMTP.

The general public is just not very interested in the implementation details
of technical products, as long as they work well enough and don't cost a lot.
If you really want incentivize companies to implement open protocols, find a
way for companies to make more money with the open protocol than with the
walled garden, because companies is where most of the engineers are and they
are the ones that will make it if the incentive is there.

------
gaius
In the old days, you were a member of an organisation such as a university or
a corporation. You had a home directory on an NFS server and in your home dir
was a folder called public_html that you put your homepage in. So your URL was
www.university.ac.uk/~username and your email address was
username@university.ac.uk

This caused problems whenever anyone moved so in the early days centralised
services were seen as a solution such as Hotmail and Geocities. No one in 1995
realised how it would turn out into abusive monopolies...

------
axilmar
I think a decentralized web is easier than what people think. In the early
days of the web, users could host their creations somewhere in their ISP's
hard disks.

The same model could work now, with a small difference: instead of storing
only data, it may also host applications that manage those data, and
collaborate with same/other applications of same/other users.

For example, a photo app could allow us to manage our photos at a specific
host of our choice; the same photo app uses a sharing protocol so as that the
photos are shared with other users of our choice.

The stored data are encrypted by our own keys and therefore not visible to
third parties, we still get to share our stuff with our mates, it's
decentralized because no central authority can lay claim on our data, it's
protected because it is encrypted for us, but the infrastructure required for
this to run is provided by a centralized entity.

------
zaarn
The UI/UX usually sucks a lot.

Bitcoin wallets for example, suffer a lot because the average user doesn't
have much of an idea that they need a strong password or what these random
numbers mean other that they might behave like bank accounts.

The ideal decentralized UI/UX will work largely by copy paste, with human
readable names (@me@mastodon.social, for example) and doesn't have too many
buttons that need more than an icon or single word to explain them.

On top of that it also needs to look modern and neat while being fast and
snappy on mobile devices without sucking up battery or limiting how the users
can interact (ie, not possible to share photos directly with the app is a no-
go)

These design limitations are solved more easily by centralized services as
they can just throw money at it.

------
aethertron
'Decentralising' means spreading around effective decision-making power. That
power comes with consequences -- negatives ones, if ineptly used. And most
users are quite inept, when it comes to running powerful general-purpose
computers, especially internet-connected ones. I don't blame them: the
internet is not a friendly environment. It's a jungle. People want to be able
to safely communicate online without the responsibility of system
administration. Many companies have stepped in to offer services with this
trade-off. Thus centralisation has prevailed.

Urbit's conceptual path to decentralisation seems sound: build a new friendly
network on top of the internet, and build a new friendly server OS on top of
Unix.

------
grizzles
It's not hard, but many in the space are working on the wrong things. We need
a consensus mechanism with the integration capability of tendermint, with
honey badger bft level performance. (eg:
[https://github.com/poanetwork/hbbft](https://github.com/poanetwork/hbbft))
That would be enough to build a fully distributed facebook clone and much
more. It will happen, but on the timing of the engineers who know what to do
to achieve it.

------
rini17
The problem here is decentralized network relies on peers to self-enforce
their decent behavior. End users don't know how to, and their vendors have no
skin in the game - then we get IoT botnets. ISPs won't bother with egress
filtering - then we get massive DoS attacks.

------
nottorp
Centralization is comfortable and in the end run, cheaper. I'd rather pay
github (or depending on how much Microsoft fucks it up, a competitor in the
future) a couple dollars per month than maintain my own git server.

Even for services where I have to run my own server, I rent a VPS at some
hosting facility rather than maintaining my own hardware or even renting bare
metal. It simply doesn't make sense to run my own.

And if I bothered to have a web presence, again, I'd host it somewhere. It
simply doesn't make sense to maintain your own except as a hobby.

Moreover, all the blockchain stuff looks to me like a solution looking for a
problem instead of something generally useful. Yes, it will make sense for
decentralized payments if the exchange rates ever stabilize. No, adding
blockchain to everything won't help.

~~~
kqr
I think key to this distinction is federation, not decentralisation. With open
standards and open source implementations, you'd be able to migrate your
GitHub to a different GitHub provider if Microsoft screws things up too badly.
GitHub would be not a monosilo, but a set of slightly smaller siloes, all
interacting seamlessly with each other.

~~~
nottorp
Well, I use github just for the repo part, no other fancy features. We run our
own issue tracker on a VPS;)

------
hasa
I don't see decentralization as ultimate goal. Why should it be?

~~~
zAy0LfpBZLC8mAC
Because humanity's experience with centralization of power has been abysmal--
and unsurprisingly so.

------
Cypher
Why do the articles on the site require sign up to read? prime example of
friction stopping the web from being decentralized.

------
cup-of-tea
Decentralisation is hard because people crave power. As long as people crave
power, people will build centralised systems of control.

~~~
WJW
Also, because people crave money. It is more difficult to extract money from
decentralized systems compared to centralized systems. Therefore centralized
solutions often have more resources available to pay developers, marketeers
and founders (and investors), which leads to increased work on the product and
(hopefully) more users. It is easy to see how this would lead to a virtuous
cycle where the centralized systems capture the majority of the market due to
having vastly superior resources and the decentralized systems capture mostly
those users and developers who care about things other than money (eg
censorship resistance, etc).

~~~
cup-of-tea
Money is power. Money is quite literally the ability to make other people work
for the things you want.

~~~
WJW
Money is a form of power, sure. There are different forms of power as well,
not all of which can (realistically) be bought. Try purchasing serious
weaponry for example (let's say a dozen tanks or fighter jets) and you will
quickly discover that money is not enough.

------
twblalock
Centralization is so useful that people will create it wherever it does not
already exist.

One example is Github, a centralized service that makes it easier for teams to
use a source control system that was originally designed to be decentralized
and distributed. When it was purchased by Microsoft and people started looking
elsewhere, they looked to competing centralized solutions like GitLab and
Bitbucket.

Given that centralization is so useful, and people want it so much, it should
not be resisted. We would be better off if the effort being put into
decentralization was instead put into better centralized services that respect
user privacy.

~~~
jessaustin
The problem solved by Github's centralization was establishing a namespace,
which could have been done in several other ways. (In fact there are a number
of other namespaces orthogonal to Github's but pointing at largely the same
resources.) The problem Github solved that convinced lots of OSS people to use
it was a reliable offsite that was free as in beer and less obnoxious than
Sourceforge. So far it seems like some combination of VC cash and paid usage
sustains Github's free service, and several other firms' free services as
well. The centralization bit doesn't seem so important.

~~~
twblalock
It's a lot more than just a reliable offsite -- it invented collaboration
features that didn't exist in pure Git. Github's pull request and code review
workflow has become pervasive in the industry.

It's also not sustained by VC cash anymore -- Microsoft is buying the company.

~~~
jessaustin
Sure the collaboration features are also valuable. It also isn't obvious that
they require centralization. One could argue that only a centralizing
organization could extract the rents that could underwrite developing those
features, but such an argument only goes so far.

