Hacker News new | past | comments | ask | show | jobs | submit login
Who Pays for the Decentralized Web? (tierion.com)
180 points by nsrediron on Oct 16, 2016 | hide | past | web | favorite | 98 comments

A decentralized web would be vastly more efficient in terms of resources. A lot of the data could be deduplicated and a lot of backbone traffic could be avoided.

Right now, if a dozen people around a block watch the same viral video, all of them will fetch it through the backbone, which is obviously wasteful. If they want to store the video for later, they will have a dozen copies wasting space on their drives, which could again be reduced by more efficient algorithms (eg. erasure codes).

There are lots of unused resources at endpoints (local network links, unused storage space and computing power on most consumer devices, etc) which could be utilized more efficiently. If we can pay for this wasteful centralized approach, why would paying for a decentralized network be more difficult?

And make no mistake, we are all paying for this "free" centralized network now. Whenever you buy a product from a company that advertises on facebook, google, whatever, you are effectively paying for it.

(Premise: I love decentralized systems.)

> A decentralized web would be vastly more efficient in terms of resources. A lot of the data could be deduplicated and a lot of backbone traffic could be avoided.

Decentralization usually brings more duplication of data. Think of email: copies of the same email are redundantly stored in many servers. And this is what we want. I have received the email, I decide if and when to delete it. Or Bitcoin. It is "decentralized", but if you want to run a normal client an be part of the federation you will need to spare quite a bit of disk space to store the exact same 90GB that comprise the blockchain.

> Right now, if a dozen people around a block watch the same viral video, all of them will fetch it through the backbone, which is obviously wasteful. If they want to store the video for later, they will have a dozen copies wasting space on their drives, which could again be reduced by more efficient algorithms (eg. erasure codes).

At the same time we do not want people to know that we are watching this video. We can get that video from our neighbors only if the ID of the video leaks somehow (if you reply "TOR", I will answer "exit nodes know"). But that is just a side problem.

The main problem is dynamic content. Almost all my open browser windows contain pages that are generated "just for me" (github, gitlab, travis, HN, a work-related 2TB ownclound installation, amazon, google search, ...). These are the pages I care about, and, sadly, these are the pages that are most difficult to decentralize in acceptable ways. Actually some of them are meant to be centralized (amazon or google search, for example).

I'd love to see more decentralized or semi-decentralized services (planets and their OPML files are great, for instance), but one has to admit that some problems just ask for centralization.

PS: weird, I had to write "google search" instead of "google". Using just the word "google" felt somehow wrong. Brand dilution, I suppose.

> Decentralization usually brings more duplication of data.

While this is true this could be vastly reduced if the internet supported source-specific IP multicast (SSM). That way even a small box in someone's basement hosting a single copy of some niche content could serve thousands worldwide.

Alas, multicast is hard and incentives are not in favor of making it available to the end-user even in limited scopes.

Tor actually only uses exit nodes if you want to connect to the regular web, if you fetch a video from a .onion address the encryption is end-to-end. And there's also Freenet.

Github/Lab and HN actually seem pretty easy to decentralize - git already is, and the rest are essentially feeds, much like email/newsgroups/RSS/FMS. It's mostly a matter of running the aggregation code in the client, rather than on the server. Google can also be decentralized and federated - see YaCy. Amazon, there wouldn't be much point, since the service would still be contracted with a single party.

It's not a trivial job (in the "I could do that in a weekend"), but it doesn't seem architecturally hard considering the existing distributed platforms.

In terms of computation and storage, maybe, but in terms of the network bandwidth, perhaps not.

The current physical topology of the Internet is already heavily biased towards backbone-to-edge connectivity, and is directionally asymmetric.

By comparison, edge-to-edge connectivity that doesn't go through any backbone is much more limited, and edge nodes often don't have the upload bandwidth to effectively deliver content.

Plus, CDNs and other distribution networks already effectively put easily cachable content closer to the edges to relieve backbone congestion.

I think edge to edge connectivity being limited is mostly a software (routing protocol architecture) issue.

For example, I live in an apartment and the ISP has a router installed in the building. Coax cables are running to each apartment. I can see 15+ wifi networks from my home. Yet to fetch the exact same front page of a news site my neighbor is reading and has a cached copy of, I am routed through the backbone probably half way around the world, or to a CDN at the other end of my country at best.

I don't think photons really care which way they travel but the shortest the path the faster they will get there and the less photons we send the cheaper it ends up being.

If the argument is that this saves on network hardware costs, then you'd think ISPs would have some incentive to invest in decentralization as well.

But I'm not so sure about the thesis that it's just a software issue. It seems wasteful to sling electrons through a backbone and back when those bits are (physically) close by, but the cost here is not the distance those bits traveled, but the marginal cost of the physical hardware to deliver those bits to your doorstep.

It's probably much cheaper, on a cost-per-bandwidth basis, to beef up the existing backbone, because significant economies of scale exist.

OTOH, there's a significant fixed cost every time you try to widen edge links (it's very expensive to dig up and replace residential cables).

My ISP would certainly love if my whole local area (building/street/district/whatever) only fetched each unique piece of data once through their network and then distributed it locally via mesh networks. It's just that they can't make it happen by themselves. It needs to be supported by network protocols, which need support from content providers and content consumers as well, so it goes way beyond the reach of ISPs.

> The current physical topology of the Internet is already heavily biased towards backbone-to-edge connectivity, and is directionally asymmetric.

Couldn't agree more. The internet as we have it today is set up for decentralized consumers and centralized producers.

There is one untapped benefit of the current system which is that there is low latency on the edges. I'm not sure what the killer app to make use of that would be though.

Is it possible we already have an entrenched system in place that squashes the possibility of a decentralized internet before it can get off the ground?

Not only is no one economically motivated to share traffic towards the edge, no one will be building physical network capacity for it either.

I wonder what projected bandwidth numbers look like (if, for instance, youtube or netflix was completely distributed) - is the edge network close to being able to handling this?

Not every thing is in the same backbone. Netflix has installed datacenters at ISPs as cache. Which is more efficient :)

Yeah, I can't find a map of Netflix's caches, but it looks like YouTube has just about every large city in the world covered except in China, Iran, and Turkey. (bottom map at http://peering.google.com/#/infrastructure)

> Whenever you buy a product from a company that advertises on facebook, google, whatever, you are effectively paying for it.

In my opinion, it's a bit more nuanced than that. When you purchase a product with cash, no personal data is exchanged. The transaction is final and complete. When you 'pay' for services by exchanging personal data and metadata about yourself, that transaction is persistent and the vendor continues collecting payment indefinitely.

With store loyalty cards and even in-store Wi-Fi tracking and other tactics, even cash payments can be viewed as less anonymous, because the vendors prefer to continue to reap the rewards of profiling and tracking customers. My point is that this isn't a black and white issue, but there are some important distinctions to make about payments in cash and payments in data, or a mixture of both.

The way I see it, when you use facebook (share personal data, view ads) you are not paying for anything in any way. You are just making counters go up (number of ad views, amount of data for targeting, etc) and thereby validating to companies that it's worth paying facebook for ads. Then, when I buy groceries for $100 and say $1 of that is used by the grocery store to buy facebook ads, I am paying for your facebook usage. And I'm not even a facebook user.

> Whenever you buy a product from a company that advertises on facebook, google, whatever, you are effectively paying for it.

Or in social media's case -- we are the product that they make money off of.

I'm always annoyed when people think the Decentralized Web needs to replace the Centralized Web, like some glorious revolution. The decentralized web doesn't need to be as big as the centralized web, nor does it have to have as many users. The distributed file and chat applications used in the '90s and 2000s (IRC, XMPP, Gnutella, etc) were mostly run by a niche set of users anyway. I just want to have an ecosystem where like-minded folks can share, communicate, and such. I also would love to see the ability for the decentralized web to "plug into" the centralized web (i.e. publish to Twitter/FB under my identities).

My friends and I have been running our own network for a bit and playing with a very grassroots version of the web. Those of us that work in software host and create services to distribute music, movies, etc. that we collect. We also process our own IoT sensors and devices. It's maintained by those of us in software, but we have friends not in software accessing the network and using its resources. It's a lot of fun!

> "I also would love to see the ability for the decentralized web to "plug into" the centralized web (i.e. publish to Twitter/FB under my identities)."

Diaspora has this functionality:


Limewire was mostly run by a niche set of users?

Relative to how ubiquitous Facebook et al is, yes.

Limewire, Bittorrent, and its ilk were fairly mainstream in the same way that driving over the speed limit, recreational drug use, and jaywalking are; and like these things it was often afoul of the law.

So as enforcement got better and legal alternatives emerged, their 'popularity' declined. There are also people who share Netflix logins, which is a (grey-market) option for those with no money, but is vastly less likely to result in DMCA notices mailed out to you.

If anything, the various decentralized networks of the 1990-2000s were the products of organic demand. There is simply less demand now than there was, or that demand is being suppressed by other mechanisms (such as laws, in the case of copyright-infringing sharing).

Perhaps there's just not a lot of demand for hosting your own identity in some decentralized, federated social network.

Micropayments again. Except Bitcoin, this time, because blockchains are magic.

As I've pointed out before, all the enthusiasm for micropayments comes from people who want to collect them. There's no vast grass-roots demand for the ability to send pennies.

Actually, we do need a more decentralized web for these use cases:

- I want to send you a big file without going through a server.

- I want to send you a text message without going through a server.

- I want to talk to my home control system from my phone without going through a server.

Technically, we need to fix it so that you can reliably get an Internet connection between any two nodes that want to talk. For starters, all mobile devices and networks need to speak IPv6, so everybody has an address.

Content payment wants more, not less, aggregation. Ultimately a broadband or content tax indexed to wealth or income. Information is a public good.


David Brin tried pitching a new micropayments concept with repudiation as magick sauce. No.


I've used bitcoin tip bots on reddit a lot. They are really neat, and I really like having them.

Of course the loudest mouths will be those looking to profit off micropayments, which then defeats the point of the system, which is why the only successful ones are the ones that start without the selfish motive to begin with.

ipv6 is not going to be a catch all answer. Despite having more addresses than number of computers we could feasibly build ever, rollouts of ipv6 are still allocating tiny blocks to home networks that don't provide enough addresses for guaranteed public IP addressing between all nodes. If Comcast only gives you 8 ipv6 addresses on your home router, your 10th device is being natted again and even on ipv6 you cannot assume a working p2p endpoint.

Comcast assigns a /56, /60, or /64 depending on the router configuration (/56 is for business class). That's 16 * 16 * 18,446,744,073,709,551,616 IPv6 addresses, respectively, not 8.

The most expensive part for any "decentralized web" is going to be data storage, so the question basically boils down to "How can a person in a P2P system incentivize other people to host their files for them?" Bittorrent sidesteps this question by focusing only on very popular files and enabling hosting by default in most clients.

So far, no established solution exists for doing this. The most promising projects working on this problem that I know of are IPFS[1] and Ethereum Swarm[2] but both are still struggling with the low level protocol that makes sure person A gets paid for hosting files from person B.

[1] https://ipfs.io/

[2] http://swarm-gateways.net/bzz:/swarm/

Given how cheap drives are I think upstream bandwidth for that storage will quickly eclipse the storage itself.

I disagree- the hardest files to deploy in a decentralized web are "long tail" files, for which there is usually very little demand on any given day (by definition) but which are extremely important to establishing a new web platform.

I think I rememeber that napster and various other file sharing services held quite a lot of the long tail but I might be wrong (never a heavy user).

IIRC the problem was more that there was an awful lot of malware + the RIAA and MAFIAA.

Napster had a tail 40,000 files long, with the decentralized web we'll need a tail 1,000,000,000 documents long or something.

> Given how cheap drives are I think upstream bandwidth for that storage will quickly eclipse the storage itself.

Which suggests a solution -- keep several of copies of everything (or copy-equivalents with erasure coding) and then download the one(s) with the cheapest/free bandwidth.

People care if you run down their battery or use up their cellular data. Not so much when the device is plugged into AC power and connected to unlimited broadband/wifi. At any given time there should be enough such devices that you don't have to make requests of any of the expensive ones.

Add to that the ability of people who don't want to worry about it (and don't want anyone they know to worry about it) to lay out a few hundred bucks once for a 10TB NAS and leave it connected to their broadband connection, and it feels like an actual solution.

Adding to the list (disclosure: I'm the author):

http://gun.js.org/ and we have a storage layer that plugs into S3 that can handle 100M+ records a day for $10 (about 100GB+ of data) https://www.youtube.com/watch?v=x_WqBuEA7s8 .

The goal is that P2P systems are "economies of scales", so the price point is orders of magnitude less than their centralized alternatives. This disruption on price is how we pay for the decentralized web.

Honestly, the real problem with the "decentralized web" isn't data storage or bandwidth, it's developers. Getting users to "pay" for storage and bandwidth is fairly simple, as they're already used to owning devices and paying for internet connectivity.

The real problem is that developers who actually build the decentralized applications need to be paid somehow. How is that going to be organized? How does the money flow?

> "The real problem is that developers who actually build the decentralized applications need to be paid somehow. How is that going to be organized? How does the money flow?"

It's possible to have decentralised app store. Apps could be hosted by developers. Metadata that could also be hosted by a developer would then determine which app stores the app should be available on. Users searching for an app using the app store could then pick up on this metadata via crawling the decentralised web, as well as sharing lists of available apps with other users. As for payment, plenty of options there (including but not limited to cryptocurrencies like Bitcoin).

I've being toying with ZeroNet and WebTorrent/web2web and I see a glimpse of the future there. Not really sure about how to achieve mass scale without having something unique and attractive to it (or an incentive as in bitcoin).

But I can imagine a kind of software or device that may become popular that uses/requires this, that will turn this into something that people will get too used to that may become something essential in the long run.

LBRY (https://lbry.io) is designed to do this. On GitHub: https://github.com/lbryio

Disclosure: I'm the CEO

Isn't this essentially IPFS plus namecoin-with-bidding? That's what I got from the reddit AMA. I'm not sure how it addresses storage.

Also, just trying to be constructive here, but you guys really need to work on branding. The AMA went about as poorly as it possibly could have because you didn't have good presentation. Here's how you have to pitch the whole "bidding for URLs" thing; "There are permanent, long-form URLs like on YouTube. But people can bid on short-form URLs if they want to as well. You have to bid on them so squatters can't take them all over. If someone outbids you, you get your money back. We make no money from this." Boom, every inane reddit comment about "corporations will buy all the video links, man!" shut down.

I agree! The AMA had a lot of confusion, exacerbated (or arguably caused) by materials that poorly explained LBRY on our end.

To address this, we've written up this FAQ answer: https://lbry.io/faq/naming

We've also communicated the truth better inside of our community, but it is tough to get that message back out to everyone that saw the AMA.

That whole bidding for URLs thing is making it dead on arrival for me. What incentive could there possibly be?

Only vanity URLs are reserved rather than owned. See this for more: https://lbry.io/faq/naming

Filecoin looks promising as well.

Everybody pays a little.

Pay for storage by sharing the disks attached to the computer at home that is connected to the Internet. Run out of space? Buy another disk or reduce the space for sharing. Same of what happens with file sharing.

Pay for bandwidth through the bill of the ISP. This could be tricky because ISPs don't usually make it easy to have servers at home.

Both storage and bandwidth can be outsourced to server farms: a machine on AWS or anywhere else would be as good as one at home. Maybe more convenient, maybe less.

But if you're a big content provider you might end up with large disks and large pipes because you want that your content is always accessible and no random guy can affect its availability.

While who pays for the decentralized web is a perfectly valid question to raise, I think it's counterproductive right now.

Much like the internet itself, this new architecture will be ushered in by hackers looking to disrupt the status quo of walled gardens and data warehouses, not corporations. So immediate monetization is not a top concern. We can't even imagine what the economics will look like, just like Tim Berners-Lee couldn't have imagined the current web 30 years ago. If we stop to worry about fitting the decentralized web into our current economic models, we will trap ourselves in a box.

It is the hubris of computer scientists to think that if we can just build it (again!) that THIS time people will surely use it, pay for it etc.

If we just make a better algorithm, people won't find our UI design intolerable.

If we just offer them some privacy, they will pay real money for it.

Long before there was a web, people were giving up privacy and freedom for the convenience of credit cards.

If the decentralized web is going to fly, it needs to START with the social sciences.

We can all nerd out over algorithms that make it happen, or use math to prove that it can't be done, only once the economists have created a model in which it might, theoretically be done. And the social scientists have created a model in which a critical mass of people will make the necessary choices, when presented with them.

As it stands, most people are not willing to pay: http://www.telegraph.co.uk/technology/news/11047801/Would-yo...

I would be willing to pay that _and more_ in a heartbeat if given the option. But, in social science terms - I understand the problems with centralization much better than most, I'm willing to put up with much worse UI than most, and I'm wealthier than average.

Heck I'd probably pay just because I think the underlying technology is neat! But that makes me a pretty extreme outlier on the broad spectrum of humanity that are choosing, en-masse to use Facebook, which means that I either need to use Facebook, or nothing.

The predictive power of the social sciences is dismal and the predictive power of economics is mediocre.

If social scientists could actually predict what people would do, they would have been the ones to start every multi-billion-dollar "why is this so valuable?" startup like twitter, snapchat, etc.

Computer scientists are and will continue to be the driving force behind techno-social change. We have no idea what people actually want, but neither does anyone else, so we should just keep trying stuff and see what sticks.

I agree - the predictive power of social sciences (including economics) is pretty terrible.

But as you point out - computer scientists don't know what people want _either_, so.. predictive power itself is not necessary for a thing to work.

But _working_ isn't sufficient either. Facebook _works_. As I understand it is WILDLY profitable. But the way it works is a matter of economics and social sciences. Building a social networking website is a SOLVED PROBLEM in computer science. (yes, I realize they're doing some creative research on scaling it up).

Someone could clone Facebook tomorrow, and offer it for free, right? But.. could they charge $10/mo for it, and say "Facebook but without the spying and advertising?" - Probably not.

I'm not saying computer science is useless here. New algorithms WILL be needed. Bitcoin or any form of distributed ledger seems extremely unlikely to scale as needed, for instance. Perhaps provably unable to scale as needed - I'd welcome a paper proving this with solid math / computer science (and there probably is one, I don't keep up).

But what IS necessary is a working economic model that is not the current economic model. You're saying that nobody can predict the future (true), so just try a bunch of stuff - so far, so good. But then when you find something that works.. how do you pay for it? Nobody in computer science is trying to figure it out... that is a consideration "for later" - the point the article is making! What happens "later" when money needs to be made.. well.. advertising and surveillance. That is literally the only answer anyone has come up with.

That is why, FIRST, predictive or not, someone (probably in the social sciences) needs to come up with the idea(s) about how it _might_ be paid for, that don't include advertising.

The article was a response to the big decentralized web conference "web 3.0" that happened recently. It was all CS people, technologists who got together to enjoy talking about technology. But economics is what will shift this, and _innovation_ perhaps will be needed in economics. And other social sciences - nudging, gamification, etc.

Bitcoin is a great example. Indeed, many (but not all) economists, particularly those of the Keynesian persuasion, said Bitcoin was economically untenable. And they were obviously wrong; Bitcoin is vastly more successful in every respect than nearly anyone would have predicted six years ago. The only way we found out was because some programmer said "I don't know if this matches the worldview of professional economists, but I'm going to try it anyway." If he'd taken your approach, we'd never have bitcoin in the first place.

(Wrt scaling, there is no theoretical reason the Bitcoin network couldn't scale to handle any number of transactions in a bounded spacial volume much larger than Earth. There are only surmountable practical limitations like processing power.)

It's also a long, long, long way from becoming a currency with properties most people actually care about. Wild price fluctuations don't help it's cause, for example.

Most people prefer some variance mostly in the up direction (Bitcoin) to a steady decline (Dollar) or a rather precipitous decline over the last few years (Euro, GBP).

And those are the most "first-world" currencies. People seem to use even more unstable currencies without issue.

> That is why, FIRST, predictive or not, someone (probably in the social sciences) needs to come up with the idea(s) about how it _might_ be paid for, that don't include advertising.

I think when it comes to this kinds of money involved, it no longer the domain of social sciences but politics. Some ppl call politics social science but there's no science in it, it's more the inevitable dystopic future than anything else. As someone said on one of the conferences with audience of open source technologists: you can ignore politics as much as you like but the politics will not ignore you.

If you look it up few founders of social networks have an extensive CS background including Facebook, LinkedIn, Instagram and Snapchat.

Disclaimer: I work for blockstack.org, a company trying to build out the decentralized Web.

I mostly agree with this line of reasoning. Users don't care about decentralization, and for developers it's much easier and more lucrative to build a centrally-hosted version of X than a fully-decentralized version of X (for most values of X). Things aren't going to change until the decentralized Web not only needs a killer feature that cannot be had on the Web today, but also needs to preserve backwards-compatibility with the Web while being at least as good in what it can offer.

That said, instead of trying to re-educate users (which would be extremely costly and probably take generations), what if we simply made it so that decentralized applications that otherwise looked like normal Web applications were significantly less expensive to write and operate for developers? If we gave each user a client-side private key paired with globally-unique user-chosen name, then we could refactor most Web applications such that they stored their users' data (signed and encrypted) on the users' personal cloud storage providers (or whatever storage media the user desired). This is great for developers, since they no longer have to host the data themselves or defend it from hackers. This also simplifies a lot of application designs somewhat, since there no longer needs to be a backend in many cases. The user wouldn't even notice the transition, since (1) cloud storage is already pretty much free, and (2) cloud storage is also reasonably performant and highly available (and nothing stops the user from mirroring data across multiple providers). There would need to be some work on the Web browser for automatically managing keys and personas, but this is certainly doable.

I'm not a game theorist, but I think each participants' incentives are appropriately aligned to make this work:

* Cloud storage providers have to compete on space and availability and are incentivized to give users the best cost/performance trade-offs.

* Users are incentivized to keep their data available for as long as they want other users can get at it (and they can take it offline the minute they don't).

* Developers are incentivized to target this development model because it lets them externalize a non-trivial portion of their operating costs without affecting performance or features.

* This design is incrementally deployable--an app can still have some centralized components (like a search index) and we get to re-use as much existing infrastructure as we'd like.

Would love to hear your (or others') thoughts.

Looking only at the cost of running a decentralized network is not sufficient. We need to look also at the cost of not running a decentralized network.

Not all incentives have an economic nature, some are political, and I think the latter applies in a greater meassure in the current case.

Just the realization that you give too much power to those running centralized services, may represent the required incentive for people to switch.

The overall material cost to run a centralized vs decentralized service may be comparable, but the political cost is not..

The web is decentralized. The data and the money is not. As long as it stays that way (and it probably will) changing some protocols doesn't make a big difference.

Really, someone tell me what is centralized about the web right now and we can figure out together that this thing actually IS decentralized, it's just that money forces people, companies and govs to work together and keep more of the power in fewer hands.

The main methods of average person paying for stuff online. Wikileaks was crippled by a boycott by Mastercard, Visa, and PayPal. Just three companies made a decision and poof!

exactly. web infrastructure is not centralized. online services on the other hand are centralized. that problem will exist even in this utopian decentralized universe

I think it's obvious. People are not going to maintain a decentralized network for copyright infringement. But imagine a NAS device that also hosts something like a Facebook account which is shared with trusted others. Your personal data is also encrypted and backed up on your friends machines - even data not part of your FB stuff. When everyone has their own NAS with the decentralized P2p/backup software then we win. How one maintains that software across the network is a mystery, but it must be up to the individuals to maintain in some way.

> People are not going to maintain a decentralized network for copyright infringement.

They do. That's pretty much what bittorrent incentivizes.

But most people don't run bit torrent 24/7

I'd say that the fact that no one agent has to is the basis of its success. In the end it isn't without its shortcomings, though, so I agree with your sentiment in general. Either that, or something is used to incentivize the maintenance of the long tail of things, e.g. filecoin.

Unfortunately bittorrent's only goal is getting a file fast, complexity was kept to a minimum to increase adoption.

Credit systems to prioritize peers who have recently shared with us and promotion of long-running nodes to ultrapeer could really help.

Yeah, that's basically it. Once home servers become ubiquitous the platform for the decentralised web will have reached maturity. That's only going to happen if they're cheap and easy to manage. I'll be watching how things develop with the ESPRESSOBin, I think it's a move in the right direction in terms of a affordable home server:


This should be the goal!

Article author here.

Wow was I surprised to see this on Hacker News! It's my view that decentralized systems will almost always be less efficient than centralized systems. This is because centralized systems can organize information in a more efficient manner and use specialized hardware. Decentralized systems give you privacy, redundancy, anonymity and other benefits at a significant additional cost.

Bitcoin has the potential to provide a mechanism for decentralized systems to set prices and allocate resources. We'll see how this plays out over time.

I have a few hypothesizes about this.

1. Decentralization won in 1970 - 2005 protocol design because reliable computers were very expensive. It was economically advantageous to have a decentralized network of unreliable computers than centralized unreliable protocols. Reliable compute has now become commoditized by the cloud services. This creates a pressure for centralization.

2. Decentralized protocols have not aged well(TCP/IP, SMTP), have deep security flaws and have proven difficult to upgrade. This has created new centralization pressures to cope with the security flaws(Gmail, Cloudflare)

3. There continue to be pressures for decentralization related to both politics and security pressures. Tor and blockchain cryptocurrencies are excellent examples of these. But decentralization is unlikely to extend all the way to edge devices and instead operate in a federated fashion.

Interesting that you describe TCP/IP as a decentralized protocol. I have the opposite opinion: That TCP/IP is the #1 forcing function for centralization in the world right now.

IP allows a maximum of 255 hops for any packet. This inherently restricts the topology of the Internet: As it stands, it can never be a world-wide decentralized mesh. Instead, you end up with large hubs and choke-points. The IP addressing scheme also makes it very difficult to have a mesh: IP addresses are assigned hierarchically. The name "Inter-Net" describes the problem directly: The Internet isn't a global network that just anyone can contribute or connect to; instead, the Internet is just a protocol for inter-connecting the world's centrally owned and operated networks. With the IP Internet forced topology, Economies-of-Scale make massive centralized services cheaper than distributed services (even if similarly massive). The obvious result: Comcast is your only ISP at home.

Disclosure: I've been working in my spare time for years on a solution, and I think an isochronous source-routed stream-based protocol is the only solution. I've got a proposed spec at IsoGrid.org

Have you seen http://www.scion-architecture.net/?

TCP/IP and Ethernet were decentralized alternatives both from a design and IP licensing point of view. But the unresolved technical debt at the transport layer is major centralizing force on the Internet right now.

I hadn't seen SCION, but just read through the FAQ and skimmed some of the Apr 2016 whitepaper. It doesn't appear to address centralization. For example from their literature: "SCION only assumes that a few top-tier ISPs in the isolation domain are trusted..." Sounds like it's trying to solve many problems, but centralization isn't one of them.

I wish them well, it's good to have lots of people working on this from all angles.

> 2. Decentralized protocols have not aged well(TCP/IP, SMTP), have deep security flaws and have proven difficult to upgrade. This has created new centralization pressures to cope with the security flaws(Gmail, Cloudflare)

Wrong, the reason centralization happened was profit. You can’t make profit with a decentralized protocol. And that’s exactly why we need more of them.

Companies trying to turn the internet into a for-profit thing are the worst possible outcome.

This is one key reason why I strongly prefer federated services over fully decentralized/distributed ones. Typically in federated services each node pays for itself, which is both fair and economical. Of course it doesn't always work perfectly, as demonstrated by email spam.

0) fix the name: 'distributed' more then just 'decentralized'

1) fix the security: https://en.wikipedia.org/wiki/Named_data_networking

2) pay by sidechain: https://bitcoinmagazine.com/articles/greg-maxwell-lightning-...

3) pay players to copy: https://www.wired.com/1994/09/superdis/

Having an incentive is a key, but I think the article is incorrect a bit on what exactly needs to be incentivized. Users are supposed to get some value from the decentralized web, it they do - they will be happy to share the cost of the infrastructure. No need for special incentives there.

The biggest problem, though, is having an incentive to promote the technology and push it to users. Both bittorrent and bitcoin have that incentive, that's why they are successful.

The same people as who have always paid for it: the users at the edge of the network via subscription fees to centralised providers (Usenet, email, etc.), or directly for web hosting, etc.

If you're not paying for something, you're the thing being sold.

It may be we end up paying central IT orgs small amounts of money to host apps (I can see people paying AWS for social network applications of their own running on Lambda, for example), but the network pays for itself by users paying for it.

If users are not prepared to do that, they are always going to be sold as commodities, as they are today in most free SaaS consumer-facing applications.

Privacy and security might sell. Freedom from government snooping and similar.

Anti-virus software shows how you motivate non-technical consumers to buy - in cynical terms you basically just scare people enough until ready to pay.

Well it seems already be a market for selling hardware without backdoors, unfortunately the price is not affordable for the majority of people. There was HN tread about this desktop system https://news.ycombinator.com/item?id=12711185

Often when people think of decentralized webs, they are thinking of bittorrent or tor, or of decentralized webs running on top of traditional web infrastructure, being used in conventional web kinds of ways.

But decentralized webs need not resemble the www. Decentralized webs are radically different.

For me a big part of decentralized webs is people using computers in person, in real life spaces, to share and interact with information. Much of the information will be transmitted in very shallow ways, via home routers and built in wireless hardware, over local networks. People sharing personal files, photos, videos, work documents and using decentralized web applications, together in person or over small, local networks.

The decentralized web could be a huge boon to local economies. For example, imagine a band that plays monthly at a local theater. When you go to a show, they release a new song over the local network, via p2p sharing protocols. Assume that people like the band, and that this is a world where people are familiar with using these kinds of applications. The experience of getting new content will provide additional incentive for people to come to the show. If it is marketed well, the band and the theater will make money from increased turnout. I think this could be huge.

The real question being asked here is: who pays for the world wide web if people aren't relying on it as much. Decentralized webs need not be world wide, though pockets of decentralized webs may interact with worldwide webs, they will not be inherently dependent on it, and will put draw far fewer of its resources.

And in this sense, decentralized webs are paid for by their users when users buy hardware. Much more than users pay for centralized webs.

Maybe there is an argument to be made that software application industries, like the web, somehow incentivize hardware manufacturing. This is probably the case to a degree. For decentralized hardware applications to carry the give people the same incentive to buy hardware as centralized applications do, they are going to need to be dependable and usable by all different kinds of people. If decentralized web applications are an uphill battle for people, then users won't buy hardware with the intention of using it in a decentralized way.

ISPs and users pay for the decentralized networks.

Decentralized networks are a cheaper way to do networking, of course it's not as fast and simple to use, but the big advantage of P2P is that the user pays for the hardware. If your software can work on user hardware (which is a lot of computers), it's a huge economic and infrastructure advantage.

The point is not only to give user the control over their data, it's also to reduce the barrier of having to pay to make a service, since it can also require knowledge and maintenance to build a service.

P2P removes the duality of the internet. The internet is not just a "series of tubes", it's also a large quantity of expensive server rooms and high grade hardware that must sustain large throughput. To me, p2p is the real "web". If today you remove gmail, youtube, google, facebook and twitter, the internet dies because users don't know what to do anything anymore.

Granted, it's not as simple and the software is complex, but if companies can do horizontal scaling in server rooms, it should also be doable with user computers.

i think we will soon be back to thin console, as you can get less than 1ms latency via fiber all you need is IO and the computation is done in the "cloud".

(This is going to seem a bit off topic at first.) I think many of the decentralization projects have issues that will limit their deployment.

A lot of these ideas limit the ability of companies like facebook, google, etc to earn money from a service and/or ignore ecommerce, etc. Take federated systems, for example.. how does a federated facebook earn money?

And while you and I might not care if facebook earns anything, it's going to be tough for existing stakeholders to back it or anything like it if the idea would result in the end of their business.

Similarly, the same limitations apply to startsups.. meaning todays startups, who eventually become tomorrows large companies, aren't going to be based around something that can't be used to generate revenue.

This isn't the 1980s anymore. There are too many users, placing too many demands on systems for it to ever be free.

So to answer the question: I think the distributed web is paid for just like today's web.. because any distributed system that succeeds, will need to be able to support the current methods of generating revenue from the internet.

  it's going to be tough for existing stakeholders to back it or anything like it if the idea would result in the end of their business.
Regardless of our inflation-positive zero-rate economy that caters to the investing class, I think people will do what they want. Obviously a decentralized internet will not be for everyone but the same can be said for democratic societies.

The desire for a decentralized and encrypted internet grows with every new data-cap by ISP's and every illegal surveillance scandal carried out by governments. The people find no recourse in their established authorities and will seek to their own devices for security.

Generating revenue is an after-thought of freed men.

I you haven't adequately addressed why we should care "if facebooks earns anything". The importance existing stakeholders is a contigent detail, not a necessary truth.

It's like evolution: things in society (inventions, ideas, protocols, whatever) exist because they popped up and then survived. One way to survive is to provide a revenue stream for profit seekers who will look after you in return.

Many thing work that way, but many things, such as human languages, and the TCP/IP protocol survive in other ways. Of course all these things do profit people, and are kept around as a result, but through mechanisms quite different to the ones you are talking about.

The old fashioned way: by providing a service.

Here is an example. We'll assume an imaginary decentralized communication platform. Alice wants to chat with Baozhai. Alice speaks English, Baozhai speaks Mandarin. They connect and chat (in their own language). Each capability here, identity, presence, and translation is provided by service providers. Alice and Baozhai have paid for these services.

And why it should be done in decentralized way?

If it's user@domain, the same answer as your email provider ? If it's P2P, basically: everyone by disk space and bandwidth.

We might need to invent an easy and secure way to pay first though :)

Edit: and a comment here gave me another idea: offline syscs to friends when your PC is online (or syncs from friends)

This is an important question. Maybe universities could play a key role as far as hosting is concerned. Anyhow, imo this question is not the most pressing one. If there is a will, there is a way. However, there is no will! The average person is totally okay with storing their correspondence on fb or google servers.

Some energy should be spend on making people want a decentralized web. Better privacy might be nice but even Snowden's revelations couldn't change people's mind. So besides privacy, what additional value would a decentralized web bring to the average person?

Ultimately, the web is centralized because the architecture of the internet (which it's built on) is created that way.

In essence users aren't part of the network the way servers are they are simply clients connected through a ISP. That's because routing is a difficult thing to do well without a planned structure.

It's hard enough to route between normal home wireless routers. Could you imagine the complexity involved in routing between wireless devices like Smartphones that physically move around, turn off all the time?

> centralized because the architecture of the internet (which it's built on) is created that way.

What are you referring to? In a hardware sense, the Internet is decentralized to a large degree. I think you mean that it's Federated, as there is a large degree of coordination within ISPs, but a fairly small degree of coordination (outside of a number of peering stations) across ISPs.

Routing is difficult, but it's not as if there is some central authority managing all routing on the Internet. There are multiple redundant pathways for any given connection, and ISPs manage these things themselves using protocols like BGP.

I wouldn't say the physical architecture of the Internet is decentralized, but it's certainly not centralized.

It'll have to be dedicated devices owned by ISP customers. It'll need to be software in their routers.

Better yet, (and I realize there are a couple logical leaps here) these devices will progress The Internet into a mesh amongst neighbors and neighborhoods. And that's when the Decentralized Web will be realized: when a decentralized, resilient, adaptive Internet is available to host it on and that Internet isn't controlled by profiteering gluttons.

there is lots of buzz around this not just to come up with the next use-case for a unicorn but also to provide break out of the costs associated with securing and maintaining the infrastructure.

server less computing would in applications where this makes sense reduce the latency down to what is dictated by the last mile. on the other hand there is a huge push towards going mobile in many industries (thanks to LTE, LTE-A), but these wireless protocols do horribly poor routing P2P traffic. (this is a problem that operators are just about getting to grips with in their fixed networks).

When it comes to use-cases I think we have some realistic opportunities if an industry is big or desperate enough to also buy data from individuals (rather than a broker). I ran recruitment firm the last few years and a recruiter would always do anything to get their hands on a larger pool of data. If you give them a tiny new cloud service to search for "niche skill candidates" they will pay for it provided they can add additional people to their database. And a distressed candidates would sure rather sell their data directly to the company (earning a few cents along the way) rather than paying linkedin to host their profile and suck up all data.

the bigger challenge (than creating a p2p network) is how to structure services on top, the article mentions btc miners as the profit model for bitcoin. it is for me perfectly logical to have a company with a centralized service/buisness model that builds data applications on top of the decentralized p2p paradigm (most trackers are not really decentralized and the total decentralized bootstrapping of the p2p network itself is also a problem which isn't totally solved afaik).

I was at the Decentralized Web Conference. We pay our local ISPs for Internet access, so the user side is taken care of. I donate little bits of bitcoin to my Gnu Social host. I pay to host my own content because I like to share and it helps both my consulting and writing businesses. This all works well.

What doesn't work well right now is supporting development for replacements of services like Google Now, Facebook, etc.

I love the principle behind the decentralized web movement, but perhaps we should start by chipping away at walled gardens and making data more portable?

Very popular nodes on distributed systems are essentially just another form of centralization. But portability of data creates a free market, low switching costs, and a more consumer-centric focus on retention.

Many of comments here are about storage but I hope that some solution will also care about content. As we see on popularity of AdBlockers and need for privacy, I hope Decentralized webs will have better business model than advertising.

Skype by authors of Kazaa was decentralized, until M$ bought it.

Groove by the author of Lotus Notes was the same, until M$ bought it.

Don't you see a pattern here? In the real world, you can start a micronation, but it will need the protection of a centralized state pretty soon. Thats why libertopias don't exist anywhere on the Earth, after all this time.

However, online, a lot has already been decentralized. Email is decentralized. The web is decentralized. In practice, we just need to make better, more user friendly servers. Why host your video on YouTube where it can be taken down?

The decentralized web has simple economics: EACH COMMUNITY PAYS FOR ITS OWN HOSTING. They can also have arrangements to have (encrypted, redundant) backup mirror hosting for each other.

What people get wrong is that the natural unit for the decentralized web is an ORGANIZATION and not an individual. Domains correspond to organizations.

We're Skype and Groove open source? If they were, it shouldn't matter who buys their maintainer.

I am not too familiar with it, but gridcoin (http://gridcoin.us) apparently verifies and rewards people donate computation power to BOINC.

There website throws a lot of jargon at me but fails (at least I couldn't find it) to tell me how I would run an application on the decentralised compute :(

I'd suggest we start thinking about a new type of business which is funded for performance instead of profitability. Like a utility, the Intercloud is a shared resource that needs to justify it's existence in connecting and processing data in our reality. Traditional business models corrupt this end goal to serve all of us equally and without exploitation.

One possible solution is the creation of a new type of corporate infrastructure entity that is disconnected from our current financial models. It should prevent outside investment or intent to create profit to keep those models from infecting it.

Also, I would point out that infrastructure can be centralized, decentralized and distributed. Whatever we end up with will likely be a combination of all three.

Isn't it a stupid question? We all pay for this. Like we pay today. Every dollar that is accounted for advertising is included in the price of the products.

Your ISP should pay for it

Guidelines | FAQ | Support | API | Security | Lists | Bookmarklet | Legal | Apply to YC | Contact