Hacker News new | past | comments | ask | show | jobs | submit login
A plan to rescue the Web from the Internet (staltz.com)
279 points by staltz on Dec 18, 2017 | hide | past | favorite | 210 comments

The reason the Web needs rescuing is because it's not a particularly well-designed system that has been patched over and over again for the last quarter century. And now it has been degraded to a delivery layer for JavaScript apps, a poor RPC protocol and an overly complex UI rendering toolkit.

It should have had protocol-level technologies for preserving historic data, for search, for offline use, for authoring and publishing. If you look closely, cloud services created tools to easily do all those things and that's how they got in control.

"The Internet was done so well that most people think of it as a natural resource like the Pacific Ocean, rather than something that was man-made. When was the last time a technology with a scale like that was so error-free? The Web, in comparison, is a joke. The Web was done by amateurs." -- Alan Kay.

A lot of web devs were enraged by his comment without listening to the context. He was talking about the lack of protocol-level solutions for resilience of the Web.

> protocol-level technologies for preserving historic data, for search, for offline use, for authoring and publishing

It does [1][2][3]. It's not the most elegant, it's not without flaws, but it's there, and enough to build against. Proprietary services weren't bound by having to code against a standard, so they cranked out something quick (and mutually incompatible), skinned an attractive interface, offered it for free (!) and attracted lots of users, who in turn attracted more users with the network effect. This is why Facebook, Wikipedia, Google Docs, Github won, and protocols like WebDAV lost.

This future was crafted by the makers of web browsers themselves: first by shifting the focus onto rendering arbitrary mediatypes delivered by HTTP from focusing on the intrinsic builtins of HTTP as a protocol, and later by trying to "standardize" the rich interactivity offered by the likes of Shockwave, Flash, and Silverlight, creating more and more APIs inside the browser that allowed client-side Javascript to communicate with the underlying system. Web developers aren't blameless, but today's web is exactly how Google, Mozilla, and Microsoft, in reverse order, steered it since the late 1990s.

[1] https://tools.ietf.org/html/rfc4918 [2] https://tools.ietf.org/html/rfc3253 [3] https://www.iana.org/assignments/link-relations/link-relatio...

Saying that WebDAV lost to Facebook is a bit like saying HTTP POST and DELETE actions lost to Facebook.

I was thinking of publishing and versioning of a different kind. For example, right now the only way to see how some website looked 10 years ago is to go to webarchive.org. 5% of all links on the web get broken every year. Isn't that something worth solving more systematically? In this respect content-addressable Web is a very interesting idea.

It's interesting, but the average web publisher doesn't care, and might actually see being able to retrieve past versions of a page as a bug rather than a feature. So it's not clear why they would implement it, or store old versions of page data on their server, or make that data available to users even if it was stored.

The web exists because it's a bunch of 90% solutions glued together. It's New Jersey style. And that's what works and survives. And we should be incredibly suspicious of attempts to do ground-up rewrites, because such things almost never end well, and typically fail spectacularly.

There were lots of other proposed protocols besides HTTP that had more functionality and features, but were more complex to implement, such that they never caught on. E.g. all those OSI-stack protocols, the majority of which were basically DOA because they tried to do too much. (CMIS could in theory do some sort of versioned document storage and retrieval, although I'm sure OSI probably had its own protocol for that, which I've just never had to work with.)

WebDAV is the right approach, in that it builds on an established, working, well-tested and thoroughly implemented protocol and doesn't attempt some sort of crazy forklift upgrade. It's evolutionary, it's backwards-compatible, and it's probably extensible enough to do document versioning if someone really wanted that.

The problem is that there's a three-part chicken-and-egg-type problem between protocols and client software and content. The protocols need to exist before client software can implement them, client software has to implement them before content can be created that uses them, but client-software developers are often reluctant to implement protocols unless there's user demand, which only comes if there's content available. So we tend to only get protocol improvements when a whole bunch of people agree that "hey this would be really beneficial" and everyone leans in to get it done. AFAIK, there's not that consensus on something like versioned web to make it work yet.

> It should have had protocol-level technologies for preserving historic data, for search, for offline use, for authoring and publishing. [...] He was talking about the lack of protocol-level solutions for resilience of the Web.

That's completely at odds with what Kay actually advocates for. His position is strongly aligned with the "mechanism, not policy" philosophy. That is, what he advocates for is less of what you want, not more. This is apparent from that Dr Dobbs interview, his work with VPRI, back to his OOPSLA speech in 1997. To wit:

> HTML on the Internet has gone back to the dark ages, because it presupposes that there should be a browser that should understand its formats. [...] You don't need a browser if you followed what this staff sergeant in the Air Force knew how to do in 1961. Just read it in, it should travel with all the things that it needs, and you don't need anything more complex than something like X Windows.

Kay has always sat on the diametrically opposite site of the table as those favoring the Principle of Least Power—one of the most underappreciated tenets of computing. Tim Berners-Lee called it out in Axioms of Web Architecture:

> At the other end of the scale is the weather information portrayed by the cunning Java applet. While this might allow a very cool user interface, it cannot be analyzed at all. The search engine finding the page will have no idea of what the data is or what it is about. The only way to find out what a Java applet means is to set it running in front of a person.

You can get a better view of what a world that more closely adhered to Kay's vision would look like in Stanislav's (an adherent's) post, "No Formats, No Wars": http://www.loper-os.org/?p=309

Having watched nearly every talk/interview of Kay I could find on YouTube (which includes the OOPSLA keynote) and having read some of his writings I think I have a reasonably good understanding of what he is advocating for.

Here is Kay talking about Internet's resilience in a bit more detail:


>His position is strongly aligned with the "mechanism, not policy" philosophy.

I don't see how this opposes or contradicts any of what I wrote above.

>and having read some of his writings I think I have a reasonably good understanding of what he is advocating for.

In the Dr Dobbs interview[1] that you pulled the quote from, Alan Kay is not talking about protocols for web resilience. Actually, there is no mention of "protocols" in that article at all.

Even outside of that particular article, AK doesn't talk much about "protocols" other than the protocol of "message passing" in Smalltalk which is a different meaning from "internet protocols".

Tim Berners-Lee is a figure that speaks of new protocols for decentralization, etc. That's not Alan Kay's angle. If you think AK is imploring the tech community for new internet protocols, you need to cite another AK source other than that Dr Dobbs interview.

>[Alan Kay] was talking about the lack of protocol-level solutions for resilience of the Web.

That's not what he's talking about. He's complaining that the "web" did not make it easy for people to do computing. One example in that interview was Wikipedia page on Logo programming language not being able to execute Logo programs.

Kay's criticism of the "web done by amateurs" looks to be the same train of thought about his ideal Dynabook enabling people to participate in computing rather than something like the Apple iPad where people just read news or watch videos.

[1] http://www.drdobbs.com/article/print?articleId=240003442&sit...

Just because Kay criticized the Web for its limits on user participation does not mean he did not criticize it for other reasons.

Here is an entire talk on complexity and robustness in software where the Web is mentioned as one of the worst offenders of design by accretion:


(I'm linking to a specific part, but the entire thing is worth listening to.)

Also, he does talk about protocols. Heck, VPRI implemented a TCP/IP implementation by treating it as a language and using a non-deterministic parser.


Page 17.

This I think is the most important question in search of a better web architecture. You could say it's the HTML vs LaTeX question. I don't mean the "hard to develop with macros and mountains of documentation and ungraspable errors and imcompatible packages" aspect to LaTeX. But rather the question of introspection. (Think also of bibtex and how to compile a document relying on it)

IMHO both approaches are crap and barely work. I don't actually think HTML supports web searches or semantics very well, and obviously it is painful to develop interactive applications in, as well. Same goes for LaTeX, even though it's coming from the other side.

I think we should have containers instead, with in-band data (somewhere linked in a for-the-web assembly language) and out-of-band metadata (in various formats, to support search engines or other enterprises).. Granted, site providers could play tricks by serving disagreeing data over these channels, but in a way that's possible even now with HTML.

Yeah, the Internet had coalesced around one standard, TCP-IP, after fighting with national standards and networks like Minitel.

Now with the Web, the approach has always been to define the front end standards for "user agents" and leave the back end standards to "someone else".

Here is what would make the Web wayyy better:

Support for content-addressable protocols via IPFS ans SAFE. You can already leverage SRI to get some of the "content addressable hashes" but you need a DHT to locate and host the stuff. That way the Web becomes resilient (no single points of failure and you can trust the content) while retaining all its front end goodness.

No more concepts of DOMAINS at all, but rather, each resource of information can be an evolving community in itself. Like people collaborating on a document.

I think this could be the standard for a new multiple-readers-multiple-writers decentralized web: https://github.com/Qbix/architecture/wiki/Internet-2.0

We are working on a mobile browser and extensions for desktop browsers to do these things.

> The reason the Web needs rescuing is because it's not a particularly well-designed system...

The problem is that you cannot predict the future. Standards have to bend and twist to move with the times. The Internet standards were decent for the time they were created. And UI fads will dictate things whether that's logical or not. Humans love fads.

The web needs rescuing not because of technical issues, but because businesspeople and nontechnicals have twisted it far from its original intent and then captured everything that could as 'added value'. "Digital colonialism" is an incredibly apt term for this, and now with NN no longer governing traffic flow, the web needs a reboot away from those that would stick their grubby mitts in the stream.

The web is heavily abused because many developers are not aware of its misguided history or even what the web is for.

To be extremely clear the web is a document (markup) delivery/rendering application. The design goal of the web is to deliver documents that contain hyperlinks, where those hyperlinks address content extension. The web had no intention to be anything else.

That said the web was hijacked, for lack of a better term, by early web browsers that intended to do things that the web was never intended to do. This is how we got JavaScript. Fortunately, it also resulted in a universally standard architecture (the DOM), and additional technologies, that extended the web's original vision.

To be completely fair the Internet was not a spontaneous miracle, but went through similar growing pains. https://en.wikipedia.org/wiki/History_of_the_Internet

The problem today is that most developers have no idea what the web is or really how it works. Now, if you ask a developer about this they will claim otherwise and will work really hard to down play their ignorance. If you are not deeply aware of coding for the DOM, lexical scope, accessibility, asynchronicity, presentation separation, and so forth you are ignorant of how these things work. These are not expert qualities, these are the basics. This largely explains why so many people think HTML is the easiest thing to learn and then written really shitty HTML.

> That said the web was hijacked, for lack of a better term, by early web browsers that intended to do things that the web was never intended to do. This is how we got JavaScript.

I'm not sure that this is accurate. Tim Berners-Lee in his 1989 proposal for the World Wide Web [1] contemplates usage that is compatible (or at least not incompatible) with the JavaScript we have today. Excerpts from his proposal:

> "Hypertext" is a term coined in the 1950s by Ted Nelson [...], which has become popular for these systems, although it is used to embrace two different ideas. One idea is the concept: "Hypertext": Human-readable information linked together in an unconstrained way.

> The other idea, which is independent and largely a question of technology and time, is of multimedia documents which include graphics, speech and video. I will not discuss this latter aspect further here, although I will use the word "Hypermedia" to indicate that one is not bound to text. [...]

> The data to which a link (or a hot spot) refers may be very static, or it may be temporary. In many cases at CERN information about the state of systems is changing all the time. Hypertext allows documents to be linked into "live" data so that every time the link is followed, the information is retrieved. If one sacrifices portability, it is possible so make following a link fire up a special application, so that diagnostic programs, for example, could be linked directly into the maintenance guide. [...]

> Many systems have been put together with little or no regard for portability, unfortunately[]. However, there are several interesting projects and more are appearing all the time. Digital's "Compound Document Architecture" (CDA) , for example, is a data model which may be extendible into a hypermedia model, and there are rumours that this is a way Digital would like to go. ['Compound document' refers to a document that combines text with non-text such as spreadsheets, pictures, video and audio, etc.]

Burners-Lee goes on to sketch out what such a system might look like, and characterizes the current browser/server architecture, emphasizing the separation between display logic and server logic, and anticipating the evolution of those display programs:

> Let us see what components a hypertext system at CERN must have. The only way in which sufficient flexibility can be incorporated is to separate the information storage software from the information display software, with a well defined interface between them. Given the requirement for network access, it is natural to let this clean interface coincide with the physical division between the user and the remote database machine.

> Therefore, an important phase in the design of the system is to define this interface. After that, the development of various forms of display program and of database server can proceed in parallel. This will have been done well if many different information sources, past, present and future, can be mapped onto the definition, and if many different human interface programs can be written over the years to take advantage of new technology and standards.

I read Burners-Lee's writing as very much anticipating the modern browser, JavaScript included.

It's also worth mentioning that Roy Fielding, one of the principal authors of the HTTP specification (along with Burners-Lee), characterized Code-on-Demand (that is, scripting/VMs, e.g. Java and JavaScript) as a core constraint of Representational State Transfer (REST) [2]. The REST architectural style, as described in Fielding's thesis published in 2000, is intended to characterize the architecture of the Web.

> A distributed hypermedia architect has only three fundamental options: 1) render the data where it is located and send a fixed-format image to the recipient; 2) encapsulate the data with a rendering engine and send both to the recipient; or, 3) send the raw data to the recipient along with metadata that describes the data type, so that the recipient can choose their own rendering engine. [...]

> REST provides a hybrid of all three options by focusing on a shared understanding of data types with metadata, but limiting the scope of what is revealed to a standardized interface. REST components communicate by transferring a representation of a resource in a format matching one of an evolving set of standard data types, selected dynamically based on the capabilities or desires of the recipient and the nature of the resource. Whether the representation is in the same format as the raw source, or is derived from the source, remains hidden behind the interface. The benefits of the mobile object style are approximated by sending a representation that consists of instructions in the standard data format of an encapsulated rendering engine (e.g., Java [45]). REST therefore gains the separation of concerns of the client-server style without the server scalability problem, allows information hiding through a generic interface to enable encapsulation and evolution of services, and provides for a diverse set of functionality through downloadable feature-engines. [...]

> The final addition to our constraint set for REST comes from the code-on-demand style of Section 3.5.3. REST allows client functionality to be extended by downloading and executing code in the form of applets or scripts. This simplifies clients by reducing the number of features required to be pre-implemented. Allowing features to be downloaded after deployment improves system extensibility.

So my take on history is that key designers of the modern Web always envisioned it as being capable of delivering and displaying dynamic content with an evolving set of capabilities.

Lastly, an excerpt from an interview with a Wired interview of Burners-Lee from 2014, characterizing JavaScript as part of the web's "fabric" and emphasizing its ability to evolve [3]:

> The good news is that the web has openness and flexibility woven into its fabric. The protocols and programming languages under the hood – including URLs, HTTP, HTML, JavaScript and many others – have nearly all been designed for evolution, so we can upgrade them as new needs, new devices and new business models expose current limitations.

[1] https://www.w3.org/History/1989/proposal.html

[2] https://www.ics.uci.edu/~fielding/pubs/dissertation/top.htm

[3] http://www.wired.co.uk/article/tim-berners-lee

> It's also worth mentioning that Roy Fielding, one of the principal authors of the HTTP specification (along with Burners-Lee), characterized Code-on-Demand (that is, scripting/VMs, e.g. Java and JavaScript) as a core constraint of Representational State Transfer (REST) [2]. The REST architectural style, as described in Fielding's thesis published in 2000, is intended to characterize the architecture of the Web.

Not exactly. REST is only semantic addressing. It becomes an architectural principle when that is the primary basis for the organization of content. That said it is additive to the organizational concept of hyperlinks, but it does not replace or disqualify the original intention of content distribution.

> Burners-Lee goes on to sketch out what such a system might look like, and characterizes the current browser/server architecture, emphasizing the separation between display logic and server logic, and anticipating the evolution of those display programs:

That is how we came to something like CSS. I remember developing for the web in the 90s and browsers completely abandoned this concept. This concept of separation didn't become a practical reality (cross-browser) until about 2006 and then was still challenging in those (IE6) days.

> Lastly, an excerpt from an interview with a Wired interview of Burners-Lee from 2014, characterizing JavaScript as part of the web's "fabric" and emphasizing its ability to evolve [3]:

Yes, the current web cannot function without something like JavaScript. That is the web adapting to the technology opposed to the technology adapting to the web.

This is not a response to your post in general, however it is "Berners-Lee" not "Burners-Lee" as your own sources will confirm.

Thank you for the correction! My, how copy/paste can amplify a typo. Looks like I wrote it correctly only in the first instance.

  Here’s the problem with IP addresses: there aren’t enough of them....
  As a consequence, the Internet has allowed intermediate
  computers to rule. These are like parasites that have grown
  too large to remove without killing the host. The technical
  flaw that favored intermediate computers prefigured a world
  where middlemen business models thrive.
The handwave is the word "prefigure". How did IPv4 and NAT play any role in the dominance of Facebook, Airbnb et cetera? This is an analogy masquerading as an argument.

  It is not fundamentally necessary to have any intermediate 
  company profiting whenever a person reads news from their 
  friends, rents an apartment from a stranger, or orders a 
  ride from a driver.
The author provides no evidence that the services of Airbnb, Uber etc. have no value added. These companies carefully designed interfaces to help us find what we need. If they did not add value, we would still be using newsgroups.

>How did IPv4 and NAT play any role in the dominance of Facebook, Airbnb et cetera?

Very directly, I would say, by making the barrier to running your own low-traffic website too high. Modern social media is all about hosting content - microblogs, photos, etc. Geocities was an early workaround to an inability to host things, and it's worth remembering that MySpace was originally positioned as a kind of evolution of that with the social aspects bolted on - it (notoriously) allowed arbitrary HTML, so that you could make your own "homepage" on the internet. Facebook further evolved this, offering a much more restricted experience. Were it not for NAT, I would have expected an explosion in tiny self-hosted personal sites around the time that dialup was replaced by always-on cable connections, and before long we would probably have had various wordpress-like frameworks to do the social aspects. Which is not to say we wouldn't have eventually had a Facebook-like entity, but it would have faced much stiffer competition than it did, and people would perhaps have been less forgiving of its restrictions if they'd had an easily-accessible wild-west to compare it to.

I don't get this habit of always blaming every problem in the world on NAT.

NAT has (almost) nothing to do with that.

The real reasons are twofold.

1. As others mentioned, the work of maintaining a self-hosted service. Nowadays you can add two extra related problems: people expect reliability (which is difficult and costly to provide, for a hobbyist or a lambda), and Internet has become an hostile territory you must defend against all time (it's "cavemen meet Wild West" level of civility, all day long, no rest).

2. The main problem with self-hosting, is that the throughput is often asymmetric, and rightfully so. From 75/1200 bps modems to ADSL, we have always needed more download than upload so systems were designed that way. When we write one message, we read hundreds of messages from hundreds of people, when we upload one picture, we watch a thousand of them, and so on. When you self-host, you reverse the roles, and your access is very quickly saturated, it gets unsuitable as soon as you have a few connexions.

Just enable the port-forwarding feature of your Internet Box, and you can host what you wish on your server(s); it takes 2 minutes in its web interface and NAT 'problem' is solved. But the other two remain.

The barrier is from the tools, not NAT. Maintaining your own server is prohibitively difficult for pretty much anyone besides niche technology enthusiasts or IT professionals.

> Were it not for NAT, I would have expected an explosion in tiny self-hosted personal sites around the time that dialup was replaced by always-on cable connections

NAT never stopped you from port forwarding 80 to your local server and many people (myself included) do this/have done this with even e.g. DynDNS to have a decent looking host address. At the end of the day managing your own server hardware/software is a far bigger barrier than NAT.

Some (many?) ISPs block forwarding of port 80 specifically. I have a third party router as my main router, with the ISP provided modem/router/wireless AP box in bridge mode, because I can't forward port 80 through the web interface of my ISP's router.

Also, port forwarding is _hard_. It might not seem like it to us more experienced people, but it's a problem that's not very googlable because there's a billion different admin interfaces for routers, and they all do it differently enough that you have to understand what you're doing, not just follow a guide, to port forward.

In contrast, if it was possible to make some simple software which regular people could install on their personal PC, and then they got a WYSIWYG website editor, and that software handled all hosting gave the user a link (ip address) to their website which they could send to friends, I imagine a lot more people would experiment with web publishing. Obviously, in order to run a site 24/7 on a dedicated server with its own domain would still be prohibitively hard for many people, but people would've at least been able to experiment and play with hosting their own simple website which they can share with friends.

Protocols need not always be client-server. The barrier raised by middleboxes is mainly to peer-to-peer applications, whose progress may have been systematically inhibited as a result, in favour of server-based i.e. centralized applications. If that hadn't happened, then your personal micropublishing social media tool might today be a peer-to-peer application, rather than a centralized advertising behemoth. There's no intrinsic expectation that end-users must feel like they're running a server in that user story.

Consider this: the application protocols that arose in the early days of the Internet were decentralized/federated systems such as NNTP, SMTP, IRC. But application-level protocols that are developed with a P2P or federated architecture today are niche rather than mainstream (like BitTorrent), or dead because a major company killed their adoption (like XMPP).

The canary was the loss of old-school active-mode FTP, broken by middleboxes impeding connections. Plenty of people saw this coming.

Maintaining your own server might be hard because there's no incentive to make it easy for technophobes to do. If there were software that was dead easy to use, there'd still be roadblocks: NAT, $ for a domain, ISPs blocking port 80, etc.

also the computer must always be on, it can't interfere with your ability to use your own computer (QoS, throttling), and backups.

The last thing you want is to host things out of your home computer[1], especially when nowadays you can run a VPS for $2.50 a month [2].

If anything, fifteen years ago (when VPSs where prohibitively expensive) we had a much more open internet than now.

[1]. For security reasons (if someone breaks into your wordpress blog they won't get access to your main computer) and uptime reasons (do you want to keep your computer on all the time? Does your home ISP have an SLA?)

[2]. If you're fine with a static blog, a few cents for AWS/Google/Azure plus CloudFlare should be enough.

> How did IPv4 and NAT play any role in the dominance of Facebook, Airbnb et cetera? This is an analogy masquerading as an argument.

Because it made peer-to-peer no longer an option. Keep in mind that NAT is a poor man's firewall and once NAT'd systems were common peer-to-peer firewall transition became a real problem.

https://www.fourmilab.ch/documents/digital-imprimatur/ (2003, still as relevant today as when it was written)

so, that means going with IPv6 will bring back p2p networking? or do you see it's already too late?

p/s: it's more than relevant, it's eerily accurate in predicting our current situation

> so, that means going with IPv6 will bring back p2p networking?

It just might, but Cisco, Juniper and other vendors are already implementing NAT on IPv6.



So what you'll see is IPv6 on the web with the LAN stuck on IPv4 because too many dumb devices don't do IPv6 and IPv4 is enough for local traffic.

That alleviates the need for more addresses on the net while keeping things roughly the same locally.

And this is important because the author misses the point as to why people get value from the "middlemen" -- to find stuff. Even with a content-centered web, there will be a need to find content and people, and new discovery engines and aggregators will emerge.

It's hard to imagine how things could be different, but for many services you don't need a middleman. For a long time, banking was thought to be a necessary middleman because you "have to" trust at least some actor to verify the transactions. Turns out you don't need to, with distributed ledgers (Bitcoin, etc).

When it comes to search engines, two things shake the centralized assumption: (1) you can crawl DHTs like https://github.com/boramalper/magnetico does for BitTorrent, and this can be done self-hosted, (2) too often we assume that searches must be global, but in many cases you are implicitly constraining your search space to regional boundaries or interest boundaries. E.g. as a westerner you are probably not interested in content from Vietnam or Pakistan. Also, when searching the Web, rarely do people browse through multiple Google search result pages. As a consequence, things like `awesome-*` lists ( https://github.com/bayandin/awesome-awesomeness ) are a good starting point for a decentralized curated catalogue that could cover that use case. So you could imagine a curated catalogue of programming websites compiled as a few hundred megabyte file, it could pretty much cover your interest boundaries, and as a file this can be shared over Dat today very easily. Want to search another 'interest boundary'? Run the same search software but on a different catalogue file. These things are possible.

In other words, centralization is overestimated. Maybe there are a couple of genuine centralized-middleman use cases but most of use cases can be very well covered by decentralization.

> Turns out you don't need to, with distributed ledgers (Bitcoin, etc).

The distributed network is the middleman in fact. The entire thing doesn't function without that middleman. To get to the end result, you have to go through that network. It fits perfectly as a middleman concept.

The notion that crypto-currencies can exist without middlemen, couldn't be further from the truth. They're entirely built on and dependent upon middlemen. The distributed ledger is one of the greatest middleman technologies ever devised.

I think there will always be a value in middle men for something, but to me the argument is about how many. As time moves forward there are fewer and fewer domains we go to for things. Facebook Google Amazon. Sure there will be competitors and that's what makes this still fair. But over time, the largest entity of each domain captures an ever larger percentage of traffic, revenue, etc..

The current method of aggregation is to empower fewer and fewer companies to control the rules, what you see, how to sell, what percentage they will allow a sale to generator for the seller, how products are shippped, and how returns work.

This works great for buyers, it always works great for buyers, but behind the scenes what is this doing to our economy and distribution of wealth. It also tends to have a major effect on the country of origin of products - everything is globabalized, so when we purchase something from Amazon we are purchasing something made also increasingly from a single country. Yes other countries will need to address that (as a separate issue too) but still the effect of letting fewer and fewer companies control the market, its no longer a free market.

Is there a way to regain independence? Can Cryptocurrency help? What other peer to peer technologies need to work in concert for us to break our dependency on Amazon, Google, Facebook?

Another way to ask the question: How do we make the online like the offline used to be (we drive down the street and go into a shop - except without the Walmart) but still make this fast-trustable and simple for the buyers.

I was struck with the same thing.

There is a reason we moved from newsgroups, to HTML and browsers to search engines. Each step became a better, more powerful tool to find content faster, more efficiently.

These days so many of the tools we're using are so entrenched, even with some of the changes the author talks about, it would still be a monumental task for people to change.

I mean, look at how bad FB is (on several levels) and look at all the alternatives which allow you to keep your information private and share what you want like Diaspora or Freenet. I tried (repeatedly) to get friends and family to join my Diaspora pod and tried to convey all the advantages of it being private. Nope, nada, no way. None of them ever joined and many just said FB has such deep roots into their lives, giving it up would have a huge effect on their lives, it was crazy.

Don’t forget that we WANT to eliminate all the middlemen if we can, because middle men = bloat. It IS undeniably cheaper to interact directly than it is through a middle man. Don’t forget that!

This is analogous to function pointers (or vtables or dynamic dispatch) in programming. It’s a really tight analog actually, almost literal. It’s slower (more expensive) to go through a cranks. You only do it for external reasons, but you should always try to eliminate them. I hope people around here saying these companies add value aren’t forgetting that we want to destroy them because in an ideal world things are more direct than they are already. We SHOULD want that too, that is the right direction to move in.

There's no way there's any "right and wrong" or any "should" about this. Also, middlemen save me money all the time. Without my grocery & retail middlemen I'd be out there trying to find an apple orchard, a dairy, a tailor, all by myself, and hassling them to sell me quantities too small to make any money on. And that's just the first dumb example I thought of. There is value in aggregation and in making connections and in brokering. My middlemen make me richer by me paying them. That's why middlemen have been around since there's been commerce and why you can't engineer them away. Engineering them away is just doing it cheaper, beating them in the "middleman services" market and becoming the new middleman.

Facebook is a shit middleman that doesn't provide enough value to me, so they're not included in this.

No right or wrong? Why not?

Well it's not always inefficient to have a middleman (sometimes it's more efficient, like in my example above), so you can't say categorically either "with a middleman" or "without a middleman" is always better, i.e. "right."

Also, let's say we're looking at one of the times when the middleman is less efficient. Even if a customer wants to take on those inefficiencies on purpose, we can't judge that person as being "wrong" because they might have other reasons. It's more complicated, is all I was really trying to say.

> FB has such deep roots into their lives, giving it up would have a huge effect on their lives

Then our task is to persuade them that the huge effect would be positive!

This reads like an episode of Black Mirror, only scarier because it's not science fiction: https://coolguy.website/writing/the-future-will-be-technical...

> Then our task is to persuade them that the huge effect would be positive!

How so? What makes Facebook so appealing is the fact that pretty much everybody is on there, just like a telephone book. As such an alternative, with a huge positive effect, has to deliver at least this "centralization" to fulfill the same needs Facebook has been serving until now.

This is a pretty common theme around IT savvy communities: People complain about Facebook and how there are so much better alternatives and how you supposedly don't even need Facebook. But the reality for over 2 billion people (among them many friends and family members of said tech-savvy folk) out there looks quite different, they use Facebook because it's what everybody uses, it wouldn't even work without that, which is what most Facebook competition comes to realize sooner or later.

I just don't see any good solution to any of this that doesn't involve some paradigm shift how we handle private information as societies and as businesses.

What if a lot of your friends and family are on it, and some interesting people you don't know yet are on it, and your favourite brands… aren't?

What if our secret advantage is that not everyone is on it?

> What if our secret advantage is that not everyone is on it?

Facebook might have started out that way, but don't think that angle is gonna help you build a new, even bigger, Facebook.

How long did Facebook actually stick to its "Only students" rule? Imho that was more marketing than anything else.

Scuttlebutt is designed deliberately so that you don't see the whole network, only the bits you care about.

Partly this is view-filtering, but also your user agent only requests and stores data if it might be relevant to you, i.e. friends of friends and replies to their messages.

It's not really one network, but a group of potentially-overlapping networks.

> It's not really one network, but a group of potentially-overlapping networks.

That might hold true for the technical implementation, but it doesn't hold true for the actual user experience.

If you are from the US and looking to befriend somebody from France, then you don't have to join "Facebook.fr" to make that connection, it all works through Facebook.com.

Not too long ago the same process looked pretty much like this:

"Are you on AIM?"

"No, are you on ICQ?"

"Nah, but I'm on MSM!"

"Sorry I'm not on MSM, but I'm on Linkedin!"

"No good for me I don't like Linkedin!"

"Guess I'm gonna make an MSM account -_-"

For that very same reason I ended up creating my first Facebook account on Facebook.de, which was a scam site back then. But decades of compartmentalization of users into "regions" convinced me that if I want to get any use out of Facebook I better sign up with their German version of the site, as most of my friends/family are German, that's how alien this whole "aggregate all your social contacts in one place" idea was back then.

> > It's not really one network, but a group of potentially-overlapping networks.

> That might hold true for the technical implementation, but it doesn't hold true for the actual user experience.

Talking about Scuttlebutt here, it's actually the other way round. It's one protocol and any two people on it could connect, but in practice you won't see everyone, because you only have a social connection to certain people.

>Then our task is to persuade them that the huge effect would be positive!

No, the task is to first create alternatives which are just as accessible and user-friendly as Facebook, and then sell them to the mainstream on terms they actually care about. Why would my 60-odd year old mother who uses Facebook to talk to her COPD support group even care about Diaspora? I don't know what the answer is, but I do know the answer has nothing to do with open source, anonymity or censorship-resistance.

> the task is to first create alternatives which are just as accessible and user-friendly as Facebook, and then sell them to the mainstream on terms they actually care about.

That's exactly my point! A term they might care about is: this is so much nicer to use than Facebook; I feel happier using this to keep in touch.

> I do know the answer has nothing to do with open source, anonymity or censorship-resistance.

Scuttlebutt is neither anonymous nor censorship-resistant. It's free of adverts and sponsored content experiences. It's a friendly, co-operative culture (hence the software is open source, but your mom only has to care about those details if she wants to help build the software).

> A term they might care about is: this is so much nicer to use than Facebook; I feel happier using this to keep in touch.

Here's the problem though - it's not really nicer to use than Facebook for someone who is already familiar with Facebook and who doesn't mind using it. The UX of these platforms tends to be rough in ways that would annoy many non-technical users, which is understandable given their newness, and the fact that technical users are early adopters, but still makes mainstream adoption difficult.

The Scuttlebutt client I downloaded has a search form at the top that will not behave as mainstream users will expect - they will wonder where their "newsfeed" is, try to search for a topic of interest and it will fail out of the box. Making an intellectual or emotional argument for why an alternative to existing social media clients is preferable won't work as long as the physical experience of using the alternatives is more complicated.

>but your mom only has to care about those details if she wants to help build the software.

But those are the details that dominate the frontpages of these platforms. She either has to care or else care enough to wade through it to find what she needs. That's a real problem.

Yes, I agree with you on all of this.

The user experience of using the app, and of discovering the app (i.e. its web presence), needs to be friendly and relevant to the user.

I don't think we want to imitate Facebook too directly, because we don't want it to be Facebook, but it needs to make sense as you use it :)

(By the way, I'm a mere end-user here, and this is just my opinion!)

I've been trying to explain this for years here. People don't care that much about privacy because they can't see it and it's hard for them to conceptualize the scope of what they're giving up. It's a selling point, but not that great of a selling point - certainly not good enough to overcome the network effect.

I forget who pointed it out, but I like the point that to disrupt a big incumbent you need to be not just 10% better, you need to be 50% better. You can get started by doing something that nobody else does which is useful. What FB did better than anyone else was provide a clean interface to social network curation that made it far more useful to the initial population than the alternatives that existed at the time.

> Even with a content-centered web, there will be a need to find content and people, and new discovery engines and aggregators will emerge.

Sure, but why should we rely on a single middleman who at some point could abuse his power? Something like Google search could possibly work in a decentralized fashion.

> Something like Google search could possibly work in a decentralized fashion.

How? Let's suppose I trust Google to crawl the Web and present a plausibly-accurate result to my query. This works because I communicate with Google using TLS (so I trust that Google is in fact Google), Google is the one performing the crawl, and I put my faith (for purposes of this exercise) in Google's editorial discretion and good judgement.

How does a distributed search work? What does my query look like? Where do I send it? How can I gauge how much I trust the response?

Here is a purely fictional example. I frankly don't know how feasible something like this would be:

The search index could be build using blockchain technolgy. Each transaction is an entry of the search index. For efficiency, such an index could then be mirrored by many different institutions. For example, universities. Each snapshot of the search index could have something like a sha256 sum. If the server you use per default returns a hash that differs from other institutions you rate trustworthy then you should be skeptical.

As for search queries, since the index should be standardized in some way there could be many open source apps that require their own syntax for queries so that they suit different purposes and preferences.

BitTorrent presents a plausible (but not complete) replacement. Most torrents nowadays are disseminated through the DHT. User clients can query the DHT to search for specific content, which is then downloaded via its content hash through trackers. Many websites are also available which provide indexes and other features (comments, virus scans, ratings, etc.) on top of this DHT database.

It would be a technical stretch to expand this to replace Google, but if you did you'd have many competing "search engines" built upon an open decentralized database.

I don't think I've expressed this idea well, but you should get the gist.

but then I'd want somebody to aggregate those competing engine results and sort them for me by relevance and applicability.

I don't see where the argument is made that there is no added value.

Instead, i think the author rightly points out that it is possible to do it a different way.

I get a completely different image from that passage in the article. I imagine both certificate authorities, and the domain system as great examples of mistakenly necessitating middlement in a systemically flawed manner.

With regards to the second passage, I suggest you address the literal wording instead of putting your own (strange?) spin on it. The author is pointing out that the value in the facebook post is from the user, your friend, aka a human. The fact that Berliners renting rooms out to a visiting Parisian will be using an AirBNB middleman versus the model a Craigslist entails, being essentially free classified ads. One can discuss WHY things are the way they are, but do try to get the basic gist of what the author is trying to communicate.

>"The handwave is the word "prefigure". How did IPv4 and NAT play any role in the dominance of Facebook, Airbnb et cetera?"

I don't see any handwaving in the word "prefigure" at all. The first middle men were arguably dial up ISPs. They literally brokered your connection to the internet. You got an IP from a dial up pool. Besides having a transit connection these companies had the pool of IP addresses you needed to be a node on the internet.

And so in some way that was the first "computer in front of your computer" business model. I think the author is drawing a through line from there and the business of brokering internet traffic. I don't believe they not saying that NAT enabled FB.

If they did not add value, we would still be using newsgroups.

I wish we were. Newsgroups were great, but they fell apart under commercial pressure; internally from spam, externally from the pretty graphics that were possible in a web browser. I really wish there were a clear successor to NNTP and a simpler, clearer markup standard - the popularity of FB and Wikipedia shows that people value simple, uncluttered interfaces almost to a fault.

> the popularity of FB and Wikipedia shows that people value simple, uncluttered interfaces

I agree with Wikipedia, but FB has a horrible interface.

It's full of dark patterns and I don't like the news feed algorithm. What I mean is that it relies on a small number of font sizes, styles and colors and is heavily standardized. Like, you could render FB in a text-based BBS software without losing much.

The author does not seem to be asserting that these services add no value, rather that similar value can be added by decentralized alternatives.

Indeed, decentralizing the algorithms that power things like Uber Pool (Lyft Line) seems like a monumental task. But, not impossible I guess.

In my view of history, P2P takes off and does well when it's faster for end-users than centralized solutions; P2P solutions fail when centralized solutions are faster.

This holds regardless of the political advantages (or disadvantages) of P2P.

BitTorrent is often faster for large files than centralized solutions, so people use it. (It's still inconvenient for a new user to kick off their first torrent, but WebTorrent will help.)

Freenet is essentially never faster than alternatives, and so it has stumbled.

If some future P2P network can outperform HTTP, then it will succeed; otherwise, it will fail.

Regardless, unless we see a breakthrough in battery technology, it will never be the case that lots of people will carry around mobile phones in their pockets whose radios are always on and participating in a mesh network. The battery would die in an hour or two.

P2P networks work best when coupled with reliable power networks. But if you have reliable power, you can probably set up a reliable wired network, too.

I think this is an accurate assessment of how things play out in the real world. It seems like many people here are attracted to decentralized solutions for ideological reasons, but don't realize/acknowledge that the vast majority of people don't care and will go with whatever is convenient for them.

Maybe there will be a time in the future when the necessary factors will all coincide. Improved battery technology, slowing Internet speeds (for Reasons), and change in the public's attitudes (probably starting with those generation or two below us).

Anything attempting to solve the problem before this time could be cursed as "before its time".

There's a lot of white on this map: [World Population Density](http://www.luminocity3d.org/WorldPopDen/) — the hard part is gonna be getting data between Europe and New Zealand without Big Wire.

I love how Staltz's solution to this goes hand-in-hand with re-personalising our interactions. In short:

1. The Next Billion haven't yet become used to the idea that useful tech services must be global, and provided for you by a single corporation.

2. They want to communicate with other people they know, physically nearby-ish.

3. Sneakernet is never slower than talking in person.

4. Isolated mesh networks and comms with days of latency are viable for these new net users.

5. Get enough people using a mesh and the networks will start to connect.

6. Where the internet is already pervasive, privacy and autonomy advocates are resisting corporate control, and choosing decentralised alternatives.

7. For now, we can exploit the existing internet to handle long-distance.

8. But eventually, enough people will have the bottom-up mindset, and the weak links in the mesh will become worrying.

9. So a solution will emerge to fill in the gaps in the mesh, using ships / beacons / balloons / satellites / modulated whalesong.

How does a mesh cope with most end-user Internet-connected devices (phones) running on battery power, especially in less developed areas? Would we be back to needing infrastructure nodes with wall power, or solar, or whatever?

This is what's keeping me from being over-the-Moon thrilled about stuff like IPFS—it's a bad fit for cell phones, which are most devices.

Each village might have 1 Raspberry Pi attached to a large hard drive and a solar panel, which stores everything — the village library.

Each mobile device would only have to store the subset of data currently relevant to that person — like a backpack.

Maybe there's also an Endless Computer at home keeping the data important to you long term: family history, photo albums, newspaper clippings — the home bookshelf.

(In Scuttlebutt terms, the village library is a Pub; the home bookshelf is a Scuttlebot following only your immediate family and friends; the backpack supports out-of-order messages and discards anything more than a few weeks old.)

I can’t help wonder if there is a way for Big Wire to exist in a way that doesn’t cause them to undermine democracy, society, free will and all the other apocalyptic end-game scenarios promised for Google and Facebook.

Is there no way for long distance telecoms to just exist as a legitimate service that people pay for?

This is how it is already. Just the other day there was a huge kerfuffle about net neutrality on the front page, which is EXACTLY people paying for long distance communications.

Building a connection across a continent just by hopping from mesh node to mesh node is like trying to drive across a country by just using dirt paths. It is clearly not impossible, but there is a reason that most people choose to use highways for long distances. It is similar with a several-hundred-hop long path through a mesh network vs a single hop over a suboceanic fiber connection.

In particular, step 5 is untrue over large distances such as across the Sahara. Step 6 is true, but does not necessarily lead to mass adoption. Step 7 is feasible, but costs a LOT of money at scale. Also, it seems that economies of scale would rapidly push out all inefficient means of communication in step 9 (assuming that sailing ships across the oceans and/or launching satellites will continue to cost money) and we would end up back with suboceanic fibres.

Finally, while the "next billion" may not be attached to their unlimited netflix just yet, it seems highly unlikely that people in developed nations are willing to accept drastic slowdowns in their connections. Streaming video takes twice the bandwidth (both downloading and reuploading) as required for just watching on EVERY mesh node between the viewer and the producer. Caching might reduce this a bit, but is by definition not effective for "long tail" content. I'd be curious to see how long the meshes hold together when it turns out you can't see Game of Thrones because the next node over has no more bandwidth to spare.

OP here. I find more than 4 hops impracticable. That's not what I was suggesting in the article. I suggested (probably not clearly enough) three ways meshes can be efficient: (1) local-first social networking where the use case is "download latest updates and read later" much more than it would be like instant messaging, (2) gossip, the eventual propagation of data over long distances, (3) satellite-based meshes to cover large distance hops.

> step 5 is untrue over large distances such as across the Sahara

Yep. Focusing on local, social connections makes this less relevant.

> Step 6 is true, but does not necessarily lead to mass adoption.

Yeah, there's more scope for growth in places that don't already have abundant internet, where the pull is practical. But there are also a growing number of enthusiasts for whom the pull is principle, and they can help make mesh networking viable for poor people in rich cities.

> Step 7 is feasible, but costs a LOT of money at scale.

Step 7 is what you and I are doing right now :) — using the existing internet. This point was supposed to mean: the existing internet, can co-exist with and compliment mesh networks. Staltz's [Multi-modal connectivity](https://viewer.scuttlebot.io/%25OryfD6hGpjiqwFkKyFnhVPZMPsZF...) puts it more eloquently.

> it seems highly unlikely that people in developed nations are willing to accept drastic slowdowns in their connections.

Never underestimate the value of cheapness! I know some people who don't have home broadband; they just use their phone's data connection. It's slower, but it's adequate, and it's cheaper.

Mesh networking doesn't have to completely supplant ISPs. I want it to become a viable alternative, suitable for certain people depending on their priorities — like Linux!

> Caching might reduce this a bit, but is by definition not effective for "long tail" content.

If we're talking about data storage here (the flatbed truck — as opposed to data delivery, the series of tubes) then HTTPS can still be a highly-available option of last resort. If you want people to see your content, and no-one else is interest in replicating it, you can host it yourself or pay someone else to. Maybe use hashbase.io; they seem generous. If you mean data delivery, the packets have to traverse the internet somehow; it's just a question of whose box they go via.

Good observations about the need for the Internet to handle long-distances. I wrote on scuttlebutt that telecommunication channels could become multi-modal like public transportation is often in Europe: https://viewer.scuttlebot.io/%25OryfD6hGpjiqwFkKyFnhVPZMPsZF...

Another way of bridging the gap in that white map is through data gossip: the person who leaves a village and comes back can bring data updates with them and share that with the village. Sneakernets in Cuba (as far as I know) work like that, but shouldn't be as fast as mesh gossip.

I'm all for p2p (really), but how about taking back control (sorry) of the Web by actually developing and enforcing declarative/markup technologies and standards instead of praising JavaScript because "it's not half bad" and adding procedural features to the Web (APIs, WASM)? With the current state of affairs wrt privacy and self-proclaimed standardization bodies, I'm not sure the Web is worth preserving.

HTML with great default styles and better built-in form and interactive elements (date picker, sortable tables) would be ideal. Screw CSS, it slows everything down, just have good defaults and don't let pages mess with them much. If we must have CSS at least jettison the complex parts with animation and such. Make it very, very simple and weak. No Javascript. Can't be trusted if it has the ability to initiate connections or modify form data, isn't useful enough without that to justify the added bloat to the client.

God, it'd feel lightning fast compared to what we have now. And it wouldn't be able to spy on you. A document-centric (once again) web that's somewhat less crippled than Gopher would be amazing, but we absolutely must cut out things like scripting in order to make the client trustworthy (and fast).

> it'd feel lightning fast compared to what we have now

I disagree with this completely. So much of what JS does is increase the (perceived) speed and sanity of the user experience. For example, in a scriptless world, your HN up/down vote can't be done without a full page load, which would come with the added headache of changing the state of the world on your page because stories and comments have changed their ordering. Solution? Return of the chronological thread (no thanks).

And there are a lot of other applications in this same boat. I don't know if you are old enough to remember webmail when the refresh button was how you checked for new mail. I'd greatly prefer what we have now, where mail just appears over a socket. Online drawing/CAD apps or games without CSS and JS? Forget it. Or go back to Flash.

Bottom line, we have the ecosystem we do today because the users demand it. For us as devs, it could be better, more consistent, etc. but we need the capability.

> For example, in a scriptless world, your HN up/down vote can't be done without a full page load

You're incorrect. The up/down vote can be a form POST, which receives a 204 No Content in response. The only thing one would lose would be the change in the vote arrow.

Even that could be achieved in a world in which HTML allows PUTs: PUT to the page URL the specific bytes required to select an arrow, and return a 204 No Content or 206 Partial Content. In the first case the browser would know exactly which bytes to update, because it PUT them; in the second it'd know exactly which bytes to update, because the server sent them.

The case of reloading only parts of a page (while retaining boilerplate and other already-received content) could trivially be solved if HTML could make use of entities (a concept from SGML on which HTML is based) in combination with HTTP cache controls. But once Netscape brought us JavaScript, any effort in this direction was doomed because it wasn't essential in an already Turing-complete scripting environmemt. As recently as earlier this year, HTML Imports were removed from the spec I believe (not entirely sure about this).

> I disagree with this completely. So much of what JS does is increase the (perceived) speed and sanity of the user experience. For example, in a scriptless world, your HN up/down vote can't be done without a full page load, which would come with the added headache of changing the state of the world on your page because stories and comments have changed their ordering. Solution? Return of the chronological thread (no thanks).

Counterpoint: when JS/Ajax-free versions of sites are available, they're usually faster in practice. Gmail, Google Calendar. Tried them in their low- to no-AJAX versions lately? I use basic HTML gmail because I got friggin' sick of how slow AJAXy gmail and Inbox were on my stupid-fast MacBook. For the specific case of upvotes on HN, IIRC this would be solvable with the appropriate 2xx response (I forget which), signaling success but not triggering a page load, without requiring any modification to current web clients. Obviously this hypothetical user-focused web successor would be well served by more flexible linking options (more method support, mostly) and built-in form capabilities—which happen to be things the current web really ought to have too, for that matter—but I don't think HTTP itself would need to change.

IMO email and chat with notifications and such belong in their own programs, if you want stuff like live notifications instead of a static page. I'd rather bloat and code that can communicate with the outside world when I haven't specifically told it to not be included with my document browser, because it makes every operation on said browser slower, (way) less safe, and less predictable.

Or hey, how about RSS/Atom for new mail notifications?

> I don't know if you are old enough to remember webmail when the refresh button was how you checked for new mail.

I was on the web well before Gmail existed, so yes, I remember the bad old days of webmail before AJAX. Most of the improvements we've seen in it would be served about as well by smarter, better client-provided form elements, and some other, smaller modifications to the client side, save update pushes. I don't think what we've gained from mixing traditional web stuff with the huge set of things a page might now do to/for you with Javascript and CSS has been even close worth the cost in trust, security, privacy, and predicability. Quarantine that stuff somewhere else, plzkthx.

I think you're quite right. Browsers would need to add quite a few capabilities in order to support the world you envision, but those capabilities would be simpler and less generic than javascript support.

HTML built-in elements—especially forms and tables—have atrophied badly, I guess because the Javascript crutch is right there. That's gotta be why things like the file picker are still awful, there's nothing like native UI date/time pickers that any sane native UI kit includes, tables don't have (re)sorting built in, and so on.

So the work would be, 1) delete like 80-85% of Gecko or Servo or whatever, 2) add back about 5% of that to patch in better form elements and such, 3) write some default styles that don't suck and a system for editing them or loading themes, maybe even per-site (nice, but not needed in the MVP), including user-submitted styles, and 4) overcoming the chicken/egg problem of getting content on it to attract users, or vice/versa (there's the tricky bit).

For 4), what if you started with clean versions of Wikipedia and things that could be built from public datasets? That would be enough content to provide a meaningful comparison experience, and if you had good content creation tools the smart set could migrate over to HyperNet or whatever this brave new world is called, originate content there, and dump it to the web as a secondary option, recruiting people through the blogosphere.

I would absolutely be up for running something like this in parallel to my existing browsers. Right now I have Chrome and Firefox going, migrating things slowly over to Brave, and Tor about half the time. Plus desktop RSS readers and other stuff. I'd love to have some clean virtual space to work in. One more open window wouldn't be a problem.

> in a scriptless world, your HN up/down vote can't be done without a full page load

In a perfect world, HN wouldn't need any web thing at all, NNTP is better fit for it. With all the features of decent newsreader of your taste.

How does one upvote comments using NNTP?

In a perfect world, there's your very own scoring rules (adaptive and AI-assisted, why not) instead of groupthink-enforcing tools like upvotes.

> For example, in a scriptless world, your HN up/down vote can't be done without a full page load, which would come with the added headache of changing the state of the world on your page because stories and comments have changed their ordering. Solution? Return of the chronological thread (no thanks).

I don't see why it would have to change the state of the world on your page. Sites like HN in a scriptless client world would still be programmatically generated on the server, and presumably could still keep track of state via cookies or via adding parameters to links when they generate the page.

Up/down vote could reload the same view you had before voting, with only the effects of your vote added.

That's true, but it would still need to reload, rather than just sending the little request in the background as you keep scrolling along.

Why does it need to reload? Why not just have the votey button hyperlink to a dummy fragment URI like "#upvote_comment_1234", and something like a ping attribute [1] to send a request in the background, and a CSS rule like

  a.votey:visited { visibility:hidden; }
to hide the button once you've clicked it?

[1] https://developer.mozilla.org/en-US/docs/Web/HTML/Element/a#...

I honestly didn't know about "PING," thank you. I have no idea how I missed such a fundamental feature, even if it is relatively new.

Users don't have any idea what they want.

They weren't the ones creating Ajax to try to fit Outlook into the browser, and everything that came after that.

> Users don't have any idea what they want.

...but they certainly know it when they see it.

Sorry, but I wouldn't like to lose the ability to build fully custom solutions. I don't want to rely on defaults. One person's sensible default is another's insane one. JavaScript is what it is today because it serves a purpose and frankly, I think it does it well enough for now. And without CSS we'll end up with Beige boxed web, all looking the same, behaving the same, and stuck in someone's idea of a sensible default.

> And without CSS we'll end up with Beige boxed web, all looking the same, behaving the same, and stuck in someone's idea of a sensible default.

That'd be amazing. I wouldn't have to manually kill awful anti-user elements on half the pages I visit, learn how some special snowflake interface works yet again, and so on. I'd know exactly when server communication was going to happen. I could set custom user styles to whatever I like, or, probably, select from the most popular user-submitted ones on some settings screen.

Write applications if you want that stuff. It's not like the current Web would go away, if you wanted to write exceptionally bad applications in e.g. Electron or write "web apps". I'd just like a program I can open to browse a document-centric web that I know won't spy on me, or use stupid amounts of resources to show me some text while hijacking scrolling, or show me popup ads, or auto-play video, make me adjust zoom levels per-page for the best reading experience, or any of that stuff that makes the Web worse and that we cannot get rid of without ditching (or crippling) Javascript and CSS. An enormous percentage of the Web would be vastly better under such a system, and most of the rest would be better as a real, native application.

Isn't it simpler to enforce-by-convention good ethical practices when developing web apps and sites? The slowness is many times not caused by the site but by the underlying extras underneath - useless and badly done ad tracking, overdose of adverts, video ads, sound - those can go. Good design however, sells. People who'd prefer beigebox internet are a minority. If they weren't the net would be different.

Of course not, but if you want to be an app, be an app. There's huge value for consumers in standardization, that was something people liked about paper encyclopaedias back in the day: a huge amount of information presented in a consistent format that's easy to navigate. By comparison, the web is a drugstore magazine rack with minimal functionality wrapped up in eye candy.

Way back in the early days of the web (said the old man) you also had control over how stuff presented int eh browser, you could customize fonts etc. to read stuff in a way that was comfortable for YOU. I absolutely hate everything being defined at the server end; large parts of the Settings page in my browser don't actually mean anything because nobody uses standardized html any more, and so client-side user interface development has stalled badly. I swear desktop Linux looked better 15 years ago than it does now because people took theming more seriously.

without CSS we'll end up with Beige boxed web, all looking the same, behaving the same

Designers generally vastly overestimate how much users actually want "design".

Engineers generally vastly underestimate how much "design" affects purchasing decisions.

You can still have design, custom design even. But it would be so nice to retain the ability to just see the content without all that (IMO) unnecessary "design" rubbish.

Reader View in Firefox is somewhat a step in this direction, but I find it removes images and some other important content.

And developers vastly overestimate how much users really want bland websites like we had 10+ years ago.

Sure, _we_ like it because it can be great for "power users" like _us_, but 99% of the population aren't power users.

It reminds me of devs wondering why many people prefer bloated, slow GUIs to CLIs.

What are you saying? That non-power-users like and are good at deciphering different designs and interaction models and layouts for every site, but they can't deal with regular, plain, bland sites because those are scary and confusing?

If anything, shouldn't that be the other way around? Power users wanting javascript experiments, and non-power-users wanting "if it ain't broke, stop messing around with it"?

No, not scary and confusing. Users (meaning non-power-users, generally) aren't always stupid. (In fact, given how well they can break our stuff, I'd say they're often smarter. But I digress...)

What I'm saying is while we might be comfortable looking at plain HTML with #000 text on a #FFF background, a user isn't going to enjoy it very much.

While we might use ctrl+f to hop around a page, a user will probably prefer a search bar with instant results that doesn't cause them to lose their place on the page.

Consider: my mother (not picking on her!) definitely notices when websites run slowly. She hates it. (I got my lack of patience from her :-) However, I'm confident she would trade "slightly slow" Facebook for "have to refresh the page after each action" Facebook.

The client would be the one customizing the appearance. You pick the client you like, or pick one which is themable as you wish.

You know, like for e-mail, or newsgroups, and so on.

> all looking the same, behaving the same

Yes, please!

I'd like to see a return to semantic markup, accompanied by pragmatic information about the purpose of the data and its intended size. The data should also be strongly typed. Then leave it to the client how to display, how to edit, and how to select the data.

The receiver should decide how to best display e.g. a list of 20000 names or a choice between 4 options on his device.

Me too, except I would say strongly duck typed. As in there should be sensible defaults when possible.

In any case, the designers won this argument a long time ago. They want to decide how you consume their page, it is not a document to them, it is a work of art. How dare you want to display it differently from their vision.

<script src=“...” usage=“spyware”>

I wonder where this would fail!

The problem with that approach is that you have to define all your possible use-cases up front. If the web had stayed as a simple document format, it wouldn't even be worth talking about now.

A much more flexible and future-proof approach is a fully programmable environment without any declarative features at all. No HTML. No CSS. Just a clean API in a tight sandbox.

> The problem with that approach is that you have to define all your possible use-cases up front. If the web had stayed as a simple document format, it wouldn't even be worth talking about now.

Everything but web apps and spy-vertising would be better (for the user) without Javascript, overall. I submit that those can live on some other platform (the current Web could keep going) while most of the rest could switch to the low-footprint client that cannot spy on you or do user-hostile crap with its interface (popups, hilariously bad layouts/styling [see: Medium] that "look cool", and so on). Keeping low-trust scumbag code strictly separate from safe sites that don't try to screw with me because they cannot would be very useful.

Ecommerce, Facebook, webmail, all that stuff would be fine without Javascript, if anyone cared to do that on this hypothetical Web alternative that treats the user with respect. Tried basic-HTML Gmail lately? It's faster, with its full-page loads on every interaction, than Ajax gmail and way faster than Inbox, and many of the most useful features it lacks from those could be added server-side with no more, or even less, difficulty than adding them client side (Inbox's categories, for example). Its worse layout issues, especially on mobile, would disappear entirely if we had sensible defaults (user overridable) for each platform's browser to which pages were forced to defer.

> Ecommerce, Facebook, webmail, all that stuff would be fine without Javascript...

But something as simple as voting on reddit would be tedious.


Every upvote would refresh the whole page.

See discussion in this thread about the "204 No Content" response code. Not all successful HTTP requests trigger a page load. And that's without having to change anything about current HTTP/HTML—except maybe giving hyperlinks and/or forms access to more HTTP methods, though for god's sake we ought to already have that in HTML and if you'd told me in 2007 that in 2017 we still wouldn't be able to PUT or DELETE on a web page without Javascript I'd have laughed at how implausible that was, so that's a long-overdue change anyway.

> See discussion in this thread about the "204 No Content" response code.

HTTP is not the problem, the problem is knowing how to update the rendered page once the upvote happened (which, sure, could be communicated to the browser by a 204.) JS is what does that now; what does that without JS?

You'd have way no indicate that your upvote succeeded with a 204 status code.

Going back to my original point, and what you're arguing for me here, is that do this properly you'd have to imagine and implement every possible use case. In 1993, nobody envisioned web pages like Reddit. And yes, without JavaScript the web would be much faster and much safer but also much less interesting and powerful. It's hard to imagine websites without simple upvote/downvote mechanisms but that's what you're asking us to do. And that's just one tiny feature that barely scratches the surface of what is possible.

We had working voting systems before AJAX (source: I have a high-but-now-considered-lowish Slashdot ID) so we can do it. Besides, visited link state seems like it'd resolve the problem, and it's been in HTML approximately forever.

I still don't see the problem. The rest of what's possible can go on the current web, which would hopefully optimize (even more) for delivering apps if that happened. I want a content reader that won't, in practice, often run malicious code when I point it at the wrong document. That means it must not be able to run scripts, or at least not scripts that can communicate with the outside world in any way, at which point you may as well save the disk, memory, and complexity by leaving scripting out. There's no reason the thing I browse Craigslist, Wikipedia, IMDB, and Facebook and Twitter for that matter on must also be able to run ports of Wolfenstein or Excel clones or whatever. Or chat clients. The loss of trust and predictability in ordinary web browsing that scripting and complex layout systems bring aren't worth it. Put them somewhere else (including right where they are now—that'd be fine. Move the web to something more suited to it and leave the apps on the current "web")

I have a site that delivers content, forums, and maybe even a few application-like features all in one. Why have two systems when one system can do all that? It'll simply never happen. The web killed gopher. The web killed NNTP. The web has almost killed email. What you described has existed and been killed off a half-dozen times already.

I'm not against having more user-protection features (heck, I have a number of plugins installed to do just that) but limiting usability and creativity is the wrong approach. It's been done and it always fails.

I'd be OK with keeping both in the same client, with warnings when transitioning from one to the other (say, you follow a link on a web page to a web app), if that's what it took. It'd be like the HTTPS lock and content-origin policies and such but for guaranteeing the page you're visiting isn't actually a "page" that's spying on you and doing all kinds of other nasty stuff, or that the link you're following may go to a page that's a god-awful pain in the ass to read and burns your battery up ("modern" designed web articles). Like a not-shit version of Google's AMP, kinda.

It'd be better separate if that worked out—if all the things that work just fine without JS and advanced CSS went to the New Web I'd be able to arrange things so I rarely need anything else. Web apps blow and I try to avoid them when possible, or, increasingly, they're better off in Electron for reasons including that it's quicker to get to them to murder them mercilessly when they go rogue, than it is with some tab somewhere. But a unified client for both that kept them strictly separate would at least be a lot better than what we have now.

Everything you describe is a usability nightmare including "with warnings when transitioning from one to the other". You might not use any web applications but major web applications are the most used sites on the web.

Would it not be better to provide full functionality and simply use better sandboxing to avoid spying and nasty stuff? Reducing functionality is like throwing the baby out with the bathwater -- you really want better privacy and more control but rather than attack that problem you want to take the web back 20 years. That's a solution but it's not a very good one.

I personally don't think that browsers shouldn't leak all the information that they do -- that should be a hole to be plugged.

> You'd have way no indicate that your upvote succeeded

Sure you do. 204 No Content for success, 4xx for an error that you (or your browser) can fix, and 5xx for an error that only the server can fix.

Still horrible usability; you've taken me away from what I was doing to show me an error page.

There is no rule in HTTP that mandates the user-agent must navigate away from what it's currently displaying to show you the payload of an HTTP error. It simply says [1] that the error "SHOULD" (in the RFC 2219 meaning of SHOULD [2]) be displayed to the user. Because of this, most browsers display the payload, but some show a built-in "friendly" error page if the payload is under a certain size.

Because of the language in the spec, the server can't rely on any particular behavior. If it reliably knew that user-agents always display the payload, the body of the 4xx or 5xx response could contain the exact HTML of the page you were just on, with the details of the error spliced in. Understandably, few people do this.

In late 1990s, early 2000s browsers, a status bar was commonplace which tended to show parsing errors, script errors, and other errors the user probably couldn't fix; nowadays these notifications have been moved to the console or log and are hidden from non-power users. A HTML form property could be used to signify that the form be submitted out-of-band, and have the results logged instead of updating the document view. With HTML5, they introduced the 'async' tag to allow linked assets to load without blocking rendering, so conceivably the same mechanism could be used to mark forms, if any user-agent were interested in supporting it.

The point is, extremely minor changes could improve usability in this particular case, using semantics that aren't breaking new ground at all, but quite simply no one has done it.

[1] https://tools.ietf.org/html/rfc7231#section-6.5 [2] https://tools.ietf.org/html/rfc2119

None of this is necessary if you have JavaScript to control the error handling exactly the way you want. If it's a critical problem, you can display an error message. You can retry behind the scenes and succeed without even having to inform the user of a failure.

Again, I make the point that a pure declarative solution only works if you know every possible use-case. You can add all kinds of new tags and attributes that could possibly exist to solve the one problem you imagine right now and that still won't be enough.

Completely agree. Declarative approaches define the palette of available options ahead of time, but if wish to do something out-of-scope, it's back to the drawing board. Imperative approaches instead give you a low-level computation environment, and hooks into manipulating the outside world.

Most facets of the original Web are declarative: HTTP is a complex state machine, HTML is a declarative language, hyperlinks are declarative ways of specifying outgoing edges of a node in a graph. Javascript enhanced that with imperativeness, which is why it's been leveraged to extend the Web in new ways -- ways that are ad-hoc reimplementations of behaviors that have not (yet) been otherwise codified.

> If the web had stayed as a simple document format, it wouldn't even be worth talking about.

Maybe in the sense that we're conditioned to always look for the "next thing". But HTML is also used for legal documents, in education, for cultural heritage, etc. Basically, for most written and published information created by mankind today. A role too important to leave it to very few monopolies.

I would love it if the internet worked like this, but I don't see how we could ever get there from where we are now. Too much is invested in the status quo and people are too used to it.

Gopher is cool. I bet if Gopher made it easier to link to other documents it would have really taken off.

Well TFA has portrayed p2p as the way to go. Relying on classic HTML and/or other markup will help in cross-media publishing to get there. Presence of much more interesting content on p2p will take people there.

A lot of the smaller blogs and personal sites I follow use CSS and Javascript in quite unique ways that give them the freedom to express themselves. I think this is a huge use case that your view misses out on in a people-centric web.

I agree that articles would be a lot easier to read if everyone followed a consistent style. And sure the internet would be a safer and faster place without javascript. But you can't just throw it all out the window.

It sounds like what you want is some sort of simple web, with nothing more than HTML and some decent default styling. Wouldn't a web based on markdown or latex be awesome?

Yes, please.

The vision of these three posts (tannhaeuser, ashark and JohnStrange) are incompatible. This thread is a tiny (but complete) clinic in why the web is such a mess.

I submit that Tannhaeuser's vision is what can replace the current web in its cross-platform-app-distribution role, leaving the document-browsing side to mine :-)

That addresses the core problem, as I see it, which is that when I want to browse some damn HTML I'm subjected to a ton of actual user-hostile behavior and lots more potential hostile behavior that I have to worry about, all while paying for it in disk space, memory, and processor cycles (so, paying in terms of battery and UX). Mixing the two is why the Web sucks so very much. Separating them into a document-browsing client and a sandboxed-app client is precisely what I wish would happen. I don't want a link in my document-browsing client to maybe open an application without telling me, nor for it to have access to all that capability when I just want to read Wikipedia or whatever, and trust that the links I follow won't lead to some "page" that spies on my every mouse movement, mines Bitcoin, auto-plays video ads, pops up "SUBSCRIBE TO MY NEWSLETTER" modals, or any crap like that.

I'm pessimistic about strict semantic markup/typing of data being viable as a web replacement, as in JohnStrange's suggestion.

So under that model, would Wikipedia, which uses javascript, be in the sandboxed-app client, and would you would be perfectly happy viewing it with javascript on, in that case?

I'm curious which sites you actually frequent which wouldn't count as "apps," but which still use javascript, and which would actually run in the static-only client you seem to prefer?

My own personal vainglorious idea about the opportunity missed with the web is much more radical yet.

Semantic markup (remember, TBL originally designed HTML this way) failed already once for a reason. Markup language was a bad choice to begin with.

Instead of an SGML-derived markup, we should have had a set of primitive s-expressions that were supported in a standard allowing people to extend the behavior of browsers by implementing renderers based on any kind of s-expression. Wanna extend your browser? Just implement a renderer. Wanna distribute a renderer? Fine; the interface of renderers are part of the standard. You can distribute the renderer along with your page.

There would never have been JS to begin with: that gets handled right out of the box with s-expressions.

The first choke point is the ISP. You are completely dependent on the ISP or mobile provider to get on the network. Once on the network there are multiple choke points around IP addresses, dns, ca authorities, registries and more.

As long as you are dependent on anyone to get on the network it by definition can't be decentralized. Consumer wireless tech is heavily regulated and governments are extremely paranoid about communication channels they don't control and can't monitor.

This is unlikely to change because there are no incentives to develop technology that truly empowers individuals, there is no profit in it, it's a social good. If developed it will be demonized and made illegal, and limited to minority dissenters, the general population is unlikely to go through hoops to get on a network.

He takes a while to get there, and doesn't quite drive the point home, but this is actually what André Staltz is proposing. Mesh-first devices. If we start at the bottom of his article, here's basically the plan:

1. Work on polishing IPFS, CJDNS, SSB[1], Beaker Bowser and some others into a "mesh-first" setup. 2. Sell, subsidize, or outright gift mobile devices to offline users in Africa (I'm going to assume these are android smartphones) setup to automatically join a mesh network, with these apps installed. 3. Users will transport these phones around, adding and creating content. Their phones will automatically sync content with other devices as they come in range. In general, only content that the user is interested in (that they follow) will be synchronized, though some may act as hubs syncing all manner of content for later redistribution. 4. As usage grows, there will be various regional and national mesh networks that build up.

I think a good comparison is to Cuba's weekly sneakernet[2] distribution, just with wireless mesh.

[1]: Secure Scuttlebut is one of the author's creations: https://www.scuttlebutt.nz/ [2]: https://www.wired.com/2017/07/inside-cubas-diy-internet-revo...

Content-addressed overlay networks work pretty well over the Internet, and they could work equally well on mesh networks (as the article posits), if not two rather awkward problems: storage at rest, and liveliness of a storage node.

These two factors even interact to make the situation worse. Because at any point, nodes can drop off and you never know if this condition is temporary or permanent, a distributed datastore has to redundantly store everything. This needs much more space than just storing everything at its origin, as is done in location-addressed networks like Web.

These are not insurmountable problems, of course; it's just that right now, conditions like storage on nodes, link asymmetry, traffic distribution asymmetry still favor centralization.

And fundamentally, the operators of individual nodes (i.e. ordinary people) are often not very selfless, and not enough of them contribute to the health of the mesh even if they discretionarily choose the mesh: seeding ratios on BitTorrent networks that are not user-adjustable (e.g. old World of Warcraft downloader, Facebook patchsets) are much, much higher than seed ratios where the peer can bow out at any time, despite all of latter users choosing BitTorrent voluntarily.

The modern CDN web stores content at hundreds of locations, but the locations are all sponsored (paid for) by the original content provider, presumably that payment is subsidized either monetarily by the end-user or by advertisers. So there's really no difference. There are a couple of cryptocurrencies aimed at settling the bill for hosting activity like this that may improve the reliability situation (by creating a monetary incentive to do that hosting)

> a distributed datastore has to redundantly store everything

Scuttlebutt gets around this: you only store and rebroadcast your friends' stuff (and anything else relevant to you).

You're actually seeing content from your local storage — which is why Scuttlebutt works seamlessly offline — so there's no need for altruism.


"a decent(ralised) secure gossip platform" [1], also referenced in the article.

[1] https://www.scuttlebutt.nz/

After reading the FCC Chairman's idea of "the Internet" in his Notice of Proposed Rulemaking (below), I think maybe a better plan would be to rescue the internet from the web. Every example he references of internet usage is web usage or email. He does mention "DNS and caching" but in the context of the potential effect of removing these from the services available to users. (para. 37)

The general tone of the NPRM seems to be that an ISP can and will block or throttle any non-web or non-email traffic. That could include any peer-to-peer innovations that seek to restore the original functionality of the internet, such as those mentioned by the author.

On the contrary, the dissent by Commissioner Clyburn specifically mentions Skype as an example of internet usage. He believes the traditional notion of "permission-less innovation" is under threat from the Chairman's proposed approach.


The author highlights the importance of distinguishing "the web" from the internet. Perhaps nothing is more important. The internet has more value than the web. The web is severely limited in functionality. The internet, still underexploited in its potential, does not suffer from the same limitations.


Before DRM, spam, i-have-to-monetize daily 5 minute of bloging into a full time pay, unskipable 30s YouTube ads, profiling every possible user behavior and other shady shit like that:

We wondered how can we improve the web.

Now, we wonder how to save it.

I would say before algorithms started showing us different content even if we are subscribed to the same feed(s).

DRM is hardly a problem unless you're using a handful of products, unskipable ads were hardly a problem before smartphones and their market stores took off, and spam was there since the Internet started.

What is new is that Facebook, Google, YouTube, Snapchat, Instagram, and increasingly Twitter are all showing us different content, and our personal preferences (things we follow) are increasingly irrelevant to the information we're receiving.

This all happened in this decade, and this is the decade where we started to fix instead of improve the web. I would point the breaking point somewhere around 2010-2011, when Google, YouTube and Facebook all decided to personalize the things we see.

Spam, scams, and bad ads have been around since the start, what internet were you using?

Spam, scams and badvertizements have been around seemingly since long before the internet.

They were, but back then, that stuff seemed much more marginal, now it's seems front and center with even major players engaging in some bad behaviors.

Since the start of the web, arguably. Usenet used to be really pleasant to use apart from the first few weeks in September when a new bunch of college students had to be integrated into the system. Spammers blew up something wonderful for a quick buck.

My wife gets increasingly pissed off when she searches for various materials she wants to purchase. A specific site keeps coming up for her that she absolutely hates and starts with an E. And there's nothing special about the links, and likely nothing important about the specific products other than the domain name that starts with E has billions of cross-links all over the web. It makes the random long-tail search terms (that mind you are so freaking obscure) all go to this same site. It's driving her nuts.

At this point SEO has been totally gamed and search is totally useless now. We NEED an alternative.

I've had a few experiences recently where I was searching for a webpage that I knew existed (because I'd been to it previously), but that was fairly niche and couldn't find it through Google even after trying several different sets of search terms and even fragments of text I remembered from the site.

That's actually a great step forward.

It needs some user-experience upgrades - currently its default is to not omit anything, should be a default in settings. It is also trying to re-write my search as I type and often causing spelling errors. But yes I agree this is what I am looking for.

Hm, what about searching Amazon? Or Bing or Duck Duck Go? Do they have the same problems?

Not sure if I am misunderstanding the problem, but are you aware you can exclude terms from the search results? For example with google "baseball cards -ebay" will omit pages where ebay is mentioned.

A content-based web doesn't solve discovering content.

I have a similar problem with SEO for recipes. There are tons of recipes SEO'd to a certain site that has a registration/paywall that frequently appear at the top of search results. Luckily they are relatively easy to avoid in the search results.

We have installed a few blockers in chrome to help with the mess, but no luck for my phone's browser which she uses quite a bit too.

It's just sad that all content for a given domain of interest is quickly aggregating on just a few domain names.

Also as a non-facebook user I am so saddened by the user experience facebook gives to people that are trying to look at a Restaurant menu where the idiotic restaurant decided to put their site up on Facebook. No employee of facebook or 66% of the users are going to complain about entering a Facebook CAPTCHA every damn-f-ing time because they are all logged in already.

I worked on Secure Scuttlebutt and then founded Beaker. Happy to answer questions.

1. Is there still going to be room for entrepreneurship on this new Web?

2. To what extent will the lack of ability to monopolise, hurt the ability to create large amounts of revenues and profits?

3. Does this community also know about projects like OpenMined which seek to democratise access to data & infra to run ML algorithms on? Any pattern like reason you can think of, for it to have not come up in André’s post?

1. Yes. Economic models are challenging, especially if ads don't work well anymore. (One thing I'd point out is that most ad revenue is sucked up by Google and Facebook, and they're the entities most ripe for disruption.)

A core motivation for Beaker is to explore new political systems which control information, software, and Web communities. If you view the existing Web as a political system, you might say that clients only have the "rights" to consume and filter information, whereas servers have the "rights" to publish, moderate, and set the software. Clients have to borrow publishing rights from the servers, and that means that all authority is derived from the citizenry of servers. A peer-to-peer Web unifies publishing and consumption rights under one class of device (the "peer") and so it equally distributes the authority over moderation and software among the peers as well.

I bring this up because it alters the service-driven packaging model for the Web. Rather than having software be delivered in a one-size-fits-all package from a given .com, the peer-to-peer Web is a customizable network of packages. There's no builtin authority over the composition of software. All users can customize and share customizations. Therefore no two devices have to share the same application code; they just need to share data formats and protocols.

I'd keep an eye out for how software is packaged for non-hacker use in the p2p Web, because that's probably one place for economic models can emerge. It makes a lot of sense for indie programmers to think of themselves as content creators, and to then cultivate patreons around their modules and packages. For larger projects, we run an Open Collective [1] and have found some initial success there.

The reality is, without a mechanism of user/data capture, it is very hard to create huge profits, and I'm not sure that's a bad thing. I'd rather trade a situation where 1% of the population gets stinking rich if it meant we get the general wealth of FOSS and open data. I think enough incentives will remain to do good new work.

1. https://opencollective.com/beaker

Ad based revenue models are based on selling user data. I think in the p2p world these services would start charging money for the actual service instead. For example, one would pay for access to the world's best search engine that has indexed all there is to index in the public parts of the p2p world; as opposed to accessing it for "free" (in which case the user is actually what is being "sold", to the advertisers)

2. See my reply to 1!

3. I'm not personally familiar with OpenMined but decentralization is a pretty active space. SSB and Dat/Beaker have a few things in common: overlap in the communities, a focus on p2p hypermedia protocols, and not a lot of love for cryptocurrency solutions. That last point may change over time, but currently the usability, performance, and waste of the coins has left us all saying, "can't we do this without them?" And we think the answer is yes.

(That's said, I respect other people trying the coins. Everybody has to place their bet somewhere.)

Any system with a fungible unit of account leads to plutocracy. Discuss :)

Thank you for the insights!

I don't have a question, just praise. It'll be interesting to see how dat addresses could be persisted or addressable; content searched, etc. Nice work on Beaker. :)

How might search engines work in a DAT enabled web?

(beaker's neat... i'll check it out)

I can't answer for pfraze, but I've thought about this a lot.

For one, even right now we badly need better client-side ways of caching, searching and indexing content. I should be able to easily cache, tag and search a wikipedia article, for example, without ever touching a remote server. On the decentralized web, these tools are critical.

As far as search goes, it'd be interesting to find a way of letting peers run anonymous queries on each others' local indices. Maybe you'd still only want trusted peers (e.g. friends of friends) to be able to query your index. Or maybe you'd want to have some privacy settings for who can search what content. Maybe this is a job for zero knowledge proofs? IDK. It's a hard problem for me to wrap my head around. How to balance privacy with fluid information retrieval.

This kind of p2p search system could be a powerful replacement of centralized search, especially in the sense that it gives users the opportunity to index content with humanly meaningful metadata.

Remember back in the day when people had web indexes? You'd visit someone's www page and they'd have a list of cool links? Think about a next level version of that, where peoples' curated indexes are stacked with meaningful metadata and well integrated with the browser.

I could also imagine interesting peer-graph indexing algorithms that worked using gather-apply-scatter kind of techniques.

macawfish is suggested distributed search indexes, which is a neat idea that YaCy [1] explores.

We're looking at adding a private search index in the browser itself. This index would be constructed by adding dat sites to your 'library,' and directing it to crawl the outbound links.

What's unique about this is, other than the private queries, we can establish trust relationships with all the data in your results. This is particularly useful for searching for users or software, since you want to trust those results. Dat is a cryptographic network - all sites, users, and apps are represented by public keys, and all data is signed - so your search results are a search against your personal Web of Trust.

When you search for "Paul Frazee," if you had previously added 'dat://beakerbrowser.com' to your library, then you'd see a result for "Paul Frazee (via beakerbrowser.com)". We can layer those signals too by matching URLs, so your result might be "Paul Frazee (via beakerbrowser.com, Alice, Bob)." Dat uses secure ledgers internally [2] so this is really a pretty decent form of key distribution, but of course it can be generalized to searching for all kinds of information. We just have to find out the limits of disk space usage.

I have more to say, but I'll add it with a reply to remain timely.

1. https://yacy.net/en/index.html

2. https://beakerbrowser.com/2017/06/19/cryptographically-secur...

Another interesting element about the local index is that it might be usable for discovery. Essentially there's a local cache of the Web in the user's browser. Much of that local cache has semi-structured data thanks to JSON, and we've begun to take advantage of that with a secondary indexer called WebDB [1]. It's feasible to imagine an API where apps ask the browser to search for files or Dat archives which match a certain spec. For instance, "Give me all user profiles which match the following JSON schema." Since the local index is curated by the user, this will naturally hit results that have some connection to the user.

The concept is of the browser as a user agent, crawling the Web from the user's input set to find relevant data, rather than an objective "map and rank the world objectively" engine like Google does.

Other than this local index and distributed indexes like YaCy (which are very neat but a lot of work) there's also the traditional hosted indexes. My hope is that people will be able to self-host their own indexes privately, as an extension of their subjective local index (but with a larger graph searched). However, there's still room for traditional search engines to crawl dat sites as well.

1. https://github.com/beakerbrowser/webdb

Please find a way to change the Google search option in the right click menu into something configurable.

Amazing tool, albeit slow on my machine (old Vista era 4GB ram laptop)

What makes people use centralized services is their utility. People will pick up new tools and repurpose them if they find them useful. In theory the web is supposed to be about communication and connection.

I think the emphasis on building a local network is a good idea. P2P mesh is cool but until it provides opportunities that don't exist otherwise it is unlikely to surpass.

The notion of building this for places without internet access is a positive angle but also tricky. Charity is seldom as successful or scalable as user driven initiatives. A lot of mobile phones now exist in places w/o "internet" per se. But from my understanding say converting a 3 year old smartphone into a mesh 1st device seems challenging from a wifi driver and power consumption and app perspective.

Balancing the project of design that is easy for non-technical people with the notion of eating your own dogfood one can theorize about building alternatives to the hierarchical Internet. This is the challenge though, figuring out how to build utility that is superior to the walled gardens and is in the hands of the users to control.

> The notion of building this for places without internet access is a positive angle but also tricky. Charity is seldom as successful or scalable as user driven initiatives.

Scuttlebutt's founder lives on a boat. There's no need for charity. We're all “us”.

> converting a 3 year old smartphone into a mesh 1st device seems challenging from a wifi driver and power consumption and app perspective.

Yeah, actually engineering the bloody thing is always the tricky bit.

> This is the challenge though, figuring out how to build utility that is superior to the walled gardens and is in the hands of the users to control.

Yeah, actually designing the bloody thing is always the tricky bit.

I like the phrase "local-first" software.

It's been happening for a while:


For us "old" people who remember the internet before the web -- one of the things that's really different about the modern internet is the very limited set of protocols and applications that the average user interacts with. It really used to be that every different service type mapped to a different protocol and HTTP (and HTTPS) has just sort of subsumed everything. Back in the old days to minimally use the internet you'd have to know at least telnet, ftp, nntp, gopher (maybe), smtp, pop and maybe a handful of others.

(okay, maybe modern users use more protocols than I'm admitting to, but it's very obscured these days but so many different applications just ride on HTTP(S) anyways).

There's really nothing preventing some motivated group to just spin up an entirely new kind of service that "fixes" all that's wrong with the web, custom protocol and application stack.

"But the network effect!"

And that's something us old timers remember, we remember lots of great services spinning up and down and even when the web was just a handful of sites. The web earned its network by having better general utility than other things that were attempted, but why can't a new better service eventually earn it?

(In the meanwhile us hacker types will enjoy having a cool new playground to muck around on for a few years).

> There's really nothing preventing some motivated group to just spin up an entirely new kind of service that "fixes" all that's wrong with the web, custom protocol and application stack.

Firewalls and security policy. The internet is a much less trusting place these days[1] and anything too new will have adoption issues once it gets out of a core hacker group.

1. Remember IRIX shopping with telnet and a guest/guest login enabled by default to “foster the spirit of collaboration”?

scuttlebutt is my main social media these days... oh and in case you're looking for git that's not wedded to github via your comments and issues, scuttlebutt plays really well with git - you push comments and code into your gossip-cloud together

plays really well with git

yes, but there are still some limitations that i've run up against (which i am trying to fix) when it comes to repos with a lot of history.

you can work around this issue by using a lot of small repos, but once you reach a certain threshold it becomes impossible to clone a repo from ssb[0]. going further, you also cannot push repos beyond a certain size[1].

these are sample repos that these claims are true for, but i still have more measuring to do so i can say what the exact threshold is.

i lean very heavily on the tech that i use to find the places that need improvement. that said, ssb is an amazing technology and quite new. it works very well for the social networking use case. with some work, we can make it fast/reliable enough for general applications.

[0]: https://git.heropunch.io/%25SI144KecoPoa4uB8kGGUbC%2Fb6Mb4eP...

[1]: https://git.alpinelinux.org/cgit/aports/

likewise, scuttlebutt is my primary digital social resource, and the #solarpunk community generally is incredibly inspiring to me.

What is it?

It's a community of people trying to build better tech to communicate with each other.


If I were to write an internet health guide, it would start with:


Then the guide would say that it's alright to want to be online, because connection through the internet is good-as-hell. I'd only ask for one thing from the reader: to stop waiting for these companies to get better. Instead, learn enough technical skills so that you're no longer reliant on them.

“You already have all the tools you need, so let's sharpen them!,” the guide would say.



Secure scuttlebutt is a totally decentralized append-only "blockchain" with a single writer, that is replicated by various peers who want to get the latest posts. Their mantra is "no global singletons".


Scuttlebutt is very good, but it would be cool to generalize it to the multiple-writers situation. Then we could have one standard for decentralized data streams and even currency tokens, without global networks:


If you asked about the GitHub alternative, it's this: https://git.scuttlebot.io/%25RPKzL382v2fAia5HuDNHD5kkFdlP7bG...

The bit about being permissionless is kind of unclear.

"This seems to work well: the SSB network thrives off of being a group of kind, respectful folks who don't push to each other's master branch. :)"

Will this hold if the community expands to 10x or 100x the size? Surely conflicts of interest will arise, or just plain assholery/trolling.

> Smartphone manufacturers sell mesh-first mobile devices for the developing world

I hope puri.sm is listening. This is what I want my Librem 5 to be!

Hi, yes, we are listening :)

First of all the Librem5 is as open as possible, i.e. if there is a way to support this or another kind of mesh networking then you will be able to add this function to the Librem5 - as long as the software needed for this is free and works on Linux on ARM (ARM64 to be more precise). There will be no restrictions from our side, none at all.

But I also have to point out that we are bound to what we find on the market, concerning hardware and drivers. The latter can be modified to support what is needed for meshes but hardware usually is hard to change. We are currently looking for as good as possible free software supported WiFi and BT chips that do not require runtime firmware and are power saving enough for a mobile device. This turns out not to be very easy. Once we settle on some type we will check thoroughly for mesh requirements.

So yes, your voice(s) are heard! And we will do our best to support mesh networking, in soft- and hardware on the Librem5.

Cheers nicole

I would, without question, purchase this their device if it had mesh-first radios on-board.

Unless I missed something, this article lacks a call to action (or multiple calls to action for different people). What can individual readers of this article do to help realize this plan? For example, if I have money, where can I give it?

If he figures out how to make a good mesh network and bootstrap it then everything else is easy. That blogpost was pretty much a long winded way of saying "we don't have a mesh network that works."

Does anyone here have any experience with MANETs? I really like the idea, but is there any way to defend against malicious actors? Are there any protocols that are clear winners? Is the reason it hasn't caught on solely because the big ISPs are against the idea?

In general, if you allow anyone to be a router then anyone can advertise a low-cost route to anywhere, and then drop the packets. You might be able to combat that with some kind of decentralizes reputation system or policies that prefer existing routes that have been in use for awhile and are known to work, or that sends a few test packets over any new "suspicious" link before using it send real data.

I don't know what current research says about this problem, or whether any current routing protocols attempt to deal with it at all.

In practice, I expect that real wireless meshes can still be somewhat secure by operating as managed clusters of routers that don't peer with other clusters of routers without the administrator explicitly enabling peering. So for instance, you might have one neighborhood with a dozen houses with routers managed by some guy, and a nearby HOA with another cluster of routers managed by someone else, and they might find out about each others' networks and agree to create a bigger network by peering. So, your network's trust model can piggyback on the trust model of the real world -- if you know someone in real life, it's a lot less likely they're going to try to take down your network than some random person you don't know.

> Is the reason it hasn't caught on solely because the big ISPs are against the idea?

While I like the idea, it seems like it would be hard to get the same consistent QOS as you do with a network built using fixed infrastructure.

The IPv4 perspective is a red herring. NATting was indeed necessitated by IP address scarcity, but a domestic installation that does NAT comes with ancillary benefits, like giving you, the home user, a single place to control access to your network.

In IPv6 it's nice that you have an address space that's not only big enough to accommodate every device, but large enough to even burn through addresses and treat them as disposable, but once IPv6 becomes widespread there will need to be some rethinking as to how to manage firewall rules between your own devices, how to segregate your portion of the network from the spurious (and sometimes malicious) traffic of everywhere else.

firewalling on a router/gateway gives you the same ability, and none of the downsides of NAT.

Nat is in no way a requirement for this.

Not to mention that you really need to focus on securing the endpoint anyway because an increasing number of your devices are mobile and aren't "home" an increasing amount of time anyway. The period of central router/gateway/NAT management made the most sense was when the majority of your home were desktops fixed in their locations in the same rooms. In a world of a majority of laptops and hand computers, worrying about the strength of your home router/gateway/NAT is increasingly silly, as those devices may spend as much time or more at your work, or the coffee shop down the street, as they do at "home".

People were sounding the same alarm in the '90s with AOL. History has shown us that, at some point, a leaner, more innovative company will surpass Facebook.

> History has shown us that, at some point, a leaner, more innovative company will surpass Facebook.

Facebook has learned the same lesson, and is explicitly trying to keep it from ever happening again.

See https://www.nytimes.com/2017/10/18/technology/frightful-five...

Well they were kinda right. AOL was merely an early iteration of what we have now. Maybe another company will surpass FB but I am pretty sure from iteration to iteration it will be harder and harder to replace an incumbent since each round's winner will be more powerful than the previous one.

AOL, for instance, was mostly an American thing. FB, otoh, has global power.

Yes, the history of the early, open web, which now is in danger of being... history.

Who says it'll be a company?

less design by commitee

If you really want to rescue the web find a way to restrict adware, spyware, and the like. I upgraded internet at my house to 1gbps (about 890mbps down and 920mbps up) and I hardly notice any speed difference surfing the web. Sad. Everything else is fast as hell though.

The good news is we have tools (browser plugins) to help us take control of our own machines while web browsing. The bad news is that sites are actively hostile to these tools. I think it's as much an incentives problem as a technological problem.

I have this too. Sadly I think it's more likely due to the fact that some of the largest websites have a speed limit per connection.

Now we just need to save the web from slow JS apps. Is there something doing that right now by chance?

You, by clicking on the "reader mode" of your browser and telling the owner of the site about it

I thought that was what the web archive and similar projects were doing...

If I wanted to rescue something from the Internet, it would not be the web

Would you kindly stop posting unsubstantive comments to HN?

Would it be the children?

Personally, I'm on team "kill it with fire". I wonder if there is a group for that?

Applications are open for YC Winter 2022

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact