Making the Internet the Web Again

[mcwhittemore]

Before it was called the internet[1], it was called the web, the World Wide Web. That name wasn’t just a label; it was a bold proclamation of intent. Unlike the desktop platforms that came before it, this new platform wasn’t isolated software. It was connections. The web let users explore and discover content made by many producers. Discovery was the core experience. The interconnected nature of the platform was so central, it became its very name.

But over time, we stopped “browsing the web” and started “using the internet.” In that shift, we lost something fundamental.

The transitions that drove to this change made sense at first. Who can “crawl the web” better than a human? A computer. So we built search engines: Yahoo, AltaVista, Google. What can discover appealing content more efficiently than clicking links and opening tab after tab? A recommendation algorithm. So we built feeds: Facebook, Instagram, TikTok.

But these sites, once tools for exploration, grew dependent on ad revenue, twisting their incentives toward a single goal: maximizing the time you spend looking at ads. What started as a better way to explore the vast content of the web became a system for keeping your eyes locked on a small subset of its content—ads.

Where does that leave those of us who want to explore? Who crave ideas that challenge our assumptions? Personally, I find myself stuck on the same few sites, longing for the original promise of the web while recognizing that version of it is dead.

But I also find myself wondering: Can we bring it back? Not to destroy the ad-driven aggregators (a lot of people seem to like them), but to build something for the rest of us. Something that makes exploring the internet feel more like surfing than doomscrolling.

If this resonates with you, let’s talk. I’ve started to ideate and build prototypes toward this goal and would love to collaborate with fellow explorers. What tools would help you explore the web more? What tools would improve the quality of your time exploring? Drop a comment below. I’d love to hear your thoughts.

[1] Okay, yes, before it was the web, it was the internet. And yes, the internet and the web are not the same thing. All valid points, but not the point. I’m talking about how the average person thinks about going online. First, it was "the web." Then, over time, the webbiness of the web, was lost.

[simpixelated]

The YouTube channel Technology Connections recently made a video [1] about this and used the term "Algorithmic Complacency" to encompass how many people would rather be fed content determined by their algo of choice than seek it out.

But for those who do want a return to the "Web", I think there are plenty of existing tools for this. RSS feeds allow you to follow independent blogs and websites. BlueSky and Mastadon can be used to curate a list of like minded people who will suggest other cool people or content to check out. YouTube has a Subscriptions tab that is not influenced by the algorithm.

That being said, I applaud anyone who wants to make the independent web more accessible and popular. We need alternatives to the infinite scroll fed to us by massive tech companies.

[1] https://youtu.be/QEJpZjg8GuA

[danmactough → simpixelated]

Funny you should mention both RSS and BlueSky/Mastodon as a return to the web, since Dave Winer (who pioneered RSS) has written about how BlueSky/Mastodon etc. are very much NOT the web. http://scripting.com/2024/12/21.html

> Threads, Bluesky and Mastodon are the IBM, Microsoft and Apple of 2024. It's ridiculous if they think this is a web.

[simpixelated → danmactough]

Makes a lot of sense and I don't disagree. They're definitely not a return to the web and I probably misrepresented them that way in my post. I was mostly using them as examples of existing platforms where you can find content without being subjected to the algorithm's suggestions, if you choose to use the "following" or "subscribed" features instead of "Discover" or "For You".

[mcwhittemore → simpixelated]

Thinking about why I've tended to avoid RSS, there is a consumption vs exploration divide in my mind that I feel RSS readers lean the wrong way on. That is probs an assumption I should challenge in my own thinking.

[simpixelated → mcwhittemore]

At some point you have to stop exploring and actually consume what you've collected. I think that's one psychological trick of infinite scroll feeds... our minds naturally want to see what's next. Personally, I spend more time searching and saving things to my various lists instead of actually watching or reading what I've saved.

[mcwhittemore → simpixelated]

How do you find new RSS feeds today? RSS readers is something I've never really loved. But I also haven't tried them in a while. I should probs go look at some RSS feed readers to try

[JohnFen → mcwhittemore]

You should. An RSS reader has become my primary window into the web these days. It really improves the experience.

[JohnFen]

> Can we bring it back? Not to destroy the ad-driven aggregators (a lot of people seem to like them), but to build something for the rest of us.

I don't think that's really possible, if what you mean is an open web like it used to be. It's too far gone, too many people don't remember or never experienced what it was in its heyday, and I doubt that the attitude that everything must be monetized is going away anytime soon.

Now, a "good web" protected behind login walls may be possible, but that's no longer the open web.

[mcwhittemore → JohnFen]

I'm not really convinced its gone. Its small. It was small 20+ years ago too though. I think the problem now is its hard to find the web where before that was all that was.

[JohnFen → mcwhittemore]

Oh, it's not gone, but as you say it's small and getting smaller. About half of my regular haunts (the ones that still exist, anyway) have withdrawn from the public web in the last couple of years and gone behind login walls in order to protect against crawlers. That's why I suspect this is the direction things are going to go.

[mcwhittemore → JohnFen]

Care to share the examples?

[mapoulos]

We've been in a world for 10-15 years where recommendation systems needed tons of data and tons of compute. That drove things towards a) large user bases and b) largely free-to-use, ad-supported models (it was only very expensive to produce audio and video where subscription based models worked, a la Spotify and Netflix).

The resurgence of long-form text (Substack) definitely seems to rhyme with the earlier days of blogs and forums. It does seem like people still like to write things and put them on the web. What's interesting is whether the technological paradigm has changed enough to where you could get interesting get discovery without getting mired in the slop of social media.

[mcwhittemore → mapoulos]

There are some interesting players taking another attempt at helping people explore the web by being aggregators - kagi.com for example. And I hope they make cool things. At the same time, I feel like we've walked that path a few times now and it hasn't made too different of an outcome. I wonder if there are laws in averages that mean it will become the same thing no matter intent. Even if that is wrong, I do worry we are stuck trying to make a faster horse.

[anenefan]

I know the feeling well.

I spent a few years on and off few days at a time as a distraction from the mid 00's in a design phase of a alternative area that embraced the notion of world wide web, web for all. Even way back then I was fed up with the endless near pointless search results offered up more and more. The idea in its simplest form was a limited tool that would help a website correctly tag each content page, with a value representing 200 or more definable characteristics each page could fall into - a global index could then faithfully use the values to better align search results. Unlike other SEO ideas, the tags were part of centralised service and very small token amount paid to validate each page. Also one of the things I considered important was after a given amount of time, each content page would be migrated from dynamic to static values for long term preservation - deletion was only something done as a last resort and special circumstances and thus aimed to see off content farms that would post, aim for attention and then are done with old stuff after a couple of months. (But times have changed, it's been a long period since running into a content farm.)

It was actually not even part way planned, realised it was better as a separate area to the regular web and ... at some point I realised that most people would not like having to take extra steps and ... competition for a good search engine is very low, the money is in providing a higher ratio of spam to useful links, more clicks equals more money, and web site owners would probably want google or other large well used search engine company to be driving their ad revenue.

Presently there may lie an answer with the use of a LLM based search helper area, even if it just has access to google's garbage, can run down the first few hundred or more results grouping said results into useful summaries and returning the handful with links that I would call truly WWW. Eventually it might inspire web designers to code more robustly or stop using external scripts that are too intrusive..

ie. You have searched for x y z ...

There are 8 results where the website has expired - ignored

There are 329 results where the website admin has not included a if no javascript option - ignored.

There are 45 results where the website will not render as the admin has included an element that is specific to certain software - ignored

There are 83 results where the website admin has not included anything more relevant than a large about content page - ignored

There are 57 results where the website admin has included SEO but the site content is mostly mismatched or is no more functional than a simple banner - ignored

There are 112 results where the website admin blocked older browsers as they might break their fragile site - ignored

There are 94 results where the website admin has set geolocation - ignored

There are 629 results where site's external scripts were found to scrape or aimed to scrape personal information not relevant to the site - ignored

Top 23 results: ...

[mcwhittemore → anenefan]

An open index of the web is an interesting idea that I've heard come up a few times now. Putting ever page into a one of 200 categories would be a pretty interesting undertaking. You'd probs need to fund it by "grants" or something from AI companies though...

One thing I feel is getting clear for me is that we often think about this problem from a "how to solve it to make it stay open" POV, but what the user ex and thus the product you are making is unclear. If you made this index, the product would be the data and so you'd be push to selling there. What is the product an end user like ourselves would pay for that helps? RSS feed readers seem to be the genearl suggestion so far of what we have today.

[anenefan → mcwhittemore]

I have given up thinking about it much these days -- I guess I'm just pretty jaded with the direction the web went and most people are happy with it. There are probably better and more practical solutions than what I had in mind - besides it had got quite complicated though I may have just got myself caught up in needless complexity.

Still I should clarify, 200 or more definable characteristics, wasn't aimed all that much at the topic ie not purely a tool that could categorise topics like a library card index.

I was more concerned with outing the rubbish within the web and thus enable quick searches and more productive for the user, as well as saving hours of a desktop or laptop being on while the user conducted an endless search while they dealt with the many fruitless and pointless results.

... and on second though removed the really long reply that roughly outlined it. I will say characteristics are things like if the site is a banner (contact, about, low signal) ... if it's a mirror / pointer type site ... if the site is just a front end to some other place ... geolocation ... dynamic vs static information ... forum / blog / etc ... a new one comes to mind present day, if the site is behind a site protection service (some are awful - I'm so tired of seeing ray id just because I prefer to use an older browser) ... the site hosts any files on offer, code or text based, rather than rely entirely on some other site, pointing the user there or some cloud source which is not theirs ... grouping of what sort of data is in site, spam, x rated, political, safe for work and around kids etc

Again I've given up on it and perhaps for the best, I figure it won't be long there will be a LLM based service which will filter the web for users however they'd like results fine tuned to make the web useful to them again, not the mere shallow depth that some of the better search switch operators could do even in the golden times of search in the 00s, but nearly at the level of determining the characteristics of the site and what the user wants to avoid. ie not a mere LLM summing up the site and information. What a dream it would be to list results for nearest physical store to blah suburb that sells / has workshop / operators manual for recent Baz 1 ton excavator ... and not waste time sifting though pages of results until the search engine won't display any more - and in the end resorting to an old phone book to search for stores directly.