I've said this a bunch of times before, but I don't see this take in a lot of Usenet postmortems, and I think it's truer than most of what people say about Usenet.
Piracy killed Usenet.
I ran a Freenix-competitive NNTP installation for a very popular ISP in Chicago in the 1990s. Usenet was by far the most demanding infrastructure we ran: we ran striped RAID arrays, read replicas†, transit servers --- all of this on the only expensive Sun hardware we had in our fleet. At one point, we independently reinvented the INN history cache, and there were a couple sites near the top of the Freenix leaderboard in part due to IRC conversations about our dumb patch. All this is just to say I feel like I have some bona fides here.
Keeping up with the Usenet everybody laments today was not this demanding. You could easily have done it with a Pentium FreeBSD server and a fast disk. But customers wouldn't let you. The open secret about Usenet was that it was a binary distribution system first and a discussion forum second.
Unlike discussion groups, keeping up with full-feed binary groups was a fucking nightmare. Usenet is the stupidest mechanism for distributing binaries you can possibly imagine: just imagine BitTorrent, but entirely Base64'd, and without the forward error correction --- your file was in 100 pieces, and if you lost one of them, you were just fucked. If users couldn't reliably download pirated games or porn from your Usenet servers, they'd loudly complain and tank your reviews for running a bad Usenet setup. And woe betide any ISP that claimed to do NNTP on anything less than a binary full feed.
The rest of the story is obvious. ISP operators had better things to do with their time and money than cater to the small minority of angry cost-center users who used Usenet this way, and so they outsourced Usenet altogether. Usenet centralized because binary groups forced it to, and once it was centralized there was no reason not to just stick it on a website.
Web forums are better than Usenet was back in the day, so arguably, over the long term, not much was really lost. Reddit today is very much like Usenet was like in the mid-1990s, and they at least have a reliable archive. Apart from the archives of my old comp.security.unix posts, all I really miss about Usenet was the competitive systems engineering aspect of it. I'd love to go back in time and take another whack at Usenet knowing some of what I know now; I didn't realize how fun a problem it was at the time: a real system people actually cared about with a leaderboard.
† These things sound very boring now but were most definitely not boring in 1995.
Some providers (namely ISPs) simply dropped the binary groups and kept the text ones. So piracy alone is clearly not what killed it. In fact, Usenet really only exists today for piracy, so if anything, piracy actually saved Usenet. Just for a different use case than originally intended.
I mentioned this in my comment. As a technical matter, you could simply not take binaries. But as a commercial matter, you could not; if you didn't have binary groups, you weren't offering real Usenet (and if your binary feeds weren't ultra-reliable, you were offering shitty Usenet). The result was a consolidation of the platform down to a small number of providers willing to invest large amounts of resources to placate binaries users.
If lots of people aren't running NNTP servers, there's not much reason to use Usenet. By the time Reddit and Digg rolled in, it was no contest.
> But as a commercial matter, you could not; if you didn't have binary groups, you weren't offering real Usenet (and if your binary feeds weren't ultra-reliable, you were offering shitty Usenet).
None of the ISPs I had from the mid '90s through 2010 ever had reliable binary news feeds. Most would be missing too many articles, or articles would expire within hours and the download could never be reassembled.
Anyone who really wanted binaries started using paid services and just use their ISPs usenet feed for text discussion.
This is exactly my point. It's why Usenet consolidated and then died: because smaller firms lost the ability to provide the whole service, which was overwhelmingly abused as a file sharing network.
Unlike discussion groups, keeping up with full-feed binary groups was a fucking nightmare.
The nice thing about NNTP was you didn't have to carry any group you didn't want to. We stopped carrying the binary hierarchy and some of the dedicated pr0n groups because of just this sort of administrative pain.
Some providers simply dropped the binary groups and kept the text ones. So piracy alone is clearly not what killed it. In fact, Usenet really only exists today for piracy, so if anything, piracy actually saved Usenet. Just for a different use case than originally intended.
Piracy killed Usenet.
I ran a Freenix-competitive NNTP installation for a very popular ISP in Chicago in the 1990s. Usenet was by far the most demanding infrastructure we ran: we ran striped RAID arrays, read replicas†, transit servers --- all of this on the only expensive Sun hardware we had in our fleet. At one point, we independently reinvented the INN history cache, and there were a couple sites near the top of the Freenix leaderboard in part due to IRC conversations about our dumb patch. All this is just to say I feel like I have some bona fides here.
Keeping up with the Usenet everybody laments today was not this demanding. You could easily have done it with a Pentium FreeBSD server and a fast disk. But customers wouldn't let you. The open secret about Usenet was that it was a binary distribution system first and a discussion forum second.
Unlike discussion groups, keeping up with full-feed binary groups was a fucking nightmare. Usenet is the stupidest mechanism for distributing binaries you can possibly imagine: just imagine BitTorrent, but entirely Base64'd, and without the forward error correction --- your file was in 100 pieces, and if you lost one of them, you were just fucked. If users couldn't reliably download pirated games or porn from your Usenet servers, they'd loudly complain and tank your reviews for running a bad Usenet setup. And woe betide any ISP that claimed to do NNTP on anything less than a binary full feed.
The rest of the story is obvious. ISP operators had better things to do with their time and money than cater to the small minority of angry cost-center users who used Usenet this way, and so they outsourced Usenet altogether. Usenet centralized because binary groups forced it to, and once it was centralized there was no reason not to just stick it on a website.
Web forums are better than Usenet was back in the day, so arguably, over the long term, not much was really lost. Reddit today is very much like Usenet was like in the mid-1990s, and they at least have a reliable archive. Apart from the archives of my old comp.security.unix posts, all I really miss about Usenet was the competitive systems engineering aspect of it. I'd love to go back in time and take another whack at Usenet knowing some of what I know now; I didn't realize how fun a problem it was at the time: a real system people actually cared about with a leaderboard.
† These things sound very boring now but were most definitely not boring in 1995.