One web page for every book ever published. It's a lofty but achievable goal.
To build Open Library, we need hundreds of millions of book records, a wiki interface, and lots of people who are willing to contribute their time and effort to building the site.
To date, we have gathered over 20 million records from a variety of large catalogs as well as single contributions, with more on the way.
Open Library is an open project: the software is open, the data are open, the documentation is open, and we welcome your contribution. Whether you fix a typo, add a book, or write a widget--it's all welcome. We have a small team of fantastic programmers who have accomplished a lot, but we can't do it alone!
They also seem to provide an API.
For an invite, please send me an email at email@example.com or go to:
# APIs & Data Dumps
- https://openlibrary.org/developers/dumps monthly data dumps for if you need bulk access and the APIs are not enough.
# Spread the word
Also, if you want to help raise awareness of this resource, please help us get the word out on twitter!
Thank you all for helping us discover some issues with our goodreads importer and search (recently migrated to Python3 + thanks @cdrini et al for these fast bug fixes! If you notice an problem, please help open an issue here:
# Learn More
- https://archive.org/details/openlibrary-tour-2020/openlibrar... if you want to learn more about Open Library, here's a short intro vid.
- https://github.com/internetarchive/openlibrary if you want to follow on github.
How do I go about claiming my author page? The current book listed there has been officially "retired" for over a decade now, and I have plenty of other books that I'd be happy to add.
As a project of the non-profit Internet Archive, having a trusted catalog is pretty paramount to what we're trying to accomplish and Aaron Swartz's original dream for an Open Library.
I've created an issue for this feature (as it's something we've discussed doing for a while). I'll also be adding this to our upcoming Tuesday 2021 roadmap discussion.
Thanks for raising this
Many of our librarian processes & FAQs are detailed here: https://openlibrary.org/librarians
Most data on Open Library is publicly editable by members. In this case, author merging is a capability that only folks who have been added to our Librarians usergroup can access because the process of reverting an accidental merge is quite time consuming (and so we have some training in place).
If you tweet @ our lead community librarian (http://twitter.com/seabelis) she can likely help you make this fix! Also, if you're interested in helping us make changes like this, we can invite you to our slack channel :)
Would anyone be interested in having an instant search experience for this books dataset, like the one I built for the 2M recipes database posted on HN earlier this week: https://news.ycombinator.com/item?id=25365397
ol_dump_latest.txt.gz 0%[ ] 28.73M 115KB/s eta 21h 54m
 https://www.goodreads.com/review/import (export)
(granted, running on older versions of firefox, but still thought it might be a bug you would like to be aware of.)
If you use the https://openlibrary.org/volunteer page and click the Librarian link, this will send an email to Lisa, our head community librarian and we can help you access our slack channel and request access to our librarian features. We also have an optional weekly call for folks to raise issues and questions (e.g. "I want book series!")
More about librarianship @ Open Library:
If you hit any issues with this process, feel invited to send me an email to firstname.lastname@example.org.
Ideally, it should probably bring you to the login page and (once you login) bring you back to the import page.
We'll see about creating an issue for this, thanks!
Although, I just tried importing my Goodreads export into Open Library and I get the following "Internal Error":
> Sorry. There seems to be a problem with what you were just looking at.
> We've noted the error xxxx-xx-xx/yyyyyy and will look into it as soon as possible. Head for home?
Anyone else facing this issue?
I can confirm it works with at least my import.
Edit: Alas, some errors regarding "Book not in collection" and "No ISBN". But I think I can probably see if there is a way to contribute those books to the repository.
The importer is a new project (as of August this year) and I think there's a lot of opportunity for improving it (including importing missing records on demand).
Other hat tips: I also want to call out the great work of https://inventaire.io, another great project in the space which uses wikidata, as they also have a goodreads import which we're learning from. Another project I'm excited about (which has goodreads import) is beta.thestorygraph.com. Not sure the status of its source code but I heard an interview from the founder and she and her work so far seems great.
Books pre ~1973 don't have ISBN. And goodreads covers far more material than just modern material.
The current Goodreads importer makes a big/unfortunate tradeoff of trying to have something which works for the majority of material on people's goodreads lists.
It doesn't preclude further efforts -- Open Library knows about millions of books that don't have ISBN. So if anyone wants to help us improve our importer (or at least register their interest by creating an issue which calls out what other identifiers should be considered -- e.g. LOC, etc) that would be very helpful.
# To open an issue:
# Here's the code for the importer
Having some free time on my hands I will take a look in the next few days. Looking for an alternative to GR and for something that tickles my interests.
Python, Search and literature. And open source.
This is one of the best implementation decisions that Aaron Swartz, Anand Chitipothu, et al made when starting Open Library.
The site's also been around since 2005, so, that speaks well to both its longevity and its integrity. Of course, that's not to say they couldn't be acquired tomorrow and paywalled, but I think it unlikely.
...And they even have an API!
Made me realize how many booksellers(especially used) put proprietary barcode stickers on top of the 'actual' barcode stickers.
And yeah, searching needs some work! That's on my task list for this month. Just this Friday I spent most of my day working on updating our search engine, Solr, from 3.6 to 8.7 (wip!). But search is a _BIG_ pain point. We're a small team with a big long list of things to do, but we are making progress! This year we updated to Python 3, switched most of our production environments to docker-based for easier deploys and to give open source contributors more control of production infra, added reading history stats for users, added a new interface for exploring books, worked on a novel recommendation system, added text selection to the online BookReader for public domain books, added GoodReads importing, grew our community, added the ability to search by classification, and much, much more (you can see highlights from our year here: https://github.com/internetarchive/openlibrary/issues/3891 ).
There is still _definitely_ a lot to do, but I think the biggest reason worth using/contributing to Open Library is likely its open source community. Anyone can jump in and help make improvements to the system (as they very often do!). Personally, I think it's more likely that a system with a community will survive/flourish than one maintained by a single person (I also wondered whether I should just create my own before contributing to and now working on Open Library!). And there are also loads of different tasks associated with a site like OL, which would be impossible for me to do if I was going it alone.
If you would be interested, checkout the GitHub repo: https://github.com/internetarchive/openlibrary . It's very active, and you can get an idea of how we work :)
How do you even get an ISBN for a candle?
There is an explanation under "About the author" section :
This "author" was created to segregate those items which have ISBNs but are not actually books. For more information, see the manual and/or start a thread in the Librarians Group.
When an item which is not a book is imported via ISBN into Goodreads, it does no good to delete it: the item will only be re-imported as long as it remains on the feeder site. (Often these are book-related items which are assigned ISBNs by book publishers so that they can be tracked through their book systems.)
“Why not? What’s the big deal?” said someone who has never worked with garbage data.
"paying $2-10k+ to add an International Standard Book Accessory Number field to their software will never get approved for a bookmark that works as a compass, or a promotional Harry Potter bookend set."
Tough to disagree.
There's very little control. E.g I just published a novel,and Amazon did not in any way validate that the ISBN I have them actually belonged to me - I had not yet registered the book data, so what I told them would not have matched anything they might have looked up.
In this case, based on the reviews on Amazon, it looks like someone changed the description of an existing product, and that the ISBN probably actually belongs to a book.
ImportBot scrapes Amazon.com, matches product id with ISBN10 (which can be converted into ISBN13 without anything else IIRC), imports product as book.
Yes. There clearly only needs to be one way to identify things, so commonly larger systems just incorporate the smaller ones wholesale. The 13-digit system just incorporates the prior 10-digit system for International Standard Book Numbers with a 978 prefix called "bookland" (most prefixes in this system are geographic, but of course books aren't really from one single geographic region, so they're from "bookland") and adds a new set of possible codes. It also incorporates the entire 12-digit American "UPC" system.
Several other (less well known) systems were gobbled up the same way as "bookland", just allocating them imaginary geographic regions in the 13-digit system.
There's actually a fourteen digit system, but the lead digit tells you about how many of something are bundled e.g. so a distributor can distinguish a truck full of Pepsi cans from just one case or a single can in terms of things you can order. Lead digit 0 means "single" so if you know the 10 digit ISBN you can not only make a 13-digit EAN for that, you can make the 14-digit GTIN that means "just one of this book" which in most cases would be what you want.
Great points here
1. The banner happens last month of the year (Wikipedia being the perfect analog). Yes, there are mixed feelings and it's not the world's best experience :P
2. Our entire data set is available to download as in bulk https://openlibrary.org/developers/dumps because we'd love to see a decentralized p2p version
3. https://github.com/mouse-reeve/bookwyrm Mouse who used to work @ Internet Archive has a decentralized version of Open Library (Bookwyrm) and it's worth checking out.
4. For the last 5 or so years the Internet Archive has been cultivating a dweb/dapp community and integrating with IIIF, Dat, IPFS, gun, bittorent, webtorrents, and others and hosting regular summits and meetups
5. The wayback machine is an interesting case study: it turns out, incentive structures (even things like FIL/filecoin) haven't been able to perfectly crack the nut on getting folks interested enough to preserve the whole wayback machine. There's petabytes of material and there's a powerlaw about what people care about today. Internet Archive realized what we care about today may not be the same as tomorrow, and so there's a cost eaten (the incentive comes from economies of scale generated by intrinsic desire rather than $). And in a way, this centralized solution (economies of scale) IS the solution a community came up with. It has flaws and advantages (tradeoffs), such as centralized points of failure, and I think the archive would be (and has been) ecstatic to explore improving these opportunities.
There's not an API directly, but https://www.howlongtobeatsteam.com looks promising. It's got an API that you can use to cache your (or all) games. Zapier might be able to do it as well w/ this scraper: https://www.npmjs.com/package/howlongtobeat (disclosure, I work at Zapier).
I used to have a CLI that would trigger a Zapier webhook that would use [this CLI app](https://github.com/xavdid/zapier-igdb) to populate data into Airtable, but I actually moved the whole process to [Airtable Apps](https://airtable.com/marketplace) since I could customize the UI a little more. I've been meaning to open source it, but it's pretty customized so I'm not sure if it'd be widely useful.
I'm curious to see where you end up with this though! It's something I've put a lot of thought into.
Time to beat is a very difficult metric. Does a Visual Novel take 10 hours to beat, or 2 minutes? I'd argue the latter, but I know plenty of people would disagree with me.
"Alright," says the webadmin, "I'll just implement a voting system and take the average". But now they'll be claiming that it takes 5 hours and 1 minute to beat, which everyone disagrees with.
The best system I've seen is SteamHunters' "Shortest/Median/Average", which gives you a pretty reliable indicator of what's possible, what's typical, and how much of a difference there is between optimal and casual play, respectively. It's by far the most comprehensive dataset out there, but unfortunately only mono-platform so probably not too useful for sites like igdb.
I haven't shared this yet -- it's more for the community, but I've tried to address various questions from the community and distill answers + resources for Open Library here:
Tried my best to include others players in the space (wikidata, inventaire, bookbrainz, worldcat, bookwyrm) who are doing great work and pay respects to readng, storygraph and other innovative services which are breaking onto the scene.
I get that Open Library doesn't have as much data as GoodReads, but I wish it would show me the data it couldn't import so I could add it manually to Open Library's data store.
Nevertheless, I love the idea and I'll be opening bug reports and maybe code contributions if something looks easy enough.
If you tag @tabshaikh who helped implement the importer and me @mekarpeles we can make sure it gets triaged and tagged correctly this week :)
Dissatisfied with how slow and clunky Goodreads is I actually thought about making my own (albeit much simpler) version of Goodreads to keep track of my reading habits. I often dig through Goodreads to find books or authors I can't remember the names of -- and Goodreads isn't great for that.
Open Library actually provides the missing piece. The fact that they offer bulk downloads also makes it easier to be a good internet citizen and not send tons of API traffic their way.
Looks like I'll have to set up a monthly donation. I'd really like see openlibrary succeed.
The project is also open source, and you can find the code (and contribute!) on GitHub: https://github.com/internetarchive/openlibrary
After some looking, there are
Some private databases with millions of # but no official site.
For books in the collections of large libraries (like the LoC) there will be a public catalogue entry with the ISBN attached, but they don't assign it.
There were also a lot of books published before ISBNs were created, and not every book has an ISBN attached even to this day.
> WorldCat is a union catalog that itemizes the collections of 17,900 libraries in 123 countries and territories that participate in the OCLC global cooperative. It is operated by OCLC, Inc. The subscribing member libraries collectively maintain WorldCat's database, the world's largest bibliographic database.
My primary interest is in recording my thoughts on books and stories I've read, so a review site is what I'm looking for. This is more challenging for short stories as they don't have a ISBN, may appear online, or in a fiction magazine or in an anthology.
So, I may put down my thoughts on short story X that I read in book Y, but if I look up book Z that also features that short story, my thoughts on the story would also appear there.
In short, I'm looking for a site that also records short stories like the ISFDB , but allows users to add reviews.
So far, I haven't found one. I'm now putting down my notes on short stories in a Zotero database.
I would like to understand the true strategic interest behind this. Is Amazon simply penny-pinching now that they’ve successfully obliterated the market for both new and used books online? There’s way more to this story than appears on the surface.
The strategic reason for Amazon is obvious. As someone else mentioned, Amazon doesn't want Goodreads data to be used to add value to their competitors' offerings.
Speaking as a developer who tried to build on top of Goodreads API, I also want to add that this was a long time coming. The API had been neglected for some time. And some of the most interesting datasets weren't even made available through the API.
This is it exactly. Goodreads was/is the best/largest source of information on books available online.
It is useless without API
Buy it so no one else can have it or buy it so you can shut it down.
I guess the writings on the wall...
Welp, time to start a better book catalog site with threaded discussions that eBook page turns can be synced with.
What I think all off these large companies are doing is pulling up the open-web ladder after they've climbed it to dominant positions. The problem with anti-trust action is that it's reactive; we wait until a company has gotten too big, and then hope we can cut it down to size. I'd love to see moves toward proactive open-data and open-algorithm requirements, so that we guarantee a level playing field. That won't be easy, but neither is trying to rein in companies with annual profits in the tens of billions.
I think you're flattening a lot of complexity here. Yes, I agree that Mark Zuckerberg is probably not terribly interested in our privacy. But one of the biggest outrages in FB privacy history, the Cambridge Analytica Scandal, was driven entirely by a researcher inappropriately accessing and sharing user data. While you might not trust Zuckerberg, and maybe you shouldn't, there is definitely some "there there" when it comes to how researchers handle our data too.
Oh, and the old IMDB API you could pay for is gone (or, gone soon), and some of the features it provided are gone permanently.
A bunch of book reviews and book recommendations that can be used separately from Amazon doesn't help Amazon.
It feels like there are a couple very nice small (not necessarily lifestyle, but not vc) businesses in that space.
I'm aware of https://readng.co and https://bingebooks.com
Had they instead chosen not to take VC: they had 35 headcount at the time of acquisition. That load probably requires $7m to $10m/year of revenue, particularly when they have to start doing data deals for all that book data / cover photos. I suspect they couldn't make the business work w/o VC given they ran two years on that $750k angel round. And even if they could, having a near-monopsony relationship with their affiliate fees partner makes them not a real business because they are highly vulnerable to Amazon predation. The same as Mozilla -- when there's just one buyer, that buyer names the price.
edit: put more succinctly: a business that either ran and grew on earned dollars (ie no vc), or a VC-backed business that didn't rely on the company you were attempting to cannibalize to play nice with you.
For example Duck Duck Go. I had thought there was no way DDG could possibly be paid at the normal affiliate pricing structure.
Anyone have insight into this
Regulators should step in when businesses/startups are being harmed by a monopoly via unfair practices such as bundling. But intervening otherwise will simply deny founders and employees a decent exit. And if the economics are not sound enough it'll be harmful for customers as well - the app will get loaded with too many ads or simply shutdown.
So IMHO not a good case for regulation.
Not that I like Amazon, but it is not philanthropy. I assume goodreads take non trivial cost to run operations, moderations and the site and just because Amazon is doing well financially it doesn't mean they have free money. This move makes it easier for Amazon to monetize on goodreads, it is as simple as that.
Effectively the only reason it's not supporting itself is precisely because Amazon bought it.
The web has to mature beyond advertising as a business model. For this to happen people are going to have to open their wallets, pay for the services they use, and support independent businesses. That’s how we build a web where indies can thrive - one that’s more village centre than financial centre. I think the shift is underway.
If I pay I want: No ads, no tracking, full access to my own data in sane export formats, schemas, no data mining, no data selling, no "sharing data with our partners", encryption options, no dumb hoops, no dark patterns, the ability to point a product at an API endpoint of my choosing, backup options that default to my infrastructure first and so on.
Actually let's add more: The data generated by my use of my data in the product. Non-canned support responses that don't ask for information I literally put in the ticket three weeks ago. Prominent indication of where (geographically and legally) data is stored and used. If/how often you do backups. If/how often you practice disaster recovery.
So really what I want to pay for is sanity and no bullshit.
Yet if I do pay, many services and companies will still do all of this shit in the background until midnight the day it's finally made illegal, all the while gaslighting me about "how much they value me as a customer" and how they "respect" our "relationship".
It's literally obscene.
A tracking free format would involve in you downloading the entire newspaper issue, then reading inside in ways of your own choosing.
Because you, a paying customer, are worth the most to the advertisers.
It is also worth noting that there is plenty of advertising in Apple products. Heck, the last time I used one they had at least two digital storefronts built in. They may not be as crass as Microsoft is (e.g. with the use of the Start Menu), but it is data driven. Whether the advertising is plastered everywhere or not simply reflects the target market, rather than how they collect and use data.
Yes, you are promoting Apple products if you carry around a Macbook with its logo, but you aren't being advertized to. The built-in stores might just barely meet the definition, but it is very different from a random ad during a TV show or on your desktop.
This is VERY different to letting third-parties advertise to you.
The have publicly discussed their plans to monetize that data.
That's not what the previous commenter was talking about at all.
Some platforms and services have been selling "no ads" as a feature. Say Apple or YouTube Premium or 100s of 1000s of applications on Android. This does not mean they'll never open up to ads in the future. Maybe a future "premium value" will just be "fewer ads". And when they do, they're more likely to generate sales because they're customers who are already paying for premium services.
The argument says "the more you spend the more you are worth to advertisers, thus eventually you will see ads". It doesn't add, "there are some categories of products for which ads aren't tolerated but for all the others this holds"
There's more than enough examples anyway where cheaper options have ads and more expensive ones don't, like airplanes or restaurants and bars.
Maybe one day these things will be standard. We have to convince the mainstream these are goals worth pursuing... As long as most people accept how shit the status quo is, it won't improve.
FWIW, I also share the GP's wishlist 100%. But we're a niche in our own industry these days. I'm not having hopes the market will deliver - on the contrary, all these points are things for unscrupulous vendors to control to extract more profit.
Like, I know it's a lot of work to jump into a codebase and make a change and that some projects are a pain to work with, etc etc, but still. This is what OSS is supposed to fix, and we're exactly the demographic who is supposed to be able to contribute.
Obviously, the above is even a bigger lift, but at the same time -- if a community of online hackers, many skilled, many experienced, many damn near independently wealthy from decades at FAAMGs, can't build an Internet 3.0 to fix the Internet 2.0 we all got rich building on top of Internet 1.0, who is supposed to?
Many of these things already are standard for EU data subjects.
The reason all of these things happen is because it's easy to slip into them in a tight financial spot and there's usually no instantaneous backlash.
Otherwise, why are there regulators for advertising or for food or government standards for anything?
I'm not talking about censorship or somehow evaluating the quality of content, I'm talking about if a company is delivering a service and I am paying for it, then the conditions under which they deliver that service should be regulated to ensure a fair and competitive marketplace.
Apple shouldn't be getting plaudits for making privacy a unique selling proposition.
All the other companies should be getting told that their business is unfair and exploitative. Privacy should be a right, the control of my personal information should be mine and consent to have it should be able to be withdrawn at any time.
Government requires thousands (literally) of people and political will in order to move any solution forward. In terms of cost and time, it is substantially less efficient than independent entities.
An independent entity could theoretically start immediately, with a small team, and focus explicitly on the stated goals of the team.
> Apple shouldn't be getting plaudits for making privacy a unique selling proposition.
Absolutely agree. The average citizen has no idea how hard they're being exploited.
> Privacy should be a right, the control of my personal information should be mine and consent to have it should be able to be withdrawn at any time.
Also agree, but the government has shown they have an interest in making that not happen. The ties between 3 letter agencies and large tech companies are just the tip of that iceberg.
In the UK the soil association is a private charity which mostly decides which food is "organic" and which food is not and it does a pretty good job. In the US, the USDA decides whether you can call your food organic and from what I understand it's kind of meaningless.
I think a private charity would work better here.
The formation of something new, and independent, means it would also be less susceptible to corruption.
Given that this is one of the primary purposes of governments, what do you suppose has been keeping them from stepping in so far?
Businesses are much faster at inventing ways to abuse people than governments are to regulate them away, even if we ignore lobbying and corruption.
Such a hypothetical organization we seek needs to be both swift and impactful. This automatically suggests some ties to government, as it's the best way to fulfill the criterion of impact. Not sure how to handle the "swift" part, though. Can't do it through regular market mechanisms, because ultimately price trumps all on mass market. The government reacts slowly for (among others) a good reason - while an unscrupulous businessman needs to invent just one weird trick, the government regulates away whole classes of tricks by default, and they need to take care to not outlaw perfectly legit new ideas.
"independent" is doing a lot of work here. Government regulators are, at least in theory, independent because they're paid from taxes collected, and have no incentive . What mechanism is supposed to keep the new 'independent' entity, actually independent?
How it's designed is important, but even under the most elaborate schemes it would be less costly and human resource intensive than having the government directly take charge of it.
It's not that a private entity can't do it - I largely trust the TÜVs and UL and other NRTLs for example. It's that most auditors - especially once you leave the "ship a physical product with verifiable physical properties" sector - instead look like E&Y, which I trust less than most of the companies they have audited.
You eventually need to have someone who will be substantially at risk if the audit is insufficient - "skin in the game" to use the modern idiom - controlling the audit. In the case of UL that's insurances companies (which are highly regulated by... the government), for other NRTLs it's the government directly.
- competitors use revenue from ads, tracking, selling data to their advantage and undercut your value based on price.
- and free users love free stuff. I had a user ask me to include ads in exchange for paid features.
- and yet people still distrust you. Once I proudly posted a new feature to Show HK and the first response was "How does this post not qualify as 'spam'?". Some people automatically think you're a bad guy since you sell something.
- without advertising, how do you get the word out?
I rely on word-of-mouth, since it carries much trust, and news articles. This means it's a slow game.
It is very hard to find mutual respect (both ways) between user and maker. Most relationships start with distrust or "what is in it for me".
I recently started a historical time-series weather data api based on ERA5/GFS and your story really resonated with me. Similarly, I don't come from weather background and initially built it for my work with building energy analysis, with service live but still at an extremely early stage.
Happy holidays. :)
You're going to trust them to have proper backups, proper disaster recovery, proper resiliency and scalability?
What you're describing works for small-time stuff like blogs, personal projects or other inconsequential things, but anything at the scale of Goodreads, where users are trusting years of data to someone just can't be hosted by random people.
I'm not saying I have the answer but "people should self-host this kind of thing on their Internet connections" is not it.
Symmetrical high speed connections allows for the option of self-hosting one's own stuff and being able to serve it to the Internet (or connect to it remotely), but it is not mandatory for people to choose that option. They may not feel confident in their owns skills so choose the option of paying someone else to be responsible for it.
I'm simply talking about having more options.
Nearly every time I contact customer services these days I'm fobbed off with obnoxious PR speak instead of just telling me straight.
GDPR's right to data portability provides much of the export functionality you're after. It must be structured, in a format that is commonly-used and machine-readable. The ICO's guidance suggests that CSV, XML and JSON best meet this requirement.
Tracking is something else that GDPR helps with. Tracking of personal information via e.g. cookies require active consent. Silence is not consent.
"sharing data with our partners" requires a lawful basis when dealing with EU data subjects. This will normally be consent where data is sold to third-parties for e.g. marketing, so data subjects will be able to make an informed decision and opt out of this. Again, silence is not consent - and burying data sharing in an unreadable legal document is not informed consent.
> the ability to point a product at an API endpoint of my choosing
The right to data portability includes this:
> Individuals have the right to ask you to transmit their personal data directly to another controller without hindrance. If it is technically feasible, you should do this.
> Actually let's add more: The data generated by my use of my data in the product.
This is in scope for a Subject Access Request.
> Non-canned support responses that don't ask for information I literally put in the ticket three weeks ago
This is difficult to solve with regulation but I think it's an entirely reasonable thing to expect for your money. GDPR does not help here
Hopefully if there are multiple competitors in the space, customer support is something that providers can compete on.
> Prominent indication of where (geographically and legally) data is stored and used
Privacy information already must contain a transparent list of data processors:
> This includes anyone that processes the personal data on your behalf, as well all other organisations.
What we really need is for other countries to start taking data protection regulation seriously.
Idea: Move all online businesses to a new digital-only currency. Let people earn that currency by donating the processing power/storage/bandwidth of their devices, like the @Home projects. Of course people could always current existing currency to the new e-currency.
Let's say an hour of donating an average laptop = an hour of using Google, Facebook, etc.
Yours is a great list of stipulations. I would just add: support for open + interoperable protocols such as activitypub and RSS.
> Non-canned support responses that don't ask for information I literally put in the ticket three weeks ago.
No, sorry. I don't want to put too fine a point on it, but you get this if you pay what I want, not what you want.
All this crying about tracking - how else the owner of the place can make product better? If I own a store - I can see where people go, how they shop, how they walk around, which basket size they prefer, what they buy. If I don't collect data on my website - I can't THINK about how to make my service better. I can only GUESS. What about data collections for simple functionality - like, when you come back to the half-filled form and I remember the values you already submitted by matching your cookies. Are you against this as well?
Sorry, but if this is the alternative - I would rather have Google know everything I do online and hope that they honestly don't store data on my Incognito browsing. If they do - worse for them.
As for legalities. It is a global world. There is no way to enforce this effectively globally. This is a pipe dream.
Also data in some cases must be shared with partners, those might be payment processor, ID checks etc.
Capitalism is about free market trade. You provide something and people choose whether they want to buy it or use it. People add greed qualifier in there so they can frame it as something illicit going on.
The fact still remains that if people cared about their privacy (and there is no evidence they do), they wouldn't use these sites.
Before it was done online. Store cards used to track purchases and spending habits in store in the same way that sites do today (however at a much greater scale) and customers were given vouchers in return.
In much the same way. Almost all the local stores have dissapeared to be replaced by large corps that can provide everything in super stores and in much the same way that is the fault of the consumer by not supporting their local stores.
> For all benefits we yield from it, it's still fair to call it capitalistic greed and notice the ethical failures the market strongly encourages.
No the failing is on us and the users of the site for using these services when we were warned by many people that this would be the case. Pretending otherwise is passing the buck.
But that does not absolve the producer. They are still using ethically questionable methods.
The market doesn’t get to decide the rights and wrongs. It just allocates resources. That we have to do collectively if we want to call ourselves democratic.
> But that does not absolve the producer. They are still using ethically questionable methods.
Most people don't care. How can it be an ethical issue if the vast majority of people are unconcerned by it?
That food? Bad. Dead. That tool? Dangerous. Dead. That work on your house? Dangerous. Dead. That car? Unroadworthy. Dead. Those aircraft parts? Counterfeit and not to spec. 300 people dead.
Every single thing humans do is already regulated in some way. Why? Because humans in the end, like all animals, try to achieve the best least effort : highest reward ratio they can.
In the modern world, these regulations need to be extended to automatically cover modern technologies and prevent inherent harm. They shouldn't be overbearing. They shouldn't be pointlessly excessive. But they are required for all things.
Many people think capitalism is greedy because they have literally spent a lifetime experiencing greed fueled capitalism first hand, not because they sit on YouTube watching propaganda videos.
Life is full of risks. Lots of things were approved by bother government and experts in the past that was bad for you.
> That food? Bad. Dead. That tool? Dangerous. Dead. That work on your house? Dangerous. Dead. That car? Unroadworthy. Dead. Those aircraft parts? Counterfeit and not to spec. 300 people dead.
The FDA has stopped people from getting medication in the US that are over the counter medicines because they haven't been approved for use. It is a double edged sword.
>Every single thing humans do is already regulated in some way. Why? Because humans in the end, like all animals, try to achieve the best least effort : highest reward ratio they can.
Unfortunately. What has the current light regulation on the web brought us cookie popups that are irritating that people just click through and a GDPR warnings that don't actually solve the problem of collecting your data. I don't hold out much hope for future regulation, which btw will favour the big tech players that have been collecting our data thus far. BTW you don't know the names of many of them, because they are B2B players and provide services to the companies we do know the name of.
As for "best result for least effort". Well it depends how it manifests itself. It can either be laziness or efficiency. The latter is not a problem.
> In the modern world, these regulations need to be extended to automatically cover modern technologies and prevent inherent harm. They shouldn't be overbearing. They shouldn't be pointlessly excessive. But they are required for all things.
Inviting any sort of regulation will involve government. Government will try to justify itself by demanding more regulation. It will always be overbearing and that will cement these players in place.
At the moment, we have the best chance of these players being toppled. People are looking at alternatives to big tech and are going to smaller players, mainly due to censorship. The trickle has now become a stream, sooner or later it will be a flood. However because of regulation on the horizon (which doesn't address any of the issues we care about)
> Many people think capitalism is greedy because they have literally spent a lifetime experiencing greed fueled capitalism first hand, not because they sit on YouTube watching propaganda videos.
I suspect you are confusing corporatism (which is a form of fascism) with capitalism (which is a party of liberty).
As for propaganda. I never said anything about Youtube. Don't put words in my mouth. I am talking about how hollywood, novelists (since the 19th century), newspapers have framed it since forever. You are soo fermeted in it you don't even realise it is propaganda.
GDPR is quite decent as laws go; the problems you mention happen because the regulation enforcement is too weak. Displaying a cookie popup was never anything but an admission that you're doing something you're not supposed to. GDPR notices again mostly give the same evidence. A lot of them aren't even compliant. I honestly wish DPAs of EU member states would start beating these companies down until this bullshit stops.
> I suspect you are confusing corporatism (which is a form of fascism) with capitalism (which is a party of liberty).
Potayto, potahto. Capitalism structurally favors something resembling corporatism, because capital compounds - the more you have of it, the easier it is to get even more. The market is a dynamic system - what matters is what it evolves over time into.
All that sites will do is do a cost assessment of whether it is worth serving those in the EU and just block the IP range and people that want to use those services will just use VPNs anyway (which is what I do when I am banned by IP from a site because of the GDPR rules).
>Potayto, potahto. Capitalism structurally favors something resembling corporatism, because capital compounds - the more you have of it, the easier it is to get even more. The market is a dynamic system - what matters is what it evolves over time into.
No it doesn't. Corporatism is a collusion with government. If governments were smaller, buying influence wouldn't be effective. You don't even understand what you are arguing.
Yes the market is a dynamic system that why if you allow it to operate freely those companies that are abusing their position will start to lose market share when other competitors that don't will be more attractive to consumers. However once you involve regulation, then that mechanism doesn't happen because you just raised the bar higher for all the would be smaller players.
Again you always want to frame it in the worst light.
Anyway. Fuck this site, dissenting opinion is frowned upon here. So much for the hacker part.
Also, consumers on average do a bad job of managing anticompetitive behaviour and harmful externalities, even if they know about them. Convenience and habit are strong motivators. And we need regulation to disincentivize companies from outright lying in the first place.
Companies and people in a fully free market system won’t magically become rational automata that behave ideally. We’re only human. Our superpower is the ability to collectively leverage our individual specialization. Foresight for negative externalities is a specialization that needs regulation to be effective.
I agree that government intervention is not ideal, as government is also often dumb, evil or incompetent. But that's more our failure to set up a political system where only the best, skilled, most ethical, least selfish people can rise to the top. We are nowhere close.
Propaganda: I wasn't putting words in your mouth intentionally, I see your point though. It was a description of how your words felt to someone making a critism of capitalism, that for me to dare criticise I must be propaganda-ised.
I don't watch movies. I don't have a TV. I don't read newspapers. I read a lot but a broad spectrum of works from a variety of times.
I like your distinction between capitalism and corporatism, it's a great point. I wonder though: Corporations exist inside of capitalism, so isn't it a failure of capitalism to bring outrageous cooperations to heel?
I dislike your "fermeted" comment, you literally followed an accusation of fallacy from me to you, with a whole bunch of actually intentioned fallacy of your own?
Anyway, have a great day!
A sort of freedom. Not free from taxes, free from interference. But that's really never the case.
Say you buy something from a store, and you expect their use of your data to follow the terms on the loyalty card. They've got a bunch of commercial-code laws that protect them. You can't pay with counterfeit money, or give yourself a 2-for-1 discount. If you use credit and don't pay, men from the state with the right to use violence come and collect for the store. They're totally legally protected. But how are you protected? Only at the end of a hugely expensive court case in the best outcome, but probably not at all.
The potential risk to the store from you is limited to the cost of goods, the risk to you is almost unbounded. You might lose healthcare coverage, or your boss might buy your data and use the store to link your id to your pornhub usage and fire you. Minimal, limited risk for them, huge unbounded and unprotected risk for you.
But then you discover after buying your gallon of milk, that they (knowingly) sold your data to someone who then sold it to, let's say an insurance company, in direct violation of the words on the back of the card. Now, how do you get made whole?
So, no. I don't actually feel that the free market meets that description for 99% of transactions. Between two citizens over a used lawnmower, yes. Between Warren Buffet and Bill Gates, yes. But between you and the supermarket - not even a little.
Not really. My perception comes from thinking about the market as a dynamic system, and not a static picture described by pro- and anti-market propagandists.
> Capitalism is about free market trade. You provide something and people choose whether they want to buy it or use it.
That's a "motte and bailey" defense. Such a perfect free trade almost doesn't exist, and very few are privileged to partake in it. Market offerings aren't independent - they're in competition, which means a lot of possible products aren't provided, and those that are face competitive pressure to provide less value for more price.
The most important thing to recognize is that the market, as a dynamic system, optimizes for profitable companies. Not for maximizing value these companies deliver to their customers. Usually, providing value is the most straightforward way for profit. But there are other ways - ways to provide no value, or even negative value while still netting additional profit - and the market takes them as much as it can. Vendor lock-in and surveillance are just few ways of making money by providing negative value-add.
> People add greed qualifier in there so they can frame it as something illicit going on.
Not illicit. Immoral. Because after all is said and done, the market is still entirely made of people and their decisions, which get to be evaluated through the lens of ethics.
> The fact still remains that if people cared about their privacy (and there is no evidence they do), they wouldn't use these sites.
They care and they will use them anyway, because the market doesn't provide any other option.
> Store cards used to track purchases and spending habits in store in the same way that sites do today (however at a much greater scale) and customers were given vouchers in return.
In the store. Not across stores. And they gave vouchers back, not shoved extra ads in your face. And that's without touching the qualitative difference between a human clerk doing the surveillance and automated systems doing the same.
> Pretending otherwise is passing the buck.
I hate to invoke the concept of "victim blaming", because it's usually invoked very unreasonably, but - you can't expect individuals to be able to rationally make market decisions while working their asses off trying to make ends meet, having their attention DDoSed, and facing against compounding improvements in manipulation techniques (courtesy of the market). I'm willing to cut regular folks some slack, and instead focus on the people running these companies, who had a clear choice, and chose to engage in abusive practices. You don't impulse-adopt business models, so you can't excuse it as a moment of weakness either.
(But then I'm willing to cut these business folks some slack too; in many cases, it's the market pressures that force to choose the abusive option - which leads us back to my original point: the market is a capitalist greed optimization engine. It promotes business people who think that, much like your margin, your ethics are their opportunity.)
Nonsense. Your framing is exactly the same. Don't gaslight me on this. I am not naive.
> That's a "motte and bailey" defense. Such a perfect free trade almost doesn't exist, and very few are privileged to partake in it.
Yes the free market doesn't exist because governments stick their noses in.
>Market offerings aren't independent - they're in competition, which means a lot of possible products aren't provided, and those that are face competitive pressure to provide less value for more price
What a load of nonsense. It is because of the free market we have niche products (Amiga accelerators would exist and that is pretty damn niche).
> The most important thing to recognize is that the market, as a dynamic system, optimizes for profitable companies. Not for maximizing value these companies deliver to their customers. Usually, providing value is the most straightforward way for profit. But there are other ways - ways to provide no value, or even negative value while still netting additional profit - and the market takes them as much as it can.
You are speaking out of both sides of your mouth.
> Vendor lock-in and surveillance are just few ways of making money by providing negative value-add.
If vendor lock in a problem if the customer is happy with it? It is up to the individual customer to decide.
> Not illicit. Immoral. Because after all is said and done, the market is still entirely made of people and their decisions, which get to be evaluated through the lens of ethics.
That is splitting hairs. No you want them to be evaluated through the lens of ethics because it benefits to do so in this argument.
> They care and they will use them anyway, because the market doesn't provide any other option.
You don't need yourtube, you don't need facebook, you don't need a lot of this nonsense.
> In the store. Not across stores. And they gave vouchers back, not shoved extra ads in your face. And that's without touching the qualitative difference between a human clerk doing the surveillance and automated systems doing the same.
You have no idea if that data wasn't sold to anyone else. The vouchers are in themselves ads.
These store cards proved two thinds. The first being that people will willingly give up their details for some trickets, and two that tracking customers and optimising via that works. It was a stepping stone.
> I hate to invoke the concept of "victim blaming", because it's usually invoked very unreasonably, but - you can't expect individuals to be able to rationally make market decisions while working their asses off trying to make ends meet, having their attention DDoSed, and facing against compounding improvements in manipulation techniques (courtesy of the market). I'm willing to cut regular folks some slack, and instead focus on the people running these companies, who had a clear choice, and chose to engage in abusive practices. You don't impulse-adopt business models, so you can't excuse it as a moment of weakness either.
Yes I do expect individuals to able to rationally make decisions. People have been told for years and year and years on end what these companies do and they don't care. So it is their fault.
No customer is happy with vendor lock-in. It is negative value to the consumer, which was the parent's point.
Also, pedantic point, I think you were looking for the phrase en masse, of french origin. See https://en.wiktionary.org/wiki/en_masse
One worry about the rising tide of anti-Facebook etc. sentiment is that it is driven by (a) a small number of vocal people who (b) say that x is horrible, but don't act like it. (I don't say whether this worry is always, or even mostly, correct. Just that it is worth considering.) The result can be problematic.
For example, like millions of people in the EU, before I use any new website, I now have to click "OK" on a GDPR button. I consider myself as caring about my privacy. But my behaviour tells me that I just click "OK" without even reading what they have to say. Obviously, my privacy is less important to me than the 10 seconds or 1 minute that it would take to read their policy and opt out from anything I dislike.
You can respond that this is a "dark pattern", and I agree. You could also say it would be better for me to be able to set my privacy once - say, via browser settings - and I indeed try to do that, having used various Firefox settings and plugins to manage my cookies. Nevertheless, the fact is that, as my behaviour reveals, I would rather get on with my life, than waste 10 minutes a day clicking on privacy policies. Given that, of course, I would also rather not be presented with the "GDPR OK button" at all. It's a fleabite of annoyance that I deal with many, many times, and every time I mutter to myself "screw the EU and screw the GDPR". I suspect I am not alone.
I also suspect I will be downvoted for this, by just the noble idealists about whom I am grumbling. Sigh.
The price of privacy sounds cheap. 10 minutes a day for privacy. How much cheaper do you need it to be.
The only problem is that the consumers want to trade the tokens at a profit instead of as purchases.
But that isn’t really a problem for the service that sold them. It is revenue. But people have an uncomfortable relationship with other people making money when they can extrapolate how much and consider that a problem.
Many services now are completely client side and use the nearest node that you connect to as the backend. They store enough variables in their smart contracts on chain and do the rest of the calculations client side. So their web service isn’t tracking you. But if you reuse addresses other people are.
I'd argue the term 'pay for the services [they use]' is too vague here to be meaningful - there are too many options that would drastically change the incentives.
Pay per API call? APIs start to need 5 calls to get all the info you'd need for one request. Subscription model? Consumers are going to have to juggle a different account for every provider they use. Subscription to aggregator who then pays content providers based on usage? We're back to the clickbait situation we were in in the early days of advertising and are arguable still in.
For me the insane thing is that there are no options. I can't universally buy any song I want DRM free from a range of providers. I can't pay per article for news from an RSS feed.
TV is an interesting one because the industry has convinced the user base to pay for content, but the subscription model is already showing some of the limitations shown above.
I feel like the biggest innovations in this space aren't so much new ideas or 'just convincing people to pay for things', it's a case of making the payments as easy and understandable as their previous counterparts.
Frictionless, permissionless micropayments are illegal. On purpose.
This is not a social or technological problem. It is a legal problem. AML/KYC does not scale.
Bitcoin has turned out to be expensive to mine. Most other distributed ledger "currencies" have been launched to make money for the creators of the artificial scarcity.
Governments cannot give up control of the money supply but they are the only ones that can establish a form of electronic currency that works the same as cash does now.
Whether they're willing to make it as anonymous as cash or allow it to be exchanged for other government's electronic currency is part of the problem that needs to be solved.
The KYC stuff is still there at the exchange level of course.
Yes but to be frank if I had to pay for everything I use I would go broke.
We're not all on SV salaries here. I have about 100€ of disposable income per month. If I put it on online things, it means I would have to stay at home (I mean pre or post pandemic times).
Sure, for one site, that $3/mo will not break the bank.
But being limited to only $100/$3 = 33 sites a month would be quite constraining.
When I'm reading about a topic or searching for a purchase, I'll typically visit that many sites in one session. Over the course of a month it is normal to visit hundreds of sites (I have counted for auditing a job), sometimes over a thousand. That's without giving it any thought, just following links and reading linked content.
(Heck, when reading HN I sometimes read more than 33 HN stories in a single sitting, but to be fair I don't often follow the links to the articles themselves ;-)
If I had to monitor my usage to keep it down to 33 sites in a month, I could do that but it would be a very different reading experience than I currently have.
Like back in the days when we had to pay per minute of online access (outside the US, using audio modems). The change to a single monthly subscription with the freedom to read as much as you want, as long as you want, was liberating and transformative. It would be a little sad to go back to having to self-police reading in order to keep costs down.
Of course this is totally made up, so it’s impossible to argue about, but my point is that what you described didn’t necessarily seem out of line to me.
AFAIK, this is the "micropayment" problem, that people were hoping bitcoin would solve.
Bitcoin solved it, and then AML/KYC made it illegal to not have bank-like overhead costs.
Compared to Visa, Bitcoin solved micropayments only if micro refers to the daily transaction volume.
To keep the comparison accurate, you can transact many times within an “accounting block”. Bitcoin has the shorter accounting block and tier 2 systems laid on top of it seem to scale Bitcoin well beyond any accounting block maximum transaction cap.
The GP is probably not referring to speed of settlement though. Bitcoin solves KYC and trust and allows frictionless micro-payment.
Companies can issue gift cards that are entirely anonymous, paid for in cash, redeemed anonymously for cash.
Unfortunately, advertising has also created an unsustainably high "standard of living", so to speak - you get so many services and applications for free these days that would realistically not exist or cost much more than you were willing to pay for them had it not been for advertising.
Personally, I don't think there's a way out of it until someone comes up with an alternative that brings the benefits of advertising without all the downsides it has, because individual consumer incentives are just not aligned. I'll gladly pay a one-time fee for some productivity app, or a small subscription for something I use almost daily. But if e.g. goodreads wanted to charge a subscription, community size would probably dwindle, and personally I'd just start keeping a spreadsheet of my books again.
If you ban ads and monetize only via affiliate links, I think you’ll see a lot more poorly written, often algorithmically generated content shilling for affiliate clicks.
People don't want to pay for content. If people pay for content and feel ripped off, they can ask for a refund. Then cheaters can pay to access the content then ask for their money back if they want to. This puts the content provider in a bad position.
If people pay for content, then they want to have that content themselves forever. In some sense, this is fair. But then they want to share that content with others. Then the other person/people don't need to pay. Now you have a problem where everyone can just get the content when one person has paid for it. This is a bad dilemma where content providers and consumers both seem to have a good case.
Since neither pay model seems to work, companies just show ads. Then people ignore ads, so the ad companies make them more attention-grabbing and intrusively targeted. So people use reader mode and ad blockers. Now no one is looking at the ads that pay for the content.
I wish I knew the answer.
> The second one should even be called piracy.
Sure, I agree. Piracy is a loaded term anyway.
As far as the media is concerned, everyone here under the age of sixty have always been the product. And it’s deeply naïve and idealistic to think that “paying for their services”—how will they do that? Oh well—will change something which is now completely fundamental to media as we know it.
But I think individual action isn't sufficient to get us over the hump. There are just too many things we use on a daily basis, and often those things use things that use things. "Free" is an illusion, but it's an illusion with a very low cognitive load. Manually supporting each and every thing I appreciate at the right level is a complex and taxing process. In practice, I'm sure I miss a lot.
In the physical world, we have some solutions for this. I don't have to subscribe to each park I use. I don't have to kick in for each sidewalk tree I walk by. I live in a neighborhood with a lot of street and alley murals, all community supported in various ways. I think the next step forward for the web involves finding ways for collective action with low individual cognitive load. It wouldn't be perfect, but it could be better than what we have now.
Switching away from ads will feel good until some do-nothing decides they'll hit their bonus targets by re-introducing ads in addition to the subscriptions.
This has happened many times. It's obviously going to happen again.
You do not want to normalize subscriptions for every old Web service, trust me.
Iow, I believe advertising is symptomatic of our current state of consciousness as a collective, and so it is not a cause.
Ofc, this could exist with actual paid services.
I’ve not used GoodReads before but it appears to be a book recommendation service. I’m surprised they can’t run the main site & API on Amazon referral links for the recommended books.
Makes me think there’s a different reason for closing the API, because Amazon could run the service for close to nothing on AWS.
Ads convince people to spend money on some product they wouldn't buy otherwise. This in term finances those ads which finances the service. I'm not convinced this is better in the long run for many than paying services directly.
I think that doesn't hold logical water.
If only 1% of people buy things of significant value based on ads, and 99% of people never buy what's advertised, it is still worth advertising while advertising is cheap.
A related phenomenon is the way online games make their money.
The vast majority of players never pay for anything. A few pay a little, and a tiny minority pay so much more than everyone else that it's the tiny minority that the game-maker depends on for their business.
Looking at the strain of our local delivery services right now, that cannot be just 10 per cent of people shopping.
I think in order for this to happen, there's going to have to be better payment models than $4-9 a month subscriptions.
Crowdfunding is probably the best solution, if it can be made to work - it would allow for some kind of monetization on a voluntary basis, while preserving free access for most users. But I don't know of any site that really uses it successfully, aside from Wikipedia.
Yes indeed. Right now news & opinion are largely limited to what is considered "advertiser friendly" - think short news cycle, and other problems.
The perspective of paying pennies for everything introduces a lot of friction. Luckily an alternative model is making roadway, mostly in Youtube circles: pay a couple of your favorite creators, and watch the rest for free.
The extension of the old "freemium" model, but this time based around stochastic balance between viewers who pay this or that creator. The paying subscribers often get access to additional material - one that's more geared towards fans or deeply interested viewers - while the regulars, free-to-watch, viewers get the general content just fine.
I've said it many times before but it's worth repeating...
The Internet survived just fine before all the ads and tracking and it will survive fine without it.
Today it’s run by people, standing on their shoulders, whose dominant motivation is making money or how to “capitalise” on something, something which they have no fundamental interests or passion for.
Obviously the outcome is going to be different.
If you allow the web to move to a model where you have to pay dollars for a service worth cents it turns into the same kind of market as the mobile phone operators market charging fees for sms messages: a total rip off.
Why does this needs to be a business anyway? The only reason is because it can be.
Is the Web a business model?
The Web really is machines serving HTML, JSON, XML,... over HTTP/HTTPS across physical connections. There are several ways of looking at this, but often enough the debate gets reduced into a dichotomy.
I'll put this into two simplified extremes.
The Web is a shared infrastructure seen as a "public commons". You can access that infrastructure, request/receive bits and bytes from other machines for free and if you want to host content yourself, you connect your own machine to that infrastructure and you share content via your own machine for free. You carry the costs of the usage of the infrastructure yourself, regardless of the direction of data traffic across the network.
The Web and it's infrastructure are commodities. Storage, maintenance, bandwidth,... are expenses that should be offloaded. The main goal of hosting and making information available on the Web is to, either directly or indirectly, make a marginal profit. You pay for the privilege of accessing someone else's machine to download data, and you get paid by those who want to gain access to information you host on your own machine.
The problem with the statement above is that it implies that both extremes are mutually exclusive, and only the latter one is viable.
This is false.
The Web is ultimately a decentralized network which is build on top of intentions and goals of humans. And those intentions and goals can be wildly differing. There are parts of the Web that operate according to the former idea, and there are parts that operate according to the latter. Both exist, and there's a spectrum in between.
In the analogue world, the same notion translates into private businesses, non-profits, cooperations, community initiatives, charities, public initiatives and so on.
Goodreads choosing to close down it's public API is just one case choosing to move towards one side of the spectrum. It's by no means an indication that the entirety of the Web and - more specifically - it's denizens decide to move towards that side.
That spectrum does emerge based on laws of economics, though.
Goodreads has always been a private business. public API's of private businesses are never truly "Public". They are either a courtesy or a business investment. And they will step away from such courtesy if the costs outstrip the benefits.
The Web isn't quite the same as public space though - parks, beaches, forests, streets, grasslands,... - because the vast amount of infrastructure is privately owned. In that regard, the notion of "The Web is a Commons" is only true to the extent that private people are willing to accept and support that idea, and are willing to carry a shared part of the costs.
It's that last part which makes all the difference. Operating a basic website with a limited number of visitors comes at a low cost, and so one could operate a small Goodreads like website with a niche of books. There are plenty of examples of people keeping freely accessible blogs and the like with their own book reviews.
Goodreads tries to turn that idea into a business model. The intention of generating a profit is very distinct in that regard. However, not only is it hard to sell the opinions of other people, it's even harder if costs generated by trying to cater to an audience of millions outstrip the revenue generated.
The Web isn't financial centre. Just like London nor New York aren't representative as to the entirety of human society. Big businesses are - ultimately - only a part of the Web, just as much as they are only a part of society. And if their business models fail to keep them operational, the Web, and society, will, ultimately, churn on without them.
As the infrastructure changes the dominating idea shapes it.
As an example there used to be (and stills is, but the change is evident) vendors selling music recordings. Vinyl, tape, cd. The market traded in tangible artifact that could change owner, could be copied (legally or not) could be put in a library and lent out.
In Sweden we even pay a special copy compensation tax when buying any device with storage capacity to tunnel some money towards “content creators” in support of this distribution form.
However as the technology has shifted, allowing for direct streaming, the trading of artifacts has disappeared. The laws and economic realities now promote a market with fewer vendors offering only a limited catalogue of recordings, and only in a form that can never leave their control effectively.
This is only an example. And I think this particular one is mostly about reviewing the legal landscape.
Another example might be how protocols like RSS, XMPP, SMTP was used for interoperability and allow different vendors to offer compatible services. As things shift, this time perhaps more due to economic realities, the dominating tendency is still to erode interoperability and dominating players shape the technology towards their more siloed reality. Perhaps we need more tax funded players, (public service?), simply competing and collaborating to tilt things back again.
In general, I doubt that the network effect can be overcome for consumer platforms. People want to share their book reviews. Why should they limit themselves to people who pay? Paying only works for those who want social filtering.
To compete with Goodreads, the data would have to be free like Wikipedia, for other competitors to emerge so that it is not just a village but a country with many villages. But then, it's a rtf reader situation where it is difficult to survive as an app creating company.
No, I don’t have a great solution other than the status quo.
The whole process doesn’t have to be expensive, but it’s certainly not free. You can build very cool stuff and give it away for free, but sustainability and scalability ultimately require revenue. The magic thing about a successful business is the ability to cover execution costs, support the development team, and still leave value on the table for users.
I think digital services have drastically different economics depending on (1) how adding a NEW user changes the value proposition for EXISTING users, and (2) something like the user’s start up/discovery cost relative to lifetime value
A direct payment model makes sense when your users’ value is independent from the size of the user base (assuming performance scales at least linearly). These services can tolerate moderate startup/discovery costs. For critical enterprise services, startup costs can be high because the lifetime value is still much larger.
If value scales with the user base, as in a social network like good reads, then startup/discovery cost must be pushed to zero to grow the user base as quickly as possible. A paywall slows user base growth and reduces value for those users that actually choose to pay.
So far, advertising is the only known way to monetize (and thus sustain) a digital service while maintaining near-zero startup/discovery costs for individual users. Micropayments, even with good UX and low fees, increase joining costs relative to advertising. Thus they will reduce the value of the service to paying users if value scales with user base size, but would benefit services where value is independent of user base size.
Federation is maybe the best way out of this dilemma IMHO. The value of the overall network grows with the user base, so adding new federation partners should be near-free. Each instance is small relative to the network as a whole, and thus can focus on individual user value rather than growing the network, which means it can charge users directly. This is why you’re willing to buy a great Twitter or Reddit client app, but would never pay for a Twitter or Reddit subscription. (Yes I know those are centralized services, but the model holds if you look at the business relationships).