> The recommendations suck, the lists suck — it’s like, 100 lists telling me to read The Handmaid’s Tale and Harry Potter.
I had the same experience with GR and also Amazon.com which constantly peddles the vampire romance books when I am looking for recommendations for horror/fantasy. Both Amazon and GR strategy make sense because best-selling books sell the best, so they should recommend them to increase profits. However, it does suck being a reader looking for new book suggestions.
I've spent a good deal of time making my own book recommendation algorithm which has been working well for me for the last two years. [1] Through it I've discovered old authors I didn't know (Ted Chiang, Clive Barker) and new authors which I wouldn't have noticed (Scott Hawkins, China Mieville). Of course, it always helps to get recommendations from friends with similar tastes, too. :)
As far as I can tell, all recommendations lists fail compared to freely-available experts.
Like, if you like literary fiction, go through the Pulitzer prize for fiction, and just read. (I'm not even half way through that list, but everything I've read on it has been really, really good.) - there's all sorts of awards for smaller niches... the nebula, the Hugo, etc...
(Actually, that's a question. What is the award for the romance genre?)
I... personally don't understand why people even try to automate making better recommendation engines when highly skilled and respected experts are already doing it for just about any niche, and releasing the results for free.
My preference is to rely on the judgement of reviewers whose taste kind of matches mine.
I'm into SF and Fantasy. In the past, I would read the reviews featured in Locus magazine by the various reviewers. Nowadays, I occasionally read Locus but also reviews from other places like tor<dot>com, the Magazine of Fantasy and Science Fiction and Interzone magazine.
> My preference is to rely on the judgement of reviewers whose taste kind of matches mine.
I wish there were a website where I could put in all of my favorite video games, books, movies, anime, etc. And it would recommend me things based on what people with similar tastes liked. Then, I could try out the recommendation and then either like or dislike it.
I would slowly acquire a recommendation network of people like me, effectively crowd sourcing the content-finding to a bunch of clones!
That's FilmAffinity, for movies. It's pretty popular in the Spanish speaking world.
You vote movies, it calculates what it calls your "soulmates" (people who voted similarly) and then you get recommendations Of movies based on what those people loved, which you can filter by genre, era, etc. You can recalculate your soulmates anytime, and there are some options to tweak the algorithm.
You can also organise movies into lists (kinda like music playlists) and those are public, so if my preferences match user X and they happen to have a list titled "movies I enjoyed this past year" I can just check that.
That's essentially what https://www.senscritique.com/ does in the french speaking world. You rate movies, books, comics, series, music, then it will suggest "éclaireurs" (scouts/recon) to follow, finally you will be advised to see/read/listen what they love and you do not know.
This used to be what Last.fm did before they turned into a MySpace-like mess. Reviews are great but in a world of access to a monumental volume of content, a way to discover obscure things is still difficult to find, and apparently, to monetize. When Netflix started streaming, they tried the long tail approach, but soon changed to only licensing a few things and then serving them up in endless configurations, which must be far more business-friendly for them.
https://www.anime-planet.com/ does recommendations like that on a per-show basis (if you liked [this] then you may like [this], [this], [this]...), the relationship recommended, voted, and commented on by other viewers.
I think Steam is working on something similar, with recommendation based on what you're playing (https://store.steampowered.com/recommender). Well it recommends only games on the store, so that's a lot of games off the table right from the start.
RateYourMusic can do exactly this; find some users with similar tastes (you can see who rates an album as what score), and add them as friends (privately, i.e they won't be notified when you add them). Then you can make lists of how your friends collectively rated music. You can also find site-wide charts for specific genres.
makes sense. I've been reading Gaitskill's "little hammer" - it's a collection of essays, but... like half of them are book reviews, so now I'm pausing to read some mailer. For that matter, it is pretty common for me to read books because they are mentioned in the front matter of other books.
Because we don't always like awarded books. I read "The Three Body Problem" a couple of years ago which had won a Nebula award and found it to be mediocre at best and much overhyped. Last year I read the "Neuromancer" which had won both a Nebula and a Hugo and was for me one of the worst SF books I've ever read. And then there are books which are brilliant, but have never won an award, like "The Martian".
A book like the "Neuromancer" really needs to be understood in it's time. If you read it now, it will look very clichéd. But that's because it introduced and popularized those very clichés!
A lot of really significant cultural works have become so absorbed in our common culture, and sometimes improved upon, that the original starts to look cliché and even naive. Is there a name for this effect?
I read Neuromancer in the early 90’s and thought it was very cliched at the time. Granted that’s ~10 years after publication, but it barrows heavily from earlier works.
I suspect people like it for the same reason they liked their first Anime, it’s an unusual style that seems very original unless you have been reading other stuff written in the same vein by say Philip K. Dick.
The focus on cyborgs a year after The Six Million Dollar Man TV show kind if shows how much a product of the times it was. But, that’s the surface.
The way it portrayed both hacking and brain machine interfaces was wildly off base and basically copied from other science fiction. Virtual reality for example goes back to 1933. Main character being a druggy is fairly common in that time period, again not a big deal. As is copping tone from other works etc.
All the big stuff is forgivable, but he also copies little things like replacing liver and kidneys to better filter the blood and thus prevent someone from getting high / poisoned etc. Sounds good, but blood takes around a minute to circulate and most of it does not hit either on the way. It might reduce how long someone stays high or improve their chances when poisoned, but it’s really not enough to prevent it.
Granted I prefer hard sci-fi, but the novel’s focus is really on style over science fiction. It’s IMO somewhere between space opera and fantasy.
>Granted I prefer hard sci-fi, but the novel’s focus is really on style over science fiction. It’s IMO somewhere between space opera and fantasy.
How I looked at Gibson's work changed completely after I read "pattern recognition" when it came out in my early '20s - It was very explicitly about style, and I went back and re-read the older stuff which I read as a child, and yeah, you could also say that neuromancer is about style and fashion. It was interesting just how much reading the later book changed how I thought about the earlier books.
Any recommendations for good hard sci-fi in the last generation or so? I'm asking because your comment strongly suggests I'd like what you like. I'm one of the very few who think that "Science Fiction and Fantasy" as a genre makes about as much sense as "Math Textbooks and Romance Novels".
I promise not to blame anyone for a recommendation that's flawed. They're all flawed. Anything where the story is based on the implications of known (well, currently accepted) science without any bogus magic is as hard as trying to figure out what will really happen in a large software project that hasn't begun yet. But what have you liked despite its flaws?
Regarding “hard” science fiction from the past 25 years, I’ve thoroughly enjoyed Stephen Baxter.
I read the first two books (Voyager and Titan) from his NASA trilogy [1]. These books are set in a near future or alternative time-line and cover inter-planetary journeys (Mars and Titan), involving the use NASA technology. Both books seem very well-researched and true-to-life.
Another book I really enjoyed was Coalescent [2]. It’s a blend of historical and science fiction: the historical part tallies with my own understanding of the late Roman Empire in Western Europe while the science part is more speculative – a human society that gradually evolves to become eusocial.
On a very different scale is, Space [3] which explores the Fermi paradox, communication between different sentient species, and the long-term survival prospects for civilisations of sentient species. Unlike the other books which have more straight-forward scientific concepts, I found some of the ideas in this book to be mind-expanding and really pushed my imagination to its limits.
From a story-telling perspective, his books are well-plotted with well-drawn, compelling characters (you really empathise with the protagonists and want to find out what happens next). I learned about a lot of diverse topics, e.g., the theories of Giordano Bruno, history of NASA projects (e.g., NERVA), the tyranny of the rocket equation, explanations of the slingshot effect, the economics of the Roman Empire, eusocial organisation and behaviour, lunar geology, Titanic meteorology, how humans could survive in a micro-gravity environment (and space in general), consequences of gamma-ray bursts, and much more.
Looking at Baxter’s Wikipedia page[3], I can see that I’ve only scratched the surface as he’s written many more books. Unfortunately, over the past decade, I’ve got out of the habit of reading novels but I really should make more of an effort.
Honestly, I have mostly given up on recent science fiction so any recommendations are welcome.
Anyway, it feels like a cop out but The Martian by Andy Weir is worth the read. The most obvious issue is the opening storm would not have done much because the atmosphere is so thin, but it is generally ok on the science side.
Agreed (unfortunately). I gave up on SF decades ago, and The Martian is the only SF book I think I've liked this century. I liked it a lot and occasionally search for others like it. In vain, it seems. I think it is an ill omen that popular culture no longer seems as excited about real science and technology, the exploration and discovery, as was the case way back in the "Space Age".
I enjoy a lot of books from Alastair Reynolds. His fiction is so and so but the science in his book is hardcore. I also liked Children of Time by Adrian Tchaikovsky (this one is two volumes).
If you don't like William Gibson... you and I have very different tastes; Neuromancer is a particularly perfect example of the sort of thing I like.
There is a certain amount of calibrating for the awards group; for my taste? the nebula is... not 100%; much like how I enjoy '80s action films rather more than Ebert. like I've never read a Pulitzer for fiction book that I didn't think was incredible, while some of the nebula books I've read were only pretty good. (I haven't read anything by Liu Cixin yet, but it sounds like my thing? I mean, it was hyped, to me at least, as genre sci-fi written from a very different cultural perspective, which is totally my thing.)
So, name five other books which are like Neuromancer and you liked them. I'll try one or two of them, if I haven't read them already. Incidentally I've read a lot of Gibson's books and Neuromancer was the one I liked the least. Probably because it was his first and not as polished as the rest. Or perhaps because by today's standards the cyber warfare he's describing sounds ludicrous. I do like his cyberpunk atmosphere though.
I've never actually read Neuromancer, but Alexander Jablokov wrote some novels similar to what I imagine Neuromancer is like. For example, Nimbus and A Deeper Sea. Near future written in the early 90s.
The writing in the English translation of The Three Body Problem is stilted and dull. Maybe it's better in Chinese? Some languages must be more expensive than others to translate well. Gave up half way through the first book; when I eventually read the synopsis on Wikipedia it sounded exciting.
I'll have to read Neuromancer again. At the time I read it I was skeptical that hacking decks and exploits would ever exist, but that was before JavaScript, internet connected critical systems, rop gadgets and nation state 0day exploit chains. The need to do scene setting for these concepts perhaps weighs on the story.
Likewise, I was pretty disappointed with The Three Body Problem after all the hype about it. Maybe it was because it was a translated work but it felt super flat for me.
I had a very hard time finding things to like in Neuromancer. It was a long time ago, but iirc the prose is pretty clunky, the characters are made of cardboard, and the plot has all the depth of a Ridley Scott movie. The world building was on point, but there's only so much 80's zeitgeist ("everything is awful, you'll be replaced piecemeal by machines, faceless megacorps rule over vast slums of pleasure junkies bathed in acid rain") that a person can channel with a straight face. Like I love me some weird dystopian cyberpunk universes, but I want that to be the starting point, not the ending point.
(Conversely, by way of making a positive contrast with something in a similar genre that , I really enjoyed A Scanner Darkly by Philip K. Dick).
To be totally clear, I think Neuromancer is probably a thoroughly enjoyable read and I'd recommend it to someone looking for what it is without hesitation. It's just pretty easy to take umbrage with.
Lol as opposed to 2019? An idiotic narcissist grifter as President, the planet on fire (southern hemisphere burning in winter), epic climate change storms, megacorp Vicodin, bikies selling crystal meth, block chain powered dark money drug markets, middle east savagery hitting new lows, internment camps in Europe and the US? China govt deploying ios 0day against their Muslim ethnic minority, and millions in camps already?
The only disappointing part is that we don't have vat grown assassins, if you want a transplant you have to go to China or Iran.
> I... personally don't understand why people even try to automate making better recommendation engines
There is a long tail of long tails: niches within niches within niches. Some of these don't have a single proven trustworthy reviewer, let alone enough that the rough edges of their opinions get sanded off by aggregation. For these ultra-niche interests, it'd still be nice to have a guide. ML can do that.
I face this problem with Netflix, Spotify and Youtube too. These algorithmic recommendations just need one improper dataset to throw everything out of the window. For weeks, my spotify is overloaded with instrumental songs. Youtube keeps on repeating the same stuff. Netflix believes that the only thing I watch is science fiction.
I call it "collapsing into the mean" - all recommendation engines (as the currently exist) will eventually corral you into the most vanilla, mass marketed set of recommendations and then fail miserably when any conflicting data is presented. We've effectively turned the web into cable TV circa 1990 - a finite set of junk food level entertainment sources that we voted for because they were "eh, good enough" and easy to find.
With as many YouTube videos that I've watched since 2005, you'd think they might recommend a video with less than 1000 views once in a while. I've found 1 new channel in the last 6 months and used the "Not Interested" option more times than I can count. And the rules are unclear. I don't want tech reviews from 5 years ago, but if I hit Not Interested, does that influence the channel, topic, age, keyword, etc recommendation frequency? I don't mind old DIY or woodworking videos, I'm subscribed to the channel, and watched the recommended video 5 years ago. (Of the current 12 YT Recommended videos, 2 are labelled Watched and only 2 are less than a year old.)
I think part of it stems from the lack of user organization features. Offer too many and you scare away users, while the people with vested interests dump money and time in to wash out any negative opinion (see Amazon reviews). Offer too few and you get poor recommendations during on-boarding/startup.
That's because the recommendation systems are optimizing for the wrong things: typically dwell time. You are highly likely to dwell longer on scifi, so heres more scifi for you!
Older recommendation systems use background and foreground timeframes to build frames of references, hence repeats.
The question to ask then is: what is the correct thing to optimize for? Dwell time is usually chosen for advertising and stickiness (you are more likely to stick with a service that plays what you like). Optimizing for novelty is difficult because the set of unknown things is much larger than the set of known things. Plus, it is risky from a business point of view.
Right. You only have four actions on Spotify as I recall (thumbs up, thumbs down, pause and fast forward). I guess some of the other apps have the larger bucket of 'who is watching/acting' based on account.
I am not sure how many actions are needed, but I can think of a few more:
It'd be nice to have an "I've already seen this" button that will mark an episode/season/show as watched so it doesn't keep coming back up. Netflix really wants me to watch Breaking Bad, but I've already seen it. I liked it; give me more like that, but not exactly that
> It'd be nice to have an "I've already seen this" button that will mark an episode/season/show as watched so it doesn't keep coming back up.
That's not just nice, that is the most obvious "feature" a normal person would design. I've stopped watching Netflix because of nonsense like this. They know I've seen it before, because I watched in their player. Also, even the tiniest, most modest application of analytics would discover that I always watch one series at the time, from start till end, or whenever I'm bored with it. In the past years, I have never ever watched a thing twice. Still, it was constantly showing me things I've already seen.
The only conclusion I can draw from this is that Netflix does not have have my best interest in mind when designing their algorithms and that they don't respect their customers. So I canceled.
Spotify has one genious feature that I use a lot: similar artists. The problem with machine learning and recommendations is that it has a few pitfalls that lead to poor results. This is on full display on Amazon which manages to recommend stuff I've already bought from them while not being able to tell the difference between hard science fiction and fantasy. The resulting recommendation bubble seems impossible to escape.
With Spotify, they do have a few features that work for me. I've discovered new artists by exploring their "Fans Also Like" feature. The nice thing about this is that they don't try to be too smart there.
Their normal recommendations suffer from the same issues that other sites have and are thrown off by the fact that my tastes are all over the place. I happily listen to sixties psychedelic rock, jazz, metal and some techno or some punk and tend to go from one to the other. Yet I'm very picky about what I listen to. Somewhere along the lines it seems to have decided I'm a middle aged guy (correct) and it consistently does not recommend me any music made this century; which is kind of frustrating if you are trying to find something new to listen to. Recommendation bubbles are a thing and escaping from them is hard.
I work around it by using the fans also like feature and using it's suggested additions to playlists. This works surprisingly well. Example based similarity search is a much simpler problem then recommendations. And it's IMHO a much more interesting feature to explore content with.
The desire for these recommendation services not to be "too smart" really resonates with me. If we can't yet give a great set of zero-effort recommendations, why don't we pull back a step and give the user a few power tools to find their own? Maybe I'm out of step with mainstream users, but I would see that as a huge improvement.
They often don't expose knobs I'd really like to tweak, either. For instance if I grew up listening to original album version of something I don't want to hear the live one. But there's only an upvote/downvote choice on Google music so what exactly am I giving the thumbs down? The artist? The song? This particular version of the song? No idea.
I absolutely agree, algorithms are terrible for recommendations. On the media database[0] my wife and I are building we're experimenting with community upvotes for suggestions. Our userbase is small so the results are inconclusive right now but hopefully time will tell if this method works.
The only good one I've ever seen was in the music business: Rdio, specifically. (Now sadly dead.) I don't know what they were doing, but their recommendations were consistently excellent at pushing me towards new artists and even new genres that I didn't expect but would up enjoying.
Back when I used to use it (about a decade ago), Pandora was good at shuffling, but their library was so small I would get the same songs over and over.
There's no reason algorithms have to be terrible, after all, most of what we do is expressible algorithmically when we give recommendations (similar interests, right level for the reader, not too advanced or too introductory, something they can use or that leads to other insights, etc.) It's just that the algorithms are generally so bad because they are optimizing for the wrong things and not any of these things that actually make for good recommendations.
Which is why the author of the article didn't find the upcoming release for the book she searched. Searching for a book we can't make money on? Pffft GTFO.
You watch one little anime with someone for high school nostalgia and suddenly Netflix thinks I'm only interested in k-pop docs, anime, and unusual shows featuring all-Asian casts... ???
Criticker has never failed me. I've been using it for years. I did hop over for a few years to a site called Jinni with really high quality UI. They closed their consumer facing services and provide their service to content providers. Criticker is still there crushing the game.
Entering "1984" yields "books similar to 1984 by Michael Dean". Entering "Nineteen Eighty-Four" gives me "Book picks similar to Nineteen Eighty-Four by Matthew Dunster". When I search for "Nineteen Eighty-Four: A Novel", the site responds with "Book picks similar to Beer in the Snooker Club by Waguih Ghali".
I'm sure Dean, Dunster, and Ghali are terrific writers but what happened to George Orwell?
"1984 orwell" finds it. Just "1984" looks like it's finding a translation or something - the cover picture clearly shows George Orwell. Maybe Dean is the translator? Still a bit odd.
Thanks for making this! I tried a bunch of searches and the recommendations are surprisingly good.
A couple of points. All my searches selected an (in my opinion) less prominent result as the top hit and I had to click the "see other results". The searches I remember doing were "Blood Meridian" (should show Cormac McCarthy as top hit and "Waiting for the Barbarians" (J. M. Coetzee).
Secondly, I'm not sure if it's sensible to show books by the same author, at least nowhere near the top.
Lastly, while I like the UI and the idea of a handwriting-type font, I find the current font a bit illegible. It's readable, but doesn't skim well.
I turned off web fonts in Firefox settings to be able to read the site easier. The recommendations seem pretty good though, so it is worth a bit of "suffering".
It takes a lot of effort to read it. I'd say that for me it's 80% unreadable. Please, at least make a button on the site to select another font if a user feels like I do.
the font is actually perfect and fits the theme of the site well, while still being easily readable. what makes it obnoxious, other than it not being some variation on fira mono or whatever?
I don't usually have an immediate positive reaction to a site, but I immediately liked yours. It's fast and simple. There is no javascript. The recommendations are reasonable.
Would you mind describing a bit of your algorithm, or is that secret info?
I applaud you for putting a site together, and I like the design look, but when I clicked on the best list of fiction from 1900-2019 Harry Potter showed up in five versions at the top of the list.
This is a hard problem because popular != good. But I would like to see data on whether recommending popular books works best. I’m sure amazon and goodreads have this data, but they are horrible at recommending.
Searched "The Glass Beads Game" got "did you mean The Everything Classical Mythology Book: Greek and Roman Gods, Goddesses, Heroes, and Monsters from Ares to Zeus by Lesley Bolton" couldnt find Hesse in additional results. Searched Inferno, could not find Dante. Kind of strange that this site misses some super important classics (well 2 of like 5 searches I tried)
Agreed. Its fixed now [1] as I added in another fuzzy search. (I was using fzf which missed some small things and I've now added Levenshtein as well to compensate).
Neat. Just a small note: On the "Best Of" page, the year selection is too narrow for my system (Chrome on Windows 10, WQHD). The two boxes only contains "20 " (twenty and some empty space); clicking/hovering one reveals those "up/down numerical selector arrows". You probably want to make them a little bit wider?
I'm really liking the recommendations on your website. I found a few good recommendations already. And like you I really struggle with finding those via the usual channels. But, that font is just awful .... :-). You do have a bit of a Harry Potter infestation on your best fiction list. It appears multiple times there. Same with game of thrones related stuff.
Out of curiousity, where do you source your data from?
Awesome! Indeed feels like a better selection of recommendations
The only tweak I’d make? Link to UK (etc, but UK for me) Amazon too! Mainly because I’m lazy but also you could then set up a UK affiliate tag. I buy a lot of books, get your slice!
your site is useful and has good recommendations. Thanks for the link. I tried looking at books with "Writing" category and some of the recommendations like "On Writing by Stephen King" are really good and what I would expect. Though I feel that you should work on your priority of order when showing recommended books. For example: when I search for books similar to "The Handmaid's Tale", I don't expect to see "Cat's eye" as a top recommendation. Something to improve upon further.
The Kindle ads are worse. I don't mind at all opening up my Kindle and seeing a book recommendation ... if its at LEAST in the genres I read. You would think Amazon would be on top of it.
How can you recommend books based on only one observation ? Wouldn't it make more sense to ask many books the user liked in order to make a better "user hyperplane" ?
By the way, can I ask how you calculate the recommendations? Your explanation on the site is that it just uses the genre, but that cannot possibly be the whole answer?
I agree. Collaborative filtering should succeed when you have sufficient of data, but we don't always have sufficient data. I think books are a tough challenge: the number of books is relatively large (vs. e.g. the number of restaurants on Yelp), the number of books people read is relatively small (vs. e.g. how many things they buy). Plus, people's tastes in them are relatively particular, and they're a big time commitment, so the bar for what counts as a "good" recommendation is high.
I think in these cases you can do better by bringing in some content-based filtering. I made an experimental book recommender using only story trope tags and I thought the results were already better than what I was getting elsewhere. It's still up at https://bookslikethis.herokuapp.com/ (but it basically only has sci-fi/fantasy titles).
Recommendations seem good for what I checked. One bit of weirdness though, searching for Gödel, Escher, Bach [1] gets me a book by "Agnes F. Vandome", that on googling is some sort of fake book of compiled wikipedia articles, and I need to click on the alternatives to find Hofstadter's book. So it looks like the search system can be successfully spammed with fake titles for reasonably notable books.
I read quite a bit and generally like Goodreads for what it is, but I agree that it's essentially not grown at all in 12 years. There are parts that are broken, there are features that are lacking, and it frustrates me to no end.
Many, many times I've thought of how I'd build a competitor, but it is pointless because Goodread's moat is too big: the integrations with the kindle go a very long way towards cultivating engagement (when you start a book, kindle will update your profile by default, it'll add your rating, mark the book as finished, all through the normal ux of the kindle).
There is no way, in my opinion, to overcome the handicap of needing users to manually update what is automatically updated by the kindle. And I can't see 'linking' accounts working because there'd be no incentive for Amazon not to block access from a competitor. And, frankly, the kindle is the only platform that matters, it likely has 95%+ of the ereader market.
Goodreads is bad because it is a monopoly, and that, frankly, sucks.
I wouldn't be so defeatist! I used Goodreads manually for a bit. Adding books only takes seconds which seems inconsequential compared to how long it takes to read them.
I didn't continue with it because there just wasn't any real value. I don't want a trophy shelf of read books or get automated recommendations by "people who have read similar books" because those titles are very predictable.
I'd need something different e.g. something facillitating deep discussion and Q&A organised by chapter or something. Get authors participating and I'm there 100%.
I've only ever used Goodreads manually, have never owned and have absolutely no interest whatsoever in e-readers. (I trust physical books I can hold and write on, thank you very much!)
I've used it a few times to show other people what I've read and help them get book recommendations or even find a book title again.
I use my Kindle a lot; it's great for travel and I can use it while on an exercise bike or while walking much easier. However, I still only ever use Goodreads manually, and I've marked every book I've read for the past 4 years (at 52+/year). I prefer it manual, and often keep wifi off on my Kindle unless I need to update Scribd downloads.
I only use it manually, and you're right, it doesn't take much to add a book. I did use the integration on the Kindle when I used a Kindle, but I switched to checking out paper books at the library last year, and I haven't missed a beat.
Honestly, I have been feeling that the integration with Kindle, and Amazon in general, is what killed it for me. They took an audience of bibliophiles (and their ratings) and expanded it to the mass market users, encouraging them to star and comment on what they just read.
What I'd really like is more access to the data and filtering mechanisms to try to build recommendations for myself.
Example: Find the highest rated books that I haven't read in the Fantasy category, as rated by an audience of people who have rated at least 7 out of 10 books from a list I provide with a score of 4 or higher, and who have never rated a Twilight book with a 3 or higher.
And if that doesn't work, maybe I tweak it a bit. Build a score threshold for excluding reviews from people based on other criteria about things I don't like. The point is that the data would help make the decisions..
Of course, that won't happen because of privacy concerns. With all the data locked up it means only Amazon can do it, and they're failing at it.
> Find the highest rated books that I haven't read in the Fantasy category, as rated by an audience of people who have rated at least 7 out of 10 books from a list I provide with a score of 4 or higher, and who have never rated a Twilight book with a 3 or higher.
People in many different jobs use Excel and SQL databases - I wonder if enough to make 'power user' but mass market apps with that sort of interface.
I once worked for a startup where we attempted this, in a sense. We built an app that let users take a picture of their bookshelf, we scanned the spines then digitized your "Shelfie". Our pitch was that we would give you the ebook of the print book that you own either for free or at a deep discount.
We built a pretty complex recommendations engine on top of this and quite frankly, it worked really well. Simplified, if you have actual photos of bookshelves it's easy to start building recommendations. Person a read a,b,c, person b also read a,b,c, however, person a read e,f so maybe person b would also like e and f.
The main issue is that publishers are archaic and building a sustainable revenue model against that is a challenge.
>> Goodreads is bad because it is a monopoly, and that, frankly, sucks.
I'd be more disappointed if I had to go to five different sites to see what different segments of my friends were reading. Sort of like...messaging...where I have to use WhatsApp, SMS, FB Messenger, Slack, and GChat because different friends/family are adamant on one particular silo.
I know monopolies are not good, but there are some benefits in scale, especially for a site like this where i'm not paying and i'm not forced into buying bundles/packages I dont want.
XMPP was a solution to all of those problems (except sms), but even the ones who supported it, broke the support one by one (starting with google and facebook).
"one protocol to suit them all" would be a lot harder with books though
> And, frankly, the kindle is the only platform that matters, it likely has 95%+ of the ereader market.
I have trouble finding up-to-date sources, but Googling has some out-of-date data indicating Kindle in the mid-80s for American marketshare and mid-50s for global marketshare.
Personally, I've found Kindle and Kobo ereaders to be broadly equivalent from a hardware perspective. Kindle has better integration with the Amazon ecosystem, while Kobo has better support for DRM-free formats and better integration with local libraries. Ebooks that Amazon and Kobo offer also seem to be broadly equivalent in selection and price. (And Kobo will pricematch Amazon, or anyone else.) Largest different I've noticed are audiobooks that are exclusive to one or the other, with Audible having more of those, but also being more expensive than Kobo's audiobook service.
This seems like a fun side project idea honestly - can I ask what’s most broken and which great features are missing?
I took a quick look and there is a official Goodreads API that you can use to pull titles from people’s accounts (once they authorize your app). Yes, Amazon could shut it down but eh, might be worth the risk
It is worth a try, but where does one get all the book cover images, ISBN etc in bulk?
I remember someone making open source version of IMDB (can't remember the link now), I think I saw it on Patreon. Would be nice to make something similar for books.
One feature I'd love as a user (specifically for non-fiction books) - given a book, show me all books referenced in that book (in the chapter, footnotes etc). I've found quite a few gems, simply by scanning the referenced books list in the books that I like. After all, the author is an expert on the subject, and it is safe to assume he/she would have read tons of books than me on the subject.
Calibre can download metadata and covers from a number of sources, mine lists: Amazon, Big Book Search, Edelweiss, Google, Google Images, OZON.ru, Open Library, and Overdrive.
Of course I'm only searching for one book at a time.
My bouts of wanting to build a competitor usually arise when I'm trying to find something I'd assume would be a feature and isn't, and I'm having a hard time remembering everything, but:
1) I desperately want a feature that'll show me the books most-reviewed among my friends, or highest reviewed with >X # of reviews. You can see what individual friends have reviewed, but there is nothing I've found to aggregate.
2) More reading stats (right now it shows number of books, and total pages at the end of the year). I wanna see breakdowns of categories, authors, fiction/nonfiction.
3) I had an idea for per-book or per-series wikis that have spoiler-bracketed info. E.g., [spoiler b2p300 "King Soandso is murdered by Assassin Soandso"] would show in a book wiki if you are past book 2, page 300. I read a bit of fantasy and have a tendency to want to google 'who is x', but that is fraught with spoilers. This is a feature that could exist as its own site, but I think it'd be a good companion to a goodreads-esque site.
4) Book Discovery seems super weak on Goodreads and I feel I could build something significantly more useful.
> This seems like a fun side project idea honestly - can I ask what’s most broken and which great features are missing?
It's basically just the momentum it has, alongside with crowdfunded book directory (you don't have to enter the metadata manually in like 99.999% of the cases). I'm not gonna say "don't bother", but I've personally used three Goodreads alternatives over the years and nobody mentioned a single one of them yet. I've settled on BookDigits[0], which is what Goodreads should be if you ask me.
> Yes, Amazon could shut it down but eh, might be worth the risk
Technically API isn't the only way users could extract their data from Goodreads. There's also export/import tool that exports data in CSV, and there's also RSS feed that you could ask for.
If there's one good thing I can say about Goodreads, it's that it's not a walled garden.
All your points are true - but I think using the API to essentially just create a “better front end” for Goodreads would be the the new idea here because of that momentum. You wouldn’t need to add your books or friends on this new app, you would just login with Goodreads and it would all be there. It could also two way sync so even if your friends want to keep using goodreads, any ratings you put on this new app would still show up on your goodreads profile.
The reason “better frontend” is in quotes is because it wouldn’t stop at just making the UI/UX better, but also do better search using Algolia, a better book recommendation engine, etc. but there’s no reason it couldn’t just use the goodreads api as essentially the backend for user profiles and ratings
Obooquity and Calibre can both be made to do some of this via LazyLibrarian. It’s giving you a nice local library rather than recommendations, but it’s pulling the wanted/read info from Goodreads into LazyLibrarian. It is a good tool for curating a library but is also pretty clearly aimed at pirating ebooks.
I don't think this is actually working world wide, my Good Reads account isn't aware of anything I purchase via my Kindle, let alone share my reviews or reading status.
If you can’t bootstrap engagement with the Kindle integrations, maybe you could bootstrap community/engagement by integrating with public library catalogs and checkout records? Patrons could opt in to sharing their activity automatically and the library website could have update options as part of the checkout management process. Might be a nice little niche entrance into a customer base there.
Our local library's website is based on a platform called BiblioCommons that actually provides a lot of Goodreads' features (including some off the author's wishlist). I can organize all media in their catalogs into various bookshelves, privately or publicly, and add ratings and comments to each.
Amazingly though, this system also doesn't integrate with checkout/-in records, but this seems like a small, easily-fixable feature gap.
I don't think its all that bad. My only gripe with Goodreads is the lack of a proper tagging system, something like NovelUpdates `Release Filtering` which uses tags and genres in addition to categories. My point is Goodreads is certainly good for getting a feel of the book with its blurb, but not so good for discovering new books via filtering.
> the integrations with the kindle go a very long way towards cultivating engagement (when you start a book, kindle will update your profile by default, it'll add your rating, mark the book as finished, all through the normal ux of the kindle
It depends to which Amazon Store your Kindle is linked. I've got the Kindle Oasis 1 and mine still doesn't have Goodreads enabled because it's linked to amazon.de
That first graphic with failed title search results reminded me of a project I did working on a library catalog search using Solr.
We tuned the relevancy ranking to work for exactly those sorts of searches, and put the things the OP was looking for on top (or at least under other books with exact same title). His examples look just like some of our QA searches for our relevancy ranking (in addition to standard tf/idf: boost adjacent words matching higher than non-adjacent or out of order; boost match of complete title higher than partial title; boost match before the subtitle colon more than after; boost title matches more than author more than other fields; etc).
And this was ~5 years ago, and this was an underfunded university library IT department where the development team consisted basically of me, just using stock Solr for relevancy ranking (the tuning came in how we constructed and boosted our indexed fields). No fancy machine learning, just configuring fields and boosts deterministically.
So, this could be done, if they cared about the site at all.
[In fact, hey, i can give you actual examples from that project. Turns out there are a lot of books called "the confession" -- we didn't do anything fancy to try to guess which one more readers would be looking for.
I build search engines for a living. While I appreciate the hacker spirit of your project with Solr, I also see this as a huge problem that leads to bad search experiences. Tuning boosts in Solr is not even close to a reasonable way to solve problems like this. Arguably not even for an underfunded library, but certainly not for a high web traffic consumer website.
For one, you need disciplined acceptance criteria in the form of both qualitative standards (things a non-technical manager can look at and say yes or no) as well as various relevancy measurements like mean reciprocal rank and normalized discounted cumulative gain (via acquiring human annotated data if needed).
When people only focus on qualitative feedback on top of boosts and hacks in an off the shelf tool, they usually end up with some weird witches’ brew of bizarre boosts and time-decay weighting that is extremely fragile and can’t be robustly changed or even understood without the qualitative performance going haywire. You need disciplined study of quantitative ranking metrics to know the drivers of performance, fall off as you move down the ranking position, and to make search index updates reproducible and make incremental improvement measurable.
Meanwhile if you only focus on quantitative metrics, you might miss obvious red flags. The relevance score used for NDCG might be biased some way. You might surface highly relevant results to only one context or sense of the words in a query (like only showing fruit for “apple” and never tech gadgets). You need people who make the subjective appraisal of quality for users to be looped in.
Here’s the point. When this is all missing, you will lose credibility with people making the decisions. They’ll hear some engineer babble about NDCG but then say the darn thing doesn’t work in the QA testing. Or they’ll say the qualitative results look OK and get angry when weird counter-examples pop up in the second or third results pages, which might have been measured with quantitative metrics.
When this happens, executives and managers just want to punt. They want the “nobody ever got fired for buying IBM” equivalent for search, and that’s how you end up with Confluence still only supporting exact title matching and having no ability for actual content relevancy search.
In this sense, the little projects showing “look what us non-specialists could cook up by hacking some boosts in Solr!” do a lot of harm and should not be considered as the plucky success stories they are often painted to be.
But it works? If you measure your tuning based on which rank an item is when a user clicks you might get feedback that would obviously help find more examples or create some kind of self-learning system, but if you only care about say, boosting common best sellers periodically, well, isn’t that good for (almost) everyone? ;-) My suggestion for GoodReads would be to rank exact title matches higher than partial or reordered titles, but that would just be a start. Of course, evaluate what’s working and what isn’t, and maybe do test cases where you see if lower rankings have improved. But any attempt at a fix is likely better than doing nothing, as you’re likely to spend energy fixing popular books folks want to find...
This is the big red flag, when non-specialists hacking on Solr boosts are claiming something works because of a few qualitative test cases.
“It works” is a statement that only applies after you’ve done qualitative and quantitative goodness of fit testing.
You wouldn’t have a random IT employee make a stock-trading algorithm and then test it on a month of data and call it a success.
For a search solution to “work,” it needs to pass quantitative and qualitative checking, and be explainable to stakeholders and be reproducible / incrementally updateable. The training and arrival at hyperparameters all need to be reproducible and based on the outcome criteria they are meant to solve.
Making some hacks into a demo that superficially looks good is not at all the same as “it works.”
I wouldn’t do this with stock trading because I’m risking everything. But I wouldn’t call the people adjusting search engine parameters completely untrained either, simply not using a methodology that tests their changes against every query. I’ve found that for libraries, at least, the search engines folks are used to are simply SO BAD, so unoptimized, that a little hand tweaking and prioritizing of exact title matches will go a long way. And you’re confusing manual testing with no testing—they would very carefully watch for counter examples with a list of known good titles to search for and get back an expected set of results and were known to rollback changes when they had unexpected consequences. Effectively they have the risk appetite to test in production because the cost to end users is minimal, and the assumption when search doesn’t work is, “oh, they must not have that book” or “oh, they need to fix this particular search” and not “oh, they broke search completely and must be fired” (no one says that last one)
> When this happens, executives and managers just want to punt. They want the “nobody ever got fired for buying IBM” equivalent for search, and that’s how you end up with Confluence still only supporting exact title matching and having no ability for actual content relevancy search
This whole post was a wild ride. But out of curiosity, have you considered walking up to Atlassian and saying “pay me $1MM a year and I’ll solve this problem for you”?
Obviously Atlassian would have to think a high-quality search would result in $1MM of additional profits for them. Judging by what Atlassian is actually doing... they apparently don't think a search better than they've got is in fact necessary for their profits or a good investment that will be returned in profits, right?
I hear you, but I firmly believe the search we did there works far better than Goodreads as outlined in the OP. And could be proven as such with the kind of formalized evaluation you suggest (I am no longer there though).
I agree that underfunded "DIY" enterprise software projects that are not properly/professionally managed/implemented with the proper expertise are a problem, in academic libraries and elsewhere, for search projects and other things.
I still don't see the problem of setting up solr indexed fields and boosts to ensure that "match as phrase" is boosted higher than non-adjacent matches (a feature built into Solr), and "match _complete_ title" is boosted highest of all. This is what the Goodreads examples failed on. It is pretty simple to set up, and I don't see much risk of this causing problems or being worse than not doing it, and would solve those horrible Goodreads results specifically.
I understand since it's what you specialize in, you see the risk of "look at what us non-specialists can set up" sending someone away from... actually I'm not sure what, hiring someone like you? (Which if I were in charge of an academic library budget, which I'm not, I'd be wiling to consider -- don't get me wrong it's not a terrible idea!). In reality, I think what it steers people away from is... a relevancy search like Goodreads has. (Goodreads is _not_ a "plucky little project", and apparently they think their horrible search is good enough! It is not! And I do think they could make it a LOT better pretty easily without having to spend millions on it).
You seem to suggest that products like Solr or ElasticSearch should not be used/configured except by people as speicalized as you backed by relatively expensive search evaluation programs. While I'm sure that would result in better searches everywhere (not being sarcastic, I fully accept that), I think it's unrealistic. If you convince people their only choices are Solr/ElasticSearch/postgres-full-text out of the box using only one indexed field with no configuration for relevance tuning; no search at all; or hiring you or an equivalently expensive search program internal or external -- you're not going to get the expensive search program you want, and you're definitely not going to get "okay, we just wont' have a search at all then", you're going to get people not touching the configuration at all, and ending up with Goodreads search.
Your search really doesn't have to be as bad as Goodreads is, without having to invest in the kind of program and expertise you are suggesting, I really believe that and stand by it. If you invest in what you are suggesting, certainly it will be even better.
(PS: If you have disciplined acceptance criteria combined with qualitative feedback from experts etc -- aren't you still gonna end up tuning your Solr configuration to achieve improvement on those evaluations? I'm confused by your suggestion that turning Solr configuration with boosts etc is not the right tool. Or are you suggesting Solr is the wrong tool for... search?)
OMG memories of when the reg went from the XT to Micron catalog! Trust me, research library patrons would have revolted had you not done your job admirably as you had ;)
1) I like about 75% of the recommendations I get, which is pretty good compared to other recommendation sites I've used.
2) I love the fact that Goodreads has an old-fashioned feeling. Modernizing it would mean a front-end JS framework - making it buggy and obnoxiously slow to use on my old laptop or phone - and a lot of aesthetic features that reduce usability. Oh, and probably light-grey text on a slightly-lighter-grey background, making it a strain on my eyes. Modern internet is meant to look good, pop on a resume, and have little or no functionality or substance.
I genuinely hope the people at Goodreads ignore this article.
I use Goodreads the way I use IMDB. To keep track of what I've read and what I want to read. I can't say I'm missing any features in that department. Not everything has to fancy. Search generally works well, and they have a pretty handy way of grouping books in a series -- which has led me to discover some that I'd missed.
The entire reading ecosystem at Amazon is stagnant. The kindle e-readers have had release after release of subpar hardware with features introduced by now-killed competitors a decade ago.
Read reviews of the brand new Oasis, a $300 product featuring a micro USB port.
Amazon does not care about readers, probably because they drive so few dollars and the market is won.
Reading does not drive prime subscriptions, and anyone with decent product management ability and influence is working on something more important.
At this point, Amazon is treating Kindle like Texas Instruments is treating their graphing calculators. Both have entrenched monopolies and have no need whatsoever to improve their product, which being able to sell them at ridiculous margin. It's laughable how much a new Kindle costs the consumer vs how much it actually costs to manufacture. If this were a non-monopolistic market then many competitors would be leaping into the fray with superior products at a fraction of the cost, but because Amazon isn't forced to grant them access to the Kindle store, it's not even possible to compete.
This is a clear example of an area where consumers are suffering from a monopolistic grip over the market.
I think some onus is on Apple, the only other player that could deliver hardware and has A complete store for books.
I understand it that they would not release any ink devices that fragment their product lineup beyond iPhone and iPad.
However, Apple‘s lack of action in providing quality reading experiences that reduce eyestrain and effectively Compete means that everyone has to deal with Amazon’s garbage.
I hate to say it but I think reading is a niche market. Not enough people do it, a lot more people want to watch YouTube then read Cormac McCarthy.
Reading is decidedly not niche in the US. Books (across all formats) are a huuuge market. They're roughly 2/3rds the size of the entire videogaming industry in this country, by revenue. The total annual revenue of the US book publishing industry is greater than YouTube's (but just barely).
To be fair, even if your claims about the market are true, books could still be niche. The reason is that books cost $20 and take a few hours to read, so let's conservatively say publishers are making $5/hr of reading time. YouTube is making pennies per hour of video watch time. So if it's true that the revenue of the book publishing industry is approximately equal to that of YouTube, then the average person is spending vastly more time watching YouTube than reading books. In comparison, books are still likely to be incredibly niche.
It's a flat-out monopoly. Amazon has a strangehold on the market. If you only had to compete on the hardware, then you'd see competitors. But you need to compete with the entire market ecosystem, which is very hard.
It's like the mobile phone industry, except instead of two unassailable players there's only a single one.
Honestly I wouldn't even mind that, but they are making kindles worse as generations go by.
First they removed the page turn buttons, then they took a small step forward by releasing the Voyage which had pseudo-buttons (you could squeeze the sides of the device and it'd vibrate to give haptic feedback) so at least you didn't have to use a touch screen to turn pages, but then they released the Oasis which has a weird as hell non-symmetrical form factor which means you have to flip the device when you change the hand you are holding it with if you want to use the buttons.
Being able to push buttons on either side of the device to turn pages was one of the best "features" a kindle had over a conventional book for me. Not having it just feels like such a huge step back
Monochrome e-ink displays are quite a mature technology at this point, and aren't very expensive. You can go on Alibaba for quotes on bulk rates for e-ink displays.
I only read DRM-free books, and I just use a 5-year-old Kindle Paperwhite. You can use Calibre to convert ePub to mobi, and then view it just fine on Kindles.
You can also get a used Kindle Paperwhite for pretty darn cheap. I wouldn't recommend going to earlier Kindle generations prior to the Paperwhite, because they didn't have backlights which makes reading in low light conditions (like bed) a lot less pleasurable.
I already have a Paperwhite and it's quite good, but the plastic degraded and is now sticky, the CPU is slow and doing anything takes a while, and it doesn't have the very handy buttons earlier versions used to have. I was wondering if there was anything available nowadays that was much better, basically...
Oh, hrm, I have no idea. My Paperwhite is in a better condition than yours and it's good enough for me.
I really wouldn't mind having physical next/prev page buttons, though. Having to touch or swipe the screen is kind of annoying. I feel like that's a step back in UI from previous gen devices.
Yeah, I have no idea why they removed them. The swipe combined with the high response time means I never know if it registered the swipe and always go back and forth in an effort to change pages.
They probably saved some trivial amount of money on manufacturing costs by removing physical switches, seeing as how they were implementing touchscreens anyway for the rest of their interface.
Smartphones initially had physical buttons that have gone by the wayside now, but at least their screens are much more responsive!
For good ereaders there's Kindle and Kobo. Kobo supports DRM-free epub, but in practical terms, it doesn't matter a whole lot with Calibre to organize your library and convert to your format of choice.
> The entire reading ecosystem at Amazon is stagnant
They are actively malignant at this point. They are breaking embargoes now[1], and clearly won't be satisfied until they kill traditional publishing.
I dropped Prime last year, and it was substantially easier than I thought it would be.
Since then, they've just kept doing disgusting, grotesque crap[2]. At this point, I won't buy anything from them. The whole site reminds me of that cleaning-product smell at Walmart - any time I'm exposed to it, I just want to leave.
That orangey-yellow color means a fight with an untrustworthy, grubby greedhead robot that consistently fucks up delivery and makes me deal with them some more. Who needs that?
Yes - Goodreads is pretty awful when it could be so much better. The searching is awful, the comment UI is grim and the suggested books lists are poor. The most useful tool I've found for finding new authors is this: https://www.literature-map.com/
I only use it for keeping track of what I've read and to look up negative reviews to see whether a book is likely to be one I won't like.
The scoring system is almost useless, as certain categories of books just get endless 5 star reviews from fans.
I'm not impressed. Philip K. Dick next to Stanisław Lem. The two lived in the same period, and there's a funny anecdote about them (Philip believed Stanisław was invented by communists to trick him, because no single person could be so resourceful. Consequently, Philip K. Dick refused to communicate with Stanisław Lem). But their literature is very different.
Dick's stories are typically short and easy to digest, with fast action. Lem's is ponderous, filled with long sentences and complex ideas. Little is happening.
Lem and Dick shared a penchant for framing abstract, deeply profound questions about society in the form of concrete science fiction stories. This is not common. Space opera has always dominated the genre, but neither of them spent time writing "bildungsromans with lasers". So, I'd call the connection is quite valid, having read both of them widely.
> I am looking for Ronan Farrow’s ‘Catch and Kill.’ It appears, inexplicably, after two books not titled ‘Catch and Kill.’
This bothers me to no end in Google maps search results. It seems like every time I'm searching for some chain> the first two results are for stores that are hundreds of miles away and in towns that I've never visited (which Google should also know!). The one that's a couple miles away / the one that I've been to (and mapped to) a dozen times? Third or fourth on the list. Insanity.
In both cases in the article it's almost certainly down to "stop word" removal.
The titles contain "the" and "and". These kind of words get filtered out because they make indexing incredibly difficult, ie. you search for "the" and get every single book containing "the" in the title (and sometimes description, depending on how deep the search is). So they exclude extremely common words to get eg. "catch kill". Then they have to rank the results for "catch kill" by some other heuristic - like popularity - and you get a list of books that people weren't looking for. Not to mention the first two books in the list were actually exact matches, so maybe they were more popular on the site.
Big search engines do try to correct for this, but it's not an easy problem. Usually results in some hard-coded exceptions for names like "The Who", but it's not trivially scalable.
In the case of your google maps results, I imagine it'll be something like (popularity * distance), which isn't necessarily the best heuristic.
The problem is that you can't please everyone with heuristics. Give power users the option to adjust the search to distance-first, popularity-first or any other of a dozen different combos and you end up with a confusing mess of a UI. Simplify it and give them your nicely hand-crafted best guess and there will always be someone complaining...
In Google Maps, I go “Home” every second trip. Yet, when I type “Home” (or whatever the French equivalent is), the only suggestions I get are “Home depot” and “Homeware shops”. I went as far as setting a “place” labelled “fghjk”. I still get other results, despite using “Fghjk” every second trip!!!
Also visible in Youtube. There’s a French guy who explained what it was to be raped by a woman, and I often refer people to it (and half a dozen other videos). When I search for his video’s title and his channel name, I systematically get 2 other results first, generally videos about how many women get raped, from the Huffington Post, often the ones with fewer views.
It really feels like Google is trying hard to dodge the correct answer.
I've noticed YouTube's been propping up mainstream media for sensitive subjects. Some of these MSM videos have hundreds of views where those with millions from independent creators are buried on page four.
I think this is to combat fake news but we all know CNN and Fox News all make stuff up from time to time to fit an agenda so they're hardly much better.
I don't know which came first, but that does appear to work in the same way. (Notwithstanding that you can't navigate to "Home" on the Android widget without enabling location history, but inputting the address manually seems to work fine.)
This seem like it should work around the poor "Home" search the previous poster was referring to.
You should see my library's search results. Trying to find People magazine (to enter a new copy) by searching for the exact title "People" means having to go half way down the second page of results. None of the previous dozens of results have "people" in the title at all.
Well, boo. Sounds like a terrible redesign is incoming for Goodreads. Personally, I like that it's a list-making app with minimal social aspects. Social networks are terrible. I don't want to "dish about the latest author gossip". I don't want algorithmic recommendations that assume I'll want to read another book that has 90% of the same keywords as the book I just read. Goodreads works fine for me.
https://www.librarything.com/ is always better for finding books that I know part of the title (or sometimes even just the full title).
I would like to be able to weight opinions (both positively and negatively) of individuals, and also based on criteria (e.g. "completely ignore the opinions of anyone who gave a low rating to The Good Soldier Svejk" and also "weight by 100 the opinions of anyone that rates The Silver Chair as the best of the Narnia stories".
The recommendation system is broken, however the social networking features can surmount it.
For example, my Goodreads "friends" are people who like the books I like. There's no social obligation. FFS, I unfriended my sister. My high-school friend who only reads YA fiction: he was unfriended years ago.
My favorite goodreads friends are a half-dozen people I've never even met, but I agree with their reviews. When they give a book 5 stars, I check it out.
That's the key to Goodreads. If Amazon can't figure out how to make better recommendations, they should look long and hard at how the social graph beats the star-ratings.
Agreed. This is how I use Goodreads for recommendations as well. There's the "Compare Books" tool to get a quick sense of whether someone reads what you read. And then reading a couple of their reviews is usually enough to get a sense of whether their recs will agree with you.
It's more labor intensive than a standard recommendation system, but the results are better. Since books are a large time commitment, and people are passionate about them, a lot of people are willing to put uncommon effort into finding good ones.
I don't know what people expect , you're going to have to do some digging to find the gems, like everything else in life. There no magical algorithm that can't be gamed or eventually skewed to be bland, boring, sales driven, and altogether less creatively interesting.
If you want real discussion nothing really beats the classic forum bulletin board.
We set out to build a better Goodreads a while back called Helloreads (my wife thinks we didn’t differentiate our name enough but whatever). It never seemed to get traction probably due to the monopoly and network effect of Goodreads. We thought and still do that Goodreads user interface is terrible, the site is slow, the reviews are too volatile. The one thing that we truly could not compete with them on is the data. The book data problem is insanely hard. New books come out and there is not authoritative database of book metadata. We had a blast building (and the iOS app) it but have put the development on hold.
I guess it's a social network, but since this is hacker news — I'd love if your front page would not be just a login screen.
The front page should say something about specific books, something that draws you in! Something from the site itself. And put that book recommendations button higher up (it's not visible on first view for me). :-)
Something that makes you begin browsing and using the site to get recommendations. Then in standard style, somewhere down the line you'll hit a snag where you have to register to use certain features, like commenting, liking, etc.
But I think your right, should open it up a bit on the main page. You don’t need to log in but I think your right, it could be more appealing on the home page. If enough interest, we may fire up the development again and see where it goes now that we know people other than us are also frustratedly with Goodreads.
I didn't intend to say the site should be inspired by HN, and that's not what I mean. I just wanted to say, since this is HN, here comes the drive by well intentioned but shallow critique. Good luck with the site. I hope it gets a more engaging first experience, and I don't hope it ends up looking like HN ;)
Interesting, just out of curiosity - does the website still work well? I just signed up with a dummy account and the images for the books are not showing up. I belive this might have something to do with AWS. (It might also be my browser settings)
They haven't built any new features for years. I just keep waiting for any meaningful privacy controls, I want to keep track of what I've read and what I thought about the book without everyone seeing that. Seeing a cover just doesn't do it any more if it's your 50th space Sci-Fi.
Keeping track if I've read the entire series or not is another feature I really miss for books that trakt.tv provides for shows.
"What Goodreads is good for is keeping your own list of books you want to read or have read this year. It’s a list-making app."
That's pretty much all I want from it, so it works great for me. The recommendations are often (but not always) good enough. I usually don't read based on recommendations from GR but occassionaly something good does come out.
Websites don't need to keep adding features, it's fine to look the same 12 years down the road. I think this website (HN) is a pretty good example of that.
I don't know about the quality of Goodreads but I don't see a big problem with her two search examples.
In the first one she searches for "the confession" and another "the confession" title is shown, together with other way more popular titles with "confession" in their titles. I'm guessing most people want fuzzy google-like searches but sure, exact matches of more obscure books can be given more relevance or search options can be given.
The second example is even worse; she searches for "title" and complaints that a book with "title" and one with "title:subtitle" are ranked higher than her "title:some other subtitle"
The problem with Goodreads’ search is definitely that it mostly only works with full titles. It’s aggravating to have to type in the full title of a popular book for it to have to be the top result; it seems like a really basic search implementation.
The article doesn't mention my biggest pet peeve about Goodreads: it makes you type reviews in HTML. It's 2019, how hard is a WYSIWYG editor? Or at least accept Markdown!
One more complex feature I'd love from an ideal-world Goodreads is ephemeral book clubs. Find people who want to read this specific book, read it together with them, and then disband.
I mean yeah you can do it on GR with a group as well, but I'm talking about a built-in book club feature with a timetable, chat, status updates, etc. Say I'm looking to read War and Peace this winter. I find a bunch of other like-minded people, and we set a timetable, read it together, then disband.
A timetable? On reddit the thread starter mentions when the next discussion is going to take place. The timetable reduces friction, but it doesn’t provide that much additional value.
Fair point. But I looked on Reddit and couldn't find the kind of thing I'm talking about (r/bookclub chooses one book a month by vote, and r/ReadingGroup seems like an overwhelming sea of random posts). I think also the fact that it contains clubs for many different media is a bit of a handicap, since the book content gets swamped, and there's a searchability problem. (Reddit's UI isn't great at the best of times, anyway.) GR has a built-in audience of a wide range of readers, so they're better positioned to do this kind of thing.
A feature I want on goodreads is "for me and a friend, show me the rarest book we have read in common". It would be great to find connections between people that would be hard to discover otherwise.
Same thing for groups - it's fun to try to find the "rarest common denominator" in the books, movies, or places visited domain.
Amazon has something somewhat related long ago called "Someone like you" that would show you books purchased by people who had similar previous reading purchases. It was odd to see someone had purchased the same combination of OpenGL books, cookbooks, and sociopolitical books as me. The recommendations that resulted were pretty good but I suspect privacy concerns ended it.
I'm not a Goodreads user, but this is something you can definitely do with Librarything. For any user in the system with a public collection, you can find the intersection of your books and sort by the number of members who own them (as well many other things). The interface is a little wonky, but very powerful.
> for me and a friend, show me the rarest book we have read in common
Got to your Friends page (https://www.goodreads.com/friend), click on "Compare books" in front on the friend with which you want to compare. You will see the list of books you have in common sorted by inverse popularity.
On a related note - what happened to the Netflix recommendation engine after they awarded the $1M prize for a 10% improvement to their already pretty good recommender? Did they just throw the whole thing out the window?
I can’t remember all the details, but from what I remember when I looked this up a few years ago, yes, Netflix did throw the whole thing out the window. They said that the winning algorithm would require more processing power to run, or expensive code changes or something, and so they decided their current engine was good enough.
Then Netflix switched strategies away from highly-personalized recommendations to a dumber algorithm that is more likely to recommend more popular movies, which ends up retaining subscribers better. The reasoning was that users were more likely to cancel their subscription if they had one obviously bad recommendation than if they had multiple so-so recommendations. So rather than recommending, for example, 80% great movies and 20% bad ones, Netflix wants to recommend 5% great movies and 95% okay ones.
I cannot believe they have any kind of recommendation engine? I just see offered more seasons the same shows I watched, or shows with similar words in the title regardless of topic. I cannot believe they have even 10% success rate recommending movies, much less improved it by that much.
What sites aren't badly broken? I just spent 30 minutes wrestling with Booking.com. I have many issues with Amazon, and Google keep changing their services under my feet.
Not to mention all the ways they want to shape the experience that I don't want, with their useless and biased suggestions being one that sticks out for me, as well as their user communities that I'm pretty sure exist to denigrate individual perspectives, since they aren't designed to let people organize and be critical, just a noisy crowd with grossly aggregated ratings.
But these big companies don't care. They dominate in their areas. Their best, only reasonable option is to drag their feet on consumer oriented features as much as possible, to save money and spread out behind the scenes. If a competitor comes along with a unique feature, they can just add it and destroy them. It's impossible for the consumer and innovation.
As we go to very high levels of integration, where our activities, connections and other personal data are involved in every decision, the only solution is to separate data and services, so I can access the data I want, process it in my own system that for any activity can be more comprehensive than they can reasonably provide — unless they all, either via backend "cooperation" or duplication have an incredibly privacy invading profile of me — and finally process it with their service. This is what Solid proposes. It's a long-term, multi perspective, standards based project, and I don't know any reasonable alternative.
Goodreads has possibly the worst search algorithm in existence today. Typing a character or two will sometimes show the book I am looking for right on top, and then typing a few more reduces the relevance of the results and hides it. I don't get it.
It astounds me that after 20+ years of Amazon, nobody's come close to creating a good way to discover books. Amazon.com is particularly bad at it, but browsing the shelves of a library is still a better experience than what I can get from Goodreads.
The single thing that bothers me most about goodreads is that they’re owned by a bookseller, the have a list of 200 books I’ve read, and every month they email me promoting romance novels, bestsellers and books about america culture. The one I just checked includes an advertisement for dresses (I’m male) If they would literally just list new releases by authors I’ve read it would be 100% more useful.
Not exactly. The moneyed interest is in driving dollars to the things that they invested heavily in, and need to get that money back. They paid big bucks for a book they expect to be a bestseller, they want attention on that book. So there’s no incentive to drive attention to the little books.
If you're looking for recs, check out librarything. In general it's worse than goodreads on almost all dimensions, but their recommendations algorithms are great. You can import your goodreads list, so it's no big ordeal.
I actually always thought librarything was strictly better than good reads as a list building/management tool. It has suffered from a similar lack of improvements recently, and I was never impressed with their "social" features, but personally wouldn't agree on the "all dimensions" part.
My wife and I experienced the same disappointments on goodreads, and many of the same flaws exist in other media rating/cataloging sites. We're attempting to solve this problem by creating a centralized collaborative media database that we always wished existed. Without any marketing yet, https://rate.house currently only has ~100 users, but we'd love to hear HN's input and know if we're heading in the right direction.
I think you've got a really nice design going on already, and basic features seem to work without JS, so that's nice. I also like that you haven't polluted it with ads (I'm far more likely to donate to sites that don't have ads).
Two thoughts:
1. I think I would find a 5 star rating system too constraining. My opinions in most areas have more gradations than just 0-5. 0-100 would be entirely too many. I think IMDB's 10 point scale is really the main reason why I've stuck with that for movies. On the other hand I prefer to only thumbs up / thumbs down books, so I'm really not sure you can do anything about this in a way that would make everyone happy.
2. I think there's either a bug with your weighted average or else you're ranking using something like "we're 90% confident the true rating is at least x". But this confidence level seems entirely too high for the number of users you have on your site. For example Avengers: Endgame appears on #13 on this page: https://rate.house/chart/movie despite the fact that it has a rating of only 3.64. I would probably lower the confidence required for now, and if you 10x or 100x your users you can raise it again.
3. Would be nice to know a bit more about what features it has on the home page before signing up. Can I import my ratings from other sites? Can I export my data in some usable format like CSV? Can the information database the users create for media entries be downloaded by users? (Even IMDB offers this.) Can I get recommendations from the site once I've rated enough items? Can I get music recommendations based on my movie ratings? (That would be cool.)
Barnes and Noble launched Browsery a little while back, that helped me find a litany of new recommendations. I now use that service in tandem with Goodreads, but GR is just my tracking list at this point since it integrates with my Kindle and migrating to something else would be a pain.
I have been using Goodreads for years. I remember when Amazon turned off API access to goodreads and how all books lost their covers and most of their data. And the shocking turn of event of Amazon taking control of Goodreads. And I am pretty happy that it hasn't evolved that much from the website it used to be.
Concerning custom shelves exclusive from read/want-to-read etc. I do have created such shelves. Not for "Did not finish" list but to have 2 kinds of want to read: books on my wishlist and books I own but have not yet read.
Features goodreads has that most competitors lack: I can have two different editions of the same book. I can store more than one date I read a book. I can export most of my data in csv.
Like the article suggests, I've been using Goodreads strictly as a list-keeping app -- the recommendations are pretty useless. The star ratings, however, are somewhat useful if you know how to read them.
The only major complaint that I have against GR is that it is quite slow.
I don't really care much for recommendations right now, because the user-created lists will often get you what you want faster.
Furthermore, I think GR does a good job of classifying the books, or rather letting their readers classify their books for them.
All in all, I think the site should be more responsive, but there are tonnes of places to have long discussions on books, genres, plots and other things over at reddit.
My list of books I want to read is way too big that I'll never catch up with it. So my problem is not finding new recommendations, it's deciding which book from the list I want to read next. That's what attracted me to Goodreads, because I can download a cab of the list and sort by different fields, etc.
This author and book are in almost every quote category last time I checked “ Criss Jami, Killosophy.” Often multiple times in the same category. He has some ok quotes, but rarely are they the same caliber of others, especially not in every category. Seems the quote area has been gamed too. I contacted goodreads and they didn’t seem to care.
When they had recommendations based on bookshelves it was amazing since I have a bunch of very niche topic bookshelves. Once they removed it and left only recommendations based on genres it is useless now. Kindle book recommendation based on similar books are a bit better, but that's only for kindle books
Anyone else remember CD Now? Human-curated lists based on "if you like this you might like". Because they were actual people who loved and knew music they worked. Amazon wrecked that too.
GoodReads and Craigslist are similar. They serve their original functions sufficiently and that's all. Personally, I like GoodReads a lot, it's the only social site I use in fact.
I wonder if people posting on Medium realize that for many of us the articles are inaccessible ("Sign up for a free Medium account, and you’ll get one more story in your member preview this month"). It seems that people missed the change in Medium's business model and still think that it's a site for posting publicly accessible articles.
To those replying with "dismiss the popup", that does not seem to be an option for everyone. For example, this is what I see: https://imgur.com/a/04W0om0
It's a shame https://telegra.ph doesnt have the ability to save/send yourself an edit link. Being able to only edit your story on the computer you made it on is a big turnoff.
I'm not entirely sure about this, but isn't the edit ability of Telegraph based on what I believe is a cookie, it seems to be called "tph_uuid" and you can see it if you press F12 in Firefox, go to "Network", reload the page, find the single POST and you can see it in the request headers.
Meaning, as long as you don't clear your cache you should be fine, and if you do, maybe you can back it up somewhere.
If you set up a non-standard browsing environment you can get around it, but it's still a trash experience for the rest of us. I no longer follow Medium links at all.
All you need is one plugin, uMatrix, to solve it. Prevent medium from loading cookies and javascript, and you're good. You do need to enable the github gist javascript though to see gists.
Not to justify them or anything but as a workaround to this many many similar awful behaviours, Reader mode on Safari is a handy way to ignore all this.
It does, but if there are images in the post, Reader mode loads blurry images because medium serves blurry previews until you scroll past the position of the image. It’s quite infuriating.
"Read smarter, not harder.
You have one story remaining in your free member preview for this month. Unlock the best of Medium when you upgrade for $5/month."
I don't know what we're doing differently. I always get an annoying pop-up trying to get me to register or to become a member, but there's always a close icon, and I never seem to be blocked from reading.
Yeah I thought it was pretty funny that someone on Medium of all places wrote an article called "X Site is Broken". Medium is the running with Yelp and Scribd for site people complain the most about.
same. Guido is currently writing a series on PEG parsers. but medium won't let me read a single one of these posts. i don't even know what the reasons are. can't be that i reached my free reading quota this fast...
Even as a member this article is paywalled. Nowadays regardless of the authors intention of keeping the article available or not, the default is paywalled and many people miss it.
Please just don't click on Medium posts then. It's that simple.
Repeating the same opinions over and over isn't providing any value at all. People on HN usually read the comments to get additional insights into a certain topic or to share their opinions on the linked content.
Opening the comments of a Medium post on HN is destined to feature at least one person who a) most likely didn't read the actual post and b) feels strongly enough about Medium to share their opinion on the platform the content was published on.
I get it. You dislike Medium. Then create a blog somewhere and post about it there. Don't make HN even more annoying than it already is.
I had the same experience with GR and also Amazon.com which constantly peddles the vampire romance books when I am looking for recommendations for horror/fantasy. Both Amazon and GR strategy make sense because best-selling books sell the best, so they should recommend them to increase profits. However, it does suck being a reader looking for new book suggestions.
I've spent a good deal of time making my own book recommendation algorithm which has been working well for me for the last two years. [1] Through it I've discovered old authors I didn't know (Ted Chiang, Clive Barker) and new authors which I wouldn't have noticed (Scott Hawkins, China Mieville). Of course, it always helps to get recommendations from friends with similar tastes, too. :)
[1]: https://nowwhatdoiread.com