Where does it get the list of books from? I thought that was always the biggest challenge with sites like Goodreads etc. Goodreads both (A) pulls data from Amazon, and (B) has an army of volunteer "librarians" who have additional privileges to merge duplicate books etc.
I was incredibly lucky to work at the Internet Archive the same time as Mouse and couldn't be more proud of their work on BookWyrm.
Open Library and its network of generous volunteers have (I hope) made a lot of positive progress towards cataloging the books that are out there and making them more accessible to the world. AND it's absolutely the case that our project exists to support innovative projects like Bookwyrm and incredible thinkers like Mouse.
Open Library can't and shouldn't be everything. It's hard enough doing well at one thing. The Open Library team is considering how we may be able to participate within the decentralized ecosystem by offering a BookWyrm instance so readers may have more ways to socially engage with each other and connect around books. If you're interested in helping us try this as an experiment, please reach out <mek@archive.org>!
I appreciate how difficult it is to run a service which gives communities voices (it requires moderation tooling, staff, and so much more). I'm impressed by the thoughtful, impressive, and creative work Mouse has done building BookWyrm and am super grateful for its progress which I see as being a win for the entire ecosystem (an ecosystem Open Library is proud to be a piece of).
Keep it up <3
P.S. the fact that many services like Mastodon or BookWyrm may have large primary servers is not a demerit. The fact that there are smaller local servers, that new servers can emerge over time, and that engineering thought is being put into how data moves through such environments is key to acknowledging the importance of creating safe communities, promoting archival strategies, and enabling accessibility. Many people use GitHub (centrally) and also use Git (centrally) and the fact that many common use-cases have been centralized do not undermine the significance of the times where small, high impact cases are able to succeed because decentralization has made them possible.
I have a half-finished project that scans barcodes with the webcam, looks up the information in openlibrary, speaks the tile out-loud and records the output to a file that tellico can import.
Some people might note that tellico can take a list of ISBNs on input, so the extra step of OL lookup seems redundant. A lot of bookstores put their own barcodes (sometimes over the ISBN) and there can be namespace collisions between those barcodes and the ISBN. Looking it up on OL and reading the name out loud lets you catch these issues when scanning. There's nothing worse than scanning hundreds of books and then having to go back through to find the dozen of them that scanned wrong; by reading out-loud the OL information, you can just set them aside immediately and enter the information manually later.
Wow, I just realized Open Library covers like 80% of Goodreads as well: read/to-read lists, ratings, yearly reading goals. Seems like https://openlibrary.org itself would be a fine Goodreads alternative if you don't want to set up your own Bookwyrm instance.
What it sadly is missing is the data. My first try, I found 2 out of 3 books in a series missing, and the first book was missing data. It looks like ImportBot [0] adds them somehow, but it’s not clear how this is done and how it could be fixed as there’s no information on the user page. As much as I’d like an open solution, if I have to add every other book I read to the database first, that will make me stop using it quickly.
Hi, can I ask what books were missing? The more examples we have, the more we can update our bots to make sure these books get either imported or fixed within our search engine.
I completely understand wanting to use a service that has the books you're looking for and would also completely understand if it's too much work to type up the examples. If you'd rather not do so publicly, happy to receive your email at <mek@archive.org> and do what I can to help. Thank you!
Hey, sure: The Series is by Glynn Stewart, Scattered Stars: Evasion, Book 1 Evasion [0]
Here it already only has the printed version, not the main kindle version (his books are KDP, so sadly Amazon-exclusive). It’s also lacking the Series title (Scattered Stars: Evasion), only having the book title "Evasion".
Missing from the series are Discretion (Scattered Stars: Evasion Book 2) [1] and the new Absolution (Scattered Stars: Evasion Book 3) [2].
Looking through his other works, they all seem to only have the print version.
I’m also not sure if the ImportBot [3] is official, or just one huge contributor, but it could really do with some kind of information, including how something like this could be fixed, or if it can.
I have been using the Open Library app for almost a year now. I am very satisfied with it.
After GR removed the ability to add books manually by users and added stringent rules for them to be added to consideration by "librarians", I stopped using GR.
I read a lot of obscure books that often don't have ISBN or an webpage (!). I can't track them or add them! It was so unfair.
So, I moved to OpenLibrary and have been using it ever since.
That's awesome. May I ask, for pre-ISBN books, do you typically look up books by title? What do you do with books when you find them? What is your primary use case / reason? Is it as a reference library (of things to read)? Keeping track of reading?
Yes, by title. And I often find out about them from friends, colleagues, and acquaintances. I also find out about a great deal of books from other books, either mentioned or cited/referred to.
I read some books cover to cover, and some are kept as references. Books are mainly on Math, Philosophy, and History. There are other topics, too.
I read in 4 languages and GR is very Anglo-centric. That's another issue.
I wanted to track my to-read and read for every book that I found.
I cannot do that with GR anymore.
(I am unsure to whether you are asking about my use case about the sites or the books. So I answered both.)
very helpful, thank you. Good to learn international use case is working okay for you (I know we can improve). If you're not on our slack already, feel free to email me @ <mek@archive.org> -- you're welcome to ask questions and weigh so we can continue to try to move in the right direction for you and others.
Copyright law is a plague on contemporary free society. I'm a composer, I work on contemporary Western classical music, and finding written music published since 1923 is a full-time job in itself. It's relatively easy (still not easy) to find extremely famous composers (e.g. Philip Glass, Gyorg Ligeti, Jennifer Higdon etc), but if you're trying to find someone lesser known, or god forbid someone unknown published by some random European music publishing house, good fucking luck. They literally won't make the publication available to you if they decide. Some works are only available for performance (i.e. if you're allowed to play this music, they "rent" the sheet to you) and no option to study the written work for other authors. It's excruciatingly difficult and holds the entire field back multiple, multiple decades. People write about ideas that were already explored and in circulation almost 4 decades ago because it's likely impossible to read their works.
Let me not even begin to talk to you about finding papers (analyses of other artists' works, musicology etc) if you're not affiliated with a university. You need a fortune to keep up with the field, or be arrrg (and it's not easy to find musicology papers in SciHub compared to other fields).
In this aspect we're significantly worse than how things were back in 1800s, 1900s or early 2000s.
> god forbid someone unknown published by some random European music publishing house
One of the most striking developments in the last 20 years is that so many of those European publishers are making a lot of their study scores free to read online. Apparently they have given up trying to make money from ordinary music lovers and are OK with selling just to performers and libraries. Back when I became a huge fan of a somewhat lesser-known European avant-garde composer, I despaired that it would cost many, many thousands of euro to buy the study scores of all his pieces. Now they are right there for free on the publisher’s website.
Otherwise, piracy largely fills the gap, although many composers have some famous piece, the score of which is impossible to ever see. Boulez’s Répons and …explosante-fixe… are my usual examples of this – all the rest of his scores have circulated in pirate circles for well over a decade. Someting like Magnus Lindberg’s KRAFT is probably not easily found because its score is a meter tall and therefore difficult to scan.
Hmm I'll be honest, I always have very difficult time finding both of the options you listed. Not aware of any publishing house that share contemporary sheet music for free, nor do I have a good source for sheet music piracy. For piracy, I try the standard sources such as Library Genesis, Pirate Bay, Google "X PDF" etc... and almost never get something for niche composers in the last 10 to 20 years.
Universal Edition, Edition Wilhelm Hansen (now part of Wise Music Classical, I think) and Boosey & Hawkes are examples of publishers who have put up a lot of scores by 20th century composers for free. The Finnish Music Information Centre website has many scores by Finnish composers of recent decades.
With regard to pirated scores, that mainly happened on filesharing networks like Soulseek and DC++.
> almost never get something for niche composers in the last 10 to 20 years.
Why don’t you just write to them and ask for a PDF? As you know, composers are usually happy to hear of any interest in their music, and (just like academics and their research papers) they don’t always agree with their scores being “behind a paywall” for personal use.
A necessary evil as long as the publishing industry continues to have a choke-hold on copyright law. IA recently lost a lawsuit trying to fight this “digital copies must be lent like physical objects” nonsense.
It's actually a lot worse than that, you're mistaken. IA recently lost a lawsuit that argued that. The court ruled that when a digital copy is shared "a new copy is made" on the computer of the person shared, so you can't even do the common practice of buying 3 physical copies of a book and digitally sharing 3 copies at any given time. It's because if you have 3 physical copies of a book, once you share it digitally 3 times, you already exhausted your privilege and not entitled to those 3 copies any more. It makes absolutely no sense, and makes US Copyright law unreasonably and extremely maximalist, even though US copyright law was already relatively maximalist compared to other countries. We live in the dark ages.
This unfortunately isn't up to them, its the publishers that are enforcing this arbitrary restriction. In fact, IA landed in some preeeety hot water recently for fudging the rules on this very thing [0]. If the IA could get away with handling out an infinite amount of e-books I'm sure they absolutely would.
It fetches data from the public APIs of openlibrary.org and inventaire.io. From my point of view, inventaire is really great as it synchronizes with wikidata and is far better at not duplicating works or authors.
I _think_ it's just things users have added. This is a fediverse app. It's not some centralized list of books that people are working from. It's not limited to English books, or even US books.
Also you can argue that the (original) filename itself is part of the metadata. It's useful to have what it was originally called when referencing elsewhere. So where to put this data? in a sidecar file during the rename process or something? We could just use content hash, but then online dashboards to redownload things from vendors, for example, won't necessarily have that there displayed on the page, or when you start downloading it in the browser.
In case it's helpful for anyone else having the issue: mmv [0] means you only have to run the command once to rename every file. It's pretty fantastic.
99% coverage is achievable via a few straightforward regexes.
Don't get me wrong though, I really like and appreciate Jellyfin, especially on Apple TV with Swiftfin, it's my daily driver for big screen entertainment and it's amazing, 10e9 times better than Chromecasting from a laptop to GoogleTV, which is just a horrible UX (no pause button on the TV) and also would randomly freeze for 5-30 seconds every few minutes.
Plex was nice too, and works great if you are okay with being at the mercy of a closed system for your media center. Though I sure don't miss those pointless forced UI "downgrade in functionality" updates!
Jellyfin parses scene naming conventions fine. I have thousands of films and hundreds of TV shows (with thousands of episodes) all in scene name format and I can think of a handful of matching errors on Jellyfish but it's usually due to a commonish film name and a wrong year or something similar.
the full win would also include 2 instances of readarr, once for ebooks and one for audio books, whisparr for your 18+ needs, stash for your 18+ needs frontend, and I think audiobookshelf and kavita to play the audiobooks and ebooks respectively. If you're into comics you might also want to throw mylar3 in there and if you have multiple users (aka a spouse and/or offspring) you may want to throw ombi in there too.
For Radarr, it's for cleaner separation and easier automation. Also, keep in mind that you cannot have the same movie with two qualities on the same instance.
My 4K instance has one root path (different from non-4K) and one quality profile that's auto-selected by default. Using the Radarr connect option, I have a quality profile on the non-4K instance that automatically adds the movie to the 4K instance. This way, you can ensure you have both 4K and non-4K copies of certain movies (e.g., for external access/transcoding). You also ~never need to actually interact with the 4K instance. See this page for details: https://trash-guides.info/Radarr/Tips/Sync-2-radarr-sonarr/
For Sonarr, it's also for separation, but less of an issue if you're on Sonarr v4. The one thing you gain even on v4 is the ability to have different quality definitions for anime and non-anime content. See: https://trash-guides.info/Sonarr/Sonarr-Quality-Settings-Fil...
There are anime specific trackers that are better for downloading anime from, especially if you're looking for non-English subs, but radarr doesn't have a way to tell it to use a specific tracker for a particular series, so having a separate radarr instance with only those trackers on it ensures it downloads from them everytime.
Pretty sure the 4k thing is a hard limitation due to not being able to select the same root path or something. I don't remember tbh.
For anime first time I've heard that suggestion but anime in general is super annoying to download due to not having real seasons or whatever often. Wonder how it helps..
I have a separate radarr for anime because there's are anime specific trackers that are much better for downloading anime, especially if you are looking for non-English subs, but radarr doesn't have a way to pick a specific tracker to download a particular series from. So my anime radarr instance only has those specific trackers on it so it will download from them everytime.
What might be easier than a whole separate Chinese tracker, Plex at least has a pretty robust feature where you can have it automatically find and download subtitles in whatever language you set. Only problem there is sometimes the timing is a little off with the actual video so the subs don't pop up at the right time, but you can change the offset with just a few clicks until it's right and then usually it's the same for every episode going forward.
If you're not using Plex or find it doesn't work so well for Chinese subs, can also try bazarr which does the same thing with subs but is a standalone software.
So with the above setup you can download the actual series from wherever and then it'll fetch Chinese subs for you and add them automatically. Otherwise for actual Chinese trackers try share.dmhy.org and mikanani.me for public trackers.
Feel free to shoot me an email(in my profile) if you need any advice or help setting anything up :)
With anime people are wayyy picker about release groups because they come part and parcel with fanmade translated subtitles, preference for which track is the default for dual-language releases, are all the honorifics written out, etc.
You either have to commit those preferences to a tag that you try to remember to put on everything (fiddly), or you just commit to them being on everything you download regardless of whether or not it is anime, which can cause weird selections when sonarr is sourcing western media.
Jesus christ. Thank you for the Unpackerr recommendation, that was pissing me off, but jesus christ.
I’d pay an extra $10/mo for my seedbox to have just a single interface for all of this without having to manage all these independent apps. Trying to debug why sonarr->prowlarr->flaresolvrr don’t work is a nightmare. That reminds me:
- flaresolverr: proxy that handles cloudflare bot checks for torrent trackers that are starting to put it up
Haha yeah, it can be definitely become a rabbit hole if you have the interest and time :)
I think the base setup of Radarr + Sonarr + SABnzbd/qBittorrent + Prowlarr is a huge improvemenet over doing things manually. A lot stuff on top of that is helpful, but the benefit vs. effort ratio diminishes quite quickly.
I haven't had to setup Flaresolverr just yet, but might do that soon.
I set up radarr + sonarr after a few years of doing it manually. Really glad I did — adding something from my phone and then having it pop into plex on my tv is delightful.
Readarr is weird. Turned it off after I started get messages from it saying it was deleting my ebooks. I don't know why it can't just work like Sonarr.
Indeed there are probably multiple paths to the full win. I like to use airsonic-advanced to feed music to my legacy sonos systems (can somebody please make an easy to use, pretty opensource alternative to the sonos multiroom software stack that can output to hifiberry's or ideally a cheaper alternative? even better would be an included best at each price level alternative speaker to double sided tape the hifiberry to). I also prefer to use beets and a cron job to convert my whole music collection into decent quality opus files and syncthing to get them onto my phone rather than using a frontend and mobile data to stream the flac files from my house.
Prowlarr is a deitysend. Instead of having to configure indexers in Sonarr, it does the heavy lifting for you. Goes in, configures an indexer, and that's your search setup done.
Prowlarr is also important if you do multiple *arr. Sonarr, Radarr, Readarr. instead of configuring your indexers (multiple) in multiple places? You point the searches at Prowlarr and then keep it updated.
I personally run a fairly modified version HTPC on a QNAP (was synology but upgraded earlier this year - better base board + dedicated graphics card)
I use the full stack and all very grateful for the products.
The remaining annoyances are:
- lack of multi language support
- there is no connection between the systems when you want to remove a movie (you remove it in one place and everything knows about that and acts accordingly)
- I still did not make to fully grasp how and where to say "I do not want this particular release". I think I saw that in radarr but it never is obvious to me where it is.
Yeah I was initially surprised when learning about the *arr stack for the first time, as my intuition was very insistently telling me: "I must be gettint it wrong, these ought to be all a single service!!"
They all definitely feel like small parts of a single package, don't look like they merit being their own thing. But it's not my thing, so what do I know.
I have no idea why Radar and Sonarr are seperate programs. Surely the difference in searching for Movies vs TV is a single line of a query search string.
Honestly I just think the concept of Son/Radarr doesn't translate well to music, I find Lidarr fiddly in general.
In particular I'd add to your list that the overnight scans to update cover art are an absolute mess. It's not so big a deal on my libraries in Sonarr and Radarr, but for Lidarr? Jesus Christ. I have reasonably sized music library (~400/500 gig), and every night Lidarr starts phoning out to check, for every single album and artist, whether the associated cover art or artist image has changed. This takes hours, and is completely unnecessary, and cannot be turned off. I've resorted to just blocking the addresses it does this on, but this breaks things when I try and use it to add new music.
> there is no connection between the systems when you want to remove a movie (you remove it in one place and everything knows about that and acts accordingly)
Not following this. Settings -> Connections in Sonarr for example.
What I meant is that there is no common management of dat. When I make changes in one service, the others do not know it, or know it after the fact (via a reindexing for instance).
Say for instance I delete a movie in Jellyfin. Radarr will then pick up that it is missing and re-upload.
Or that I deleted a torrent in deluge. Radar will restart it.
Most of the things are doable, one just need to know what to do where, exactly (otherwise the other pieces may try to recover).
Right now I am making the chnages in Radarr (despite actually seeing them in Jellyfin). This is a real problem when I have, say, two uploaded versions of a movie and want to keep only one. I have to be very careful to track down in JF what I will remove in Radarr.
It would have been great if there was a common indexing mechanism across all the suite.
What you pointed me are notifiers - they work fine (but are shoot-and-forget kind of services)
As rPi 4Bs are hard to get still / overpriced, unless you’re absolutely married to the rPi, check out OrangePi 5B for a much faster CPU, more RAM, eMMC, and good wifi OR get the 5+ for the same minus wifi (it has an E-key m.2 to add it) but adds a 2280 m.2 so you could throw a 4TB m.2 SSD on there (which are about $200 now) and now you’ve got a pretty dang good little NAS that’s fast and fanless for about $350-400 all-in. Did I mention the 5+ has 2x 2.5G Ethernet?
Not OP, but you pretty much have to run the Celeron ones. I don't think the docker image would work with the Realtek ones (the ones that end in j). I have a DS220+ (J4025 with dual core only). It works ok, you pretty much max out one of the cores running a 4K stream. I would recommend separating the storage and server if you can afford it. The price difference between the quad core (4 bay) and 2 core (has 2 bay) is enough to get a 2 bay + a N95 mini pc that can handle 4 streams of 4K.
Another idea is to get a thin client with an i7 or i9 (you probably want at least 10th gen at this point) and either an external enclosure for a few SSDs or if you find one with a PCIE slot, a PCIE card to fit maybe 4x m.2 SSDs. Some good deals are out there on U.2 SSDs if you look as well, like 8TB for $400 from Intel or WD good.
Don’t forget that Asrock Rack and Supermicro sell Atom and Xeon-D boards, as well as some Ryzen AM4/5 models if you want to DIY. There are great cases out there (eg Fractal Node 304/804) these days that support full-size modular PSUs with 80+ Titanium ratings to sip power. That’s been my biggest gripe with x86 over ARM: idle power usage for something I expect to have on 24x7 with PG&Es 50c/kWh. I just rebuilt my old desktop 5950x into a NAS using a Silverstone RM44 with air cooling, but it’s made to support liquid as well. That’s got plenty of room to fit 4X full-size GPUs and a power supply to match if you dabble with AI on the side. RTX 4060s are coming soon for $300 and that should be more than enough power for transcodes for the whole family.
I actually think buying a first gen Mac Mini M1 is a better idea with a thunderbolt drive storage. I have friends in California that does this for the reasons you mentioned. The utility price has made homelab servers and 3D printing pretty much non viable unless you want to pay a $300+ electricity bill.
He runs Jellyfin straight up from the downloads page [1]. I'm not entirely sure if it runs on Rosetta but he hasn't had any issues with multiple streams. For storage you have a couple options but enabling file sharing on macos + a large drive of your choice is your best bet.
If anyone wants to try this, be wary of transcode acceleration / HW passthru with Docker, esp on M1/M2. In general I’d love it if there was a GUI-less stripped-down MacOS Server edition instead of running the full-fat consumer OS as an always-on server.
MS-DOS was not really what people would today consider to be a "true" kernel, as it didn't implement any sort of access controls, boundaries, or other process management functionality. It was more of a set of shared libraries, and also included some standards such as a file system, and program loader format.
Even Windows 1.0, which more directly sat on top of MS-DOS, had to implement considerable amounts of functionality that would be considered more of a kernel than what MS-DOS offered.
I use "DS Audio", the web/native app that comes with Synology home NAS. It works pretty well. Probably not as well as Plex or Jellyfin. Has anyone used both?
I came here to say this as well. Synology's Video Station app is also decent.
Both of them are surprisingly good in a lot of ways, but also...kind of ugly. Not enough so to not use them, but I wish these two apps had the same UI polish that some of the other first-party Synology apps have.
