Hacker News new | past | comments | ask | show | jobs | submit login
Kindle, ePub, and Amazon’s love of reinventing wheels (hackaday.com)
193 points by lxm on May 18, 2022 | hide | past | favorite | 159 comments



This article brought back memories of my time working in Kindle.

Amazon has a culture of promotion focused “engineering”. This rings true across product management and software development.

Rather than solving real customer facing problems, many organizations including Kindle, emphasize “what do I need to do to get promoted?”

This is not only SDEs, but L7/Senior engineering managers and L8/directors, who latch onto a sales pitch and put all their eggs in one basket.

The template is like this:

> I’m going to build a new framework, platform, format, or some other “thing” that everyone should now use to solve {{problem}}. Then, I can claim a larger scope and significant impact, no matter how pointless, costly, or annoying this thing is to customers.

They write documents. These are narratives where they spend hours and hours reviewing and nit picking how “crisp” a sentence is, or how there are too many commas, or some other superfluous bullshit.

There’s an OP1 document, where an L7 writes a fancy narrative, justify more head count and growing their organization. It’s their ticket to a director promotion.

If you’re a developer, you flock to the same project. For L4s and L5s, it’s your ticket to the next level. For an L6, you spend your time selling the absolute hell out of this thing, going on a road show and pitching your idea to get buy in from other teams. You’re careful not to write much of the code and mainly come off as being the architect and consultant. This is your ticket to getting to the Principal level. If this is someone else’s idea, you’ll do whatever it takes to take credit when this succeeds but blame everyone else when it fails.

Best case, the thing you deliver gets you promoted. Then, you simply just move onto to another organization. You were in Kindle and this new piece of shit has made on call hell? Oh well, you already got your promotion. Now you can just move on to AWS (or find another team in any of Amazons legacy businesses where you can coast).

Kindle was an army of H1Bs with an extremely toxic culture. They refuse to hire or even interview white, black, or Hispanic candidates. You can pay H1Bs less, and they’re happy with it because they get to live in Redmond, WA instead of India.

I could go on and on, but I feel anxiety right now even thinking about my time working there. Absolute shithole.


My impression from working at Google is that people are similarly promoted based on implementing new features rather than maintaining existing features, including bug fixes. It's been 6 years now since I left, but I can still tell when there's a promo cycle because the UI on my Android apps change arbitrarily and become slightly buggier.

Just to rant about both companies, Amazon Prime's Chromecast support seems to be currently bugged out. It plays for a few seconds then gives up with a content not available error. Maybe it will work again in a year?


Prime video has always been extremely buggy in my experience. It’s baffling that a product backed by someone as big as Amazon and part of their core prime offering can be so difficult to use. Then again Netflix has been getting much less usable over time and HBO is similarly buggy so maybe that’s just par for the course in streaming.


I'm ex-Kindle (Doppler/30) and I had exactly the same experience. It was shocking how many requests we got for better ePub compatibility and just a better experience overall. Never happened. Customer needs were absolutely not a focus unless a senior manager's wife/husband had some random issue and then it was a major priority (but good luck explaining that in a PRFAQ or via the Release Management team.


> They refuse to hire or even interview white, black, or Hispanic candidates.

This is complete BS, the org has as many white developers as asian and indian and even opened a big branch in Madrid

> You can pay H1Bs less, and they’re happy with it because they get to live in Redmond, WA instead of India.

Salary is a _direct_ computation from performance ratings which are the same % for every org. For H1Bs to be paid lower, they need to be systemically rated lower by the same managers who "refuse to hire white, black, hispanic" candidates.


There are some really cool openings at Amazon I was considering moving into but this makes me rethink. Thanks for the write up.


My wife works at AWS (for the last ~5 years) and has generally loved being there. She says the parent post has truths (people write documents), and a lot of exaggerations.

So if that was the _only_ anecdote that made you rethink, hopefully this is a second anecdote that helps you make a choice!

(side note: If you dislike working with Indians it can be a terrible place, a good number of her colleagues have been Indians (she is one as well).)


I always see people saying these kind of thing about Amazon, but personally, I don't think this is the case. I think there is actual technical reason why Amazon don't adapt EPUB natively (the Kindle Direct Publishing accept EPUB as input for ages).

EPUB has a lot of features. A lot. I don't think anything (except maybe Adobe Digital Edition) support everything. I actually just realised yesterday that Moon Reader+, one of the best epub reader for Android, doesn't support vertical-rl layout (mostly used in Japanese book).

Even Kobo doesn't use EPUB as-is. Its native format is KEPUB, which has tons of id-tagged <span> inside the content. Moreover, KEPUB and EPUB on Kobo use different renderer (EPUB use RMSDK, KEPUB use Kobo's own) with different supported features. Isn't this a mess?

The new KFX format has one major advantage over EPUB. The images can be stored out-of-band. This allows you to start reading image-heavy book quickly, while images continue to load in the background. For fixed layout magazine/comic/manga where the entire file can be 100MB or more, this helps a lot.


Never had an issue with a EPUB on a kobo or a Nook reader, I clearly never tried any edge case.

Not saying there might not be any issues with EPUB but the fact that in 10 years they can't be bothered to add at least a basic reader for me just mean they want to keep this incompatibility and keep people in this separated ecosystem with amazon/Kindle. I believe this choice is anything but a technical one.

Since I never had much trouble using Calibre it means I'll probably never buy a Kindle just because of it but many people can't be hassled to do that.


EPUB is like PDF.

If you're going with just the very basic stuff: text, maybe a few images - it really doesn't matter what kind of bare-bones reader you use.

BUT when you're opening a PDF that actually uses the features it's a different story. You've got a file with umpteen layers of information that needs to be displayed in a very specific way (I've seen blueprints do this for example). Most bare-bones PDF viewers either choke completely or display everything wrong.


Not sure where you are going with this :), it seems you imply that a bare bone EPUB reader is way more useful than a PDF one.

Which I completely believe, PDF support on readers is always an issue (there is even a discussion on it in this thread).

Then it support the position that if they added support for PDF but not a basic EPUB then this choice was not made for technical reason.


There's two different question here.

1) Why Amazon keep re-inventing the wheel?

As I explained in my original comment, KFX does have technical advantages over EPUB.

2) Why doesn't Kindle support EPUB natively?

Probably because there isn't a need for one, as they are using KFX anyway. Why invest time in something that is only for people who aren't using your platform?


I was reacting to (should have added a citation) : > I think there is actual technical reason why Amazon don't adapt EPUB natively

Kindle is out since 2007, KFX was released in 2015, hopefully when you create your own format there is some advantages.

In the end it appears we agree that the position of Amazon is : "Why invest time in something that is only for people who aren't using your platform?" But we probably disagree on the answer to it :D.


I was saying it's a technical reason that Amazon didn't adapt EPUB, not a technical reason why they don't implement EPUB support.


Then surely, Amazon will show good faith and open source their superior, albeit, proprietary format and back it with a standards body that makes it less monopolistic, no?


The feature that is missing and most important is called out in the article: a standardized form of DRM. If you're selling a subsidized product and depending on consumable content (like razors and blades), DRM seems like something you would want. Likewise, if you sell a digital product that can be copied, DRM would be seen as an advantage.

Other online media, notably streaming services and Xbox, DRM everything. I don't think Amazon is an outlier here.


> The new KFX format has one major advantage over EPUB. The images can be stored out-of-band.

EPUB is just a ZIP file. It's perfectly feasible to read the HTML/CSS and image portions separately.


Then you need to preprocess EPUB to know which order to load images, along with complicated client side download to download part by part and reassemble the .EPUB after all download are done.

Not sure if that is strictly easier than just new format (which allow you to add more features you wanted too)


That just depends on how the client consumes it. I'm the developer of Kavita and our reader doesn't download the epub at all. We read one page of the epub and rewrite the html and send it to the UI to be rendered. This way it loads like a normal html page (without having the iframe mess), images can load later, Javascript can still work within it, etc.

https://kavitareader.com for those interested.


But at that point your client isn't really reading EPUB isn't it?

Amazon backend accept EPUB (either from KDP or from Send to Kindle). Amazon then process EPUB into Amazon own format, and deliver that to Kindle device. Which sounds a lot like what you described.

Btw your project is interesting. Does it support vertical-rl EPUB? After finding out Moon Reader+ doesn't it make me doubting everything now.


And yet Apple Books have no qualms about it and support epub.


Part of the reason I went with a kobo over the kindle is that they support ePubs. The kindle honestly seemed a bit nicer hardware wise but I like the ability to build and read my library of DRM free books well into the future. No complaints about the kobo so far and it's lasted me nearly a decade by this point. Once it finally kicks the bucket I'll definitely stick with them.


Calibre makes converting epubs and loading them onto Kindles. It handles a wide range of different ebook formats. I probably would work with your Kobo as well if you have some non-epub source documuments.


You can use Calibre to convert with kindles, but some documents don't convert well without a lot of effort/tuning and you really shouldn't have to. Spend your money elsewhere.

The biggest downside of Kobo for me has been Rakuten's update cadence. The system updates opt-out doesn't actually work from what I can tell and my device only comes out of airplane mode once every few months. Every damn time I do so it has to download an update, lagging out for a long time, and changing the UI in exciting new ways.


Whenever I had a problem, the Epub book was not built right and had the exact same issues before conversion. PDF doesn't work well, but I don't know anyone does PDF well on e Reader space

Calibre is king, set things right and you can just email books to your Kindle.



Maybe it works but why someone would want to use 3rd party service with no privacy policy, TOS, owners etc.


Because all you give them is a (usually) public book and your kindle email. Most of us aren't afraid of anything that can be done with that information.


There's a reason most states in the US have laws protecting library records. I don't know who runs this site or what privacy laws and policies (if any) they operate under, so I don't know how they might feel about certain books I might send through it and what they might do about it.

I don't want to send a copy of "And the Band Played On" (for example) and start getting Chick Tracts.


To do that you need to disable airplane mode on your kindle which means you have to see ads :(


I’ve had Kindles with ads. They are almost unnoticeable. If you are tight on your budget it is not unreasonable to choose the ad-supported model. Often the screen saver ads are offers for books that match your interests.


Or you could give up the subsidy Amazon provided by seeing ads and then ads are gone permanently.


I’ve never had any problem converting an epub to a mobi


You'd generally expect that to be the case as they're fairly similar formats internally. Nevertheless, I did probably thousands of conversions in a decade+ of owning various kindles and encountered issues more than a few times. When my last kindle finally kicked the bucket I simply switched to a kobo.


That's a pretty low failure rate then


You can get a Kindle for 89 euro. Amazon, smartly, subsidises their products. The cheapest competitor is at least double that.


In the U.S. the cheapest Kobo is just under 100 USD: https://us.kobobooks.com/collections/ereaders. They go on sale somewhat regularly.


A subsidized Kindle + Library Genesis is a potent combination.


I've also heard (but not verified) that kobo's firmware isn't locked down the way amazon's is, so you can compile and install your own apps and modify system scripts. In other words it's your device, not Amazon's.


Kobo's firmware is very open, although it's technical enough that your average (non-developer) end-user would have trouble modifying. You upload a `KoboRoot.tgz` file to a hidden folder when it's plugged into USB, which is extracted to the root directory of the device (which is linux + busybox, with a QT gui 'nickel'). Newer FW versions also have a development telnet server which can be enabled in settings.

The MobileRead forums are where most of the unofficial discussion/development happens: https://www.mobileread.com/forums/forumdisplay.php?f=223

https://www.mobileread.com/forums/forumdisplay.php?f=247


IMO this is how it should work everywhere; the device works well out of the box for normal users, but if the owner wants to modify it, the means is right there and is actually pretty straightforward if you're technical enough to care. (There is room in the middle but dividing people into "wants to mod their device" vs "doesn't know what a tarball is" seems sane)


> Kobo's firmware is very open, although it's technical enough that your average (non-developer) end-user would have trouble modifying

That's good to hear. I have no problem with a device vendor focusing on just making sure their device works well, as long as they're not trying to close it off and actively working against external developers. I'd heard stuff like Kindle blows physical fuses so that one can't install older firmware and stuff like that so they actively prevent users from having control of their device. I have no desire to be a customer of such types of vendors.


I have a pocketbook touch hd3 [0]. It's nice, tbh I feel that my Kindle Keyboard (died after 10 years) was higher quality, mainly because the buttons on the hd3 can move a bit. Other than that it's a nice device, it can read all formats and has a feature to make the front light more red.

[0]: https://www.amazon.com/Pocketbook-Touch-HD-Metallic-Grey/dp/...


It's true, and the open source koreader software is great.


Made my Aura much more responsive, with better UI. Easy install too.


If you have a kobo, I highly recommend checking out koreader[1], which is an open-source ebook reader and is highly configurable (to the point where you can even write custom css rules from withing the e-reader), integrates with calibre natively (you can even connect to your Calibre library over Wifi), has a solid support for CBZ (with features like auto-cropping the border around your comic, automatically or manually), and more.

Having used it, I don't think I could go back to Kobo's reader (and even less to Amazon's, which is a joke in term of configuration options).

[1] : https://github.com/koreader/koreader


Yeah I use KOReader on my Kindle Paperwhite 5 and it’s definitely superior in every way.


Yep. Same here.

Oh, and not to mention, how easier... is to just poke things? At least with Clara HD, don't have to fear about bricking it, because worst case scenario, I can just reflash the system image, and have it working again.

And few days ago I just went and replaced internal microSD card with a bigger one (First making image of original one, then flashing it to bigger, and extending partition) and it just worked.

Sadly or not, I like to hoard things locally, on the reader itself. And that includes manga. So yeah, being able to replace microSD Card was very convenient.

And the device itself is a tiny bit more responsive overall too, alongside better transfer speeds :D


A friend sold me an original kindle for maybe $20 one time. I used it to read about 75% of Orson Scott Card's novels within the Ender's Game series. After about a year, it stopped turning on. It was probably 7 years old by then, so I just got rid of it.

I bought a 2" thick sci-fi book a few years ago, "Pandora's Star". It seemed pretty interesting, but way too much book to carry around. I got a kobo and put what I vaguely recall was named "koreader" on it. It was pretty nice. It stopped working after 6 months, though. I took it apart, and at least one power management IC is shorted out. Its e-ink display permanently says, "Sleeping..." It's been a paperweight on my nightstand for about 2 years, and a minor source of anxiety for me. I don't plan on buying another kobo device any time soon because I hate e-waste.

My wife got me a kindle paperwhite to replace it, and it didn't take much work to get the same book onto it. I think I'm maybe 25% of the way through the book, but I can't really say because I don't understand what the numbers in the lower corner mean. I also printed the Rust programming language manual to PDF and managed to copy it onto the kindle without much fuss.


> I don't understand what the numbers in the lower corner mean

If you tap on the numbers in the lower corner, you can cycle through various options including % read. My personal preference is for time left in chapter.


My preference is for number of pages left in chapter, but for some reason Amazon doesn't seem to be able to implement that.


Some Amazon Kindle books DO have "real" page numbers. It's an additional feature that must be supported. The problem of page numbers on reflowable text is much more difficult than it may appear.


Not totally trivial, sure, but I have a hard time believing that it's difficult. Maybe for complex documents, but just plain text? Not really. I use Moonreader on Android, it shows page numbers with no trouble.


It depends on the book. Some have them, some do not.


Ok, well it has been available for zero books I've read on Kindle, and 100% of books I've read on moon reader. Whatever the challenge might be, other people have figured it out and Amazon hasn't. It's embarrassing.


After a certain OS version kindles all support plain (DRM-free) ePub files just fine, Amazon just has yet to update their "Send to Kindle" tools (both the Windows File Explorer extension tool and the "email to an amazon email address you setup" tool) to allow the .epub extension, but you can copy the files directly over USB and there are silly workarounds like renaming from .epub to something the tools do recognize. There's file extensions they will load just fine under.

The interesting question is why if the systems support ePub, Amazon doesn't update the "Send to Kindle" tools to also support the extension? Those tools support .doc, .txt, .pdf, and other random things, just explicitly not .epub. It could just be that those tools are low maintenance and don't update often in general. Maybe it's simply as sinister as they don't want people to know kindles support ePubs, even though they do (because they have to) and the "Send to Kindle" tools are the primary interface people use for "what does my kindle support?" as very few users bother to directly USB their kindles and rely on wifi updates.

It's strange.


Read the article: apparently the good news is that the "Send to Kindle" tools are announced to support .epub file extensions directly soon. The article makes the assumption that "Send to Kindle" will be reformatting .epub files to Kindle native formats and uses that assumption as the basis to rant about the Kindle native formats, but anecdata (such as the USB support and file rename workaround) again suggest kindles already support ePub well enough and that assumption may not be valid and "Send to Kindle" may not wind up reformatting ePub files (much).


Ya, I love my kobo. I recently cracked the screen on mine (after about 8 plus years). I went straight to best buy the next day and got a new one.


Kobo isn't perfect but they have other nice features as well, such as Overdrive integration to check out books directly from the device, and Pocket integration for reading webpages offline.

My only complaint is no equivalent of Send to Kindle for Kobo, so I always have to plug the device in if I want to copy an EPUB to it.


I have kindle, but it's so hard for me to read programming books that I have in PDF (because it's too small). I end up having to read them on the laptop / PC. I'm even thinking to get an iPad because of the amount of programming books I have in pdf and want to read. Anyone is having the same issue?


As others have said: the problem is the device's display size.

If you compare the size of an e-reader to actual published books, you'll find that the smallest devices are roughly the size of a 4x6 index card. Most trade books measure about 9" diagonally, and in my experience, you'd want a 9" -- 10" device for most reading, possibly a 6" -- 8" minimum. Given that an actual device loses some area to bezels and controls, the translation isn't direct, and you're suffering either larger physical device dimensions or smaller screen size.

I purchased a 13.3" device specifically for use with textbooks and scans of small-font, multi-column articles. It's a bit on the large size for standard text (though can be read landscape-mode in 2-up format), but the ability to read almost anything without requiring in-page zoom and navigation (which is well-supported) is a major benefit.

The device has also convinced me that virtually all the supposed disadvantages of PDF or similar fixed-dimension document formats (e.g., DJVU) is not the document format but the display properties.

I'd recommend:

- Typical fiction or nonfiction text: 6" minimmum, 8" preferred.

- Technical books, some tables / diagrams: 8" minimum, 10" preferred.

- Textbooks, technical articles, old scans: 10" minimum, 13" preferred.

Comic and manga fans will also probably prefer 10" devices.

Then there's colour, though AFAIU size formats are more limited (8" and/or 10"). The fidelity is low --- more a low-saturation 1950's-era palette, though with much better resolution. Colour cuts resolution by a factor of three, so 220--300 dpi (typical of current devices) falls to about 75--100 dpi. I've not experienced this directly, so can't say what the net appearance is.

I wouldn't mind colour for some articles, mostly for graphics and visuals where colour differentiates values being presented. Though I'm rather enjoying my desaturated B&W world.


Since nobody has mentioned it yet, KOReader is an open source reader application for ereaders, kindles & others, and has the ability to reflow PDF's. While not perfect, it can work good enough to read well formatted PDF's.


KOReader reflows pdf's for ereaders using k2pdfopt, see https://www.willus.com/k2pdfopt/

Kindles need to be jailbroken to use koreader, and Amazon apparantly makes that harder with each device :-/

I have this as a right-click action for files matching *.pdf in my file browser (thunar):

    k2pdfopt -ui- -dev kp3 %F
So if I right-click on a pdf I can immediately reflow it into a kindle-readable version.

(%F is the quoted file name, in your terminal you would `k2pdfopt -ui- -dev kp3 'something.pdf'`)


> Amazon apparantly makes that harder with each device

Alternative phrasing: Amazon is continuing to fix security bugs in Kindle device.


What threat is the security for?


The technical detail about latest WatchThis hasn't been published yet, but the previous KindleBrake seems to be using malicious JXR image payload via built-in browser. That sounds like pretty serious security flaw to me.


IMO using a Kindle for anything else than linear fiction is an exercise in futility.

For stuff you read from start to beginning the Kindle (and other eInk readers) are amazing.

For books you need to jump around in it's just painful. Use an iPad or something else with a smoother UI and display.


Small e-ink screens (e.g. 6 inch) are only suitable for paperback-sized, text-only books (fiction or non-fiction). Small e-ink screens are simply not suitable for reading PDFs as you have discovered. Even ePub or Kindle programming books are often poorly converted from the print edition of a book.

A general rule of thumb when deciding whether to purchase an e-book: If the physical book is larger than a small paperback size, then it is unlikely to be suitable for an e-ink screen (unless the book is text-only).

Small e-ink readers are not suitable for any books with complex layouts, layouts designed for larger books, or books with colour graphics: charts, diagrams, photos, etc. Amazon, in particular, encourage publishers to convert as many books as possible to Kindle format regardless of whether those books are suitable for e-ink screens.


The kindle format, really any kind of text-based format, is not great for programming books in general. Code snippets end up having all kinds of hard to read wrapping, and also images and other things don’t really have a nice user experience on a Kindle. It’s really best when you’re just reading straight text, prose.


I agree. I write programming books for various languages and having to reformat text in sometimes funky ways to fit page width is a challenge.


I switched to using an IPad (mini) for that reason better rendering for programing books (both syntax highlighting and diagrams)


> I have kindle, but it's so hard for me to read programming books that I have in PDF (because it's too small).

Then just get a bigger e-reader.

8" Kobo Sage is the largest I still find comfortable holding in hands for longer periods of time:

https://us.kobobooks.com/products/kobo-sage

10" Kobo Elipsa should be even better for PDFs, but I would only use it with a stand:

https://us.kobobooks.com/products/kobo-elipsa


I have an 8" Pocketbook Inkpad 3 which is already decent size for reading PDFs. They now also have a 10" version which should be perfect for it.


13.3" Onxy MAX Lumi, and it's fine for lap reading.


I use a remarkable for this reason, and so I can scribble notes in the margin.


Have one too. Love the feel of writing on it. I came from a graphics tablet and surface go 2. I like it for drawing wire prototypes of apps/mechanical designs just for conceptualizing.


>Anyone is having the same issue?

honestly, I am a bit perplexed by your wording. there are no issues, you are simply using a very very very small screen. just buy a big e-ink reader like the Kobo Elipsa


As a former Kindle DX owner, a larger screen is not a solution for the problem of PDF readability. The issue is with how the eink display is not fundamentally not suitable for a format that relies on scrolling more than reflowing.


> The issue is with how the eink display is not fundamentally not suitable for a format that relies on scrolling more than reflowing.

I'm trying to understand your meaning of "format that relies on scrolling". How can this be the case since pdf was designed for printers?

"A PDF file is actually a PostScript file which has already been interpreted by a RIP and made into clearly defined objects.

Postscript is a file format supported by almost all high-end printers and many business-class laser printers. With these printers, you can simply send a postscript file to them over USB—no drivers required—and they’ll print it perfectly.

"


I will try to explain. Files meant for printing are typically created with a resolution target of 300dpi or above. There could be some fine details that are harder to see at normal reading distance, and this is easily solved by bringing the pages closer to the eye.

Files meant for screen display used to be standardized around a 72dpi resolution. Any detail not visible at the normal viewing distance is lost unless you are able to re-render the file at a higher resolution a.k.a. zooming in. And once you start doing it, scrolling becomes necessary in order to read through the document since the formatting is fixed or as you say "made into clearly defined objects". And this often works very poorly due to the inherent limitations of eink display when it comes to frequent screen refreshes.

Nowadays most displays, big or small, can do better than 72dpi but they still have a long way to go before they could match printed material.

Small screen e-readers get around the problem by aggressively reflowing the text of books with larger, sharper font and more line spacing to improve readability . Yet non of these techniques really work for PDF. Kindle DX failed rather miserably in the textbook and technical document market because of its limited resolution. I do not own the Kobo Elipsa, however I have spent some time with a couple of Dasung eink displays of a similar pixel density. And my current personal verdict is these panels are still not good enough for reading pdf files.


I don't know dude, I bought a Kobo Elipsa and finally I can study from my maths and physics textbooks that only come in PDF. I agree with you that it's terrible for reading in random access mode stuff like documentation. But for linear reading like literature or study material it's just wonderful.


I had the same problem (Kindle 4). So I switched to an E-Ink notebook/reader (Onyx NoteAir) and I never looked back. Admittedly, it's a 10" screen,so that may be a problem for some. PDF is not only consumable now but I can annotate everything.


I just googled Onyx. Seems good! But the price tag is expensive though.. that’s almost as pricey as ipad air for an ebook reader :/


One of my main use cases for my large iPad Pro is reading PDFs of tech articles with lots of math, code, and figures. That said, I bought a Lenovo Duet Chromebook tablet with keyboard and pen included (for 10% of the total iPad Pro package) and the slightly smaller Chromebook does almost as well for reading tech papers (and has useable Linux containers).

Shop around. You should be able to load up a test PDF from the web on demo units at stores like Best Buy, etc.


I do too, but that's simply an issue of reading books that simply aren't designed to be read at A5 size


I have the Fujitsu Quaderno A4 (13,3" screen). Great for reading scientific publications it whatever pdf. Also had a great Wacom stylus to take notes. I am great fan of these devices. Before the Quaderno I had the Sony DPT-RP1, which is very similar except the stylus is not so great.


Is it a custom operating system? Some of their competitors use Android which is the last thing I want.

Is the screen on the Fujitsu lit?

Does the note taking application index your text so you can search? Is it easy to get the notes off? Can you highlight and annotate books and then easily export your annotations and highlights?


I have a BOOX reader, runs Android, happy with it so far.

Although the font could be better, prettier maybe. So much better than Kindle for daily use.


You should be able to put any TTF/OTF font in a /Fonts directory and use it directly.

I like the TeX Gyre Pagella version of Palatino, myself.


I've never understood the hubbub. You can easily perfectly convert between the formats with any number of tools. I use Calibre and I've never had a problem with a converted document.


I had a 15 year old ereader from sony that could open pdf, epub, rtf, txt whatever you wanted and text reflow would work for everything bar pdfs.

Kindles are actively walled to force customers into the amazon ecosystem.

I could just plug this sony e-reader to my PC and copy whatever I wanted to read. The e-ink panels have gotten substantially better but the experience is now objectively worse with Amazon and more expensive, most customers not don't realize what it could be.


So true. I still own a Sony ereader and although the battery is a little worn off, the whole ease of use with epub or pdf is remarkable (its nearly 10 years old now). I really wonder why they stopped producing them


The last I looked Calibre, including its CLI tool ebook-convert, is literally the only local option, paid or free, for consumers to convert EPUB to AZW3. If you know of anything else please let me know.


I ran into issues with sending the converted files to my Kindle and there were no error messages to debug by. I was probably doing something wrong but still.


I would root/"hack" mine just to get rid of the damn lock screen ad that takes like 10 seconds to go away.


I definitely have run into issues once or twice, but you're right that it mostly just works.


The DRM bothered me until I discovered the Libby app’s excellent Kindle integration, now I don’t pay for anything since I have a great local library.


Wait until you discover LibGen and ZLib.

There's also Gutenberg, Archive.org, and numerous other public-domain e-book collections such as StandardEbooks (https://standardebooks.org/)


Take a look at previous hacker News post 'Software Jailbreak for Any Kindle' and installable app 'KOReader' allowing for reading ePub and other formats.

https://news.ycombinator.com/item?id=31220553

I gave it a shot and it works well although instructions wern't obvious at first.


Yes, KOReader is open source and way better on the Kindle.

Here's a clear how-to guide on Jailbreaking your Kindle https://www.mobileread.com/forums/showthread.php?t=346037&hi...


As is often the case, it's the perfect intersection of corporate greed, bored engineers with a NIH syndrome and device limitations


It would be very interesting to find out how much data Amazon has on pirated books being uploaded onto the Kindle.

The article ends with a discussion of Calibre. It would also be very interesting to know how much Calibre is used with pirated books.

Amazon must be following an interesting dance with copyright, piracy and the Kindle. They know if it gets too out of hand that's the end. But if they were to entirely lock down the Kindle to anything without DRM they would also know that many people would shift devices.


It's a very intelligent dance on Amazon's part I find.

The objective of antipiracy restrictions shouldn't be to restrict the top 5% of most technical power users. That takes far too much time and effort, and frankly power users provide word of mouth recommendations.

The objective of antipiracy restrictions is to stop 95% of users going to the trouble of piracy.


Calibre is good for library management, but for simply adding the odd book I've found it surprisingly easy to simply direct download a .mobi from any easily googled website and email the file to my kindle email address.

This obviously feels very sketchy, not only because you might be including Amazon in some very transparent piracy, but because it works TOO well. By which I mean, these files aren't simply added to your device, but to your Amazon library; these are then readily available OTA to all devices you own and any you add later.


It's a little weird that we're impressed when a relatively tiny book can be stored in the cloud and synced to every device. I get the same feeling from Google Play Books or iBooks. Yet this is a standard feature of [insert name of every cloud storage service ever] and I don't think that much of it at all in those contexts, except maybe when iCloud or OneDrive or Dropbox seems to magically put files on my desktop.

I wonder if the difference is that with the exception of comic books (sigh), most ebooks are small enough that they download quickly, yet have good metadata and often excellent cover art, so they replicate quickly. And they rarely get buried inside of unnecessary folders the way other files do.

It'd be pretty awesome if say, you could visit Humble Bundle and after buying a DRM-free collection of ebooks, just hit a button to transfer the entire bundle to a Google Drive folder or iCloud Drive folder without having to download and upload things. That's still maybe a missing thing. But it's not terrible to have to send files manually. It's just extra work. (How lazy can I get, apparently?)


They aren't helping by charging $15 for fiction ebooks with near zero distribution costs and marking up paperbacks even higher to make the ebook look like a "deal".


I asked the authors of The Expanse series which format put the most money in their pockets, per sale. They said that it was even between all formats, even though the cost to distribute the ebook version was significantly less.

I don't mind paying $15 for a book if the author(s) are paid well.


>It would be very interesting to find out how much data Amazon has on pirated books being uploaded onto the Kindle.

I don't see how Amazon could know for sure. I have never uploaded a pirated book to my Kindles, but have uploaded dozens of books via Calibre, either public library books that aren't available in native Kindle format for some reason, or obtained elsewhere (notably Tor.com's regular offers of free ebooks).


The lack of support for ePub is very annoying, when my 2013 Paperwhite finally breaks I'll definitely replace it with a non-Amazon ebook reader with better file support. Probably one with Android so it has proper browser support since there is a web frontend for Caliber and it might be nice to access Readarr directly too.


I've been waiting for my paper white to break, but it just keeps on going. I might just have to upgrade anyhow.


I'm just surprised my 2012 Paperwhite hasn't broken. It's still chugging along like a king. There's a thin line across the screen about 10% off the top bezel, but it's easy to ignore. I don't think the battery life has degraded too much either.

Perhaps because it is relatively simple, this has been one of the longest-lasting and best-aging devices I've ever owned.


I mainly use Kindle for technical books in English which would cost more and take more time to get in my country compared to e-books.

Otherwise, I much prefer real books. I find them more comfortable to read, more pleasant to read (I get a better feeling turning pages and looking at paper than pressing buttons and looking at a screen) and I can pass them to my sons.

Also, for children interacting with a real book is a better experience.


I'm currently using reMarkable 2 tablet and I love that it can render pretty much everything I need from it, except Kindle books.


Topaz was also used for long tail books, as there was a pipeline that could convert the high resolution search inside the book scans with minimal amount of human work. At the time OCR wasn't an option, so it used glyph binning similar to JBIG. The OCR'd text was included for searching. The biggest downside was no font changes


Amazon popularised e-readers and they pretty much own digital publishing. For normal users it doesn't matter that the books they buy from Amazon aren't epub- seriously do you think they even know what that means lol?

And for the rest of us Calibre converts anything in 30 seconds.


Wasn't that long ago that "Microsoft owned desktop computing". You couldn't do anything without Windows. Barely anything worked with Mac. Linux was just a baby.

The reason you fight monopolies, is because monopolies cannot last forever. There are all sorts of bad consequences with having digital publishing owned by a trillion dollar monopoly.

In order to have a free press at all, which is important in democracy, which is important for society to stay resilient, we need a diverse ecosystem of digital publishing options.


> Calibre converts anything in 30 seconds

Calibre, with the DeDRM tools installed, is unreliable for Amazon's newest books. There are workarounds that involve getting the file in an older format, but then you lose the typography improvements that are only offered in KFX files.

It feels like Amazon is finally starting to close the DeDRM hole.


Comments mention Pocketbook. How good is it? I’m tempted by 10.3” screen since I read technical pdf’s (think O’Reilly or Packt) a lot and no Kindle is good for that.


While its not the same product, my 10" e-ink reader (an Onyx Boox Note 2) is the single best note-taking, annotation and technical document reader I've ever used. Its genuinely fantastic for those purposes.

I read all my Kindle books on it, it's entirely replaced my paper notebooks for note taking, and all my technical books, manuals and so on have been excellent on it.

Your intuition that this form factor is better for technical PDFs is correct, I think.


I have an 8" Pocketbook Inkpad 3 which is already decent size for reading PDFs. They now also have a 10" version which should be perfect for it. Although if you are also annotating a lot, I would not recommend it, as it can be slow. (but i guess this is true for any E-Ink Reader)


Where does one go to buy ePUBs of popular books like, say, Brandon Sanderson’s works or Dune or older nonfiction like Zen and the Art of Motorcycle Maintenance?


For Sanderson specifically, his books are DRM-free wherever they are sold (apple/google/etc). Same goes for TOR publishing.

Past that, most books aren't available for DRM-free purchase. Breaking the DRM on an otherwise legally-purchased book is the next easiest thing. DeDRM tools are just a search away!


ebooks.com has DRM as a search filter so it is easier to find DRM-free ePUBs. Since you mentioned Sanderson, I believe Tor books (or is it Tom Doherty Associates?) actually publishes all of their books as DRM-free so there is a large selection of sci-fi and fantasy books to choose from. That being said, most mainstream stuff is just published with DRM enabled. Now for really older books that are public domain I like standard ebooks and project Gutenberg.


kobo, google, and some smaller ebook sites (which might just serve as an interface to kobo's catalog like walmart did). They almost always use adobe digital editions drm, but that's removable by the buyer with effort.

For free epub ebooks, there's overdrive and the library ecosystem; legal libraries also DRM-wrap their epubs.


> legal libraries also DRM-wrap their epubs

This is the one case where I'm OK with DRM -- you're receiving the library's book on loan (it's never your book), and they need a way to enforce the length of the loan.


https://www.libreture.com/bookshops/ has a list, though what we really need is a search across all these sites – who here needs a weekend project? :)


Doesn't happen often anymore but (something).com actually provides a useful result.

ebooks.com

edit: actually, every something I .com is pretty good. Haven't done that since the early days.


100000_scifi_epubs_a-e.torrent usually has what I'm looking for.


The fact that epub is going to be dropped from the send to kindle feature is finally going to get me out of the Kindle ecosystem.

I really liked the wireless mechanism of pushing books to my kindle, is there a similar feature for koreader / remarkable / Kobo / anything else?


Am I missing something or is this the opposite of what is happening?

> Beginning in late 2022, you will be able to send EPUB (.EPUB) documents to your Kindle library from the Kindle app.

Before this point, it was never possible to send e-pub documents via Send to Kindle.


In the Netherlands, anything you buy from Bol.com is automatically synced to your Kobo device. Not sure about other retailers.


It's because they're affiliates. Same goes for FNAC


Libby + Calibre + Apprentice Alf is how I handle my books these days. I really like my Forma.


in what world is kindle the defacto Ereader? here in the Netherlands the book market is dominated by bol.com, which has a deal with Kobo, so here it's all epub and Kobo e-readers.


Well, the US, obviously. I imported a kindle myself about ten years ago because they were both cheaper and better than the competitors at the time.


Worldwide they have a 80% market share, apparently. I'm staying away from Kindles because of their spying and their lack of EPUB support.


When I looked into this before, a lot of the "spying" was sending information back to Amazon in order to synchronize state between devices.

With that being said, I have an Oasis and keep it in airplane mode 99% of the time.


Anyone else enjoy kindle book <-> audible narration feature? Curious if the Kindle platform is the only solid option for this style of reading/listening.


I use this almost every day. Listen to Audible books when commuting, reading when i'm at home.

It's probably the one "killer" feature for me.


> Alexandria on your hard drive

That image brings to mind: How much space would you need to store every out-of-copyright book written in English?


Alexandria was strongly anti-copyright. Not that copyright existed, but they allegedly forced visitors to donate copies of any books they had to the library (scribes copied them and gave back the copies).

As of last year, someone on reddit determined that l--g-- had 2.7 TB of fiction and 40 TB of nonfiction, and double that size of scientific articles and magazines.

It's missing a fair amount of books, which would make the true total size larger, but it also has a lot of duplicates and scans that are unnecessarily large compared to true ebooks, which would make the true total size smaller.


In an age where books had to be hand copied, copyright really doesn't even make sense to apply as a concept. They weren't anti-copyright, since that concept doesn't even make sense from their viewpoint. The physical book itself was what was valuable, and wasn't separable from the 'work' of the original author. Since they were craft-made objects, the skill of the scribe & bookbinder was as important as the original text.

This is much later historically, but in medieval conception 2 identical texts could be worth vastly different amounts based on the quality of the binding and the illumination in the book.

This is an example of why applying modern thinking in a historical concept can lead us astray. The Library of Alexandria wasn't pro or anti copyright; that makes as much sense as saying they were pro x86 and anti ARM. It just wasn't even a concept that had been developed.

Edit: upon rereading, my tone seemed a bit combative. Not trying to call anyone out, just wanted to point out that applying modern sensibilities to history can cause distortions. It is a pet peeve of mine that people consider ancient people's stupid, and this is one of the major reasons why. We forget that we have the benefit of thousands of generations of previous people improving their understanding of the world, and what seems obvious to us is only obvious in hindsight with the benefit of that prior knowledge.


I agree there were all kinds of other considerations. The education required to write, and the cost of hand-copying books, meant that the market was nonexistent and those who wrote anything worthwhile were intellectuals and had some other job or patronage. The cost of contacting most authors might've been impractical, even more than the cost of the scribes to copy books, even in cases where the authors were still alive. Still, there was a thriving intellectual culture, and nobody thought about this modern "moral right of the author" to extract rent from the transmission of knowledge.

By modern standards, there's hardly anything more anti-copyright than forcing someone to let you copy their personal library. It disrespects not only the modern rent-extraction copyright by the author, but also the idea that written material in any form is anyone's property at all.


Whilst the Gutenberg Project doesn't have every single book, it is a substantial effort to compile them. The last time I downloaded the complete English collection, it was around 40Gb in plaintext form, and a little closer to 60Gb in HTML, which should be pretty comparable to ePub.


Compressed or uncompressed? Books should compress pretty well


Compressed. Sorry, that's an important detail that slipped my mind.

zstandard via the ZIM [0] format.

[0] https://wiki.openzim.org/wiki/ZIM_file_format


That's a lot of CD's to burn.


Stick them on a single blue ray


There’s a joke I futurama where they go to the library and it’s just two discs: fiction and non-fiction.

Honestly I wouldn’t be surprised if the modern version is just two SD cards. At least if you only want the text.


There are a few estimates of total books ever published. Those range from the 40 million or so within the US Library of Congress collection to a Google estimate from a few years back of 140 million books existing, with about 2--5 million new ones published per year.

Bowker, the US issuer of ISBNs, has registered roughly 300k new titles annually, since at least the 1950s, and beginning in the aughts, about 1--2 million "nontraditional" (self-published) titles.

In straight text, assuming 250 pages of 250 words at 8 bytes/word, 40 million books works out to about 20 TB. This is a likely close to the lower bound as there are texts with image and other content which bump this value up. It works out to about 0.5 MB/book, which in my experience is too low.

At 150 million books, we'd get 75 TB, and at 200 million, 100 TB. Nice round numbers....

A more realistic per-book size is 5 MB. This is typical of a mostly-text, formatted PDF. At 40, 150, and 200 million texts: 250, 750, and 1,000 TB.

A data-heavy / graphics-heavy book might weigh in at 20--100 MB, though those tend to be less common.

Scanned-in books with both page images and OCR text typically run about 5--10 MB. The largest I've seen (a copy of Lyell's Geography was >300 MB.

Apple has offered iPads with > 1 TB onboard storage for at least several years. This seems to be the higher end of available tablet storage. (For some reason, the Cloud-and-Surveillance-oriented Google skimps with many devices still at 16--32 MB. Remarkable, which also went Cloud, tops out at 16 MB.) If fully devoted to texts at 5 MB each, that would offer storage for up to 200,000 documents, roughly the size of a well-stocked municipal public library.

As to how many public-domain works might exist, the 1927 Report of the Librarian of Congress notes that the Library had assigned 4,503,164 copyrights over the preceding 57 years (presumably 1869--1926), the time in which such registrations were the work of the Library. Those works should all presently be in the Public Domain, and would total 22.5 TB at 5 MB/work.

Annual report of the Librarian of Congress yr.1927, Washington, Library of Congress; for sale by the Supt. of Docs., U.S. Govt. Print. Off., p. 22.

https://babel.hathitrust.org/cgi/pt?id=uc1.a0013520523&view=...

The Library of Congress's total holdings was 3,420,345 million as of end of FY 1926 (p. 23 ibid). Presently, works published prior to 1 January 1927 in the United States are in the public domain.

An additional roughly 1 million maps, 1 million volumes and/or pieces of music, and ~500k prints were also in the library's collection as of 1926.

A full set of the Librarian's reports, dating to the 1860s, is available at Hathi Trust. From at least the 1950s, annual holdings and aquisitions are broken out by Library of Congress classification, which makes for some interesting data, for those who find that sort of thing interesting.

http://catalog.hathitrust.org/Record/000072049


Thank you! We are not far in time (10 years?) from being able to provide a complete set with every computing device, though we would need them all converted and formatted, etc., and many will (unfortunately) think we should just use the cloud instead.

I will add this study published in 2010, which estimated 129 million 'editions' published in all languages. Any text can have multiple editions (I don't recall how 'edition' was defined, but IIRC the study was of published objects not of texts), and "includes serials and sets but excludes kits, mixed media, and periodicals such as newspapers". It also contains a useful description of methods to bulk process bibliographic data. (Though maybe this is the Google study you talked about, because IIRC their data is involved.)

J-B Michel, et al, "Quantitative Analysis of Culture Using Millions of Digitized Books", Science (16 Dec 2010) https://www.science.org/doi/10.1126/science.1199644

By the way, I greatly appreciate your informed comments. It's some of the most substantive and interestin reading on HN.


I still can't get over the fact that the Kindle Oasis their flagship device is still Micro USB.


So what happens on the day my kindle updates? Do I lose access to anything in Mobi format already on my kindle? That’s a PITA


No, why would you?


Truth is every last company on earth wants to monopolize and lock its customers. There's a reason why king of pretend-openness Google doesn't support carddav and caldav



In Android I meant (sorry I wasn't clear) Your comment further shows how disingenuous their stance is


> Your comment further shows how disingenuous their stance is

Does it really? I think it just shows that you didn't know about something they offer, and decided to attribute that to them being malicious.


It's the publishers that have handed Amazon their (near) monopoly on ebooks. If they wouldn't have insisted on DRM or at least had insisted on non-proprietary DRM, you would be able to buy books anywhere and read them anywhere.

I prefer reading on my Kindle to paper books but I also love browsing and supporting book stores and the people who work there. I've always wished that I could browse a bookstore, take a stack of books to the front and have them scan a QR code (or something like that) on my ereader and sell me digital copies of books rather than the paper copies.




Consider applying for YC's Fall 2025 batch! Applications are open till Aug 4

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: