
Backing Up My Kindle Ebooks (2018) - exolymph
https://www.sonyaellenmann.com/2018/09/back-up-kindle-files-mac.html
======
maskros
Calibre is great, if you configure it properly.

By default, converting books to another format will change the layout!
Destroying the original formatting by default is unacceptable behaviour to be
honest, so user beware.

For Kindle, I suggest using the KindleUnpack plugin instead; which can extract
the EPUB file embedded in the AZW/KF8 format file losslessly for use with EPUB
based readers without having to worry about Calibre's conversion messing up
the formatting.

~~~
jxdxbx
I had no idea that an EPUB file was embedded in the weird Amazon formats.

I mostly agree that you want to avoid "transcoding" an EPUB. But the
formatting on some official EPUBs is so bad that I like to use readers that
can override the built-in formatting. I like to choose the font, spacing,
hyphenation, and justification myself. (Full justification and hyphenation are
my bugbears--I think they are never appropriate on smaller screens, as they
lead to large rivers and too many words broken up, but are fine on large
ones.)

~~~
jhbadger
There isn't an EPUB "embedded" in Kindle formats. The Kindle format is based
on the pre-Kindle Mobipocket format (that's why unDRMed Kindle books often
have an extension of .mobi). EPUB didn't even exist when this format was
created. Making an EPUB (or any other format) from a Kindle book involves
converting, which is always going to introduce changes in formatting. People
can argue that one program or another does a better job at doing it, but
there's no way around it.

~~~
rrix2
There's another layer to the format onion, even. The .mobi file, without DRM,
is a PalmOS resource collection file
[https://en.wikipedia.org/wiki/PRC_(Palm_OS)](https://en.wikipedia.org/wiki/PRC_\(Palm_OS\))
since mobipocket

~~~
rrix2
(I just looked and saw that I didn't finish that sentence: since mobipocket
was primarily a PalmOS application at its peak.)

------
superkuh
Sometimes it's just better to pirate the books even if you've paid for them.
The formatting and error correction is usually better and they stick to epub
and mobi standards.

As an aside: I was helping a friend out who owned a Kindle recently and
shocked to learn it can't read mobi or epub. Why would anyone buy such a crap
e-reader?

~~~
kranner
Kindles are awesome e-readers. They can read .mobi files, the battery time is
in weeks and the e-ink display is very easy on the eyes.

~~~
superhuzza
Yes but they don't support epub, which is frustrating (at least mine doesn't).
Easy problem to fix but it seems the majority of ebooks online follow this
format.

~~~
captn3m0
Installing KOreader after a jailbreak let’s you read whatever you want. (And
connect to OPDS servers)

------
bsharitt
While I try to buy non-DRM books when I can, I'm still willing to buy DRM'd
books as long as the DRM is pretty breakable(and I back them up immediately
most of the time). If the books on Amazon, Google, B&N, etc were to suddenly
have unbroken DRM, I'd stop buying DRM'd ebooks until that issue was "fixed".
Every ebook DRM format(except maybe Apple) is already trivially removable, so
I wish they'd drop the farce and go DRM free already.

~~~
criddell
> Every ebook DRM format(except maybe Apple) is already trivially removable

AFAIK, Amazon's latest DRM on KFX files hasn't been broken yet. That came out
in 2015. Some people get around it by getting the book in the older format,
but then you lose all the advanced typography stuff that KFX supports.

~~~
calcifer
Removing DRM from KFX works since March [1].

[1] [https://apprenticealf.wordpress.com/2019/03/30/dedrm-
tools-6...](https://apprenticealf.wordpress.com/2019/03/30/dedrm-
tools-6-6-3-released/)

~~~
criddell
I think Amazon patched their PC client and e-reader firmware to fix the hole.
I don't think the DeDRM works on a KFX you download today.

------
ben7799
I've backed up my eBooks from time to time & cracked some books over the years
that I had purchased & wanted to put on another device but a lot of the time I
don't worry about it.

I got my first kindle in 2010 and bounced over to Barnes & Noble for a while
with a Nook but then came back to the Amazon ecosystem. I've bought hundreds
of books over the years.

But just like paper books there are a lot of books that I read once and never
look at again. They aren't priceless treasured objects for me.

The thing with Amazon.. people complain a lot, but their ecosystem just works
the best in the eBook space. I actually find it a lot less annoying than some
of the shenanigans with other digital stores like the iTunes store over the
years. I haven't tried Kobo but B&N was always a hot mess compared to Amazon.
Tons of annoyances in the store & device software that Amazon had right from
day 1. And you were paying more for a worse experience.

~~~
komali2
Fair, but just like I've never needed my photo backups, when I _do_ need them
(if I ever do), I'll be very, very, very glad I put in the ~3 hours of work or
so a month it takes to ensure my photos are all backed up.

~~~
bduerst
Doesn't Amazon let you re-download your books on your kindle?

That's a little different than photos in that once the photo is gone, you
don't have the opportunity to capture it again.

~~~
calcifer
Assuming Amazon doesn't just decide to delete it [1], sure.

[1]
[https://www.nytimes.com/2009/07/18/technology/companies/18am...](https://www.nytimes.com/2009/07/18/technology/companies/18amazon.html)

~~~
bduerst
Seems that those books were added to the store by a company that didn't have
the rights. I'm not sure that's a widespread problem.

~~~
calcifer
I'm not sure the distinction is relevant. The fact is, Amazon can and will
remove your books at any time and there is nothing you can do about it.

~~~
bduerst
Right, but in setting a pattern of behavior that they will remove books
whenever they want, Amazon only seems to do it when there are bad actors
involved.

------
ggm
Some of the DRM failures are real: kindle DRM updates and some forms don't
strip in calibre IIRC. But that aside, this is all pretty much what I do.

I feel uncomfortable about the legalities but I do not like the social contact
being written in this way, against any consumer interest and without
discussion.

Buying books is important. Authors deserve their pay. DRM is like book cancer.

------
farss
Don't forget to backup your marks and highlights also. I made that mistake
once, and then lost my Kindle. Lost years worth of highlights and notes, so
now I set up a bash script to automatically sync them to a Git repo when my
Kindle is plugged into my laptop.

~~~
neves
Your bookmarks and highlights are synced to the cloud. You should be able to
read them online on:
[http://read.amazon.com/notebook](http://read.amazon.com/notebook)

From the kindle you can also export all the highlights of a book to your
email. It's a lot better to read it on your email. The kindle device reader
note reading experience is subpar.

~~~
farss
Yes, but only for the books you purchase through Amazon. If you import DRM
free books (or presumably re-import DRM free versions of Amazon purchases) or
personal docs, the notes on those won't be saved.

~~~
preek
And that’s a big use case. I use Kindle for most of my reading and import many
other documents (papers, DRM free books I bought and such).

------
macinjosh
By coincidence I was doing this last night because I got a new eReader (non-
Kindle). I found the most reliable method was to go the the "Manage Content
and Devices" page on my Amazon account and manually download each book. This
method downloads a single file for each book to your machine and its in a
format Calibre and DeDRM can work with. It took a while even though I only had
~80 books but it was worth the effort.

~~~
pmontra
I went to that page but there is no way to download the books. What's the name
of the link/button to use?

~~~
darkpuma
That page should have a list of books you own. On the left side of each row
for each book, there should be a light gray button that looks like: [...] It's
under the "Actions" column, second to the left. Click that and there should be
a popup that contains a link _" Download & transfer via USB"_

I believe this link will _only_ be available if your account has a real kindle
(e.g. eink or fire) registered to it. It encrypts the downloaded file for
whichever registered kindle you select from a list of all your registered
kindles and apps, with the apps grayed out so you can only select actual
kindles.

------
jxdxbx
The easiest ebook format to strip DRM from is the Adobe DRM that various
stores use. I only buy ebooks from the Google Play book store now, since DRM
removal is so straightforward. (Same deDRM calibre plugin, but you just get a
single encrypted ePub you can download, and adding it to calibre gives you a
DRM-free ePub, which is a more industry-standard format. Converting to or from
KFX or AZW3 can be error-prone.)

~~~
exolymph
I mentioned this in the post :)

------
manojlds
What are folks using for Audible? Open Audible? It wasn't pulling my library
correctly last time I tried.

~~~
anatoly
1\. Download your own AAX files (using the Audible download manager or search
around for some ways to do it w/o the download manager). 2\. Find out your
'activation keys' using ffprobe and the inAudible-NG project, see instructions
here: [https://github.com/inAudible-
NG/tables/blob/master/README.md](https://github.com/inAudible-
NG/tables/blob/master/README.md). Needs to be done only once for your Audible
account. 3\. Use ffmpeg with --activation_bytes to convert AAX to MP3 or other
formats, either directly or with a nice script like this one that'll divide
into chapters:
[https://github.com/KrumpetPirate/AAXtoMP3](https://github.com/KrumpetPirate/AAXtoMP3)

OpenAudible looks like a nice tool that'll automate all of these steps for
you.

~~~
Wowfunhappy
Why is transcoding necessary even if the AAC option is used? (I was under the
impression that audible used aac under the hood.) Is this avoidable?

~~~
anatoly
I'm not sure. Possibly if you try ffmpeg's -a copy and such, it'll simply
extract the AAC. Let me know if it works for you. I've always wanted mp3 in
the past so I didn't mind the transcoding.

------
jamescampbell
I figured out how to finally get this to work. For Mac OS, you have to install
an old version of the Kindle App. Version 1.23.1 appears to be the last
version that downloads AZW3 formatted ebooks that can then be converted to
EPUB easily with the DeDRM plugin. All later versions appear to be broken.
Also, the Kindle app will silently update to the latest version unless you
turn off the auto-update option. I converted all of my paid books to epub and
not buying any more through Amazon. This post inspired me to do this before it
is too late and the old version is no longer compatible to future OS versions.

------
carlosgg
In April, Microsoft announced customers who bought books through its ebook
store would lose access starting in July.

[https://www.bbc.com/news/technology-47810367](https://www.bbc.com/news/technology-47810367)

------
Yizahi
There are other reasons for doing this. Even if Kindle market would function
forever and Amazon won't delete books remotely there is still a possibility
that your account could be breached, for example by social engineering means.
And if malicious person can access your account he can delete everything
there, forever. If someone will steal your Steam account you can restore it
completely, most likely. If Amazon acc is stolen - goodbye everything. And
support won't restore or refund anything. (support itself is good)

------
Causality1
It's a neat exercise but one I gave up on years ago. Now I avoid the hassle by
buying a book and then pirating a copy for my archive.

------
misiti3780
this only works with purchased books. if you sent an epub to your kindle, it
will not show up in the devices section of the amazon website.

~~~
jpindar
But then you must already have the file as an epub... so what are you trying
to accomplish?

------
umvi
I have this great idea for a business: "ewhiteout" for ebooks.

Basically you can selectively "patch" ebooks to censor profanity, sex scenes,
violence, (and more) depending on your tastes and on what you like (and don't
like) to consume.

However, this is extremely unpopular with authors/publishers for the same
reason that censoring movies is extremely unpopular with Hollywood (hence why
companies like VidAngel are constantly mired in legal issues despite
widespread popularity with consumers).

Honestly, I don't understand why since at the end of the day everyone gets
paid, and indeed get paid _more_ than they normally would have (I guess it's
more of a "you are altering my art so people can consume it in ways I didn't
intend" attitude). But in my mind it's no different from using whiteout on a
physical book or picking out the mushrooms (or whatever you don't like) in
some chef's dish.

Basically, I've already built the tech to do this for myself privately (and
have patch files for ASoIaF and a dozen other epubs), but I will never share
it (I've even considered FOSS) except with close family and friends even
though I know it would be wildly popular in places like Utah because I fear
legal pursuit from authors/publishers (and since you must break DRM in order
for the tech to work, I don't see a way out).

~~~
jedimastert
> Honestly, I don't understand why since at the end of the day everyone gets
> paid, and indeed get paid more than they normally would have (I guess it's
> more of a "you are altering my art so people can consume it in ways I didn't
> intend" attitude).

The problem with "you are altering my art so people can consume it in ways I
didn't intend" can be a purely economic one, as it's misrepresenting the
"brand" that is the author. If you removed the "vulgar" things than you are
changing the product in a non-controlled way. It's why some directors or
writers remove their names after producers or whatever external controllers
get a hold of it. It doesn't represent their creative abilities. They could
put out two versions, but I think that the opinions of most authors that if
the sex scene could be removed, it didn't need to be there, and it wouldn't
have been there in the first place.

It's removing an aspect of the brand.

~~~
umvi
The way I see it: it's not me or my company altering the brand, it's the
_consumers themselves_ altering the brand so that they can enjoy it more
fully.

How is a consumer altering a movie experience or book experience any different
from a consumer altering a:

\- music experience (remix, etc.)

\- food experience (customizing a chef's dish)

\- video game experience (with game genie, mods, etc.)

\- picture experience (with photoshop, memes, etc.)

?

People create derivative works all the time of all media types to suit their
tastes, dreams, and whims - I don't see why that should be illegal as long as
they aren't trying to resell it or create a competing brand.

~~~
Skunkleton
Why not just consume media you don't find offensive in the first place? Would
you feel differently if I wanted to _insert_ sex scenes in to movies that
didn't have them?

~~~
umvi
> Why not just consume media you don't find offensive in the first place?

FOMO, mainly? You feel left out when all of your coworkers are talking about
GoT everyday... if only there were a way to consume it in a way that met your
standards and you could participate in the discussions too...

> Would you feel differently if I wanted to _insert_ sex scenes in to movies
> that didn't have them?

Not at all! If that's what you want to do, you should be able to do it on your
own copy! Or if you wanted to replace the main character with Thomas the Tank
Engine, whatever floats your boat, that's great.

------
m463
I find that a number of Kindle books I've purchased at the beginning say:

"The author and publisher have provided this e-book to you without Digital
Rights Management software (DRM) applied so that you can enjoy reading it on
your personal devices"...

So I assume either they don't have kindle DRM, or it is ethical (possibly
legally allowed) to remove the kindle DRM.

~~~
chipotle_coyote
Publishers (including self-publishers) can choose whether to apply DRM to
books on Amazon as part of the uploading process, and not all publishers do
so. Tor and Baen, for instance, don't use DRM on any of their titles.

------
BuckRogers
I did a review of the varying e-readers 9 or 10 years ago and decided on the
Nook based on standards support. I still use a Nook and never had any problems
like this.

------
tuesday20
How legal is stripping the DRM like this? Can one get in trouble for this?

~~~
moviuro
> _Use of Kindle Content_. [...]. _Kindle Content is licensed, not sold, to
> you_ by the Content Provider. The Content Provider may include additional
> terms for use within its Kindle Content. Those terms will also apply, but
> this Agreement will govern in the event of a conflict. Some Kindle Content,
> such as interactive or highly formatted content, may not be available to you
> on all Kindle Applications.

> [...]

> _Termination_. Your rights under this Agreement will automatically terminate
> if you fail to comply with any term of this Agreement. [...].

[https://www.amazon.com/gp/help/customer/display.html/ref=hp_...](https://www.amazon.com/gp/help/customer/display.html/ref=hp_left_v4_sib?ie=UTF8&nodeId=201014950)

So you may get in trouble if you're caught doing stupid things, yes. Will they
bother though? Unlikely.

~~~
ptah
physical books are so much better for this reason and generally cheaper

~~~
atemerev
They are not “generally cheaper” (ebooks are usually about 30-50% cheaper),
and much less convenient if you travel a lot.

~~~
ptah
also you have to buy a device to read them on

~~~
aw3c2
Once.

------
pnathan
I periodically go through and back up my Kindle to Calibre.

I have a fairly enormous ebook library, and I don't care to have it exploded
due to, e.g., some error in the kindle software api.

------
aswanson
Anyone know of a good way to browse & put pdfs into the kindle? The Amazon PC
application purports to do as much but is really bad at it.

~~~
simcop2387
Easiest way I know of is to email it to the kindle,
[https://www.amazon.com/gp/sendtokindle/email](https://www.amazon.com/gp/sendtokindle/email)

Haven't done it often, but it seems to work fine.

~~~
jxdxbx
You can use this app, too

[https://www.willus.com/k2pdfopt/](https://www.willus.com/k2pdfopt/)

to pre-process them in various ways so they are actually readable on a Kindle.
(Crop whitespace, flip to landscape etc)

------
Havoc
The ones that fail are the new encryption.

To remove that you need another add on and the serial from the kindle as key

Don’t have the name of it with me now sorry

~~~
b15h0p
If I remember correctly the process is described in the README of the DeDRM
plugin that can be downloaded from GitHub:
[https://github.com/apprenticeharper/DeDRM_tools/blob/master/...](https://github.com/apprenticeharper/DeDRM_tools/blob/master/README.md)
(and I am talking about the README file inside the zip you download there).

------
Symmetry
I've downloaded all my books but I'm holding off cracking the DRM until I need
to.

~~~
criddell
> I'm holding off cracking the DRM until I need to

Last time I checked, there was no crack for Amazon's latest version of their
DRM (KF8 files). A workaround exists where you get Amazon to wrap the content
in the old DRM, but then you lose all the typography improvements in the KF8
file.

------
mobilio
This doesn't work anymore.

You need to download old Kindle app where it's possible.

~~~
jpindar
Or download the files from your "Manage Your Content and Devices" page using
the "download and transfer using USB" link which gives you the older AZW3
format.

------
dwighttk
anyone know how to do this for audible books?

------
jtbayly
2018

------
philtar
Why is this article so high up? It's a blog post by an Amazon associate who
fails at following basic instructions (dedrm tells you how to remove the new
DRM and how to do it without a physical device)

"hacker news"

~~~
dewey
It's probably not high up because it's a great blog post, but it sparked a
discussion and with the news from a few days ago that some big provider's
ebooks will stop working because of DRM issues it's a topic on people's minds.

------
santafe
Or instead of messing with propertiary programs one could set up an rsync job
to pull the books from the actual device. DeDRM still required, btw. But do
humans really requires a guide how to backup stuff from a frickin' mass
storage device? I hate this like the 1000 different "10 hammock mistakes"
video on youtube, where all of them telling the same things. Ah, she is member
of "Amazon Associates" program so this article is just a bs for more clicks.

~~~
jauke
She didn't have a phsyical kindle device available.

