
Ebooks, you’re doing it wrong  - barredo
http://blog.thomasrhiel.com/post/6014862098/ebooks-youre-doing-it-wrong
======
acabal
Agreed--not just typography but in many cases the entire ebook experience is
just terrible. Some works are very clearly just OCR'd from the print version,
with no proofreading done (you can tell if you're reading one if you spot an
obviously out-of-place word that would still pass a spellcheck). Others may be
mostly presentable, but retain their print ephemera--for example, hyphenation
in the middle of a paragraph, because there was a line break in the print
version at that point.

It's not always the publisher's fault--I've developed a few ebooks for
different platforms, and ebook readers today are where Mozilla and IE were
around the IE5 browser wars. Developing a book that looks good across all
platforms is difficult, if not impossible in some cases.

It wouldn't suck so much if we weren't paying almost the same price as a print
book. Why would I pay ~$13 for a brand-new, often terrible-quality ebook (and
a DRM'd one at that), when I could buy two print books for the same price, and
get Amazon to ship them free to my door?

~~~
w1ntermute
> Some works are very clearly just OCR'd from the print version

Why are books even OCRed to create ebooks? Do they not have the original
digital copy of them to work from?

~~~
pak
If they are subcontracting the ebook dev process to a third party, perhaps
not. And also, the whole editing to typesetting process is probably a hellish
mix of Word and InDesign files, which leaves no "pristine copy" to pick the
structured text out of. (Imagine: edits late in the process are made to the
typeset InDesign files, after the structure has been lost.) You need
structured text for creating an ebook because the ebook software does its own
typesetting. So to pick up all the late edits, the only choice might be to OCR
the final book and re-apply the structure. Yep, it's sad.

~~~
w1ntermute
> If they are subcontracting the ebook dev process to a third party, perhaps
> not.

So they're not willing to provide the original source files, or what?

> edits late in the process are made to the typeset InDesign files, after the
> structure has been lost.

Wow, they do this so haphazardly that they don't even bother to go back to the
original Word file and add the same edits?

Is InDesign used for all books? I don't see why you would need it for
something without any figures/graphs/tables. You could just use Word, and
things would work out fine.

~~~
wtallis
Word's typographical capabilities are just as bad as current ebooks. Word is
absolutely not capable of producing professional quality text. So, while it
may be useful for drafting a book, it is worthless for getting it ready for
press.

As for why OCR is used, just look at the range of books that are now available
electronically. There are a _lot_ of books from the 1980s and earlier that are
now available as EPUBs. Do you really think that it would have been easier to
use the source files off floppies? (Keep in mind that the floppies are, for
the most part, long gone.)

~~~
w1ntermute
> As for why OCR is used, just look at the range of books that are now
> available electronically. There are a lot of books from the 1980s and
> earlier that are now available as EPUBs. Do you really think that it would
> have been easier to use the source files off floppies?

How are reprints of such books achieved?

~~~
wtallis
Most likely, by using PostScript (edit: sometimes, even high-resolution
raster) files that have no semantic markup and cannot be automatically
reformatted in any way other than simple zooming/scaling. Such files are not
much easier to transform into ebooks than scans are, but are probably harder
to get access to.

~~~
w1ntermute
This state of things is quite apalling. Given how small the source files are,
I would've thought that publishers would keep an up-to-date copy of all of
their books in a centralized repository.

~~~
wtallis
Some publishers might do this. But it would be very surprising if any major
publisher has been completely standardized on using the exact same toolchain
and file formats for more than 20 years. Most publishers also have no
incentive to modernize the markup for a book that's not getting any content
updates.

------
gizmo
I can't stand ebook typography either. Almost half the kindle ebooks I buy are
formatted so poorly I can't read them. It's a combination of awful technical
decisions and very lazy typesetting.

Awful technical decisions such as: text justification and margins are fixed
and inconsistent, spacing between words is awful, footnotes break the
"navigate to last page" feature, pictures show up in the wrong place, tables
look awful, charts and diagrams are full of jpeg compression artifacts) and so
on.

It's really infuriating. Nowadays I check all 1-star and 2-star reviews for
mention of "ebook" or "kindle". That usually keeps me from making a bad
purchase. With the combination of lousy typesetting and comparatively high
prices amazon really shows contempt for their customers.

~~~
dfc

        "Almost half the kindle ebooks I buy are formatted so
        poorly I can't (sic) read them."
    

Honestly? I think the typesetting is quite awful too but it has not interfered
with my ability to parse the words on the screen.

I really wish that the typesetting was better but I am not sure if this is
something that we will see any time soon. Can you imagine running latex on the
kindle cpu everytime you flipped a page? On the other hand the ebook files
would be huge if you prerendering all of the pages. You would have to
prerender a separate page starting at every "location" and in each font size.
That is an enormous number of combinations.

~~~
wtallis
I haven't used a recent Kindle, but the original eInk screens were slow enough
that the tablet most certainly _could_ have run TeX on the next page in the
time it took to clear the screen.

~~~
dfc
Do you have benchmarks for running a tex engine on a tablet CPU? Everytime
this topic comes upo on comp.tex the consensus seems to be that tablets are
not yet capable of running tex on the fly.

~~~
wtallis
Are tablet CPUs fast enough?

Probably not for a whole book, but for a single chapter? Almost certainly. And
if you eschew the features of XeTeX or LuaTeX and just stick with pdfTeX,
things could be _very_ quick: For the example I posted elsewhere in this
thread of typesetting a novel in 30s with XeLaTeX, pdfTeX does it in about 6s.
A single chapter (32k of text): 0.8s, and I/O bound at that. Even if a
tablet's 800Mhz Cortex A8 does that 5 times slower than my Core 2, it's fast
enough to be done speculatively. It's certainly no slower than loading a web
page over the internet.

~~~
dfc
Is the single chapter time with all of the modern typographic features the
main article is about or without them (pdftex)?

~~~
wtallis
For the single chapter test, the book was stored with each chapter in a
separate tex file. All but one of the \input statements was commented out, but
the preamble stayed the same as in the previous test: pdfLaTeX with microtype,
memoir class, but no fontspec. The output was 25 pages. Enabling SyncTeX gives
about a 30% slowdown in wall time, but only about 10% slowdown in CPU time.

(For comparison, when run on the slowest CPU I have handy, a 1.5Ghz Athlon XP,
the single chapter takes 1.5s wall time and 1.1s CPU time.)

------
natch
There are two issues here. The author doesn't really seem to care about
distinguishing them, so I will:

1) Shitty rendering engines used in (some) ebook readers. If the ebook
original has an em dash and it's being converted and displayed as hyphens,
that's just bad rendering. Same thing if it has two hyphens together in a
prose text context surrounded by spaces, and it's not being shown as an em
dash, bad.

2) ebooks that start out bad. In this case, it's garbage in, garbage out. For
this, the blame goes to the publishers, as he correctly noted.

Has anyone done side by side comparisons of the same books, from the same
publishers (I'm not talking about Gutenberg books that may have been converted
using differing technologies, but major publishers), on different ebook
readers from different companies?

------
kleiba
I think there's a parallel to hacking though. On our weekend projects, our
babies, our github projects we're most proud of, we like to make sure that we
get it right: clean code, beautiful abstractions, efficient implementation,
etc -- because we, as hackers, get a kick out of that. We have a gut feeling
what is the _right_ way to write a certain program. Like a painter knows where
to draw the next line.

...but does that mean that we do the same in our day to day jobs? The code
that you have to fire out to meet next Friday's deadline? The program that
will make it into the next product that needs to hit the market _now_? I'd bet
almost all of us know the feeling of, "ah, if only I had a little more time, I
know exactly how to code this the way it should be". But this is business so
(a) the time you need just isn't there and (b) what the heck, you don't care
as much about the accounting app your company sells anyway.

So, yes, I'm sure that ebook publishers aren't idiots. And they know that
ebooks can be done differently from printed books. But they are businessmen,
too, and so the question is always: will doing these changes give them more
profit? That is, will the additional costs of e.g. moving the glossy-paper
pictures from a central position directly to the parts of the books that refer
to them result in more sales? Or sales at a higher price?

I figure if the answer were _yes_ , the publishers would do it. But as long as
they get away with sloppy editing and one-to-one copies of paper prints, why
would they?

The additional costs are not material costs, but that people have to sit down
and make these edits. And that's not just one guy spending half a day to do
that.

~~~
tutysara
this should change when many people start telling how they would have liked
the ebook version more if it would have been polished a bit, most of the time
business think they are doing fine unless someone complains or bring these
difficulty to their attention.

this will also change when more and more people are opting ebook version to
the printed copy. as of now the printed copy sells more compared to ebook
format AFAIK and the ebook format is catching up fast for the many convenience
stated in the posts here. We can expect a change soon as many people are
getting ebooks and raising their concerns.

this could also change when some really nice product(device or software) to
read ebooks comes to the market, I dream of a ebook reader where I can
annotate, mark and make notes using a stylus like device. while many of these
features are available in the ebook readers today they don't give the same
feel as using a printed copy, for example (making handwritten notes)

~~~
kleiba
Very good points!

But rationally speaking, if someone who bought the ebook tells the publisher
that they would have liked it better if it had feature _X_ , the incentive for
the publisher to implement that feature is still pretty low - after all, the
guy complaining bought the ebook anyway.

It may have a long time effect though, because the same customer might not be
willing to buy the next ebook from the same publisher unless the perceived
annoyances go away.

And competition from other publishers might help too.

~~~
ToastOpt
Publishers are keenly aware of these effects in print media.

It's chicken and egg. Until there's a large enough market, they won't care.
Low quality negatively influences market growth.

------
azov
Think of it in context. A few short years ago nobody had e-readers because
books that people wanted were not available, and nobody cared to publish books
in electronic format because nobody owned e-readers. That's why Amazon and
others had to come up with fast and cheap ways to scan thousands of paper
books. There was not enough sales to pay for adapting each title for the new
format, but prices for ebooks had to be high enough so that ebooks weren't
jeopardizing paper book sales (otherwise publishers won't agree to have them
scanned).

Of course it's unfortunate that consumers are now left with sub-par product,
but I think it was necessary to get the ecosystem going. Beautiful typography
and ebook-appropriate formatting will get there, just give it a little time.

------
wtallis
Typesetting a longish (~800k plain text) novel using XeLaTeX and the microtype
package takes my 2Ghz CPU about 30 seconds, and produces a 1.8MB PDF.

Current tablets should be able to re-render an individual chapter in a second
or two, and it would not be any trouble for ebook distributors like B&N and
Amazon to pre-render portrait and landscape versions at several different type
sizes.

Fixing the abysmal typography of ebooks really wouldn't take much work, and
would probably be the best thing ebook publishers could do to woo serious
book-lovers who aren't also geeks. Why aren't they even trying?

~~~
pak
I'm with you; I think ebooks should be distributed in some variant of XeLaTeX
and rendered using its toolchain. You retain the structure and can change the
size of the text, font, etc. as the user desires. It would be leaps and bounds
better than the current level of typography you see on the iPad and Kindle.

The only thing I can think of is that normal people, even book-lovers, don't
care. We have been trained by reading webpages so much of the time that we
have low standards. But the sad thing is, ebooks generally have worse
typography than your average blog post read in a typical web browser. I'm
looking at you, iBooks. Jesus.

~~~
dfc
XeLaTeX? Why do you prefer xelatex over lualatex? I would be interested in
hearing if you actually had any concrete reasons for your preference.

~~~
pak
Hmm, I had no need for the programming additions in LuaTeX; my main concern
was using UTF-8 and having better integration between LaTeX and my system
fonts. XeLaTeX offers a great out-of-the-box experience for using commercial,
"pro" fonts, particularly on a Mac; check out the 'fontspec' package. However,
I've heard the LuaTeX can do microtypography really nicely, and I haven't
checked if they've made using system fonts easier, so maybe it's work a second
look.

There is a beta 'microtype' package for XeTeX but it doesn't have all the
features that LuaTeX has (font expansion?).

~~~
dfc
That's kind of what I expected you would say;) You like fontspec and utf, both
of which work in lualatex.

Lualatex/pdflatex are the future.

------
johnnyn
You have to understand the publishing industry to understand why eBooks fail
so terribly in design and typography. Most publishers could care less about
the eBook format. They still have this idealistic vision that eBooks will die
off. Publishers are truly idiots. I work for a startup in the digital book
space and we get so many ePubs that do not meet standards and are formatted
incorrectly. We end up having to hack our code to cater to each publisher.
Some publishers are better than others but most of them honestly do not care
about eBooks.

------
kbatten
I've read a lot of ebooks (even long before I got a kindle) and I have to say
that I read them for the story, not the presentation.

What is wrong with ebooks is that there are still books that I _cannot_ buy. I
want to give the publishers/writers my money and they simply will not accept
it. For me its e or nothing, I don't have the space to store paper books, and
to be honest I just like ebooks better.

------
gbog
I find it strange that such a complaint about typography is done on a web-page
I have hard time reading with my Chrome. The font is too small, and trying to
enlarge it make the column narrower but the font size stay the same. When
selecting text has some weird contours. There are some CSS tricks behind this,
I guess. They seem to imply that the author is not fully aware of the number 1
rule of typography: it must be done according to the medium.

Books and newspapers don't have the same typographic rules. The web have other
rules. Ebooks probably have other yet-to-be-invented rules.

But I agree with the claim: most of the books I have read on my Kindle have
bad typography. It should be fixed on two sides: The reader's software can be
improved[1], and ebooks have to be proofread and adapted to this medium.

[1] Oh I wish Amazon could iterate on the Kindle (I mean the real one, with
e-ink), improve its reader software, its browser, etc., instead of running
after iPad's success.

------
reaganing
I've come across a number of Kindle eBooks with numerous typos or other
errors. It's like they simply OCR'd a printed book and didn't bother to check
for issues ('Too Big to Fail' by Andrew Ross Sorkin is the most egregious I've
bought).

Many of these books are eventually corrected and Amazon will email letting you
know this so you can get an updated copy, but this has generally been months,
or years[1], after I've put up with the problems and finished reading the
book.

And it would be nice if pictures were handled better. I think ideally they
should all be put at the end of the eBook but have links in the text that take
you to the relevant picture, so you can quickly look at it and hit "back" to
go back to the text where you left of.

[1]: Just this morning, I got an email from Amazon telling me a book I
purchased in May 2009 has been updated with corrections.

~~~
dfc
Your suggestion to put images at the end creates the same problem that
footnotes do. Namely it breaks sync to last page read. The author of the
linked post laments this very problem.

------
TheCowboy
Thomas, it would be nice if you could zoom properly and have the font-size
adjust on your blog. (Using Chrome)

------
bhickey
To deal with this issue I've taken a few Gutenberg texts, converted them to
LaTex and then rendered then into PDFs designed for my reader (nook Simple
Touch). It's laborious, but it looks great.

~~~
maxerickson
Have you looked at GutenMark?

(It purports to take the work out of Gutenberg->Latex->PDF; I've not used it,
simply had it buried in the back of my head somewhere)

~~~
docgnome
I took a look and it seems to work well enough... I'm not sure how well what
it does would translate to an e-reader.

------
tnicola
I am quite passionate about this and my husband and I are trying to fix it. If
you want to be kept in the loop <http://www.pixelpublish.com>

I write popular fiction right into Tex and use pdflatex to typeset my books.
But that is also not a solution. As error proof as it is, it is rather
cumbersome and not very user friendly.

The problem of (most) of today'e ebooks is not that they are typeset in Word,
but that they are converted from Word.

Contrary to common sense, authors' write (produce text) in Word (WYSIWYG
editor -- not text) and then they upload their .doc files into epub creators
to produce epubs (text). Yes, we go from text to text via something that isn't
text. Bound to be a few glitches.

Word has hidden commands that an average user cannot anticipate unless they
are surgically careful with their formatting. Most of today's epublishers have
written an entire book length instructions on how to do this error free. It's
exhausting.

In my opinion, writing, as a process, must be revolutionized for this to be
fixed.

------
brnstz
I've read a lot of non-technical books on the Kindle app for iPhone, and I
haven't noticed any serious flaws. Generally I find the experience to be
extremely pleasant, particularly compared to random PDFs I've tried to read
through iBooks. Perhaps this is more of a problem with books with different
typesetting for code vs. prose, etc?

~~~
slipperyp
FWIW, I agree most prose works great and think you're right that the technical
book experience is where it's most clearly lacking (not being able to quickly
flip to an appendix or back / forth 5 pages is a big annoyance of mine).

There are ways I think it works poorly for prose, though, too. The most
obvious example to me is the George RR Martin Fire & Ice books. When I'm
reading those, I very frequently want to flip from a section of the book to
the maps of the world described in the book. I read a physical copy of the
first two books but went to Kindle (both the 3rd generation Kindle and Android
app) for the third and it's tedious to set a book mark, navigate to marks and
notes, find the map, and then navigate back to the bookmark. Additionally, the
map images in the ebook are of such low quality that they are nearly
unreadable. Lastly, there's an appendix of the families represented in these
books at the back (who's who). When reading the first two books, I often would
consult that, too, but once I'd done that with the Kindle book, it made it
really painful to switch back and forth between reading on my phone vs. my
kindle, because the "sync" function is "sync to furthest page read" - which is
at the end of the book, not my current place in the book.

All of the above gripes with that book would be plain to see for anyone who
bothered trying to read the book on a kindle - and it should be obvious to the
kindle team that "sync to furthest page read" breaks when a book has an
appendix / index or something that readers are likely to consult.

~~~
brnstz
Interesting you mention Fire & Ice, I read the first three primarily on Kindle
for iPhone.

I did get into the habit of bookmarking my current page to reference the maps
and appendix. Generally my biggest problem with "sync to furthest page read"
was reading while underground (subway). Usually I just fire up the iPhone
briefly so it will sync forward if I want to read on a different device.

There is also the "Book Extras" section which, for Ice & Fire, includes
content from Shelfari with some appendix info and does not break the furthest
page read. It would be nice if the official appendix and maps were included in
a similar format.

~~~
slipperyp
I worked around the syncing but it was tedious. Primarily I would read on my
phone on the bus during commutes and want to use my Kindle at home. However,
having viewed the appendices of families, I could not simply sync. This was
tedious, but less tedious than lugging around that tome :) I think annoyances
like that are really about the software, though, and I expect it to continue
to get better, though the changes in Kindle have not been on details like
that, they've been the library lending thing, getting some free books into
prime, and lending. I'd love to see some smaller fit & finish.

I haven't noticed the "Book Extras" you mention and will have to investigate -
thanks for the tip!

------
pasbesoin
If my Kobo would just (word)wrap code samples in ePubs, I'd be happy. It lets
them run off the right edge, with no control to horizontally scroll to see the
rest (awkward as that would be).

I guess once I grok ePub a bit better, I can editing the texts to change the
code samples' formatting.

Also annoying: It will landscape PDF's, but not ePubs.

/grump

------
slipperyp
Yawn, this old saw? Despite the fact that every person who cares about this
has a blog where they complain about it, the market is indicating pretty
clearly that this is not so important that it needs to be solved today. I'm
sure this will get better over time and I suppose posts from people like this
will help motivate it, but don't we all get it by now?

Recalibrate your expectations a little - buying an ebook is a lot like buying
a physical book, but it's not the same thing. Maybe I should start blogging
about how unhappy I am with the portability of physical books or how I was in
an airport and wanted to rip the first chapter out of a book in the store to
read on the plane but the store owner TOTALLY wouldn't let me!

~~~
marquis
Until a publisher comes along who takes the effort to present eBooks properly,
grabs a big share of the market and others follow suit. (see: iPhone for an
example of how manufacturers take note of selling trends).

~~~
slipperyp
What I'm saying is that I'm not convinced that this would move a significant
chunk of the market. I could be totally wrong but my impression is that this
is a lot more like the a gold rush with all parties scrambling to lock in
business with the factors that are most compelling to potential customers and
I believe those factors are things like selection, platform cost, and maybe
other functionality of the device. Whether the fonts are beautiful or whether
the books are rendered with more than one style of the "-" glyph is probably
somewhere like priority #153.

Which is NOT to say that I disagree that it matters.

I just expect it's going to get better, because the platform / technology is
relatively immature. And I think it matters a lot less today than people who
blog about things like this (or those who downvote dissenting comments about
things like this rather than trying to be persuasive) might think.

~~~
marquis
Sure, you're probably right about vendors trying get lock-in, that makes
sense. I'm personally curious about what premium we'll end up paying for
physical books, like how CDs are no longer the primary music delivery
mechanism yet music lovers pay a premium for vinyl.

------
mjschultz
I don't have a Kindle or other ebook reader, so I can't speak from experience
but are some publishers better than others?

I mean, there are _many_ very bad physical book publishers out there that
don't (or minimally) edit the copy sent to them before sending it to the
presses. That translates into the digital realm too, but there is a much lower
barrier for entry for bad publishers and it's hard to differentiate between
bad and good publishers in these early days.

If no publisher out there is doing a good job, it might be something to invest
time into and maybe make some money.

------
rbanffy
Am I the only on perfectly happy with the epub files O'Reilly makes?

Also, when you are talking about ebooks, shouldn't things like font selection
be left for the reader (the being, not the machine) to decide?

~~~
viraptor
I'd be glad with a nice default (chosen by publisher) that can be changed.
Technical books are usually using different font than novels, than cookbooks,
than children stories, ...

~~~
rbanffy
Sure, but the styling has to be semantic or it will end up breaking. I am not
sure most non-technical authors even know what the "style" drop-down does on
their word processors.

------
illumen
I've been working on an Ebook for a while now, and it is quite hard. So I'm
not surprised that many books have bad typography.

Our ebook is for the open source/FOSS project PyGame.
<http://www.pygamezine.com/>

PyGameZine is in Beta, and we're launching today.

We made the book in html, and converted that into PDFs. The process was quite
hard over all. Haven't started on the process of turning it into an EPUB yet.

~~~
adrianN
Why did you choose HTML over something like DocBook that makes it easy to
convert it to other formats?

~~~
illumen
That probably would have been a saner way to go.

However, we felt that authors would find html easier to write, and designers
would find it easier to design in.

Maybe something like restructured text would have been ok to do too. Since it
is a python magazine, and many python programmers would know restructured
text.

If we make it to another issue, we might think about using a separate
toolchain.

------
8ig8
It comes down to resources and throwing them at the highest return. For all
the people complaining about typography in ebooks, if they were indeed hand
set for the new format, you'd have people complaining that certain books
aren't available in an electronic format fast enough. It's impossible to
please everyone.

~~~
larrik
Nonsense. It is months between when the physical book is typeset and it hits
the stores. Plenty of time to tweak the ebook version.

~~~
capnrefsmmat
Surely the majority of the typesetting would be done by the e-reader software
itself, anyway. Hyphenation, justification, and spacing can be handled almost
entirely in software, except perhaps hints for strange words or difficult
situations. LaTeX can certainly handle it without help, at least.

~~~
8ig8
I agree, but in this post, the author says:

> With a little effort, the makers of The Publisher’s digital incarnation
> could have one-upped the printed book by positioning these photographs to
> accompany relevant bits of Brinkley’s text, illustrating, e.g., a
> description of Henry Luce’s childhood with a photo of Luce as a toddler.

I don't know anything about typesetting, but this specific example seems
difficult to automate with software. A human would need to decide where best
to place these photos.

Again, a guess, but it seems that this book started life in a pre-electronic
book production process. To get it right would require more manual effort for
the e-book.

Just like websites are evolving to be adaptive/responsive, I think book
production will follow. We're just not there yet. Websites are still being
launched today that are close to impossible to read on a mobile phone.

~~~
wtallis
In LaTeX, switching between putting all the figures on their own page or
putting them inline takes the equivalent of an #ifdef. It's really easy, and
doesn't require any special knowledge of typesetting.

Remember, TeX started out with the goal of being able to automate the
typesetting of a very large, very complicated multi-volume book (The Art of
Computer Programming). Novels are almost too easy, and adding a few pictures
to a biography is mostly trivial.

------
zokier
Many people are complaining about Kindle e-books here. Are eg. B&N ebooks any
better or worse?

------
canes123456
The only way we will get good ebooks is if publisher make the book for
HTML/ePub first. They can than export for the page size they are printing and
tweak it. They will not put in the work on an already created book to make it
re-sizable.

------
eisa01
If we're going to get a stop to this, people must complain and demand their
money back. I did so with one book I bought from Amazon, were some words
consistently had spaces inserted into them e.g. "carry ing".

~~~
waqf
I just cut out the middleman and don't buy ebooks in the first place. Less
effort, same result.

------
obtino
This is the reason that I choose buy O'reilly e-books from the O'reilly
website. They are the only publisher (that I know of) who puts in the effort
to make typography in their e-books beautiful.

------
Freestyler_3
"I just scan books, I don't get paid for that."

I can hear it. Meanwhile someone is rubbing his hands (or should I say wallet)
and tells people that they are now available on ebooks.

