
How I wrote and published my novel using only open source tools - ggambetta
https://medium.com/@gabrielgambetta/how-i-wrote-and-published-my-novel-using-only-open-source-tools-5cdfbd7c00ca
======
jccalhoun
It seems odd to me that he would write and publish a novel using open source
tool and then write and publish about that using a closed source platform. I
wonder why he chose medium (I wonder why anyone chooses medium over their own
site)

~~~
jmcomets
Probably for simplicity of use, as always. Although it isn't _that_ simpler
than building your own blog, at least for a tech aficionado. Wordpress if you
don't know web development, one of the hundreds of blog engines otherwise.

I've used Jekyll on Github pages for quite a bit and it suited me perfectly.
Being able to write a post in markdown and `git push` to deploy is all the
simplicity I need.

~~~
skoczymroczny
Github is a closed platform too

~~~
manquer
you don't need github pages to deploy jekyll. While source code of github
pages itself is not available, since it supports a open standard, which can
deployed to a webserver/CDN directly without major changes, I would consider
it to be open.

~~~
jefurii
Just to follow this to its logical conclusion, the webserver may be running
Apache or Nginx on Linux, but there are likely to be proprietary blobs in the
drivers and the firmware is probably closed. Plus, much of the hosting
company's networking hardware is probably running proprietary software.

IANANE (I Am Not A Network Engineer) but I imagine with open firmware like
Coreboot and networking distros like Cumulus and pfSense (BSD) and the whole
Open Compute Project it might be possible to run a completely open-source
hosting company. Well, no. Your UPS's and other auxiliary hardware are
probably running closed OSes. Darn.

~~~
mcpherrinm
Cumulus contains a large portion of closed-source code, including all the
driver code provided by Broadcom, which is where much of the interesting stuff
happens. (I assume their Mellanox products are the same, though I'm not
familiar with them)

I'm not aware of any efforts to write open source drivers for any high speed
ethernet chips, though I haven't looked much. Intel's (formerly Fulcrum)
switch chips have open data sheets (but not the reference code, simulators,
etc) so it would at least be plausible to start there.

------
binarymax
Nice writeup.

I'm currently writing a (technical) book in markdown, and I know I am going to
pay for it later, but I don't care because I'm able to write quickly without
anything getting in the way. I'm able to use git to commit changes and a host
of tools to preview the work.

Also, using markdown now will force me to edit it again when putting a final
version together for a real editor.

For the skeptics: I don't see how using markdown is any worse than using a
typewriter. Its an interim medium that allows for quick capture of structure
and thought.

~~~
geerlingguy
If you use LeanPub, you can fully publish both ebook and paperback versions
from the Markdown source. And for editing, if you have an editor who's willing
to not use Word, you can use a tool like Authorea to track changes, and then
incorporate those back into the markdown source.

I really don't get the hate on Markdown; it's IMO the fastest way to get words
from brain into text, more so than rST and other formats, because of its
simplicity. And for slightly more involved formatting, with LeanPub, you can
put down what you need, then fix formatting by previewing the published copy
until it looks just right.

~~~
lkiux
I never understood this argument about rST. The syntax for the same MD
features in rST is only /marginally/ different, certainly not harder. I use
both, depending on the context (which forces me to use one or the other). Most
of the time you don't even notice the difference, because both have a really
good baseline.

However MD falls short for writing technical documentation, which is why
pretty much anyone has to dope the "standard" (GH flavored MD, and so on). The
syntax for the extensions is not any better than what rST offers. rST
generally allows a more human-readable writing style, whereas MD with
extensions has a lot of very verbose inline formatting.

In fact, rST _CAN_ have a slight edge in my mind as a human readable document
if you use the appropriate formatting, whereas MD with extensions is easier
for a program to parse.

But, I repeat myself, for the same set of MD features, rST is hardly
different. If you don't believe this, use pandoc and try to transform a
vanilla MD document into rST.

Now pick any rST page from sphinx documentation, and move back to MD. Hardly
any better. sphinx is often criticized for being "hard to write" or illegible,
transferring the blame to rST, but for the same set of formatting MD is not
better. sphinx allows a whole slew of extra features which require a lot of
annotation, and MD doesn't save the day.

What's really annoying if having to switch between the two. We have projects
which have a good majority of text documents written in MD and then API
references in rST. I hate that. There's constant mismatch.

Just pick one :(

------
PieterH
I use a similar flow for my own books, except Perl scripts instead of Pandoc.
Starting with a markdown-style text format is so nice.

One thing with LibreOffice is that you can generate documents, they are just
XML zipped up. So my tools produce ODF files with proper formatting (ready for
CreateSpace), automatically. The only thing I can't automate is producing the
table of contents; for that I have to open the document once and update the
ToC, then export the PDF.

My tools are on github
([https://github.com/booksbyus/mkbook](https://github.com/booksbyus/mkbook))
and there's an example book using them
([https://github.com/booksbyus/scalable-c](https://github.com/booksbyus/scalable-c)).

Also to note:

* gitbook works really well as a front-end to show Markdown books off a git repo and collect edits. See [https://www.gitbook.com/book/hintjens/social-architecture/de...](https://www.gitbook.com/book/hintjens/social-architecture/details).

* If you write your own filters (text to whatever) you can do really interesting things. It's worth learning Perl just for this kind of work.

* CreateSpace is excellent, once you have cracked how to produce perfect PDFs for the cover and interior. Highly recommended.

* Amazon's author tools are overall good. A bit clunky sometimes (e.g. lots of separate logins), yet they work and can produce decent income for a self-published author.

* LibreOffice produces excellent PDFs. Really top class, once you've figured out the details. The way I use it in the mkbook tool is by templating the format; i.e. design a book by hand, then slice out the various pieces of XML, and then generate the body of the book text using the template. This gives me 100% control over the results.

* When you automate this, it can work really quickly, as in a few hours to to make a publishable draft, and perhaps 30 minutes to release a new version of the text to CreateSpace.

* Get a professional cover designer. It isn't that expensive and makes a big difference.

~~~
teach
Recommendations for finding a professional cover designer? That's one of the
things missing from the original article that I was interested in.

~~~
ggambetta
The cover for my novel, which may be the most professional bit about my novel,
was done by an artist I found through 99designs. I loved the process, I had
over a hundred designs to choose from, tons of variants of these, etc. And I'm
super happy with the result!

------
chatmasta
Congratulations on a great accomplishment. For what it's worth, I found your
post on how you reverse engineered Dan Brown [0] far more interesting than
this one. Very cool!

But it seems writing was the easy part; how do you plan to market the book?
After four years of writing, surely you want to make sure it gets maximum
exposure. Do you have any concrete plans here?

Did you ever consider shopping the book around to publishers, or were you
always set on publishing it yourself? Any comment on the fee structure of
Amazon?

[0] [https://medium.com/@gabrielgambetta/how-i-wrote-my-first-
nov...](https://medium.com/@gabrielgambetta/how-i-wrote-my-first-novel-during-
my-daily-commute-e1d02c9447b9#.6i2bynt95)

~~~
lintiness
> how do you plan to market the book?

that's what he's doing here.

~~~
ggambetta
You're not entirely wrong, but not for the reasons you think :) I hope this
post raises the profile of the book a but, but I'm much, much more interested
in getting the book into the hands of more people, than in selling more
copies. In other words, I want more readers, not more customers.

I can't just post a link to the book because of Amazon's terms, but as I
offered at the bottom of the post, I'm giving free copies to anyone who sends
me an email.

------
vog
I'm glad this all works well for the author, but still ...

 _Why Markdown?_

I know that it is widely used, but that's only because GitHub pushed it so
much.

 _AsciiDoc_ is superior in almost every aspect. For larger texts I can
strongly recommend it. The toolchain is mature, there are many modules for
everything, even included UML from text description, if you need it.

Moreover, in addition to HTML output, AsciiDoc has proper PDF and LaTeX output
as well.

Really, Markdown is only good within its niche, while AsciiDoc has a mature
toolchain and has a proper extension system, so that new features are
additional modules for the main tool, rather than incompatible forks of the
Markdown processor.

There is the original AsciiDoc tool, but for modern use I recommend
Asciidoctor.

[http://asciidoctor.org/](http://asciidoctor.org/)

~~~
talideon
Markdown was popular _long_ before Github.

The reason for using Markdown is simple: pandoc. Pandoc is just that damned
good.

~~~
erlehmann_
Pandoc can also process reStructuredText [1], which looks similar to Markdown,
but has more features and a single non-ambiguous specification. Me and my co-
author rejected Markdown for our book about internet memes [2][3] after a
short evaluation, mainly due to ambiguities [4] and lack of features.

[1]
[https://en.wikipedia.org/wiki/ReStructuredText](https://en.wikipedia.org/wiki/ReStructuredText)

[2] The book can be downloaded for free:
[http://internetmeme.de](http://internetmeme.de)

[3] About the writing process:
[https://news.ycombinator.com/item?id=12311546](https://news.ycombinator.com/item?id=12311546)

[4] [http://roopc.net/posts/2014/eval-stmd/](http://roopc.net/posts/2014/eval-
stmd/)

~~~
talideon
I know. Personally, I prefer reST, but Markdown is more approachable than reST
for a lot of people.

I wouldn't say reST is all that similar looking to Markdown though: they have
very different philosophies, and really only coincide on the fact that they're
used for marking up plaintext documents.

~~~
erlehmann_
What do you mean with “more approachable” ? Markdown surely is more popular –
but which constructs differ much in usability?

~~~
talideon
Off the top of my head, the complaints I've gotten have referred to
directives, the link syntax, roles, not being able to nest inline markup.

None of this bothers _me_ , and directives and roles the source of much of
reST's extensibility and power, but apparently those are the things that make
reST a bit intimidating compared to the likes of Markdown.

~~~
erlehmann_
Please elaborate on the complaints. What about the directives do users
consider problematic?

~~~
talideon
The two nicest things about directives once you get the point behind them are
that they're _consistent_ and _flexible_ : once you know the syntax for one
directive, you know most of the syntax for the rest.

The problem with that is that the syntax is comparatively heavyweight. Part of
this is due to reST's alignment rules and partly down to the generality of
directives.

Let's take 'image' as an example. The simple way to embed an image in reST is
this:

    
    
        .. image:: foo.png
    

And in Markdown:

    
    
        ![](foo.png)
    

And with some alt text:

    
    
        .. image:: foo.png
           :alt: Foo!
    

And in Markdown:

    
    
        ![Foo!](foo.png)
    

I've had people struggle with image directives simply because it doesn't sink
in that the `:alt:` attribute has to be aligned with the directive name,
whereas the rough equivalent in Markdown has no such issues.

OTOH, once you know how to embed an image in reST, you're 90% of the way to
knowing how to embed a code block, whereas in Markdown you need to learn a
little bit more syntax. The downside of that is that this:

    
    
        .. code::
    
           print("Hello, world!")
    

Is more heavyweight than:

    
    
        ``
        print("Hello, world!")
        ``
    

And people prefer the latter as it's less to type to the former, even if the
former introduces no special-purpose syntax.

This is where Markdown's ad hoc nature has benefits over reST's more
structured nature: if you want to embed an image in Markdown, you learn how to
embed an image; in reST, you learn the directive syntax so you can embed
images. That gives Markdown and reST different learning curves. Markdown gives
you lots of little easy-to-learn tools, but requires that you be constantly
learning new tools as you go along, whereas reST gives you fewer, slightly
more complex tools you have to learn up front, and all the new stuff you learn
later is based off of those. Moreover, Markdown has less impact on the text
due to its ad hoc markup being more compact, unlike reST, which takes the
route of being more general at the cost of being more verbose.

Somebody starting out confronted with either reST or Markdown will look at
both and see that the latter looks more like plain text than the former, and
thus there is less of an initial barrier to entry. That's why the likes of
MkDocs exist in spite of the likes of Sphinx existing long before MkDocs and
its ilk did.

TL;DR: Markdown makes easy stuff super easy and hard stuff possible; reST
makes the easy stuff a little harder than Markdown, but lets you lift
mountains.

Edit: s/codeblock/code/

------
Johnny_Brahms
I saw this a couple of weeks ago:
[https://www.youtube.com/watch?v=FtieBc3KptU](https://www.youtube.com/watch?v=FtieBc3KptU)

I really like how he has no idea how to pronounce stuff, but he still managed
to bend emacs after his will.

~~~
Normal_gaussian
This is great, and hilarious. A good place to get an idea of the video from is
about 13 minutes in until 15 minutes.

~~~
Johnny_Brahms
And actually quite inspiring. He got me to set up abbrev mode properly and
actually start using emacs for _everything_ text related (even this comment,
using "it's all text" for firefox).

I set up it's all text with a bunch of different file endings depending on
what language I use, so that abbrev mode works correctly. It's a godsend.

------
midgetjones
Hey, OP! I admire the fact that you took all this on yourself, and also from
reading the linked blog post[0] it seems like you took the hacker's approach
to the planning and writing of it too.

Honestly, I'm not sure it's a book I'd read - not a big Dan Brown fan - but I
am deeply envious that you've completed such a huge undertaking :)

[0] [https://medium.com/@gabrielgambetta/how-i-wrote-my-first-
nov...](https://medium.com/@gabrielgambetta/how-i-wrote-my-first-novel-during-
my-daily-commute-e1d02c9447b9#.y4os6qg6e)

~~~
harscoat
... then OP, could you share this structure? "Soon the structure underlying
The Da Vinci Code and Angels and Demons and The Lost Symbol was laying bare
before my eyes. I could see why the stories worked. I had reverse-engineered
Dan Brown."

~~~
madaxe_again
Multiple mcguffins, drip-feed expository dialogue and prose, cliff-hanger in
30 minute reading time chunks. Always keep at least one good mystery open.
Start in media res, slow the pace for the second act, which is where you
consolidate to accelerate to a frantic conclusion where you draw seemingly
unrelated threads together. Essentially, give the reader a reason to keep
reading, and to keep coming back.

It's not Brown's recipe, it's how pretty much all compelling fiction hangs
together.

~~~
ggambetta
Pretty much, plus mini-cliffhangers at the end of each chapter. And tons of
chapters. My novel has around 70, so roughly 5 pages on average. It helps keep
the pace, and because of the psychological need to read "just one more
chapter", it makes it a "page turner" or "a book you just can't put down".

------
weeksie
You _can_ do it that way, but using something that's made for the job makes
life a lot easier. Between Scrivener for writing and Vellum for compiling to
ebook formats there's no reason to spend your time futzing around. If you want
to format for print, either pay someone or learn InDesign because nice print
layout is tougher than it appears.

Then again, a lot of the self-pub strategy these days is specifically geared
around getting as much out as quickly as possible, so that's not necessarily
an invalid approach.

In any case, I prefer to spend most of my time writing rather than screwing
about with tooling.

~~~
jnbiche
> learn InDesign

Ahem, or learn the open-source Tex/Latex, from which InDesign took
inspiration. If you can format a beautiful scientific/mathematical textbook
for print using Tex, you can format anything. And Pandoc is a great adjunct.

But yes, it's not for people who wish to avoid screwing around with tooling.
It's for perfectionists who want absolute control over layout/notation.

~~~
weeksie
Writing a novel in LaTeX is not the best approach if you're looking to write a
lot of text and don't have to worry about notation. But Neil Stephenson
famously wrote Cryptonomicon in Emacs and LaTeX, so hey, it works for some
people.

Edited to add that InDesign is much easier for back matter, front matter,
laying out images, covers, etc. . . . And that's coming from someone who LOVES
LaTeX. I set my old software company with some sweet templates and all of our
proposals were typeset in LaTeX. It was awesome. But if you're writing a novel
for print I really wouldn't recommend it.

~~~
wodenokoto
I don't think you are supposed to write in LaTex anymore than you are supposed
to write in InDesign

~~~
weeksie
Spoken like someone who has never encountered `auctex-mode`

------
rachel11201
I have a friend that writes porn using open source tools. Her stuff is on
AMazon as well. She also told me how their numbers work. A book at this
author's rank sells maybe 1 book a month. That's not good. Anyway...she uses
openoffice to write her docs, then uploads to kindle. She authored some kinky
shit like Control - A Billionaire Romance series that got high sales and
turned on lots of folks. If you're into that sort of thing, like bondage and
such.

~~~
creshal
It's kind of impressive just how high sales you get with not even that good
smut fiction. ( _coughs_ 50 Shades. _coughs_ )

I've picked the wrong job.

~~~
rachel11201
Yeah...me too. Wish I wrote her books. Her name is M.P. Lodi in Amazon.

~~~
erlehmann_
If she has written a blog post or something about her experiences, please link
to it.

------
lifeisstillgood
Let me recommend "working copy" for fantastic GitHub integration - basically
it's a git client on iOS (and android?) and it has a good enough editor that I
can write .rst and git push and I have published an article - this went up
today -
[http://www.mikadosoftware.com/articles/alwaysgoodships](http://www.mikadosoftware.com/articles/alwaysgoodships)

~~~
ggambetta
Oooh, that sounds great. I'll give it a try. Thanks!

Any idea whether it has good support for large files (e.g. a 300k character
novel)? I've tried a ton of iPad editors but most of them can't handle large
files, or don't have good scrolling support so it's a PITA to go to the end of
the text.

~~~
lifeisstillgood
I have never tried anything so large. I suspect it's just pretty much iOS
basic editor (whatever that is)

Why not divide up the novel into chunks (chapters?) - I do this and stitch the
chapters together before running through docutils - it's trivial and let's me
divide and conquer (so far it's an unfinished manual for Dev Leads and
managers)

~~~
ggambetta
Yeah, that's a good point. I'll give it a try!

------
ktRolster
I used Latex to write my own book, in case anyone cares:
[http://www.zerobugsandprogramfaster.net/](http://www.zerobugsandprogramfaster.net/)

If I had it to do again, I would consider using just tex and Makefiles.

------
imroot
I'm a HUGE fan of using softcover.io to build documentation for runbooks and
other technical documentation. You can write the text in Markdown (with some
LaTeX and other goodies), and it automatically converts it to both HTML as
well as epub, mobi, pdf. I like using it because you can make mobile
documentation that can be pushed out to Mobile devices (and optionally managed
by Apple's tools) so your on call resource has the ability to pull up the
documentation about the alert that they are receiving on their mobile device
so they know how to remediate/mitigate that issue.

------
erlehmann_
Here are some lessons I learned writing the German-language O'Reilly book
about internet memes [1]:

• reStructuredText [2] looks superficially similar to Markdown, but is vastly
more useful: It has more features and a defined specification without
ambiguities. Many RST processors do not follow the specification in edge
cases, but at least it is clear how the document should be layouted. The
author of Pandoc [3] fixes bugs very quickly. The Sphinx document processing
system [4] can export to HTML, PDF, LaTeX, Epub etc. and can check included
URLs automatically (we did not use it for the final version).

• Using Git is a good idea even if you are writing alone: The web interface
for our repository and Sparkleshare – a DropBox-like interface for Git [5] –
helped immensely with reviews. Gitstats [6] helped me to find out that
crunchtime before a deadline always led to at least a few days of writer's
block, which enabled me to gauge what amount of text I could write in a
sustained manner.

• If possible, save everything relevant to the book in your repository. One
example: I investigated the origin of the first lolcats (“Happy Cat” [7], it
came from promotional materials of Russian distributors for the “Happy Cat”
cat food brand), but have lost some of the emails related to the
investigation. Similarly, footnotes in our printed book are now exhibiting
link rot. Both of this could have been prevented by importing all source
materials in the repository for our book and making sure that all the web
pages we mentioned are archived in the Wayback Machine [8].

• The file format of the text matters for interoperability, but which text
editor you use does not. Do not learn a new editor for the project if you are
already comfortable with one. I used GNU Emacs while my co-author used GNU
nano; we never had any problems due to that.

• Me any my co-author worked remotely for the most time. Each evening we had a
short chat about what each of us finished during the day, what we were working
on, and what plans we had for the next few days. At least for me, telling
someone else what I did boosted my motivation; having such a routine process
helped me to overcome writer's block.

• There seem to be at least two different writing modes. I think a long time
about what I want to write and then write the text, making a few cosmetic
changes later. My co-author writes a rough draft and then refines it with each
pass. This means that my text is always of the same quality, but he will
always meet the deadline. Using Git, we found out he wrote around twice the
amount of text as I did, as he re-wrote almost everything at least once. We
had no problems with this because we clearly allocated who was responsible for
each chapter in the beginning.

• Avoid changing your workflow mid-way or towards the end of the project if
you can avoid it, it probably leads to more stress than it is worth. Thanks to
publisher shenanigans, we had to send in our texts as multiple ODT files and
got it back as one layouted PDF that we had to annotate and send back. One
problem was that there seemed to be no software to merge two PDFs with
annotations. I quickly hacked up something with Ghostscript only to be told
much later that we unintentionally overwhelmed the publisher's pipeline as
each one of our around 1000 annotations to the PDF text was apparently faxed
to the layouter.

• Get a proper advance. We did not want one and I think it was a failure. You
will most likely not make much money with your book – even if it is good and
about a popular topic. Three years after the book is published, only around 15
copies are sold each quarter; I suspect it might be one university course
using it as classroom material.

• Include a section in your contract about free licensing in case the book is
not available. We told O'Reilly we would only accept the contract with a
provision that the book would be CC BY-NC-SA and the licensing would switch to
CC BY-SA (same as Wikipedia) as soon as O'Reilly stops selling the book. Now
that the German section of O'Reilly went under the book is still sold, but by
another company that uses the O'Reilly imprint [9].

• Get your hands on as many specimen copies as you can get away with, those
are some of your best promotion material in terms of effort vs results.
O'Reilly did not do much to promote our book, but they did provide us with
copies.

• Do not neglect your loved ones because of work. I regret not having spent
more time with the ex-girlfriends I was together with during the time I wrote
the book.

[1] The book can be downloaded for free on
[http://internetmeme.de](http://internetmeme.de).

[2]
[https://en.wikipedia.org/wiki/ReStructuredText](https://en.wikipedia.org/wiki/ReStructuredText)

[3] [http://pandoc.org/](http://pandoc.org/)

[4] [http://www.sphinx-doc.org/](http://www.sphinx-doc.org/)

[5] [https://www.sparkleshare.org/](https://www.sparkleshare.org/)

[6] [http://gitstats.sourceforge.net/](http://gitstats.sourceforge.net/)

[7] [http://knowyourmeme.com/memes/happy-
cat](http://knowyourmeme.com/memes/happy-cat)

[8]
[https://en.wikipedia.org/wiki/Wayback_Machine](https://en.wikipedia.org/wiki/Wayback_Machine)

[9]
[https://en.wikipedia.org/wiki/Imprint_(trade_name)](https://en.wikipedia.org/wiki/Imprint_\(trade_name\))

~~~
Noseshine
Since shutting down my own Internet presence for good, which also took down
technical documents I had written years ago that - to my great surprise - I
found were actually cited a lot according to Google Scholar - I'm thinking
maybe linking to the archive.org link is better than linking to the actual
URL? Any thoughts? My documents are still available under the orginal URL
there, so if all those footnotes and citations pointing to one of my docs
pointed there they would still work. It also has the added advantage of being
able to link the concrete version as it was when you included the link.

Example for the first link in the parent comment, pointing to the (right now
randomly selected) 26 March, 2016 version of the webpage:

[https://web.archive.org/web/20160326165458/https://en.wikipe...](https://web.archive.org/web/20160326165458/https://en.wikipedia.org/wiki/ReStructuredText)

TL;DR I recommend to always cite sources using their archive.org document
version.

~~~
erlehmann_
A HTTP GET to a URL prefixed with
[http://web.archive.org/save/](http://web.archive.org/save/) archives the
content behind that URL.

I use the conkeror web browser [1] and have included the following code in my
$HOME/.conkerorrc/config.js and execute “wbsave” for many documents (even my
own) whenever I send the URL to someone else.

    
    
      define_webjump("wbsave", function(url) { return "http://web.archive.org/save/"+url; } );
    

[1]
[https://en.wikipedia.org/wiki/Conkeror](https://en.wikipedia.org/wiki/Conkeror)

~~~
ashitlerferad
Probably best to use HEAD instead of GET to save downloading the result.

------
emodendroket
Most novels in the history of the world have been written using exclusively
open-source tools.

~~~
dredmorbius
I'd be careful with that assumption. Tthere aare over one million novels
written every year now.

The novel as a literary form dates only to the 1500s. Cervantes' _Don Quixote_
is generally considered the first novel.

I suspect a quite large share, possibly majority, ofnovels have been written
since the word processor era.

[http://www.forbes.com/sites/nickmorgan/2013/01/08/thinking-o...](http://www.forbes.com/sites/nickmorgan/2013/01/08/thinking-
of-self-publishing-your-book-in-2013-heres-what-you-need-to-
know/#75c05e3929f4)

------
lifeisstillgood
A little more interesting was how he reverse engineered Dan Brown novels (see
[https://medium.com/@gabrielgambetta/how-i-wrote-my-first-
nov...](https://medium.com/@gabrielgambetta/how-i-wrote-my-first-novel-during-
my-daily-commute-e1d02c9447b9#.248qy79fb))

I would love to see his spreadsheet on what makes a Dan Brown novel - it's
like distilling literary evil.

~~~
ggambetta
_" How I distilled literary evil"_ sounds like an amazing title for a follow-
up article :D

------
ryanmarsh
I'd like to know what people are using for writing non-fiction. Mainly I'd
like to know how people keep track of their research and cross referencing it
from their editor.

Here's what I have:

* Many words typed up in One Note

* Highlights and annotations in iBooks, Kindle, and Instapaper (although Instapaper allows export to MD)

* Notes from PDF exported to Markdown using the Highlights app[0]

* Images

* Tons of shit to read in pinboard and other places

Here's the kicker, I travel a lot (every week) so I need most of my tools to
work well offline/slow wifi and preferably on an iPad/iPhone.

What do people use for organizing notes and research (literature of all
sorts), cross referencing, organization, and first draft?

0: [http://highlightsapp.net/](http://highlightsapp.net/)

~~~
jseliger
1\. I use Devonthink Pro for notes / research:
[http://www.stevenberlinjohnson.com/movabletype/archives/0002...](http://www.stevenberlinjohnson.com/movabletype/archives/000230.html).

2\. Many use Scrivener for writing, but I have never quite managed to agree
with it properly, so I just use Word.

3\. I use Bookends: [http://sonnysoftware.com/](http://sonnysoftware.com/) for
citations. No one it seems really likes their citation manager, but I like
this one best.

See also [http://blogs.plos.org/neurotribes/2011/06/02/practical-
tips-...](http://blogs.plos.org/neurotribes/2011/06/02/practical-tips-on-
writing-a-book-from-22-brilliant-authors/).

~~~
ryanmarsh
2\. Scrivener is great, wish it could do .md natively 3\. Yes I hate Papers.
Citation managers seem to only be useful for academic papers anyway.

------
joeax
I also wrote a book called the Final Six Days. If you visit his Golden Legacy
book page on Amazon you will see my book in the "Others Have Viewed" section.
It's mostly due to me checking on his and other novels.

I was planning a Medium article about my (similar) experience. I'm not sure
why he'd chose markdown when epub is just structured HTML. If you're a
software dev just stick with HTML. Also, to anyone planning a novel: get ready
for annoying inconsistencies in display across the various Kindle + iOS +
android devices. Fonts not displaying correctly, sizing is off, things not
centering, some CSS selectors not being applied at all. It is IE/Netscape
circa 1997 all over again.

~~~
munificent
> I'm not sure why he'd chose markdown when epub is just structured HTML.

Markdown's a lot simpler/faster to read and write than HTML, and books rarely
need the extra features HTML provides. When you do, you can always do inline
HTML in Markdown.

> Also, to anyone planning a novel: get ready for annoying inconsistencies in
> display across the various Kindle + iOS + android devices. Fonts not
> displaying correctly, sizing is off, things not centering, some CSS
> selectors not being applied at all.

Oh yeah. It's even worse if you write a technical book.[1]

[1]: [http://journal.stuffwithstuff.com/2014/11/03/bringing-my-
web...](http://journal.stuffwithstuff.com/2014/11/03/bringing-my-web-book-to-
print-and-ebook/)

~~~
joeax
I fought getting fixed-width fonts to work for weeks across all these devices.
Believe it or not, despite being a sci-fi novel it does have code snippets :)

------
fs111
Can somebody show him how to take a screenshot on Linux please?

~~~
ggambetta
I know how to take screenshots on Linux :D Taking pictures of the screen felt
more "authentic", in the same way that I took pictures of the printed book
instead of pasting a PDF.

------
manifestsilence
I have started to write a novel using almost exactly this setup. It's
encouraging to hear it worked out for someone. So far I've been using plain
markdown and just rendering it to HTML for a pretty view, but with plans to
turn it into e-pub later. As mentioned in the article, the controlled sync is
what makes this beat Google Docs. I set up Vim and Mercurial on my phone, so I
can sync and edit the text anywhere. Fun tools to play with!

------
cstross
Go back to the early 2000s. I'm kind of obsessive about data portability, and
the lifespan of a novel is closer to 30 years than the 3-5 months typical of
business documents. Alas, I'm selling novels to Big Six publishers who like to
chew on a Word file. So ...

Writing tool of choice: vim. Markup format: Perl's POD (plain old document)
format, because (a) I can mess around with the formatter if I need to (old
Perl hand), (b) it can export to a variety of formats, (c) it has the minimum
necessary set of directives -- headings, lists, emphasis -- that I need
(novels are typographically simple unless you're playing Stupid Layout Tricks,
hello Alfred Bester and Samuel Delany) and (d) Markdown didn't exist back
then.

Version control: a novel with a single author is piss-easy. I used rcs,
because it was ubiquitous, easy to understand, and to the extent that I was
keeping projects on multiple computers I was fine using rsync to keep a
directory tree updated; remember, a novelist working alone has no need for
multiple user IDs, no forking of multiple versions, virtually _never_ any need
to diff or merge -- just the ability to roll back to a previous point if I
took a wrong turn in writing (in other words, undo checkpoints with names). No
cloud services, of course. (Dropbox didn't come along for a few years.)

Build: makefiles and some homebrew perl scripts FTW. Type "make", check out
latest draft and generate HTML, RTF and (eventually, via an external
toolchain) Word files I can ship to my editors.

Back in those days, copy edits showed up as a bunch of paper print-outs with
red ink on them and you mailed them back to production after you added your
own chicken scratchings. (If you were smart you scanned/photocopied the pile
first, for insurance: this saved my ass on two occasions when CEMs went
missing in the post, thank you for nothing, Royal Mail). Page proofs ditto. So
I was able to use my homebrew toolchain happily for several years during which
I wrote "Singularity Sky", "Iron Sunrise", "Accelerando", "The Atrocity
Archives", and the first three Merchant Princes novels using it.

Then the publishers began moving to Microsoft Word tracked changes for
processing copy-edits, and annotated PDFs for the page proofs, and it was all
over bar the shouting; Word tracked changes suck, but trying to check the
changes on a _large_ document in a third party word processor like LibreOffice
sucks even harder (one sleeper bug that nobody triggered before can cost you a
couple of months work), forcing me into Word for the post-writing workflow.
And then a Better Way came along for writing books in the shape of Scrivener,
for which there is nothing remotely like an open source equivalent and which
is well worth the price. Alas.

Which is why my open source novel writing days are over. (But if I had to do
it again these days? Markdown, Pandoc, and Git, probably with all the fancy
Vim plugins I could lay my hands on.)

~~~
david-given
At some point I'd be really interested to read a writeup on Scrivener and your
writing workflow.

~~~
cstross
Sure: here you are!

[http://www.antipope.org/charlie/blog-
static/2012/07/writing-...](http://www.antipope.org/charlie/blog-
static/2012/07/writing-a-novel-in-scrivener-e.html)

~~~
david-given
Awesome --- thanks!

------
watersb
FWIW - Scrivener is a plausible novel-writing tool, mentioned a couple of
times in these comments. It took a Long Time, but their iOS app was released a
couple of weeks ago.

Scrivener by Literature & Latte
[https://appsto.re/us/jqx95.i](https://appsto.re/us/jqx95.i)

------
sputknick
You probably don't want to give away specific numbers, but in generally, how
is the profitability of selling on Kindle versus paperback? The price
difference looks to me to be less than the cost difference, thereby making
Kindle more profitable. Is that accurate?

~~~
searine
>You probably don't want to give away specific numbers, but in generally, how
is the profitability of selling on Kindle versus paperback?

Not OP, but I've sold thousands of ebooks, but only a few dozen paperbacks.

Paperbacks are for people with distribution networks and warehouses. As a
self-publisher, we are relying on electronic distribution to cut the overhead
costs. Profit per unit is about the same between the two, but we can sell more
ebooks because the cost to the customer is only a bout a 1/3 compared to
print.

Mostly print copies are for promotion. Handing them out as free prizes/gifts
are a great way to build readers.

------
tn13
There handful of proprietary software I actually pay for because I personally
think they are by far the best tools of the trade. For creative writing I
think I have found the best solution in form of Scrinver.

It has been simply the best tool I have used to write my novels.

------
ksk
This is pretty cool. But I would be interested to know how the author would
have handled other requirements. Embedding images, other scripts, typesetting
different column layouts fonts, spacings, etc.

~~~
dredmorbius
Almost none of that matters in _a novel_. Which has little structure other
than sentences, paragraphs, and chapters. Many won't even include italic text
(though some might).

If your output is simple, stick to simple tools. Tool tyranny -- decisions
you're _forced_ to make, because your tools dictate them -- is a significant
cognitive overhead.

(Incidentally, this is why I found Lyx actually _harder_ to use than LaTeX.
Lyx forced me, _from the beginning_ , to consider the document format and
structure, whilst LaTeX lets you just start writing paragraphs and sorting out
the rest _later_.)

The more extended answer: if you've got other shit you need to do, you address
it when the time comes.

Markdown, specifically, allows for inline inclusion of other formatting
languages, particularly LaTeX and HTML. It also has native support for images,
tables, lists, etc.

You've got the option for Roman, bold, italic, and monospace text. If your
document needs more than this, address it in layout.

In the workflow mentioned here, the author actually answers your question:
images, etc., were added (for the cover) in LibreOffice and through ePub
management tools, including a Python script to reorganise the output file as
needed.

Again, the advantage of Markdown (or rsT, or AsciiDoc, or ...) is that _the
markup gets out of your fucking way_. Instead of obsessing over format, _you
write your fucking story_ (or article, or essay, or rant, or ...). Then deal
with the layout _later_.

------
Animats
LibreOffice has a plug-in for generating ePub files. You can do the whole job
in one step.

Not only that, with LibreOffice, your book can have _pictures_!

~~~
dredmorbius
Markdown: ![arbitraary text](image-uri)

------
newman8r
I like this idea of this.

Would be great to have a website site that shows entire open source workflows
and the optimal software/ approach. Anything like this exit?

------
fitzwatermellow
Ultimately, any text processing toolchain that uses ReGex for parsing will
inherently be difficult to extend. And slow. And perhaps even broken. The
disconnect lies in the human-machine readability gap. The only way to do it
right, I believe, is custom text formats, scanned via structured strings into
FSM grammers.

------
itaysk
You do use a mac though...

------
amelius
Interesting to see that open source DTP is still a challenge in 2016.

------
jlebrech
"self published" _

------
kazinator
ガンベッタさんは、良く頑張った、ね。

------
syngrog66
vim, pandoc

------
caub
no git?

------
RubyMyDear
I did this back in 1990. Using Emacs and TeX/Metafont. And so did many, many
other people.

------
fensipens
> Ex Google Zürich

Badge of Honor

~~~
ggambetta
Relevant, because [https://medium.com/@gabrielgambetta/how-i-wrote-my-first-
nov...](https://medium.com/@gabrielgambetta/how-i-wrote-my-first-novel-during-
my-daily-commute-e1d02c9447b9)

------
garaetjjte
"To create this file, I used Pandoc again, this time to convert the Markdown
files to ODT (OpenDocument Text), which can be read by LibreOffice."

Ouch. WYSIWYGs are terrible.

~~~
reidrac
I think he says he changed styles only, which is closer to the WYSIWYM ("what
you see is what you mean") idea used by Lyx, for example.

I agree that WYSIWYG type of editors are terrible, specially for big
documents, but limiting them to the use of styles, is manageable (in my
experience at least).

------
tjic
Meh.

I wrote two novels in emacs.

(now I just need to find some time to finish the final proof read and actually
PUBLISH the damned things).

~~~
microcolonel
You managed to call one of your novels "recursion". Proof that the medium
colours the work.

------
frostburg
Is it inappropriate to point out that most likely there were a lot of closed
source binary blobs involved in making his computer actually work?

~~~
liw
Does pointing that out advance the discussion in a useful way? Does it help
having a constructive debate? Does it make anyone's day or life better? Are
you just posting in the hope of getting some upvotes for making a funny,
snarky short comment? Is your comment actually relevant to the discussion?
Does the fact that you ask that question indicate that you already know the
answer?

~~~
frostburg
I did not write that from my Lemote, channeling Stallman, nor I have any
general enmity towards closed source software (that I both use and write). I
just don't like inaccurate proclamations of purity.

