"What kept me from [writing my own tool] was a calm voice in my head telling me that I’m here to write a book, not a preprocessor. [...] Now I have written two books and zero tools, which I consider a success."
Many people have problems focusing on the task they set out to do. It's a type of procrastination. Insert that famous gif from Malcolm in the Middle.
This is relevant to startups as well (and business in general): Being able to decide whether something is in scope of a project or not. You are encountering a roadblock, how do you react? Do you set out to solve it perfectly or do you work around it (or even, do you give up)?
The quote you pasted is the perfect way to put it into words. Another common theme I hear about is people asking: "How do I become a successful [profession here]?" (eg. "streamer", "youtuber", "musician", "programmer", "writer", ...). The answer is always the same: "Start".
For me, that time started about two years ago. At 31, a full 25 years since first learning how to program I finally felt like I could be trusted to make the right calls on when to invest the time to abstract or create generation or code parsing tools; and when to do things the old fashion way.
After two years of spending about 20 to 40% of my time on tooling I'd wager I'm about 10x faster than I was at 30 and about 20x faster than I was at 25.
The problem now isn't writing the code. The problem now is that the code is so fast that I don't have the same amount of passive time to think about features / strategy. I have to intentionally set aside time for it and I find it unnatural. Also, I know too much about cybersecurity which is good for my end users, but bad for my feature productivity.
 Poorly. My mum taught me how to code QBASIC. I altered video games.
Indeed yak shaving should only be done when one has the breathing room to do so. It's just good common sense, except there are exceptions. Slack was a yak shaving effort. The most valuable things I've shipped have all been yak shaving efforts. I guess I'm just better at building tools than anything else. Perhaps I enjoy it more. shrug
To each their own, but if you're trying to ship a product instead of a bald yak you're going to have to do it the ugly way, with tools you could fix, etc...
That recursively applies to each tool used to make the product as well.
I think I’m going to frame that and put it on the wall. Did you come up with that or did you hear it somewhere?
I'll get my clients to give me access to an S3 bucket that I can push their data to.
I'll make sure I get my clients to give me a password or public key to encrypt their data before I push it to the S3 bucket because many people fuck up security on those things.
To my end user, it's obviously more secure, but it isn't lining my pocketbook and most end users don't care about security, even if they should. Same thing with security headers or anything, really. Security slows you down at some point.
I don't agree at all. It seems to me that the most interesting advances of mankind have happened precisely when people were trying to do something else, and had to invent new tools. The tools turn out to be more important than the final result because they can be used by other people.
(Author here.) My point was not that you shouldn't invent tools if you need them. By all means, go ahead. My heart beats for creating tools (especially tools to create other tools...).
What I wanted to express was this: if you want to write _a book_, first write the book and only then worry about the tools. If you then notice that you need a custom toolchain to get the book in exactly the shape you need it to be, do it.
I've had people tell me that they would love to write a book, but, well, the existing tools are not sufficient enough. And I think that's the wrong approach to writing a book: write something and you'll quickly realize that the tooling is not your biggest hurdle.
I agree 100% with this. Following the case of Knuth/TAoCP/TeX, in fact the first edition of TAoCP was not written in TeX, he made TeX to improve his already written book.
That’s hilarious. Humans were hand writing books on paper and giving them to other humans to hand write copies. A good craftsman never blames his tools.
Not very many were, and a big reason for that was the available tooling.
> A good craftsman never blames his tools
Someone who writes a book by blames the available tooling support for the low quality of its content is a craftsman (but, per the saying, not a good one) blaming his tools.
A person who chooses not to write a book at all because the anticipated opportunity cost given the existing tooling support and that person’s personal utility function is not.
I managed to write a 25,000 word story in a word processor once, but it was such a jumble that I just closed the file and forgot about writing for a few years. I know now there are ways to organize with folders and spreadsheets, but now I do most of my writing on the phone. Scrivener's binder makes that easy on a small screen.
Tools are super valuable; it's arguable that many of the things I list above are sort of tools. I just don't think that a tool that is quickly slapped together in the service of another project is likely to be a game-changer.
In fact, I'd say that one of the keys to growing your personal impact is to find the best work that's already been done, and use it to further your goal. Even Newton said he was standing on the shoulders of giants.
Then there are the three branches of modern mathematics, according to V.I. Arnold:
1) modern hydrodynamics and PDE, are just guys procrastinating while they are paid by the manufacturers of nuclear submarines
2) celestial mechanics, control theory and ODE, are just guys procrastinating while they are paid by the manufacturers of rockets and missiles
3) number theory, are just guys procrastinating while they are paid by the CIA to develop cryptographic tools
agriculture : this kid is just playing with small seeds instead of collecting nuts for winter
computers : just tools to speed-up the important computations of nuclear reactors
TeX : the guy just had the write a damn book about algorithms but spent 10 years procrastinating
Bach : the guy just wanted to glorify God
I stand by my point that nearly all important advances of mankind are tools developed to solve a long forgotten goal.
If we just open up the idea of "tool" wide enough, everything becomes a tool. Writing a book isn't actually a goal, it's just a tool for acquiring money and fame. Calculating the orbit of planets was just a tool for Newton to establish social credit and historical significance. Agriculture is just a tool for eating, but eating is just a tool for living, but living is just a tool for procreating, but procreating is just a tool for perpetuating genes.
Why are we perpetuating genes? Really, no one knows. Solve that problem, I guess, and finally we'll have the one project enriquto can't easily dismiss as tool development.
That is indeed true.
Of course, almost all of the goals I have accomplished for myself have been accomplished by actively engaging in pursuing that goal rather than pursuing some side channel, while the side projects I've created have generally been a lot less useful to myself and humanity than, say Bach. :D
If you're trying to write a book, and you want to write a book, write the damn book. There's an infinitesimal chance that the tool you're writing instead of your book will change the world.
If you're trying to write that tool, and want to write that tool, then power to you, but don't lie to yourself and tell yourself it's "for the book". Just admit you're switching focus.
A close friend of mine has been trying to create a game for .. a decade or so. Every time he starts, there's always something. Instead of using, say, Unity, he tried his hand at writing a game engine. A physics engine. A programming language. Because none of the tools are "quite right". Of course, none of those get completed either because they're a shitton of work. And his work didn't change the world, because it's never completed. At the end of the day, much like many in the software engineering world, he's a brilliant mind busy optimizing ads. And he doesn't have a working game engine, a working programming language, or a game.
I don't want to sound demeaning, because it's still very good experience to acquire. And in general I don't want to tell people "don't write your own a programming language". I think these things are super cool. But then if that's what you set out to do, make that your project instead, it'll have a much higher chance to actually complete, instead of sitting in an incomplete state, in a hidden unfinished repo with nobody to share it with.
The common thread between all of mankind's most interesting advances: They're finished. How do you finish a project? Well... start.
> If you're trying to write a book, and you want to write a book, write the damn book. There's an infinitesimal chance that the tool you're writing instead of your book will change the world.
Not the person you were replying to, but their point certainly rings true to me, from all the history I've read or otherwise heard of over the years. They could just as easily ask you for citations.
"if what you say is true, it doesn't change anything at an individual level" -- yes it does. It's usually individuals that come up with the important tools and inventions. Those development are very significant because they make systemic advances.
"There's an infinitesimal chance that the tool you're writing instead of your book will change the world" - it doesn't matter how small the odds are for any one individual.
There's two points here. One is that the effects can be cumulative -- even small advances and bits of knowledge gained contribute, over a longer period of time, to large benefits.
The other is that the historical reality is that tools that were developed as side effects of other goals have made a very significant overall effect, and this can't be denied by a blinkered focus on the probabilities concerning individual cases.
For one thing, at the time, Unix and all its associated tools were really not prevalent outside of Bell Labs and a few places — troff would not have been available to run on the computer systems Knuth had available to him in 1977. (The ones at SAIL, which were PDP-10 machines running either TOPS-10 or WAITS, I forget what I read.) He didn't have the source code of troff either; all he had was published descriptions: and those he did use, e.g. the paper by Kernighan and Cherry describing `eqn` came out in 1975 and he used ideas from it while designing TeX's syntax for math.
For another thing, troff was designed to control line printers and other such devices that were typically attached to computers; it simply wasn't designed to drive a digital typesetter of the kind Knuth was thinking of (Alphatype CRS). So he'd need a new program anyway. In fact, he says that until he saw Patrick Winston's book on AI in 1977 (which was one of the early books produced on a digital typesetter, i.e. where the actual technology for getting ink onto the page was based on 0s and 1s, instead of being based on either pieces of lead (hot metal typesetting) or light and lenses (phototypesetting)), he had simply not associated output from computers with “real books”. Things like troff were designed to control the simplistic devices that came with computers, that could move by discrete (large!) amounts on the paper (e.g. some could move by full “lines” only), and strike down specific symbols (from a limited font repertoire that came with the device) onto the page. This wouldn't have seemed relevant to Knuth, who primarily wanted to create his own digital fonts (as he did with METAFONT), and TeX was among other things simply a way to use those fonts.
So, regardless of whether one considers troff to have very beautiful typesetting or not, it simply wouldn't have seemed applicable to the problem at the time, to be rejected as "not good enough" or not. Now of course with output going to generic page-description formats like PDF (even TeX's DVI was not Knuth's idea—he was thinking TeX would just write code for the device directly—and he had to be convinced to do it by Fuchs: https://tug.org/interviews/fuchs.html), and high-resolution printers (fairly high DPI, even if not 5000+ as with the CRS then) available to typical consumers, we don't appreciate all this. :-)
Of course, that's the ideal. I can't use my preferred setup when I'm working on our legacy Windows codebase. I also won't pretend that I'm immune to chasing the new shiny thing to distract me from doing real work, it happens with some frequency. I just think that editors (and perhaps a good tiling window manager) are the exception(s) to the rule.
1. Book is entirely written in LaTeX, every chapter in it's own file, master doc with everything combined. I really like having different gutters for left and right pages, ultimate control over formatting, fancy things like Lettrine, header layout and design etc. When I want to do something across the entire doc I write a new command and can use it anywhere.
I Create perfectly formatted print-ready pdf to go to printers directly from this.
2. Run LaTeX file through pandoc to create an ePub.
A little known trick that is now supported by pandoc is conditionals in the LaTeX doc, so when formatting gets hairy I declare commands like this in the master LaTeX document:
3. Un-pack the ePub and tune some of the html/css to my personal taste, then pack it back together. I run it through kindlegen to make sure it's valid, and I have an epub/mobi ready to upload to stores.
I like this workflow so much I'm using it again for my next book
My book was a non-fiction novel, so that made life much easier, I only had some tricky image layouts to deal with.
I use Sublime Text, sometimes in distraction free mode, but usually not because I often want to Google sciencey things.
I have a single file called "story.txt" that has all the important notes - the names, ages and visual descriptions of the main characters, a detailed (historical) timeline, and chapter outlines I wrote before I started when I knew roughly what the overall plot was going to be. The chapter numbering has changed drastically but the overall outline has stayed pretty true to the original idea.
I really like this system. Keep it simple and focus on writing!
It was written by the same guy who created the original Rails tutorial. You'll be able to generate books that look and feel like this https://www.railstutorial.org/book.
I've released a couple of PDFs using it, and it's pretty decent to work with. There's room for improvement but I haven't come across anything better.
The biggest challenge here, to me, isn't writing the book or figuring out what tools to use, but it's marketing. Email lists, FB ads, etc. are really what drive the success or failure of your book. Nathan Barry's writing on this in Authority is my bible for that.
Also, if we're talking tools, shout out to Leanpub. Love that platform, especially for developer books!
EDIT: Nathan Barry doesn't specifically talk about FB Ads, so in terms of a marketing bible, I guess courses and private FB groups have helped us the most. This guy is actually the main goto for fiction authors: https://selfpublishingformula.com/
I've looked into creating a mailing list with MailChimp and/or its competitors, since I think it'd work great for my target niche. The thing that stopped me is that you need to provide your real name and location. Though I understand the reasoning, I'm publishing under a pseudonymous identity and do not wish to tie my real identity to it. Bummer.
Maybe you’ll get some ideas from the podcast :)
For https://viewsourcecode.org/snaptoken/kilo/02.enteringRawMode... , the code is kept in a git repo where each commit is a step in the tutorial. This way steps can be modified, reordered, or split into more steps using git interactive rebase (git rebase -i).
I'm working on a tool which abstracts away git rebase, and gives you a nicer interface for manipulating the series of steps in your tutorial. For example, you can run `leg 3` to checkout step 3, make edits to it, and then run `leg amend` to apply the changes, resolving conflicts as the changes make their way through the rest of the tutorial's steps.
The tool is still in very early development, but here's a quick tutorial to get a better idea of how it works: https://github.com/snaptoken/leg/blob/master/TUTORIAL.md
Also, I'm playing around with a "literate diff" file format that might be of interest: https://github.com/snaptoken/tgc-tutorial/blob/master/doc/02...
This is 99% of what I need! In the best case it's just files/folders I can edit manually or with the help of a CLI.
> The tool is still in very early development, but here's a quick tutorial to get a better idea of how it works: https://github.com/snaptoken/leg/blob/master/TUTORIAL.md
Now, if those diffs are rendered nicely and the splitting up of a step into multiples (or: explain separate parts of one step) works, I think that really is 99% of what I need. The 1% being the grouping of steps into a chapter and some other small things probably
"But, most importantly, the code is also duplicated: one version lives in a Markdown file and one (or more) lives in the code folder that comes with the book. If I want to update a snippet of code presented in the book, I have to manually update every copy of it. Yes, cumbersome."
I had a similar problem in writing a book about gnuplot: keeping code samples, code in the text, and the generated plots synchronized. I decided doing it manually was too tedious and error-prone, so I wrote some Pandoc filters to do it for me: https://lee-phillips.org/panflute-gnuplot/
Orgmode Babel: https://orgmode.org/worg/org-contrib/babel/
- "literate programming" where the output is actually commented code and the eventual documentation
- "reproducible science" articles, where small independent snippets are executed, even piped into each other, to produce "results" in the form of tables, charts, ...
The author seems to need to highlight differences, which may not fit without hacking its own features, which he appeared to be reluctant to.
I should write a blog post like this. My use case is different, since I write role-playing game books.
I made a blank epub template with 50 empty chapters in it, so when starting a new book, I copy that and write new IDs and strings to the metadata, and if I finish it, I delete the unused chapters and remove their entries from the metadata files. It's easier to delete empties than add new chapters, and doing the metadata by hand ensures there's no auto-generated cruft in my file.
For illustrations I use Paint or GIMP or Inkscape, but the template doesn't have more than just a cover image in it.
After testing various ebook readers, there's no way I would recommend xhtml+css zipped to epub for anything technical. The only reader that actually renders text they way I expect, right out of the box, is Edge. Epub is for novels, not textbooks, papers, or manuals.
The problem I found with it was that I would add the snippet, then later in the chapter, I would refactor the code to better fit the story, and that worked, as long as you only needed one copy of the file. But, it didn't work if you introduced a basic version of the code and then later elaborated and expanded that code and wanted it to reside in the same file. Additionally, you have to keep the code in the same repository, which I didn't want to do since I published companion repositories (which couldn't contain the sources for the book, obviously) and I didn't want to maintain and synchronize two copies of a file.
So I wrote a little preprocessor that ingested a subtle variation on the snippets O'Reilly pioneered. Normal snippets work. But also, you could specify a file and a SHA hash, and if it saw that, it would pull the file at the commit (just using git show). Then I could update the file inside the repository as the code progressed, and still have all the sample code. And my repository always had working code, no matter what branch or point in time I had checked out.
It's here: https://github.com/xrd/oreilly-snippets/blob/master/README.m...
I have the entire book in a GitHub repository , including the export script , which may be of interest to those wishing to write a book a similar way.
(Pardon the new account, but my "normal" HN account is more closely tied to my IRL identity than my Nocturnal identity.)
Has anyone come up with a method for doing what I want? Maybe the ```lang code blocks in md could take parameters like repo, file, line range; then an update to pandoc would fix it all right up. Thoughts?
As I wrote in the post, I didn't use it because it doesn't allow me to include a portion of something and then comment out a part of that, but today I got thinking: if it could include a portion of a function and also the relevant (semantic?) context, it would be 99% of what I need. But the "relevant context" is the tricky part, I guess. I like what git diff does, when it shows the signature of a changed function (does it simply use the first of the previous lines that are less intended?), but I guess it's hard to get really right...
edit: GitChapter is also a really interesting project. If it didn't use a real git repository, but something like folders that resemble single commits, I think I'd use it. https://github.com/chrissound/GitChapter/
Personally, I'm too error-prone to pull that off. I change the code snippets frequently after chapters are done (often in response to bug reports from readers) and having to manually update the snippets in the text would drive me bonkers.
Also, I want to rigorously, automatically validate that the code in the book is correct and contains everything you need.
To do that, I wrote a Python script  that weaves the code and the Markdown together. In the Markdown files, you use a marker like (here in the chapter "Strings"):
^code memory-include-object (2 before, 2 after)
//> Strings memory-include-object
//< Strings memory-include-object
The build script reads the "^code" markers and finds the snippet of code in the sources that correspond to it, as well as any desired surrounding lines. But it can also do a lot of other useful things with the snippet markers:
* If some lines of code are marked as being part of a chapter, but that chapter doesn't have a "^code" marker to include them, it reports an error. This way I can't accidentally drop code on the floor.
* It can generate a separate copy of the entire codebase that only includes the snippets that are in a given chapter (and the preceding ones). In other words it can output "what does all the code look like by the time the reader reaches the end of chapter ___" for every chapter.
I output the code for each chapter and then compile that and run all of the tests. This way, I can ensure that by the end of each chapter, the program supports what it's supposed to support by then. I've found countless bugs doing this -- places where I inadvertently put code in a later chapter that is actually needed earlier.
However, this build script is not a generic "build a book" system. It's a bespoke program that only supports this one specific book. That helps keep the complexity down and makes it much easier to put little hacks in when I run into weird edge cases my specific book has. I think that's a reasonable trade-off to keep me producing a high-quality book without too much yak shaving.
One of the days, I'll write a full blog post going through this in more detail.
Of course, none of this is a criticism of Thorsten's system. The simple most important skill in writing a book is figuring how to get those pages done. It doesn't matter if that means waking up at three in the morning and writing on parchment with a red pencil. Whatever works for you, works.
Garmin's own programming language for device apps is called Monkey C.
It was previously called Monkey but then changed to Monkey X, which allowed me to use Monkey. Fun fact: I originally wanted to call it Tiger, because they're my favorite animals, but not two days after I wrote that name down, I opened Appel's Modern Compiler Implementation in ML and discovered that the language implemented in the book is also called Tiger. Thus, a Tiger became a Monkey.
I think KDP allows setting different pricing for different countries.
I wonder how well this method would work for something like an illustrated children’s book?
What I really like about AsciiDoc is that it's right near Markdown in terms of simplicity, but with a bit more beef behind it to handle more complex things like referencing figures. If you don't need them, you can stick with the basic syntax, but the power is there when you want it.
Markdown is limited, but it's very lightweight and doesn't break your flow. It's a piece of cake to type a couple of "*" when you want to italicize something.
Also, the limitations are a benefit because they keep you from getting distracted by the hundred different ways you could structure or format a paragraph.
I understand the appeal of markdown but I avoid it for writing coding tutorials because it doesn't have an easy way to annotate code with graphics like arrows and floating callouts. Here are some visual examples of code enhanced by arrows: 
Also, all types of text files besides programming syntax can also be explained with graphical annotations such as configuration files or log output files. E.g.
Therefore, I use MS Word because it makes annotation easy. (LaTeX can also do annotation but that has its own workflow issues such as the typical programmer not knowing it which means the team can't easily update the documentation.)
IMO, annotation is important because if we performed eye-tracking and heatmap tests with people reading code tutorials, we'd see that a significant amount of time is wasted with the eyes constantly shifting back and forth between the code and the subsequent paragraphs explaining that code that's located a few inches down the page. I also think this is one reason why live coding demonstrations can sometimes be more effective than reading a book because the presenter is wiggling the mouse (or laser pointer) around the exact spot of the syntax while he's talking.
This can be somewhat mitigated with adding inline code comments in the example but often, it still doesn't point to the exact point in the line that needs to be studied. Code comments also don't easily illustrate (because it has no spatial cues) how separate lines are related to each other.
One workaround of markdown's limitation would be to rasterize the code's text into an image and then use Photoshop to annotate. However, with MS Word, that cumbersome step isn't required.
Missing is everything about the writing process, structuring content and editing.
Every dictionary I just consulted considers the primary definition to be the physical artifact: pages bound together. Some have secondary definitions that refer to any long written work. I found none that excluded non-fiction texts from it.
Are you claiming that phone books are not books? Recipe books are not books? Have you ever once heard a human say, "Look up their number in the phone text?"