This is awesome, and (since it looks like he came up with the setup) I declare Tim Wheeler personally to be awesome.
The thing kind of reminds me of the setup that Robert Sedgewick used for his algorithms textbooks. They've got code snippets, plus a ridiculous number of diagrams that visualize the algorithms step by step on toy inputs. They're an incredible aid to understanding, and nobody would possibly have the patience to draw all those. The book even has extra-wide margins to hold them all!
Somewhere in the book (the preface?) he explains how the hell he drew all of those, and it's a real story. He wrote the code snippets as full self-contained programs that `printf()` some algorithm visualization data as they go along. Then he used some literate programming tool to chop out the inessential parts for the book -- but he runs the programs also, somewhere in his makefile, and gets them to spit out their big list of numbers. He then converts this to a diagram using a widely-known command-line tool called "cat": he wrote a bunch of diagram-drawing programs in PostScript, and they'll have the necessary data waiting on the stack if he just appends them to the output of the code snippets.
I weirdly love reading about other people's workflow. This is really cool thanks for sharing.
One issue I have with literate programming is that its difficult enough to write code, now you have to worry about writing code in the middle of writing prose. For that reason I prefer to just have all my code output a bunch of latex files so I can separate the code-writing from the paper-writing. Which I guess is against the philosophy of literate programming as a whole...
How did you get around this? Do you share the same frustration that you are constantly being distracted by one or the other?
Julia turns out to be a very nice language for problems like this. Many of the algorithms end up looking very much like the pseudocode from the original paper. We can use unicode as well, which makes a lot of the math (like gradient symbols) easier to read. We drop the type annotations to make it more legible. "Real" implementations could definitely improve what we have in terms of error checking and efficiency improvements, but we optimized for understanding and clarity.
Very often getting the figures to generate properly helped us catch mistakes in the code. I usually did this in a Jupyter notebook, because I can iterate there a lot faster than making a small in the book source and running the compilation process there.
I guess we didn't really have to get around the problem - the algorithm was of principle importance and then we had to figure out how to best produce figures for it.
What is the license for the code samples? I seem to remember that the code in Numerical Recipes, which is stated as a similar book to some degree, is actually fairly restrictive [0].
I am not a lawyer, but I for one would be thrilled to hear of any of this being used for good by anyone, whether for personal or commercial uses. I don't see anything other than the standard MIT Press protection clause behind the cover at the moment but will definitely look into it. Thanks!
This looks like a great piece of hard work covering topics that are not easy to grasp. I am wondering why the author(s) chose to write their code samples in Julia. Most algorithm textbooks I have come across use their own "pseudocode" which often ends up being challenging to read, but in a perfect world, I would selfishly like to have all of the code snippets in Python. Even though this textbook doesn't do that (although let's not rule out future optimizations!), the amount of work that went into this was clearly astounding and well done.
Mykel Kochenderfer is a coauthor on the book and the PhD advisor to Tim. The video [1] below describes how Kochenderfer came to be using Julia (it's faster than Python, and has an AST that the formal verification guys like).
Many of the algorithms would be very laborious to implement and explain in a language like C or C++. Julia has the advantage that the actual code looks very much like the pseudocode you mention. In fact, the guideline implementation for the ACAS X collision avoidance software that my prof developed, that is to become the de-facto issued by the FAA, has been produced in Julia rather than in pseudocode because it actually runs. Real code has no ambiguity. (assuming you avoid undefined behavior).
Julia is a fantastic language. Really hope it catches on.
I really wish that all of these books had a wiki so that users could post their own solutions with corresponding discussion. Particularly practical implementation details that don't belong in the text of the book.
As a grad student that has started and re-started writing a textbook for a couple of years, this is super inspiring! I love hearing about clever LaTeX tooling too.
The thing kind of reminds me of the setup that Robert Sedgewick used for his algorithms textbooks. They've got code snippets, plus a ridiculous number of diagrams that visualize the algorithms step by step on toy inputs. They're an incredible aid to understanding, and nobody would possibly have the patience to draw all those. The book even has extra-wide margins to hold them all!
Somewhere in the book (the preface?) he explains how the hell he drew all of those, and it's a real story. He wrote the code snippets as full self-contained programs that `printf()` some algorithm visualization data as they go along. Then he used some literate programming tool to chop out the inessential parts for the book -- but he runs the programs also, somewhere in his makefile, and gets them to spit out their big list of numbers. He then converts this to a diagram using a widely-known command-line tool called "cat": he wrote a bunch of diagram-drawing programs in PostScript, and they'll have the necessary data waiting on the stack if he just appends them to the output of the code snippets.