I would love a reference about the yellow legal note pad thing.
In reality, writing a program in longhand is something everybody could do -- but because nowadays programming is mostly plumbing, you have to empirically test everything.
I saw it with my own eyes when I was in his office to discuss some typesetter-interfacing issues, but I suppose that's not enough of a reference. Perhaps the story is repeated in an old issue of TuGboat, the journal of the TeX Users Group, but I couldn't find it. Here's a more extensive description from a decade ago though: https://groups.google.com/d/msg/comp.text.tex/9quGg7j6U0k/tl...
In reality, writing a program in longhand is something everybody could do -- but because nowadays programming is mostly plumbing, you have to empirically test everything.