
For Donald Knuth, good coding is synonymous with beautiful expression - theafh
https://www.quantamagazine.org/computer-scientist-donald-knuth-cant-stop-telling-stories-20200416/
======
bitwize
People often say "TAOCP is a dense, technical book, you don't just sit down
and read it cover to cover." It is dense and it is technical, but it's also a
joy to sit down and read and this is why. Knuth doesn't present an algorithm
and annotate it with (Foobar 1968) and that's that. He says "Bazquux came up
with an early version of this algorithm in 1962 while working on missile
trajectories for the Department of Defense, then Foobar came up with the
linear-time optimization that we all use today." Stuff like that.

~~~
pmiller2
I agree that Knuth's exposition is a joy to read, and there are a lot of
significant mathematical insights to be had within those volumes. I just wish
he could get away from MIX/MMIX. I don't find it very useful or interesting as
a pseudocode, and I think it gets in the way of understanding.

~~~
kragen
On the contrary, I think it's extremely useful to have the algorithms realized
in a form that has a cost model free of handwaving. The books are to a great
extent focused on computational costs, and high-level languages obscure those
costs.

The only exception is that the books only rarely dive into the variable
computational costs due to operating on values of different sizes; heapsort,
for example, is only O(N log N) time if comparisons take constant time, while,
in fact, as key cardinality approaches infinity, key size grows as O(log N).
(Which is why O(N) time sorting is possible.) This kind of thing is growing
more important as hardware design becomes more accessible; there are probably
more people writing VHDL and Verilog now than were writing any kind of
software when Volume 1 came out.

So I think using a high-level language would get in the way of understanding
those costs, and even to some extent how to implement the algorithm in other
contexts.

If you just want to implement a red-black tree or something, then sure, the
MMIX code isn't going to help you. But he also presents all the algorithms in
pseudocode, so you can use that.

~~~
pmiller2
There's no reason you can't take into account things like varying key size,
memory allocation, etc. in your computational model when it matters. For
comparison sorting, the reason it isn't done is that if you're comparing keys
of the same length, in the worst case, you have to look at the entirety of
both keys. O(n log n) might be misleading, but it's a white lie at best.

~~~
kragen
There are a lot of very interesting issues there. For example, surely you have
to look at the entirety of both keys at least once, yes; but you don't have to
do it on every comparison, for example in mergesort where you can store a
longest common prefix between pairs of adjacent keys and avoid comparing them
byte by byte most of the time, or even storing them. The algorithm is tricky
both to implement and to analyze, and I don't think anyone has done it,
because there are better options for long keys; but I conjecture that it
actually gets mergesort down to a real O(N log N) instead of the O(N log N log
N) of the usual mergesorts.

I suspect that isn't the reason this kind of analysis isn't usually done,
though. If you measure the running time of a standard comparison sorting
algorithm on real data on almost any general-purpose computer from 1955 to
1985, you will find that, above very small sizes, it is quite precisely
proportional to N log N, where N is the number of records. The extra factor of
log N doesn't show up. Why is that?

It's because almost all of those computers, even bit-serial designs like the
RCA 1802 and the LGP-30, aren't bit-addressable; the smallest amount of data
an instruction can manipulate is a character, typically 6-9 bits, but often 32
bits. And almost none of them could manage more than 4 gigabytes of data,
because that would have been too expensive. So in practice the average time to
compare two separately stored keys does not vary by a factor of 100 or so over
the data sets the machine could process, as you would expect from looking at
key unique prefix length. It might vary by a factor of 2 to 4, but more often
was actually constant. MIX in particular had 5-character memory words, but
only 4000 words of memory, so its comparison time for internal sorting is
quite precisely constant for any realistic data.

After 1985, caches and massive parallelism complicate the picture a bit,
initially for largish machines (though Stretch too had cache, such monsters
were exceptions) and finally for almost all types of computers, though perhaps
not yet the numerical majority of computers, which are probably mostly 8051s
and Cortex-Ms and things like that.

Anyway, back to the issue at hand: assembly language exposes all the
computational costs by default, which is what you want if you're going to
prove theorems about them, while high-level languages obscure them by default,
so more of your theorems will be incorrect. And that's Knuth's primary
concern.

That level of single-minded pursuit of excellence is the reason we can learn
things from books Knuth wrote in the 1960s that are still useful for
programming computers that are literally a billion times bigger than the
machines Knuth used as his model. It's also the reason he's not done yet, 55
years later, with what he thought would be a single book, done in a year or
two.

~~~
pmiller2
>I conjecture that it actually gets mergesort down to a real O(N log N)
instead of the O(N log N log N) of the usual mergesorts.

I don't think it does in the worst case. I'd bet money you could construct an
adversarial input that would force you to look at more of each key than you'd
have to in order to get down to "real" O(n log n).

>MIX in particular had 5-character memory words, but only 4000 words of
memory, so its comparison time for internal sorting is quite precisely
constant for any realistic data.

If we're being precise here, then it is certainly _bounded above by_ a
constant. But, this is also the same sense in which there is literally no such
thing as a Turing machine (all physical machines, even one the size of the
universe, are linear bounded automata, at best).

> Anyway, back to the issue at hand: assembly language exposes all the
> computational costs by default, which is what you want if you're going to
> prove theorems about them, while high-level languages obscure them by
> default, so more of your theorems will be incorrect. And that's Knuth's
> primary concern.

Except, no, you don't. Both you and I just got through explaining why.

> That level of single-minded pursuit of excellence is the reason we can learn
> things from books Knuth wrote in the 1960s that are still useful for
> programming computers that are literally a billion times bigger than the
> machines Knuth used as his model. It's also the reason he's not done yet, 55
> years later, with what he thought would be a single book, done in a year or
> two.

No, it most certainly is not. I'm assuming you've read at least some of TAOCP.
Knuth certainly introduced some mathematics and techniques for the analysis of
algorithms in TAOCP, but literally none of the algorithms themselves were
first published there. This is not a research monograph. It's a reference. The
algorithms themselves were published and proven, including runtimes, before
they ever made it into those pages.

And, yes, there is plenty that almost anyone can learn from these books and
the subsequent fascicles, but it's nothing to do with the computational model.
Not one result in TAOCP contradicts the previously published work. We can
learn from it because there is simply so much there to learn from. There's a
reason that when Steve Jobs told Don Knuth "I've read all your books," Knuth
responded that jobs was "full of shit." [0]

\---

[0]:[https://www.folklore.org/StoryView.py?project=Macintosh&stor...](https://www.folklore.org/StoryView.py?project=Macintosh&story=Close_Encounters_of_the_Steve_Kind.txt)

------
markarranz
> I write an average of five new programs every week. Poets have to write
> poems. I have to write computer programs.

It seems more obvious now that I read it. A lot of programmers out there,
including myself, are looking for ways to improve their skills and I never
thought to myself to write more programs. And not just programs for the sake
of programs, but programs that force you to explain to the computer that you
understand certain concepts. This man is and will continue to be a huge
inspiration.

~~~
2OEH8eoCRo0
>The ceramics teacher announced on opening day that he was dividing the class
into two groups. All those on the left side of the studio, he said, would be
graded solely on the quantity of work they produced, all those on the right
solely on its quality. His procedure was simple: on the final day of class he
would bring in his bathroom scales and weigh the work of the "quantity" group:
fifty pound of pots rated an "A", forty pounds a "B", and so on. Those being
graded on "quality", however, needed to produce only one pot - albeit a
perfect one - to get an "A". Well, came grading time and a curious fact
emerged: the works of highest quality were all produced by the group being
graded for quantity. It seems that while the "quantity" group was busily
churning out piles of work - and learning from their mistakes - the "quality"
group had sat theorizing about perfection, and in the end had little more to
show for their efforts than grandiose theories and a pile of dead clay.

~~~
gowld
Cool but fake.

~~~
2OEH8eoCRo0
You should really post a source.

~~~
gowld
A source that disproves a made up unsourced story is fake?

~~~
MR4D
Here's two that prove it.

[0] - [https://blog.codinghorror.com/quantity-always-trumps-
quality...](https://blog.codinghorror.com/quantity-always-trumps-quality/)

[1] -
[https://www.amazon.com/dp/0961454733/?tag=codihorr-20](https://www.amazon.com/dp/0961454733/?tag=codihorr-20)

~~~
detaro
prove what? That someone printed the anecdote in a book?

~~~
sincerely
What would suffice, video recordings of every class?

~~~
gowld
Naming the teacher, the school, the year, the city, something that would give
a hint for someone to follow up for verification.

------
third_I
As Paul Dirac so eloquently put,

> _“I think that there is a moral to this story, namely that it is more
> important to have beauty in one 's equations that to have them fit
> experiment. If Schrödinger had been more confident of his work, he could
> have published it some months earlier, and he could have published a more
> accurate equation. It seems that if one is working from the point of view of
> getting beauty in one's equations, and if one has really a sound insight,
> one is on a sure line of progress. If there is not complete agreement
> between the results of one's work and experiment, one should not allow
> oneself to be too discouraged, because the discrepancy may well be due to
> minor features that are not properly taken into account and that will get
> cleared up with further development of the theory.”_

> Scientific American, May 1963.

By that token, coding should be beautiful first, in elegance of structure and
forms. _And then we can iron out the bugs._

I tend to agree.

~~~
typon
Isn't this line of thinking which has held physics back for the past few
decades? I thought that physics is gradually moving to being more data driven
than being driven by ideas of beauty. Sabine Hossenfelder has written quite a
bit about this.

~~~
third_I
The problem with theoretical physics, in my external perception, is more
political, within academia.

\- selecting for the H index, that is, what will be most cited, leading to
strong convergence around "consensual" or "popular" research, at a huge
detriment to diversity of ideas

\- one negative effect is a major push for incremental publication (equivalent
of a LOC KPI for coders... it's just wrong). “What did you publish this year?
How many times was your name in a paper somewhere?”

\- putting funds and institutional support behind people you like as opposed
to people with the most promising ideas. That's pretty much the story of
string theory, and see how despite the failure they are still "winning"
politically, occupying top positions and feeding the mainstream vulgarization,
whereas those who "lost" decades ago are still AWOL even as their ideas are
freshened up by a new generation, whose future against the institutional
pyramid is all but guaranteed.

We might still be there (a situation of total stagnation, since the 1970s, at
the theoretical level) by 2040 if something doesn't change profoundly in the
way university politics have such a compelling handle on research, thus on our
progress in science in general.

My 2cts but I'm parroting, people who now speak up in academia.

What I do know about, ML, is all but dead currently in MIT, Stanford, etc.
It's just monkey coding applying whatever the industry likes for short-term
benefits (again this "incremental"-only approach, playing it safe with 0.001%
gains that are, sure, predictably doable). There's no actual research on "AI"
let alone the general notion of "intelligence". Not even in neuro. I fear
we've lost sight of the importance of basic research, as basic as it gets in a
given field, because industry deals run the show in terms of funding. Yet look
what Bell Labs or MIT did in the 1970-80s, surfing on the last wave of
fundamental breakthroughs of the 50-60s. These days are long gone.

------
senthil_rajasek
Whenever people bring up aesthetics in coding they forget to balance it. Here
is the man himself on how he feels about code aesthetics, “I do think issues
of style do come through and make certain programs a genuine pleasure to read.
Probably not, however, to the extent that they would give me any
transcendental emotions.” [1]

[1] Knuth, D. E. Things a Computer Scientist Rarely Talks About. Center for
the Study of Language and Information, Stanford, California. 2001.

------
plg
I like this:

"A person’s success in life is determined by having a high minimum, not a high
maximum. If you can do something really well but there are other things at
which you’re failing, the latter will hold you back. But if almost everything
you do is up there, then you’ve got a good life."

~~~
circlefavshape
I've heard it said (maybe by Frank Zappa? can't remember for sure) that the
difference between a professional musician and an amateur is not how well they
play at their best, but how well they play at their worst.

~~~
heinrichhartman
The hardest part about professional music, is that you have to perform at a
schedule. It takes a lot of effort to perpare your body physically and
mentally for peak performance ... and often-times you just fail. Have a bad
day. That happens.

There will be many concerts, where you are just at 70% of your peak ability.

As a professional musician, you have to make sure your 70%-version is good
enough that people gladly pay for seeing it.

~~~
coldtea
> _As a professional musician, you have to make sure your 70%-version is good
> enough that people gladly pay for seeing it._

Or you just do electronic music, so that's not an issue!

~~~
snazz
A lot of live electronic music performances involve some pretty serious
showmanship.

~~~
strbean
One of my favorite "laptop DJ" moments -

I saw Brock Hampton at a festival in 2018. Hadn't heard of them before, and
they were described to me as a boy band so I had low expectations. Their
performance was bizarrely wonderful - they were all painted blue and wearing
orange jumpsuits. Anyways, they have a DJ with a Macbook and a bunch of
rappers. At one point mid-song they have a bar of silence, and I swear I saw
the DJ lean forward and hit spacebar to pause the track and then resume it
after.

------
mkchoi212
I feel like when people start talking about TAOCP, the first thing that comes
up is the fact that you need to know a good amount of math to understand
Knuth's language. But what people are forgetting is the fact that his books
are just bunch of stories told in mathematical language. You need just the
slightest of math knowledge to be able to follow his stories. Maybe hard in
the beginning - because yes, it's dense - but once you get used to his way of
storytelling, you get hooked.

------
lordleft
Donald Knuth manages to be brilliant, well-read and also just a nice person.
Sort of refreshing given our field's tendency to let mean-spirited people off
the hook if they happen to be good at what they do.

~~~
tartoran
Even here on HN I've seen quite a bit of criticism of Donald Knuth alluding
that somehow he's become irrelevant. Personally I appreciate Donald Knuth the
scientist as well as Donald Knuth the person. Mean spirited people will get
better at what they do but will probably miss the mark on becoming a great and
humble person such as Donald Knuth. I'm really grateful that he's still among
us and still working on his his books and that he shares his wisdom with us.

------
kragen
A thing I never knew before: Donald Knuth, the paragon of scholarship and of a
life well spent, not only cleans his own toilets but bought a specific
janitorial uniform to make the process more efficient.

I find this inspiring enough that I am going to clean my toilet now.

~~~
7thaccount
Who else would clean his toilets? Is he wealthy? I assumed no.

~~~
kragen
Yes, that's how he was able to build his house around a custom-built pipe
organ. Also, though, many men of his generation leave the toilet-cleaning up
to their wives. But he says he and Jill each have uniforms for this purpose.

(My toilet looks and smells much better now.)

~~~
7thaccount
Makes a bit more sense.

------
philiplu
I learned programming from TAOCP, but that was in the mid-70s, when good
expository texts were rarer. I mostly skipped over the math then, and tried to
figure out the algorithms. Actually wrote a MIX assembler/simulator, which
taught me a ton of low-level programming.

If you're into the math side of this, I'd highly recommend the book "Concrete
Mathematics", which Knuth wrote with Graham and Patashnik. I got it to get
better at solving problems over at Project Euler, but it's a fun read, like
TAOCP. As deep as it is, it's amazingly light-hearted, down to the graffiti in
the margins from the students who proof-tested the material in a Stanford
course.

------
smitty1e
There is a joy that shines through everything Knuth does. Those who have ears
hear the reason for this.

~~~
mcphage
> Those who have ears hear the reason for this.

You think that's it?

~~~
smitty1e
Absolutely.

------
plg
Man for an older person he is using a really small font on his screen! Also:
looks like a GNU/Linux variant not windows or macOS.

~~~
sasaf5
True. The editor on the left side does look like emacs.

~~~
alxlaz
That's the FVWM window manager. His FVWM config (or at least some version of
it) is available if you're curious: [https://www-cs-
faculty.stanford.edu/~knuth/programs/.fvwm2rc](https://www-cs-
faculty.stanford.edu/~knuth/programs/.fvwm2rc) .

~~~
sasaf5
Thanks so much for this link! It also explains what I am seeing in that
picture:

"Now comes the fun part: Buttons to push that will take me from one desktop to
another.

Being a control freak, I am not trusting FvwmButtons to find the correct
button layout; I'm building it myself. The goal is to have a 64x64 ASClock at
the upper right, preceded by four 32x32 buttons that will aim my display at
another desktop, all above a 16x128 CPU load display. Geometry-wise, I
consider it to be an 8x5 grid of 16x16 squares (although I could have regarded
it as a 4x5 grid of 32x16s)."

------
mkchoi212
So much respect for the man. Hope he stays safe during these times and lives
long enough to finish the series :D

------
wglb
An interesting book interviewing high-level programmers is
[https://en.wikipedia.org/wiki/Coders_at_work](https://en.wikipedia.org/wiki/Coders_at_work).
The book starts out with JWZ and ends with Knuth. In addition to covering
their work, Peter also asks each one whether or not they have read TaoCP, how
they feel about C++, and whether the use a debugger.

Only one of the people has read the book, with even Knuth saying that he
hasn't read it. Of course, he is joking.

But this is a fun read.

------
legedemon
Does someone know which are the books lying on his bookshelf? Alternatively,
is there an interview with Donald Knuth where he mentions the books he
studies.

------
galaxyLogic
I think a point is that when you create software there are so many choices how
you could create a given piece of program. Trying to figure the best way to do
it before you actually do it could take very long, and you still would not be
very much wiser because it would all be in your head not in software. Whereas
if you just do it you have something you can learn from, and make
incrementally better

------
ScannerSparkly
Has anyone read his books? Seems like an interesting way to get into
programming. I enjoy reading, but I never thought that a book about
programming would present it's ideas and technical details through stories.

~~~
dhosek
My first major programs were written under the influence of Knuth—I spent a
lot of time reading the source code of TeX and its related programs and
learned a lot from that. In fact, I only ever took one computer science class
in my life (I went three times and got a C), so I'd have to say that Knuth was
by far the most formative influence on my early programming. The other day I
was actually looking to see if there was any trace of the DVI previewer I
wrote for VM/CMS back in the 80s around on the internet. As near as I can
tell, there is not. It'd probably be embarrassing to see (among other things,
I didn't do any caching of font bitmaps, mostly because I didn't know how and
didn't have time to learn, so every character displayed on screen re-read the
bitmap data from disk).

~~~
kragen
How do you get pixels on a screen from VM/CMS?

~~~
dhosek
There were two graphic output options in the previewer. There were specialized
graphic terminals using the GDDM protocol (this code was actually written by
someone in Germany who sent me their changes). The original code that I wrote
used Tektronics graphics protocols available in the terminal driver that
connected to the mainframe via a protocol converter that enabled the use of
cheap ASCII terminals instead of the standard dedicated IBM terminals.

~~~
kragen
Oh man, you were drawing pixel fonts on a 4014 storage tube? That must have
taken _forever_.

~~~
dhosek
It was an emulation of it on a PC. VT100+Tektronix was a common graphics
option on terminals of the era and the PC terminal software provided that as
its graphics choice. I had some optimizations like replacing any characters
below a certain threshold size with a solid box based on the bitmap's bounding
box. It was reasonably fast given the speed of the connection.

------
Tomis02
> Science is much easier to learn if you know the sequence of discoveries.

Is there a math book that gives the historical context (what is the exact
problem this piece of math is trying to solve)?

------
ssivark
Here’s the funny thing... If Knuth (or a modern avatar) were applying for a
faculty position today, or coming up for tenure, would s/he succeed?

------
knolax
How does that gel with TeX, whose syntax creates expressions that are nearly
universally regarded as ugly?

------
pkhamre
I tend to be a good writer and able to express specific feelings easily.
Should I learn to code?

~~~
RMPR
Coding is a valuable skill for everyone, not only computer scientist, a
popular recommendation of mine for those who want to learn what it looks like
while having a decent introduction is: Automate The Boring Stuff With Python
[https://nostarch.com/automatestuff2](https://nostarch.com/automatestuff2)

------
m23khan
Donald Knuth and Larry Wall - I find these guys in the field of Computer
Science the most intriguing.

