
Knuth on Huang's Sensitivity Proof: “I've got the proof down to one page” [pdf] - espeed
https://www.scottaaronson.com/blog/?p=4229#comment-1815290
======
svat
A few random comments:

• Obviously, this is typeset with TeX.

• Though originally Knuth created TeX for books rather than single-page
articles, he's most familiar with this tool so it's unsurprising that he'd use
it to just type something out. (I remember reading somewhere that Joel
Spolsky, who was PM on Excel, used Excel for everything.)

• To create the PDF, where most modern TeX users might just use pdftex, he
seems to first created a DVI file with tex (see the PDF's title “huang.dvi”),
then gone via dvips (version 5.98, from 2009) to convert to PostScript, then
(perhaps on another computer?) “Acrobat Distiller 19.0 (Macintosh)” to go from
PS to PDF.

• If you find it different from the “typical” paper typeset with LaTeX,
remember that Knuth doesn't use LaTeX; this is typeset in plain TeX. :-)
Unlike LaTeX which aims to be a “document preparation system” with
“logical”/“structured” (“semantic”) markup rather than visual formatting, for
Knuth TeX is just a tool; typically he works with pencil and paper and uses a
computer/TeX only for the final typesetting, where all he needs is to control
the formatting.

• Despite being typeset with TeX which is supposed to produce beautiful
results, the document may appear very poor on your computer screen (at least
it did when I first viewed it on a Linux desktop; on a Mac laptop with Retina
display it looks much better though somewhat “light”). But if you zoom in
quite a bit, or print it, it looks great. The reason is that Knuth uses bitmap
(raster) fonts, not vector fonts like the rest of the world. Once bitten by
“advances” in font technology (his original motivation to create TeX &
METAFONT), he now prefers to use bitmap fonts and completely specify the
appearance (when printed/viewed on a sufficiently high-resolution device
anyway), rather than use vector fonts where the precise rasterization is up to
the PDF viewer.

• An extension of the same point: everything in his workflow is optimized for
print, not onscreen rendering. For instance, the PDF title is left as
“huang.dvi” (because no one can look at it when printed), the characters are
not copyable, etc. (All these problems are fixable with TeX too these days.)

• Note what Knuth has done here: he's taken a published paper, understood it
well, thought hard about it, and come up with (what he feels is) the “best”
way to present this result. This has been his primary activity all his life,
with _The Art of Computer Programming_ , etc. Every page of TAOCP is full of
results from the research literature that Knuth has often understood better
than even the original authors, and presented in a great and uniform style —
those who say TAOCP is hard to read or boring(!) just have to compare against
the original papers to understand Knuth's achievement. He's basically
“digested” the entire literature, passed it through his personal
interestingness filter, and presented it an engaging style with enthusiasm to
explain and share.

> when Knuth won the Kyoto Prize after TAOCP Volume 3, there was a faculty
> reception at Stanford. McCarthy congratulated Knuth and said, "You must have
> read 500 papers before writing it." Knuth answered, "Actually, it was
> 5,000." Ever since, I look at TAOCP and consider that each page is the witty
> and insightful synthesis of ten scholarly papers, with added Knuth insights
> and inventions.

([https://blog.computationalcomplexity.org/2011/10/john-
mccart...](https://blog.computationalcomplexity.org/2011/10/john-
mccarthy-1927-2011.html?showComment=1319546990817#c6154784930906980717))

• I remember a lunchtime conversation with some colleagues at work a few years
ago, where the topic of the Turing Award came up. Someone mentioned that Knuth
won the Turing Award for writing (3 volumes of) TAOCP, and the other person
did not find it plausible, and said something like “The Turing Award is not
given for writing textbooks; it's given for doing important research...” — but
in fact Knuth did receive the award for writing TAOCP; writing and summarizing
other people's work is his way of doing research, advancing the field by
unifying many disparate ideas and extending them. When he invented the Knuth-
Morris-Pratt algorithm in his mind he was “merely” applying Cook's theorem on
automata to a special case, when he invented LR parsing he was “merely”
summarizing various approaches he had collected for writing his book on
compilers, etc. Even his recent volumes/fascicles of TAOCP are breaking new
ground (e.g. currently simply trying to write about Dancing Links as well as
he can, he's coming up with applying it to min-cost exact covers, etc.

Sorry for long comment, got carried away :-)

~~~
svat
Looks like no one complained about the long comment, so some further trivia I
omitted mentioning:

• The problem that one cannot copy text from a PDF created via dvips and using
METAFONT-generated bitmap fonts has recently been fixed — the original author
of dvips, Tomas Rokicki ([1], [2]) has “come out of retirement” (as far as
this program is concerned anyway) to fix this and is giving a talk about it
next week at the TeX Users Group conference ([3], [4]):

> Admittedly this is a rare path these days; most people are using pdfTeX or
> using Type 1 fonts with dvips, but at least one prominent user continues to
> use bitmap fonts.

So in the future (when/if Knuth upgrades!) his PDFs too will be searchable.
:-)

• In some sense, even Knuth's work on TeX and METAFONT can be seen as an
extension of his drive to understand and explain (in his own unique way)
others' work: at one point, suddenly having to worry about the appearance of
his books, he took the time to learn intensively about typesetting and font
design, then experiment and encode whatever he had learned into programs of
production quality (given constraints of the time). This is in keeping with
his philosophy: “Science is what we understand well enough to explain to a
computer. Art is everything else we do.” and (paraphrasing from a few mentions
like [5] and [6]) “The best way to understand something is to teach it _to a
computer_ ”.

• Finally returning (somewhat) to the topic, and looking at the 2/3rds-page
proof that Knuth posted [7], one may ask, is it really any “better”, or
“simpler”, than Huang's original proof [8]? After all, Huang's proof is
already very short: just about a page and a half, for a major open problem for
30 years; see the recent Quanta article ([9], HN discussion [10]). And by
using Cauchy’s Interlace Theorem, graph terminology, and eigenvalues, it puts
the theorem in context and (to researchers in the field) a “natural” setting,
compared to Knuth's proof that cuts through all that and keeps only the
unavoidable bare essentials. This is a valid objection; my response to that
would be: different readers are different, and there are surely some readers
to whom a proof that does not even involve eigenvalues is really more
accessible. A personal story: in grad school I “learned” the simplex algorithm
for linear programming. Actually I never quite learned it, and couldn't answer
basic questions about it. Then more recently I discovered Knuth's “literate
program” implementing and explaining the simplex algorithm [11], and that one
I understood much better.

> The famous simplex procedure is subtle yet not difficult to fathom, even
> when we are careful to avoid infinite loops. But I always tend to forget the
> details a short time after seeing them explained in a book. Therefore I will
> try here to present the algorithm in my own favorite way—which tends to be
> algebraic and combinatoric rather than geometric—in hopes that the ideas
> will then be forever memorable, at least in my own mind.

I can relate: although the simplex algorithm has an elegant geometrical
interpretation about what happens when it does pivoting etc., and this is the
way one “ought” to think about it, somehow I am more comfortable with symbol-
pushing, having an underdeveloped intuition for geometry and better intuition
for computational processes (algorithms). Reading Knuth's exposition, which
may seem pointless to someone more comfortable with the geometrical
presentation, “clicked” for me in a way nothing had before.

This is one reason I am so fascinated by the work of Don Knuth: though I
cannot hope to compare myself in either ability (even his exploits as a
college kid are legendary [12]) or productivity or taste, I _can_ relate to
some of his aesthetic preferences such as for certain areas/styles of
mathematics/programming over others, and being able to so well “relate” to
someone this way gives you hope that maybe by adopting some of the same habits
that worked for them (e.g.: somewhere, Knuth mentions that tries to start
every day by doing whichever thing he's been dreading the most), you'll be
able to move a few steps in somewhat the same direction, and if nothing else,
this puts me in mind of what Bhavabhuti said many centuries ago [13] about
finding someone with the same spirit, so to speak.

[1]: [https://tomas.rokicki.com](https://tomas.rokicki.com) [2]:
[https://www.maa.org/sites/default/files/pdf/upload_library/2...](https://www.maa.org/sites/default/files/pdf/upload_library/2/Joyner-
CMJ-2015.pdf) [3]: [http://tug.org/tug2019/preprints/rokicki-
pdfbitmap.pdf](http://tug.org/tug2019/preprints/rokicki-pdfbitmap.pdf) [4]:
[https://github.com/rokicki/type3search/blob/a70b5f3/README.m...](https://github.com/rokicki/type3search/blob/a70b5f3/README.md#introduction)
[5]:
[https://www.maa.org/sites/default/files/pdf/upload_library/2...](https://www.maa.org/sites/default/files/pdf/upload_library/22/Ford/DonaldKnuth.pdf#page=5)
[6]:
[https://youtu.be/eDs4mRPJonU?t=1514](https://youtu.be/eDs4mRPJonU?t=1514)
(25:14 to 26:46) [7]:
[https://www.cs.stanford.edu/~knuth/papers/huang.pdf](https://www.cs.stanford.edu/~knuth/papers/huang.pdf)
[8]:
[http://www.mathcs.emory.edu/~hhuan30/papers/sensitivity_1.pd...](http://www.mathcs.emory.edu/~hhuan30/papers/sensitivity_1.pdf)
[9]: [https://www.quantamagazine.org/mathematician-solves-
computer...](https://www.quantamagazine.org/mathematician-solves-computer-
science-conjecture-in-two-pages-20190725/) [10]:
[https://news.ycombinator.com/item?id=20531987](https://news.ycombinator.com/item?id=20531987)
[11]: [https://github.com/shreevatsa/knuth-literate-
programs/blob/9...](https://github.com/shreevatsa/knuth-literate-
programs/blob/9b46afe/programs/lp.pdf) [12]: [http://ed-thelen.org/comp-
hist/B5000-AlgolRWaychoff.html#7](http://ed-thelen.org/comp-
hist/B5000-AlgolRWaychoff.html#7) [13]:
[https://shreevatsa.wordpress.com/2015/06/16/bhavabhuti-on-
fi...](https://shreevatsa.wordpress.com/2015/06/16/bhavabhuti-on-finding-a-
reader/)

~~~
vanderZwan
> The problem that one cannot copy text from a PDF created via dvips and using
> METAFONT-generated bitmap fonts has recently been fixed — the original
> author of dvips, Tomas Rokicki ([1], [2]) has “come out of retirement” (as
> far as this program is concerned anyway) to fix this and is giving a talk
> about it next week at the TeX Users Group conference ([3], [4])

Hope that will be filmed and put online, sounds like an intriguing talk to
watch!

~~~
svat
Unfortunately the talks at TUG were not recorded this year, but you can read
the preprint ([http://tug.org/tug2019/preprints/rokicki-
pdfbitmap.pdf](http://tug.org/tug2019/preprints/rokicki-pdfbitmap.pdf)) which
will probably be published in the next issue of TUGboat.

------
cromwellian
I met Knuth a few months ago helping my wife do portrait photography of him (I
got to hold lighting/reflectors :) ), and got to chat with him on machine
learning and other research computer science results. I was floored at the
sheer amount of papers and author names he could recall on the fly, he had
already read every citation I had. At his age, his mind is as sharp as a tack,
and watching him code and demo some stuff to me, he was incredibly adept in
his programming environment, far more productive than I would be. I really
hope he can finish all of the volumes of his books, it will truly be a gift to
humanity.

~~~
svat
Thanks for sharing. That was great to hear, and like you I too hope for many
more volumes coming out.

Your comment reminded me of the following from Herbert Wilf, 2002:
[https://www.math.upenn.edu/~wilf/website/dek.pdf](https://www.math.upenn.edu/~wilf/website/dek.pdf)
(which I believe even better after reading your comment):

 _Here’s a little story of math at 35,000 feet. […] I wrote to Don and sent
him a few more related results that Neil and I had gotten […]. As a result, I
got a letter from him that I found to very moving indeed. Of course it was in
his familiar pencilled scrawl, written at the bottom of the note that I had
sent him. It began as follows:_

> > _Dear Herb, I am writing this on an airplane while flying to Austin to
> celebrate Dijkstra’s 70th birthday. This whole subject is beautiful and
> still ripe for research (after more than 150 years of progress!), so I will
> tell you what I know about it (or can recall while on a plane) in hopes that
> it will be useful._

> _There followed four pages of tightly handwritten information that was a
> gold mine of the history of the sequence b(n) and related matters. It
> contained references to de Rham, Dijkstra, Carlitz, Neil Sloane, Stern (of
> Stern-Brocot trees fame), Eisenstein, Conway, Zagier, Schönhage, Hardy,
> Ramanujan, and so forth. It did not merely mention the names. It contained
> precise statements of the results of interest, of their relationship with
> the result of Calkin and myself and with each other. It contained a number
> of Don’s own observations about related questions, and had several
> conjectures, with background, to offer._

> _It was a letter, ladies and gentlemen, that was written by a man who very
> much loves his subject and the history of his subject, and who knows more
> about that than any five other human beings that you could name. His
> enthusiasm, his background knowledge, and his mathematical values, that
> impelled him to write such a letter under such conditions, can be taken as
> examples for the rest of us to aspire to._

~~~
ignoramous
You might enjoy this news.yc discussion:
[https://news.ycombinator.com/item?id=18698651](https://news.ycombinator.com/item?id=18698651)

------
userbinator
Congratulations Scott, you are one of the _very_ few people to have _Knuth_
make a comment on your blog. Printing out the comment and framing it on your
wall may be very appropriate. ;-)

(I know he gives out reward checks, but this is the first time I've seen such
a comment. I wonder if anyone knows of any others.)

------
zcbenz
Huang's comment is also very interesting:

> Regarding how long it took me to find this proof, here is the timeline for
> those who are interested.

[https://www.scottaaronson.com/blog/?p=4229#comment-1813116](https://www.scottaaronson.com/blog/?p=4229#comment-1813116)

~~~
jxramos
I find it interesting too in so far as it references Math Overflow...

> Yet at that time, I was hoping for developing something more general using
> the eigenspace decomposition of the adjacency matrix, like in this
> unanswered MO question:

> [https://mathoverflow.net/questions/331825/generalization-
> of-...](https://mathoverflow.net/questions/331825/generalization-of-cauchys-
> eigenvalue-interlacing-theorem)

There's some serious stuff on Math Overflow and the Mathematics stack exchange
([https://math.stackexchange.com/](https://math.stackexchange.com/)). I
haven't figured out what the difference between the two sites is however in
focus

~~~
mauricioc
The line can be blurry sometimes, but it's roughly the difference between
research and teaching. MathOverflow is meant to help mathematics researchers.
At its best, it's like being able to walk over to a colleague's room in a
prestigious math department to ask for insight. Math.SE, on the other hand, is
for "people studying math at any level and professionals in related fields"
and should be used for questions that didn't originate in research, such as
hard textbook exercises. Both are super interesting, and even mathematicians
will have plenty of questions that more suitable for Math.SE than
MathOverflow.

~~~
jxramos
Thank you for clarifying. I've been incredibly impressed with the adoption by
the mathematics community with these tools.

------
espeed
NB: Proof announcement thread from a few weeks ago...

Sensitivity Conjecture Resolved
[https://news.ycombinator.com/item?id=20338281](https://news.ycombinator.com/item?id=20338281)

------
Procrastes
Talk about a coincidence. I just interviewed (for an article) someone today
who happened to mention out of the blue that he's mentioned in the "Art of
Computer Programming". He asked me if I "had heard of Donald Knuth." He didn't
know if people still read those volumes. :) I let him know folks are very much
still interested.

~~~
pradn
In what way is he mentioned? I'm curious.

------
quxbar
:') it's like watching a really great game of baseball

~~~
gHosts
Golf.[https://code-golf.io/](https://code-golf.io/)

Mathematical golfing

~~~
shubrigast
Related: dwitter.net

------
AnimalMuppet
More like two thirds of a page - the bottom third is "notes" which are not
really part of the proof!

~~~
TheRealPomax
When it comes to a proof, there are no notes. Just "Weirdly small typeset text
off to the side that should have been in the main body".

~~~
usmannk
In this case the notes don’t support the proof but rather explore
alternatives.

------
tw1010
This is especially interesting to think about in light of the whole school of
self-teaching Knuth has pushed out into the world in fragmented pieces here
and there.

------
utopcell
Not even a page: 1/3rd of it is notes!

~~~
bytematic
Could fit that proof on a notecard!

~~~
dimtion
Or in a book margin.

------
doe88
One thing is certain reading a mathematical proof from Knuth is TeX and its
beauty will be used ;)

~~~
pdpi
I find it quite amusing that the venue where he announced[0] his more compact
proof mangled word wrapping as badly as it did.

0\. [https://imgur.com/E1Xrzvf](https://imgur.com/E1Xrzvf)

~~~
devnulloverflow
You are amused that the Web has worse typography than TeX???

~~~
lonelappde
How would TeX render that sentence with a long link in a small column?

~~~
krastanov
One of the fallbacks is to switch from justified to right aligned text for
that one line. Most importantly, the compiler issues a warning so that you can
look it up yourself.

------
TrueCarry
I'm sorry, I still don't understand practical applications of that paper. Does
that mean we now can write "smart" function with input of booleans and
sensitivity and get results much faster than if we just iterate over booleans?

~~~
devnulloverflow
While I generally thing that looking for practical applications of research is
nonsense, this one seems to be in an obviously useful area.

It's all about different ways of measuring how good a function is at
scrambling the input data. I'm guessing if you want to break (or make) a hash
or cryptosystem you will use these measures over various aspects of it to look
for weaknesses or some such.

This _particular_ proof seems to be saying that the measure called
_sensitivity_ will give you similar answer to a bunch of other measures.

On the one hand, that's disappointing (a measure that gave totally different
results might enlighten whole new ways of attacking/strengthening you crypto).
On the other hand it is encouraging because if a whole bunch of very different
measures agree, then that's a sign that they are on to something real.

------
oneepic
_The Don._ Quite a fast turnaround too.

~~~
copperx
Isn't Knuth about 80 years old? If that isn't humbling, I don't know what is.

~~~
antupis
I would donate my kidney if I would be half of bright as Knuth in 81 years
old.

~~~
pjmorris
I would donate my kidney if it'd help Knuth live long enough to finish his
series!

~~~
aryamaan
Not to innuade the value from his books; but I do have a question.

Are his books useful for someone who are into improving his daily-job-
programming skills? I gave a look at the indexes of his books, they seems more
relevant for when I was into algorithmic programming competitions.

~~~
commandlinefan
I read all of the first three volumes and attempted every exercise. It took me
a little over three years, and most of the focus was on very low-level stuff
like random number generation, arbitrary precision math, sorting (of course)
and memory management. The first book is half math and then a lot of the rest
discusses his assembler language MIX. The middle third or so of volume 3
discusses how to optimally sort files that are stored on tape (!) that are too
large to fit into main memory. So, no... rather than to say the books aren't
_useful_ for a practicing programmer, I'd say that there are far, far more
productive uses of your time. However, I really enjoyed reading them, and I'm
glad I did, and I recommend them to everybody. I think they've made me a
better programmer in the sense that I can appreciate what goes into the
routines that I'm using, but I doubt I'll ever have cause to prove the runtime
bounds of an algorithm like Knuth does.

I have peeked at the two new volumes (4a & 4b) and it looks like they spend a
lot of time on graph algorithms, which _are_ contemporary and something you
might apply directly. All of the examples are in MMIX (a more modern, but
still imaginary, assembler dialect), so if you wanted to jump right into 4A,
you'll have to learn MMIX first. You can download the MMIX description off of
Knuth's website ([https://www-cs-
faculty.stanford.edu/~knuth/mmix.html](https://www-cs-
faculty.stanford.edu/~knuth/mmix.html)) if you want to dip your toes into
Knuth before committing to one of the books.

~~~
copperx
What is your Math background, if I may ask? I want to know how mathematically
mature one has to be to attempt the exercises.

~~~
commandlinefan
I had an MSCS when I started, but if you did a STEM undergrad, you should be
fine - he actually explains everything you need. Don’t get me wrong: the
problems are brain-bendingly difficult, but you don’t need to know how to do
them before you start.

------
hoten
what a legend. he's golfing.

------
estomagordo
This is amazing.

------
dang
We changed the URL from
[https://www.cs.stanford.edu/~knuth/papers/huang.pdf](https://www.cs.stanford.edu/~knuth/papers/huang.pdf)
to where Knuth actually said that.

------
emmelaich
Not a link to a pdf, despite the HN title.

It's a link to Knuth's comment which has a link to the pdf.

[edit; saw dang's comment just now which says the HN link has changed.]

~~~
m463
[https://www.cs.stanford.edu/~knuth/papers/huang.pdf](https://www.cs.stanford.edu/~knuth/papers/huang.pdf)

~~~
emmelaich
Yep, I posted that comment because some (including me) tend not to click on
pdf links.

------
sunstone
Ok so now you're just showing off. :)

------
m463
Any proof can fit on one page.

(for the set k where k is the inventor of tex)

~~~
LegitShady
prove it on one page.

------
sjg007
Does this have any implications for P vs NP? I guess the question would be is
the sensitivity of a 3-SAT problem related to finding a solution?

