Hacker News new | past | comments | ask | show | jobs | submit login
Ask HN: Is Knuth's TAOCP worth the time and effort?
255 points by toinewx on Nov 28, 2023 | hide | past | favorite | 182 comments
I'm starting the first volume. Right at the start, Knuth introduces Induction Mathematical Proof, and more specifically tries to show that we can describe it as an `algorithm mathematical proof`.

I showed it to a friend who is quite good at math, and he told me the book may be trying too hard especially in the examples variety, and how it might not be needed for comprehension's sake.

Would you still recommend this book, and if yes, in what circumstances?




A little bit of history about the book series may help understand what is in it.

In 1956, Knuth graduated high school and entered college, where he encountered a computer for the first time (the IBM 650, to which the series of books is dedicated). He took to programming like a fish to water, and by the time he finished college in 1960, he was a legendary programmer, single-handedly writing several compilers on par with or better than professionals (and making good money too). In 1962 when he was a graduate student (and also, on the side, a consultant to Burroughs Corporation), the publisher Addison Wesley approached him with a proposal to write a book about writing compilers (given his reputation), as these techniques were not well-known. He thought about it and decided that the scope ought to be broader: programming techniques were themselves not well-known, so he would write about everything: “the art of computer programming”.

This was a time when programming a computer meant writing in that computer's machine code (or in an assembly language for that machine) — and some of those computers were little more than simple calculators with branches and load/store instructions. The techniques he would have to explain were things like functions/subroutines (a reusable block of assembly code, with some calling conventions), data structures like lists and tries, how to do arithmetic (multiplying integers and floating-point numbers and polynomials), etc. He wrote up a 12-chapter outline (culminating in "compiler techniques" in the final chapter), and wrote a draft against it. When it was realized the draft was too long, the plan became to publish it in 7 volumes.

He had started the work with the idea that he would just be a “journalist” documenting the tricks and techniques of other programmers without any special angle of his own, but unavoidably he came up with his own angle (the analysis of algorithms) — he suggested to the publishers to rename the book to “the analysis of algorithms”, but they said it wouldn't sell so ACP (now abbreviated TAOCP) it remained.

He polished up and published the first three volumes in 1968, 1969, and 1973, and his work was so exhaustive and thorough that he basically created the (sub)field. For example, he won a Turing Award in 1974 (for writing a textbook, in his free time, separate from his research job!). He has been continually polishing these books (e.g. Vols 1 and 2 are in their third edition that came out in 1997, and already nearly the 50th different printing of each), offering rewards for errors and suggestions, and Volume 4A came out in 2011 and Volume 4B in 2023 (late 2022 actually).

Now: what is in these books? You can look at the chapter outlines here: https://en.wikipedia.org/w/index.php?title=The_Art_of_Comput... — the topics are low-level (he is interested in practical algorithms that one could conceivably want to write in machine code and actually run, to get answers) but covered in amazing detail. For example, you may think that there's nothing more to say about the idea of “sequential search” than “look through an array till you find the element”, but he has 10 pages of careful study of it, followed by 6 pages of exercises and solutions in small print. Then follow even more pages devoted to binary search. And so on.

(The new volumes on combinatorial algorithms are also like that: I thought I'd written lots of backtracking programs for programming contests and whatnot, and “knew” backtracking, but Knuth exhausted everything I knew in under a page, and followed it with dozens and dozens of pages.)

If you are a certain sort of person, you will enjoy this a lot. Every page is full of lots of clever and deep ideas: Knuth has basically taken the entire published literature in computer science on each topic he covers, digested it thoroughly, passed it through his personal interestingness filter, added some of his own ideas, and published it in carefully written pages of charming, playful, prose. It does require some mathematical maturity (say at the level of decent college student, or strong high school student) to read the mathematical sections, or you can skim through them and just get the ideas.

But you won't learn about, say, writing a React frontend, or a CRUD app, or how to work with Git, or API design for software-engineering in large teams, or any number of things relevant to computer programmers today.

Some ways you could answer for yourself whether it's worth the time and effort:

• Would you read it even if it wasn't called “The Art of Computer Programming”, but was called “The Analysis of Algorithms” or “Don Knuth's big book of super-deep study of some ideas in computer programming”?

• Take a look at some of the recent “pre-fascicles” online, and see if you enjoy them. (E.g. https://cs.stanford.edu/~knuth/fasc5b.ps.gz is the one about backtracking, and an early draft of part of Volume 4B. https://cs.stanford.edu/~knuth/fasc1a.ps.gz is “Bitwise tricks and techniques” — think “Hacker's Delight” — published as part of Volume 4A. Etc.)

• See what other people got out of the books, e.g. these posts: https://commandlinefanatic.com/cgi-bin/showarticle.cgi?artic... https://commandlinefanatic.com/cgi-bin/showarticle.cgi?artic... https://commandlinefanatic.com/cgi-bin/showarticle.cgi?artic... are by someone who read the first three volumes in 3 years. For a while I attended a reading group (some recordings at https://www.youtube.com/channel/UCHOHy9Rjl3MlEfZ2HI0AD3g but I doubt they'll be useful to anyone who didn't attend), and we read about 0.5–2 pages an hour on average IIRC. And so on.

I find reading these books (even if dipping into only a few pages here and there) a more rewarding use of time than social media or HN, for instance, and wish I could make more time for them. But everyone's tastes will differ.


His first publication was "Potrzebie System of Weights and Measures" for Mad Magazine in June 1967 when he was 19-years old.

https://silezukuk.tumblr.com/image/616657913


The story there is that he had written this in high school, combining the style of MAD magazine with the textbook “system of weights and measures”, as one of his submissions to the Westinghouse Science Talent Search (1956). A few months later when he was in college he sent it to MAD magazine with (basically) “you guys may like this”, and to his surprise they treated it as a submission and decided to publish it (with illustrations by Wallace Wood); it came out in June 1957. Later when writing his CV Knuth decided to start counting from this as his first publication.


I don't know anything about that, but he was born in 1938, so he wasn't 19 years old.


MAD Magazine 1957, not 1967.


I would add that one thing which is very relevant today but not covered much is memory hierarchies and parallel/distributed programming. For that you might want to accompany TAOCP with one of the heavyweight teaching books on Parallel Algorithms.


> For that you might want to accompany TAOCP with one of the heavyweight teaching books on Parallel Algorithms.

Any recommendations/suggestions?


I was trying to find the book I had at university, which was excellent. However I can't find it right now. I would suggest looking at book recommendations from reputable university courses on parallel architectures.


Of course but i am curious to know somebody's personal preference (i have my own collection and preference) particularly when they juxtapose it with TAOCP. It would be nice if you could find the book which seems to have made a significant impression on you and add it here so it is useful to others.


Found the book I had at college, it was: https://dl.acm.org/doi/book/10.5555/156619 https://www.amazon.co.uk/Introduction-Parallel-Computing-Ana...


Nice! I didn't know of this particular book and hence now have to hunt it up.


Do you have any recommendations of your own that you can share?


Sure. IMO it is important to have an overall picture of the fields of Concurrent/Parallel/Distributed architecture/programming before delving into the details of a language/library implementation. To that end i have found the following books useful;

1) Foundations of Multithreaded, Parallel, and Distributed Programming by Gregory R. Andrews - This is one of my favourites even though old. You get to learn/compare the different paradigms in one single book.

2) Parallel Programming: Concepts and Practice by Bertil Schmidt et al. - This is a more recent book with good explanations/coverage including CUDA.

3) The Art of Multiprocessor Programming by Maurice Herlihy et al. - Well known classic though with a main focus on shared-memory architectures.

Along with the above you also need more detailed language/library specific implementation books;

a) C++ Concurrency in Action by Anthony Williams.

b) Programming with POSIX Threads by David Butenhof.

c) UNIX Systems Programming: Communication, Concurrency and Threads by Kay Robbins and Steve Robbins.

Finally; for studying concurrency at the OS level where software meets hardware, i have found nothing better than;

i) Unix Systems for Modern Architectures: Symmetric Multiprocessing and Caching for Kernel Programmers by Curt Schimmel.


Thanks for the great comment. It made me desire again to buy the books. But I know they will just sit on a shelf mostly unread.


I've wanted to buy these since the late 90s when I used to enjoy thinking about algorithms in the abstract, but never bought them because at the start of my career I felt quite poor and it didn't seem like a good use of my money.

In 2015, I decided I was no longer poor and should buy the box-set as a treat. I set aside time to read a chapter in the first week after it arrived, but realised that it would take years to read and actually understand the material in that one book, let alone the other two. I decided I'd just dig in from time to time, but found that I never actually did, because in all honestly I'm less interested in abstract algorithms now, and prefer to spend free time doing other things in life that don't involve computers.

So, my copy is still essentially untouched after almost a decade.


Maybe try putting a volume in the lavatory? That’s my algorithm for getting things going with a stuck read.


Don't feel too bad -- I strongly believe that 99.99% of the copies of TAOCP are never read beyond the first chapter.

I'd love to own them and be the kind of person to read them, but I know I would also struggle to give them the proper attention (not with the hundred other books to read on my shelf).


TAOCP is best referred to, not read in it's entirety.


Wanna sell it?


Weirdly, probably not. I know if I sold it, then I'd suddenly get the urge to read it.

Also, it's really heavy (I'd say at least over 1kg), so by the time I sold it at less than the current price because it's no longer new (also mine is 1-4A, the current edition is 1-4B) and paid postage to package and ship it somewhere, it's probably not really worth the effort.


Older editions might actually fetch good prices. That's because newer editions aren't just new material plus minor edits and corrections. They also drop old material.

For example if you are interested in large integer arithmetic the 2nd edition of Volume 2 has some exercises that develop some methods that, if I recall correctly, I found more suited to my needs than the methods in the text when I was working on a large integer library.

The 3rd edition dropped those. I've long forgotten most of what I learned when working on my large integer library, but if I had to learn that subject again I'd probably actually start with the 2nd edition, and then check the 3rd edition to add to that.


Memento Mori.

You should imagine that you are forced to sell it every day. Eventually you'll either sell it (or donate it (your local library might appreciate it)) or start reading it.


> You should imagine that you are forced to sell it every day.

Semi-serious question: Which of the following do you mean:

A. Every day, you should imagine that you are forced to sell it; or

B. You should imagine that you will, every day, be forced to sell it.

(I teach law students to spot and fix ambiguities in draft contracts; this would be a nice practice example :-) )


A.

OP imagines that there is a point in the future where they read the book without commiting to it. Selling/donating it would put an end to that possibility and force OP to imagine a future where they cannot read it, and hopefully helps them assessing one va the other, and prioritizing accordingly.


Since it's not possible to sell the same physical item more than once without some intervening event, it's not that ambiguous!


https://en.wikipedia.org/wiki/Prometheus ?


Not sure of your location but if you are mailing materials like this in the USA it's really cheap. USPS Media Mail.

https://about.usps.com/notices/not121/not121_tech.htm

I've used it to buy/sell large tomes like this and it is really cheap ship things this way. It's not the speediest of delivery methods.


This is a really nice comment, but I find the comparison at the end so strange. You shouldn't compare reading these books to using social media, a better comparison would be: given a limited amount of time, should you read these books, or some other one?


> This is a really nice comment, but I find the comparison at the end so strange. You shouldn't compare reading these books to using social media, a better comparison would be: given a limited amount of time, should you read these books, or some other one?

I don't get what you find strange about comparing reading TAOCP vs reading social media. Your time is limited, and each activity that you choose bears an opportunity cost. Of course it is an interesting question to ask whether you should read another book instead, but I bet that most people waste a lot more time on social media than reading a book that is not TAOCP, so svat's comparison does make a lot of sense in my opinion.


I'm not the one you replied to, but for me:

I read social media when I'm too tired to really engage with anything in a serious way.

I (try to) read TAOCP sometimes when I have a surplus of energy and ambition.

Admittedly, sometimes ambition comes along a little bit after I force myself to do something challenging, but TAOCP is so difficult for me that I really have to be feeling it to make any headway.


Reading professional material means you want to improve yourself/your craft/your knowledge, while reading social media is shown to have mostly negative effects, so obviously nobody is going to advocate the latter.


Yeah good point, the reason for the strange comparison is that I started to write “I wish I could make more time for [TAOCP]” and was immediately forced to confront my time spent on social media/HN, so I did. :)

There is also a contrast between the two: they lie on opposite ends of a spectrum from shallow to deep, of things that make you really think / put in honest effort, and also give a deeper enjoyment.

As for reading other (good) books: like anything else, it's ultimately a question of what gives you most joy, I guess. Learning (say) real analysis from Rudin would also be deep and make you work, but Knuth has more jokes.


So it's like the principia mathematica equivalent for computer programming


Principia was an attempt to derive math from logical ground up, motivated by ideas about the foundations of mathematics and logic (wrong ones, it turned out). Knuth is more like A Magical Mystery Tour of Algorithms, wherein Knuth is both tour guide and man behind the curtain.


"Wrong" is too strong. The fundamental bases it used are not generally used today, but it was the first of its kind and inspired much. Many of its details are still fine.

If you are interested in the underlying goal of Principia Mathematica, I urge you to check out the Metamath Proof Explorer (MPE): https://us.metamath.org/mpeuni/mmset.html

By itself, Metamath doesn't have built-in axioms. MPE uses Metamath to first state a small set of widely-accepted axioms, namely classical logic and ZFC set theory, and then proves a tremendous amount of things, building up to a lot of mathematics from careful formal proofs that rigorously prove every step.

Some things cannot be proven, but that doesn't mean that proof is dead.

https://us.metamath.org/mpeuni/mmset.html


Yes, it was a tad tendentious, but I don’t think anyone really buys the logicist program anymore.

Don’t get me wrong, PM is marvelous and there’s no gainsaying its enormous historical impact.

If your characterization of Metamath is correct, I don’t think that’s in the spirit of PM at all. One of the major problems PM had was the rejection of (what later became) the Axiom of Choice in favor of Russell’s convoluted theory of types. Indeed, that set theory (ZFC) is needed to get the rest of the way R&W were trying to go is one of the signal failures of the logicist program behind PM.


If you believe the big advantage of Principia Mathematica was that it starts with a very few axioms and then manages to formally and exactly prove many things, then MPE is a worthy successor. I'm in that camp.

However, if you think the main point of Principia Mathematica was the very specific set of axioms that they chose, then that's different. The PM authors chose to use a "ramified" theory of types, which is complex. It does have sets, it's just not ZFC sets. Few like its complexities. Later on Quine found a simplification of their approach and explained it in "New Foundations for Mathematical Logic". There's a Metamath database for that "New Foundations" axiom set as well (it's not as popular, but it certainly exists): https://us.metamath.org/index.html

More Metamath databases are listed here, along with some other info: https://us.metamath.org/index.html


Although N.B.: it’s still an open question whether NF is consistent.


Strictly speaking that's true for any system that can handle arithmetic (as proved by Goedel). You can show an inconsistency, but you can't prove consistency. No one's found an inconsistency.

There are pros and cons for both systems.


Yes, I was too loose with me words again. I was gesturing at relative consistency (especially, at least, when compared to the lack of serious doubt about ZF/C’s consistency), as well as the work of folks like RB Jensen and Randall Holmes.

The biggest problem NF has is, as usual, a social one: there just ain’t a lot of people working on, or interested in working on, NF compared to other set theories.


Also check out the Principia Rewrite project, which aims to use the interactive theorem prover Coq to ensure each proof step is a valid step according to Principia’s axioms and that no steps are skipped, even by accident.

https://www.principiarewrite.com/


It's a bit extreme imo to call Principia wrong. It isn't wrong so much as it will never completely succeed. These are two different things. The logical perspective presented in Principia is still sound and a relatively useful framework for understanding most of mathematics, and it's a monumental and impressive piece of work. I find it hard to believe anyone who has read it wouldn't leave with a better overall comprehension of what exactly it is that we are doing when we do mathematics.


I didn’t say PM was wrong. I said the ideas behind it were—-although I should’ve said idea (singular) insofar as I was thinking in particular of the strong form of logicism motivating Russell.


My understanding of the flaws of PM is in its attempts to avoid self-reference, which was sort of folly from the beginning as proven by Godel. I learned this from I Am A Strange Loop, and I'm not sure how accurate it is historically. But Godel's Incompleteness Theorem is one of the most interesting things I've ever read about.


The principal flaw of PM if you were to read it now is that it is an evolutionary dead end.

The elementary vernacular foundations of modern mathematics is (more or less) naive set theory; the starting tools of serious foundational work (as arcane as it is even within maths as a whole) are logic and more rigorous set theory, perhaps with some computability mixed in; the main tool of a mathematician who wants to reason in great generality is category theory (with some handwaving in the direction of the previous point about universe hierarchies and whatnot). There are some signs of convergence between these (and of mainstream mathematicians starting to once again take foundations seriously), but at the basic level those are what you’ll be dealing with.

None of them existed in their current form at the time PM was written, even logic (no Kripke semantics! no forcing! and no Gödel of course). Some did not yet exist at all. Some changed quite drastically in direct response to PM. And of course PM is the origin of (embryonic) type theory, which is the inspiration of the unified approach I referenced above. So as a historically important text, sure, if that’s what you want, but as a gateway to understanding more interesting maths it’d be terribly inefficient.

In that respect TAoCP was uniquely lucky. It was also a self-obsoleting book: it ceased to be comprehensive months after it was published, exactly because it told you everything there was to know about algorithms to date. Yet none of the stuff that’s in it is itself obsolete, there’s just immesurably more stuff now. PM, on the other hand, was attempt at “rationalization” in the 19th-century sense, and mostly a failed one except for serving as fertilizer for all of the later ones.


It wasn't a folly from the beginning because it seemed like it could be done when Russel and Whitehead started on the PM. It was only after it was published that Gödel proved it to have been a folly.


In some ways, it's the opposite: Knuth is super practical and any theory is only in the service of whatever is actually relevant to writing better programs and getting answers. See e.g. this older comment of mine comparing CLRS and TAOCP: https://news.ycombinator.com/item?id=21924265 — where others may say O(lg n), Knuth will say that a trie search inspects an average of lg N + 1.33 bits for a successful search and lg N − 0.11 for an unsuccessful one.

But then again, as a result of all this work from him, pulling together and categorizing and cataloguing and analyzing all the disparate ad-hoc “tricks” that programmers had come up with, Knuth basically put (this area of) computer science on good foundations and made it a respectable academic discipline. So in that sense he successfully executed a Principia-like project.

To put it another way, the computer programming/algorithms world is generally divided into hackers who derive satisfaction from elegant ways of bending computers to their will, and theorists who are happy to talk asymptotics, treat their problems abstractly as a branch of mathematics, and often never write a single program. What sets Knuth apart is that he straddles both words: is firmly a hacker at heart, but is able to bring prodigious theoretical skill to bear on the area.


It continues to surprise me that Principia Mathematica (PM) still gets mentioned so regularly in discussions related to computer science topics. As far as I can tell, PM was one of the least influential works in any branch of mathematics or logic or philosophy or computer science. It is must be one of the least-read books (3 volumes, 1994 pages) ever written. PM's sole claim to fame is that it was mentioned in Kurt Goedel's famous paper "Ueber formal unentscheidbare Saetze der Principia Mathematica und verwandter Systeme I". The emphasis strongly is on "verwandter Systeme" - related systems, of which PM is just one example.


Apparently its well marketed (Russel is good at that), because it's one of the only math related books HN knows.


Nope. Principia is purely logical theory. This is practical book, although foundational.


I really hope at some point he stops writing and speed sketches the rest of the "book" so that it can be completed after his inevitable passing at the level of his vision.

---

PS- not to be "that guy" but I do think that a future edition should port the MMIX to RISC-V because it'll be actually runnable and I'd bet money that RISC-V is the educational standard going forward for the next 50 years.


MMIX has educational qualities that RISC-V does not.


Excellent write-up! Should be stickied as "bestof" or something.


Well, I haven't read it but it sounds like it could easily be a multi episode documentary on YouTube or even Netflix.


I've read most of Volumes 1-3 of TAOCP.

I think you can read at various levels though.

When I say I've read most of Volumes 1-3, I mean literally that, I read the text through. At that level it isn't too difficult to read, and you'll learn a lot like that.

However some bits I've studied in much greater detail, like for instance the multi-precision arithmetic algorithms which I had to implement. For that I read it multiple times, implemented the algorithms, did some of the exercises. That takes much much longer than just reading it through.

It is rather like reading a research paper which you do roughly like this, stopping after any step where you think it is no longer providing value to you.

    1. Read the title
    2. Read the abstract
    3. Read the paper through quickly.
    4. Read the paper through slowly being more critical, making notes.
    5. Read the paper through again working through the maths, implementing the algorithms, making sure you understand everything.
To get the most out of it, you'll need to go to level 5, but you can get a great deal out of it at the earlier levels.

So I've read most of TAOCP at level 3, half a volume at level 4 and only 50 pages at level 5.

If nothing else I've loaded the index in my head!

PS the examples are notorious in TAOCP. The easy ones will take a few minutes and the hard ones are unsolved research problems!


the hard ones are unsolved research problems?

I haven't read the book, so he poses research questions as exercises to the reader...?


One level 50 problem was Exercise 1.2.6.63:

"Develop computer programs for simplifying sums that involve binomial coefficients"

Which was solved in the book titled "A = B":

https://www2.math.upenn.edu/~wilf/AeqB.html

or

https://www.amazon.com/B-Marko-Petkovsek/dp/1568810636


He doesn't just spring them on the reader. Each exercise has a difficulty label. Level 50, IIRC, is for unsolved questions.


Yes. He has a system for assigning difficulty ratings to exercises. I don't have the books at hand, but the highest level were open research problems, potentially suitable for PhD work and the like.


> He has a system for assigning difficulty ratings

He's like a depth-first system guy... I suspect he only planned the 7 volumes because it was a requirement to start out wide.


Who wouldn't?


D. Richard Hipp used TACOP to implement a B-tree for SQLite:

> "Nobody ever taught me about a B-tree. I had heard of it. When I went to write my own B-tree, on the bookshelf behind me, I’ve got Don Knuth’s The Art of Computer Programming, so I just pulled that down, I flipped to the chapter on searching and looked up B-trees and he described the algorithm. That’s what I did. Funny thing, Don gives us details on the algorithm for searching a B-tree and for inserting into a B-tree. He does not provide an algorithm for deleting from the B-tree. That’s an exercise at the end of the chapter, so before I wrote my own B-tree I had to solve the exercise at the end. Thanks, Don. I really appreciate it."

https://corecursive.com/066-sqlite-with-richard-hipp/#b-tree...


Two of my favorite names in computer programming, working across time. Love to see it.


This is the first article or interview I've enjoyed reading in months. Thanks for sharing!


The interviewer is on HN: https://news.ycombinator.com/user?id=adamgordonbell

My favorite episode is https://corecursive.com/remote-developer/


Top notch podcast, one of my favorites.


I recommend you read through volumes 1 and 3 cursorily, skipping things you can't immediately understand.

I read through volume 1 several times. I then skimmed through the arithmetics part of volume 2 and read only the section about random numbers, which is very fascinating but you will be able to live without unless you are doing simulations or cryptography/cryptoanalysis.

Whether you will need volumes 4a, b, c depends on whether your work has much to do with combinatorics. I'm a professor and I did not need most of it, but had a good laugh about the hidden gems and jokes like the exercise about Knuths personal pipe organ.

As a software engineer, you may occasionally consult volumes 3 and 1 (searching, sorting, data structures).

[Side node: If I was Knuth, I don't think I would have created three subvolumes instead of the planned single volume 4. While it is true that the material exploded, that will always be the case, and is also true for the other chapters. In his shoes, I would have worked on succession planning and trying to map out the remaining volumes so that the book can be completed one day. Under Knuths approach (esp. going back to update the existing volumes repeatedly on the way) TAOCP will unlikely ever be completed, which would be a shame.]


Yeah, this is how I read TAOCP. I was a grad student working on computational chemistry, so I had lots of downtime waiting for code to compile or long calculations to run (and I didn't have the fancy cloud to run tons of stuff in parallel). I got TAOCP out of the library and just read through it from beginning to end. I didn't work any problems and I skipped the "hard" stuff like proofs (side note: that's how I read all math, I read what they're proving, but don't follow along with the proof, I just tuck it away in my head what was proved and then if I come across a problem where I need it then I go back and try to understand it).

One day I was working on a problem and I realized that I was storing a lot of 0's in my matrix and I remembered that I read about sparse matrices in TAOCP (this was before you could just go to numpy and make a sparse matrix). So I went back to that chapter and used that to solve my problem.

That's happened to me many times in my career now, I remember that I read the general idea in TAOCP and then when I realize I need it, I go back and learn it for real.

If you want to be someone who understands computation (using computers to solve real problem), then it's probably good to have "read" TAOCP just to see what kinds of problems other people have encountered and know that there is at least one good solution to them. Then if you ever need to solve that problem, you know where to look for a start.

A little more philosophical: a culture needs a common core set of ideas that everyone can reference. When working together as a group, having a set of common ideas is a great shortcut in communicating. I'm not religious, but I see the bible as a historical way that this worked for the western cultures - there was a set of "givens" that everyone could hang their ideas off of. Every field of knowledge needs this common understanding. I view TAOCP as this for the field of computing. There are better ways to solve many of the problems now, or common libraries that provide the solutions, but having the problems named and discussed gives everyone a good starting place to talk.

tl;dr - yes, TAOCP is good, read it to learn the concepts and ideas that people smarter than you solved years ago.


Are you suggesting that Knuth's approach is "depth first"?


It's interesting how far he takes particular topics. One on binary decision diagrams ended up with novel algorithms related to zero-suppressed binary decision diagrams. I think he even "broke" counting records for various hard counting problems (and submitted the numbers at OEIS).

It does feel that sometimes he goes deeper than what you'd expect from a 7 volume book.

As with any complicated topic, you need to give yourself some sleep to be able to truly intuit it.


Many other commenters apparently take the view that TAOCP isn't much help to a modern, professional programmer.

That's probably true, in the sense that it deals largely with the kind of functionality modern programmer should be calling from a library, not implementing by hand. But I think it's hugely enriching to read these books. I read 1-3 back in the 70s, and was particularly inspired by Sorting and Searching, and especially the chapter on random numbers. RNGs have intrigued me ever since.

I find Knuth's writing style very pleasant. He's funny. So is it worth the time and effort? Not if "worth it" means "makes you a better Javascript programmer". But it's worth it in the sense that reading books about the history and geography of a country is "worth it" before visiting; your appreciation of the place is deepened, possibly transformatively. It could even be that you would never have visited had you not read those books.

I found TAOCP easy to read, much more so than most textbooks. I don't think I tackled many of the exercises; certainly none of the challenging ones. So for me, the "effort" aspect of "is it worth the time and effort" was quite low, so the ROI was quite high. If you find the books hard to read, then don't bother (and perhaps you aren't really interested in low-level algorithms).


I worked through parts of TAOCP when I was a graduate student (and by parts I mean a very small, tiny fraction of the book). It was highly useful to demonstrate rigorous techniques of how we can analyze algorithms, and I used it a couple times as a reference material. I would only recommend the book to people who want a career in academic CS or are looking for an exhaustive theoretical understanding of the topics covered. A "proper" read of one TAOCP volume would probably take in excess of a year for a well-prepared reader.

TAOCP is extremely rigorous and detailed, the discussion of any topic there will cover every detail. For practical software engineering, it's not useful, you don't deal need that level of theoretical understanding as a software engineer (if you're one of the few who need it, you'd know you are). And you need a very solid mathematical background to actually understand it.

If you want to understand the theory of CS algorithms, I think the CLRS "Introduction to Algorithms" is an excellent textbook. It covers about everything most people are likely to encounter, has comprehensive mathematical analysis but is much easier to follow than TAOCP. For an understanding of the mathematics involved, Concrete Mathematics is a good book, it teaches the mathematical concepts and tools that are the foundation of other, more rigorous books.


TAOCP is about like reading an encyclopedia; a very well-written, black and white encyclopedia with not-necessarily-clear illustrations.

For instance, I just opened randomly to the binary tree traversal material in Vol. I. It's very mathematic or formal, e.g. "T2. [P = Λ?] If P = Λ, go to step T4." and so on. To fish out some of the more interesting details, you have to look through the questions and then flip to the answers.

Meanwhile, https://en.wikipedia.org/wiki/Tree_traversal has the same information but somehow more clearly presented.

If I flip to other random pages and scan for 30 seconds, I can't find much that is actually interesting---particularly in the early volumes.

-----

The Art of Electronics is more my speed, as it's a collection of real-world applications, deep dives into each component type, quick comparison tables, and clear overview of a specific design's pros and cons. It's as much interesting to read through as it is useful as a reference.

I can flip to basically any random page of The Art of Electronics and find something eliciting, "Oh cool." I can't say the same for TAOCP.


One suggestion: if the mathematical preliminary section in Volume 1 of TAOCP is too difficult, you might benefit from the textbook Concrete Mathematics by Graham, Knuth, and Patashnik. Concrete Mathematics is essentially a gentler (but still rigorous) introduction to the mathematical concepts covered in the first volume of TAOCP.


I confirm it has a similar general style, very high quality, rigorous and readable, but in the format of a textbook rather than an encyclopedia.


And it also includes full solutions in the back so it's great for self-study if you have the prereqs (probably calculus and maybe an earlier exposure to discrete math).


If you are at all interested in computer science, read volume 1. If you want to go further after that look at the other volumes.

> he told me the book may be trying too hard

Knuth isn't trying to write a novel or something to learn from, rather a concise, comprehensive attempt to cover the field of Computer Science. It's a repository of knowledge. He marks some exercises at 50 which, from memory, means that if you can solve it fully you are in the running for a Turing award. One of the Python core developers was once asked what the developers did in their sparetime: We read Knuth so you don't have to [0].

[0] https://www.norvig.com/quotations.html


This is certainly a matter of taste. I saw many prominent people everywhere praising TAOCP, but I was disappointed when I finally read the first three books some 10+ years ago. So much disappointed that I regret taking all the time to read it - not because they're bad books, but because I think that time would have been better spent learning things a different way. What bothered me most is MIX. I think using a very low-level language to teach algorithms is a pretty bad choice. Algorithms are high-level concepts, and using an assembly-like language just distracts from those concepts; it requires you to solve so many micro-problems orthogonal to the real problem, and forces you to repeat yourself over and over. I prefer to implement abstract ideas in abstract code. Using MIX just causes suffering for no point. I learned a lot about math from TAOCP (but I've never needed it for anything).


When the books were written, the target audience would all be writing programs in (very different) machine languages, so the “mix” he created of existing computer architectures was a good one (https://retrocomputing.stackexchange.com/a/18176). The specific MIX language is no longer similar to present-day machines (self-modifying code is out of style, for instance), which is why he came up with "MMIX" and there's a book (“The MMIX supplement” by Martin Ruckert, but carefully proofread by Knuth) with the MMIX equivalents of all the MIX programs.

But in any case the algorithms are taught in English not MIX; the MIX programs are only a very tiny part of the books (https://news.ycombinator.com/item?id=14520230 — the 3000+ pages of Vols 1–4A have only 90 assembly programs), used only when the low-level details are actually relevant. So they can be skipped if one is not interested in them.


For my argument, MMIX is not an improvement because it's still a low-level assembly language. True, you can simply skip over the MIX/MMIX source code, but that doesn't help me to like TAOCP. For me, (abstract) source code is the primary source of understanding a concept, and English text is just an accompanying explanation, not the other way round.

Your comments put TAOCP in a historical context. Yes, for the time, TAOCP was a huge achievement and it is a magnificient work. History is interesting, but the question here is about today, and I don't think it's the best way to learn the stuff today. (Again, just my opinion, there's no right and no wrong here.)


Yeah as you say it's just opinion/preference, but re:

> source code is the primary source of understanding a concept, and English text is just an accompanying explanation, not the other way round

note that most algorithms in TAOCP are only described in English (pseudocode), no (MIX) source code. (So if you want source code as primary you simply won't find it… but then again, a typical algorithm textbook like CLRS also would have only pseudocode).

The TAOCP pseudocode is also in a unique style that's not similar to modern programming languages — e.g. here's a note that Knuth wrote just earlier this year: https://cs.stanford.edu/~knuth/papers/cvm-note.pdf explaining (and generalizing/correcting) in his own style, the algorithm from a recent paper (https://arxiv.org/pdf/2301.10191v1.pdf): compare how the algorithm is described in Knuth (first page) vs the paper (second page): Knuth is simultaneously more high-level and more low-level (and also uses “goto” rather than “structured programming” for-loops, about which heretical preference of his I can write a long essay some other time :D).

It is possible to have different opinions about which one is the best way to learn the stuff today, but clearly Knuth rewrote it in his style because he thought it was better. ;)


Did nobody create a MIX virtual machine or compiler in all this time?


Yes, and also for MMIX, the successor version.


Your milage will vary. I got zero value from the books.

They do make a good doorstop though.


Same. I bought physical copies years ago but ended up giving them away.

The main problem is that algorithm examples are specified in a fake machine code. It’s not that useful when things can just be Googled now.


Well, the Turing machine is also “fake.” It is used for some analytical research. Similarly, Knuth uses his MIX for the analysis of algorithms.


It is poorly adapted to that use case though. Either use an abstract machine that captures the essence of the algorithm better, or use a more realistic yet still simple machine (e.g. MIPS or RISC-V). MIX is a middle ground that fails on both fronts.


book 3, especially sorting and hashing is pretty much contemporary. The linear probe, alone, and deletion shift of 6.4 is more than worth it.


By contemporary you mean relevant? That’s not my complaint. TAOCP is pedagogically bad and the content is better learned from other sources.

People just seem to elevate Knuth to divine status and treat his books like the Bible. Why, idk.


Thanks for being honest about it. Sad to see that it's downvoted. I guess people spent the time and effort and don't want to admit it might not have been valuable.


The comment above clearly and honestly stated that "your mileage may vary".

Generally people that continue reading through TAOCP and invest the time and effort in doing so do get value from that or are foolish to keep going.

I read the early volumes thoroughly in the 80s and 90s and got a lot of insight into, for one example, methods for constraining error when doing chained numerical calculations.

His approach alone was inspiring at the time even if specific chapters weren't of any immediate interest.

They're not especially useful for general journeyman programmers and they're but one star in the firmament for people aiming higher .. but they are worth the effort if the material is of interest.


But you’d get all that info, and more, from a numerical analysis textbook.

When I read TAOCP it was right after university at the culmination of an 8 year process where I had self-studied a literal heap of graduate level texts and papers. I had been wrongly advised that TAOCP would be better saved as a capstone, where I could understand better its depths.

It had no depth, and literally everything in it was better and more thoroughly explained by other works I had already engaged with.

Oh, I did learn one very valuable thing: not to blindly trust the advice of others. I wish I had learned that earlier, in which case I could have thrown it on the DNF pile and gotten that month of my life back.


How do you measure the value? I treasure these books, they've brought me so much joy over the years. They've likely not affected my career trajectory much, but they've absolutely enriched my personal life.


There wasn’t anything useful in these books I didn’t already know from an extensive binge into CS theory and practice. There was no new insights either.

There was a lot of useless knowledge that wasted time to cover, like how to optimize for his totally made up machine language.


All machine languages are totally made up.


Sure, but the span of machine languages that are in actual use occupy a small cluster of possible design space. MIX, the language invented by Knuth, is not within this cluster. Maybe computer architectures in the 1960's looked like this, idk. I wouldn't know: I'm not that old.

But it's also a weirdly esoteric, crusty machine language. If it was a Turing tape machine, or an idealized stack machine, that I'd understand. Those are invented architectures that are amenable to analysis and in which the analysis (e.g. maximum stack size, maximum tape use) reflects fundamental properties of the algorithm.

But MIX is just this weird thing that neither resembles CPUs you are likely to use, nor is it particularly useful for analysis.


When Knuth wrote MIX, it was indeed a mix of existing machine languages (https://retrocomputing.stackexchange.com/a/18176) and very similar to them — but an improvement over them for pedagogy, e.g. even something as simple as his abstracting away the detail of whether the machine is a binary or a decimal computer.

Of course decimal computers and self-modifying code, and a lot of other things besides, went out of fashion, so he designed MMIX to replace MIX, during 1999–2011. The MMIX update to TAOCP started being put online in 1999 and was published in 2005 (Fascicle 1) and all the individual programs were finally put in book form by Martin Ruckert in 2015 (MMIX Supplement). The design of MMIX is close enough to (some/many) actual CPUs of the present/future — it's basically a RISC architecture like MIPS or RISC-V; in fact Knuth closely worked with Hennessy and Dick Sites in designing it. Yes it is “nicer” in some ways to write programs in than real CPUs (e.g. it has a whopping 256 registers, handles alignment automatically), but that's a fine choice for pedagogy IMO (avoid dealing with register spilling etc, while still being able to look into machine-level concerns like pipelining etc).

Maybe you read TAOCP at some point before 2005 (or missed the “don't bother learning MIX” part of the preface), in which case your complaint is valid (and of course it's not very convenient to read some parts from a different book), but IMO the machine language being a weird one has not been a problem for over two decades.


I think you’re missing the forest for the trees here. Knuth shouldn’t have been using a machine language for his examples, full-stop. Nothing was gained by giving examples in needlessly complex and obtuse assembly language.


It's not clear whether your argument is that:

(1) TAOCP shouldn't have used an assembly language at all,

or

(2) the specific assembly language used in TAOCP (MIX in the 1960s, MMIX since the 1990s) has a bad design.

In the comment I replied to, it seemed the argument was (2) rather than (1), so that's what I replied to.

As for (1), it has been discussed many times before, but apart from the reasons he himself gives (https://cs.stanford.edu/~knuth/mmix.html#:~:text=Why%20have%... and the preface to TAOCP and to the MMIX supplement — “Indeed, Tony Hoare once told me that I should never even think of condensing these books by removing the machine-language parts, because of their educational value”), in short:

• [quantitative] Most of the book is in English pseudocode, dropping down to assembly language only on occasions when the concreteness is relevant. (See also https://news.ycombinator.com/item?id=38444482 above.)

• [historical] The job he was hired by Addison-Wesley to do (you've written a few compilers in machine language — e.g. https://ed-thelen.org/comp-hist/B5000-AlgolRWaychoff.html#7 — write a book for us showing others how to do the same), meant a book for working programmers writing real programs (most of whom would be writing in languages for various different machines): no one in the 1960s was writing production compilers in FORTRAN or ALGOL (or COBOL or BASIC!); I may be wrong but I imagine very few compilers were written in “high-level” languages (PL/I notwithstanding) until C took off in the latter half of the 1980s, and Turbo Pascal was written in assembly even in the 1990s. A language that is similar to what serious readers would use, but abstracts away some of their finicky details and is a bit nicer for teaching in, is exactly what MIX and MMIX are. (Yes MIX was getting seriously out of date by the 1990s; that's valid criticism and why he spent so many years replacing it with MMIX.)

• [cultural] Among people seriously interested in computing/programming/algorithms, on the one hand there are the theorists, most of mainstream academia, who are happy thinking abstractly and asymptotically, not bothering too much with constant factors or machine-level considerations. Knuth, despite being part of academia and having to a great extent spawned this field, is at heart very much on the other side (here not counting the even large number of programmers who simply do it as a job): of hackers with “mechanical sympathy” who simply care about accomplishing a bit more with computers, getting their programs to run faster, and so on. In fact, this subculture is somewhat underground (occasionally surfacing like HAKMEM) and Knuth may be one of its few “respected” representatives in academia. For example: his presentation of circular lists of course includes the XOR trick to save one field (https://en.wikipedia.org/wiki/XOR_linked_list) from the very first edition (Exercise 2.2.4–18). “Packing” tries to put the children of one nodes among the “holes” of others? Of course (Exercise 6.3–4, and how his student Liang and him did hyphenation in TeX, and what he used for the word count program) (unlike CLRS which barely even mentions tries https://news.ycombinator.com/item?id=21924265). He has written (still being published) multiple volumes devoted to “backtracking” programs that many theorists are not interested in because the programs are only applicable in special instances, but Knuth is super excited to be able to count (say) the exact number of knight tours; there are hundreds of pages (each) on binary decision diagrams and on “dancing links”, a low-level way of implementing the “undo” operation in backtracking programs. A lot of what he writes is backed by his actually having written programs and seeing how they run; his volume on SAT solvers gives detailed measurements in “mems”. He has spoken multiple times about how skill in programming is being able to zoom in/out across multiple level of abstractions, knowing how some high-level goal of your program is being achieved by a specific register changing its contents. From this perspective, there is often value in being concrete about what the machine is actually doing.


> I may be wrong but I imagine very few compilers were written in “high-level” languages (PL/I notwithstanding) until C took off in the latter half of the 1980s,

There was an interesting time in the late 70's when many vendors had their own high-level systems software development programming languages: CDC's Cybil, Sperry-UNIVAC's PLUS, &c., and these were often used for compiler development as a replacement for assembly language. At Cray, the CFT compiler was in assembly language, and CFT77 was in Pascal.


Exactly. It’s hard define value. Did some random thing you read lead to some other insight? Can you link them directly? I think societies relentless pursuit of productivity and ROI has killed the creativity that comes from general knowledge.


There's hardly a chance that one would both (a) read through all of TAoCP and (b) not extract any value from that. The attention required to wade through the books can't be sustained if you have no interest.


Well, I did read through it all and got zero value.


So what do you do with your combinatorics, math, and logic skills that you got from reading?


I’m not sure what you’re asking? I do a fair amount of algorithm development in my work. I’ve implemented C++ containers before.

I just learned what I needed for that from sources other than Knuth, and those sources did a better and more thorough job explaining it.


Maybe people are still in a denial that all companies want is some LeetCode and funny scripting languages.


Or maybe people don't care just about that but instead have a higher ambition in furthering their craft. Be it for joy, some sentiment of "getting better" or whatever. Yeah, sure, most of my days I'm a glorified data converter. I take data in whatever shoddy system and format the customer has it and bang on it long enough so that it fits into our own systems and can be used there.

Does TAOCP help me with that? Well, maybe, mostly no though. I still treasure the parts I've read (I freely admit not having read all of it, but some parts and done some of the exercises) even if from some strict capitalist time = money point of view they probably wouldn't have been worth it.


I think LeetCode is one of the activities where it is worth reading TAOCP, more than for most coders actual jobs.


That's the only answer; SICP and TAOCP are totally useless in an average modern setting. People just copilot glue drivel all day long. You have to be interested in fundamentals and basically write low level stuff (OS/DB/PLD) to really benefit from these things and most, by a large margin, will never touch that stuff.

Edit: for instance, there is now a SIMD article on HN #1; how many people actually are really interested in that? Those should overlap the people who would enjoy these books.


TAoCP is a delightfully written comprehensive encyclopedia of computer science for those interested in learning how and why stuff works; it is fundamental theory at its best. Without having read it I would have spent my life without even knowing something like a biased base 3 numeric representation even existed (each digit of a number is represented by 1, 0, or -1 using a radix of 3 -- wonderful!). I have yet to find a practical application for much of what I've read, but maybe that's why it's got "art" right there in the title.

If you're looking for some browser scripting code to cut and paste, you can go to Stack Overflow or Chat GPT to have your thinking done for you. If you want to understand what you're doing, pull out your dog-eared well-worn copy of TAoCP and spend some warm quality time with D. E. Knuth.


Have you read Euclid's Elements? Did you enjoy the proofs? And the general process? The conclusions in the Elements are what any 10th grade geometry student should already know. It's not the conclusions though, but the journey. Some people still swear by Euclid, claiming the Elements helps teach a person how to think.

TAOCP is in that vein. As you have identified, it is not a programming book, and it's not a computer engineering book, and it's not really a college textbook either. It is a computer science book in the old tradition, where computer science is treated as a branch of mathematics.

It won't teach you to program. As another user said, you should probably be using the algorithm your language's standard library provides. Now, if you want to know why a library's author chose a particular algorithm, the trade-offs between the options, and an open-ended and in-depth exploration of such topics, then this is the book for you. The author of that optimized tree search function you're using probably read TAOCP at some point, so you don't have to.


Does a christian needs to read the bible to a be christian not really. Same with TAOCP and programmers, but the more you read about CS the more complete of a programmer you become.

For example i never thought i would use the basic compiler knowledge i gained during my bachelor. But at one of my first consulting gigs it did helped knowing about abstract syntax tree, where you can add and remove nodes. This knowledge helped me create a highly automated refactoring tool for an old but massive enterprise code base.

This was the only situation in my whole career for now where i needed this knowledge, the rest has all been about creating webpages and rest apis :p


Scientists dont need bibles.


He might not need it to do scientific work, but he might want to read the bible, koran etc for his/her spiritual development.


[flagged]


You do know that Knuth is a devout Lutheran, right?


And that certainly doesn't mean the bible doesn't subjugate women and condone slavery and child murder (which you can easily verify by reading it yourself), it simply means he's steered clear of addressing those points (which is a pity).

His primary interest lies in the patterns, structures, and the mathematical beauty of the scriptures, rather than in theological or moral interpretations, which are still there and deeply influence and corrupt society, despite his ignoring them.

And that shows that expertise and rigor in one field doesn't necessarily translate into expertise and rigor in another field.


> His primary interest lies in the patterns, structures, and the mathematical beauty of the scriptures, rather than in theological or moral interpretations,

Knuth hasn't written much about theology, but he has written some. How much of it have you read? Reading it myself, I don't get the impression he's at all uninterested in the theological or moral implications of scripture. (But I do get the impression he's used the tools of his trade in his public presentations on the subject.)


The perfect segue to another classic:

Knuths Organ Works https://www.youtube.com/watch?v=OBO613Q8hAw


As a pastime, maybe. For a practitioner CLRS offers better value for time spent IMO, and is not devoid of mathematical rigor either.


Cormen, Leiserson, Rivest, Stein : Introduction to Algorithms

https://en.wikipedia.org/wiki/Introduction_to_Algorithms


If you're a mapper[1] like me, it sure is. I ignored the math I couldn't grok, and got all those lovely well described computer fundamentals to grind against the things I already knew, and discover new connections.

[1] https://wiki.c2.com/?MappersVsPackers


I've read SICP - loved it until it got into the later object orientation parts - so I'm wondering in if you LOVED X IS Y FOR YOU sort of comparison -

What books if you liked will you like TAOCP

If you hated skip TAOCP.

Me:

Loved SICP

totally wore out Godel, Escher, Bach

Somewhat angry at A New Kind of Science


Are you me? After SICP and GEB, get "The computational beauty of nature".


I'd recommend it, but I wouldn't recommend reading it cover to cover. It really depends on what you need. Here is what I read selected chapters for:

- For expanding my conceptual understanding of algorithms. I still remember the thrill of learning how the Joseph's problem can be modeled as base-n modulo arithmetics. Or how powerful it is to use generating functions to compute algorithm complexities. And I read the vol 4 for a pleasant study of how we can code back-tracking in an almost mechanical way.

- For historical context. Knuth is amazing at tracing the roots of algorithms, and show how the solutions to a problem evolve over time. You'll get high when seeing how a seemingly impossible problem gets solved in such elegance and ingenuity over generations of attempts by great minds. Check Vol 4 for all the wonders of combinatoric algorithms, for example.

- For thorough study and pure awe. For instance, one probably learns a bit about ordered binary decision diagram (OBDD) to reduce state space by thousands of times in a formal verification system. But what if you want to see all the amazing details of OBDD and therefore getting hours of dopamine high? Vol 4 is the go-to place.

- For reference. At least for a student. I got stuck implementing a fibonacci heap for my project on memory management in my OS class. And Knuth's books came to rescue. I guess professionals would still refer to Knuth for their implementation from time to time.


This is a book that is literally famous for being one of the most rigourus and comprehensive computer science books ever written. I feel like if you don't already know what you're getting into then it probably isn't what you are looking for.


It's not usually useful, per se, unless you are able to refer to it during programming competitions, or if you're implementing a runtime library and want to brush up on the tradeoffs of some low level algorithms - but even then, the state of the art for e.g. sorting is outside the books.

I think the reader is improved for having read it, because it can open one's eyes to the fractal detail of solutions to coding problems.


I haven't actually read TAOCP (yet!) so take this with all the salt you have on hand, but most fields are divisible into primary and secondary works. The ones you should read depend on how interested you are in the field and whether you want practical knowledge or deep conceptual knowledge.

Take mathematics as an example. It has a rich history of primary sources written by famed mathematicians. Most people never even glance at these sources and gain a practical understanding of mathematics strictly through secondary sources like textbooks. This is perfectly fine for most people that only need to use math mechanically and don't care too much about a deep theoretical understanding. If you want the latter however, imo reading primary sources and understanding the history of a field is really important.

To me, TAOCP has always been more of a primary source. If you're just looking for surface level knowledge and advice that's immediately applicable, look elsewhere. If you're looking to understand some of the fundamental concepts in the field of computing and their history, it might be a good fit.


Apparently there is another more accessible several volumes algorithm books from another Stanford University professor, Algorithm Illuminated series by Tim Roughgarden:

https://theory.stanford.edu/~tim/won1samplefinal.pdf


They are great books about some combinatorial mathematics inspired by programming problems. Not the best investment of time if you want to learn programming itself, because:

1. Most of the time you are not implementing foundational algorithms like sorting or SAT solving. You use mature implementation in libraries.

2. If you are in fact implementing foundational algorithms, then the existing volumes of Knuth cover only a very limited set of problem areas.

3. If you are implementing something in an area covered by Knuth, the books are worth looking into as a reference, but often writing performant and robust code requires considerations not in Knuth. This is because Knuth works with an idealised machine ignoring things like parallelism, memory hierarchy etc. and understandably does not get into software design issues.


I'm not sure your point "3." is wholly correct.

They do also go much wider than just combinatorial math.


Totally worthwhile. If you don't want to sit down and read them linearly, that's understandable.

Keep them in the bathroom and open them at random at idle moments. Like the Bible or a "Complete Shakespeare" there's something worth your time on almost any page you pick.


If you want to pursue an academic career in computer science, then yes, probably, I don't know. If you just want to "grok" algorithms to prepare for an interview, then there are much more time efficient ways to do it.


What are your goals? Knuth's books are a very rigorous tour through computer science. You will almost never need this amount of rigor as a software engineer, or indeed, as a computer scientist.


Check one out from a library and see how you feel after a few chapters.


This is the easiest answer. Check out the first (or second, or third) volume and skim-read it.

You'll either be interested, or return it and not feel a desire to get it again. And maybe years later you'll want to dive in.


Occasionally when I'm doing weird algorithmic stuff I end up with my nose in one of these books, combinatorial algorithms more often than many others.

I'm a big fan, but I think you already have to be pretty deep into the world of algorithms and discrete math to get a lot out of them. I cheated my way into that world by starting my math minor a few years ago with Combinatorics, specifically because I felt like learning how to count things in weird ways would help my autrocious mental math (it did!).


> book may be trying too hard especially in the examples variety, and how it might not be needed for comprehension's sake

Had a giggle. Sounds exactly like something a mathematician would say.


I don't think most of us here are the intended audience for a book like this. TAOCP is meant to be a comprehensive compilation of the field of computer science. Such content is useful for people like researchers, lecturers, and those who work on computer languages. But for software engineers - there are much better books that will help you. Some of my favorites:

- pragmatic programmer

- code complete

- hackers and painters

That's not to say his works are 'bad' -- but they are very specialized.


>> I'm starting the first volume

you've told us about your friend (hello there friend), but not said anything about your background / the context in which you're "starting". So...

It's generally aimed for those studying Computer Science and/or have deep interest in Theory of programming + computation.

Knuth loves algorithms, he even has algorithm on how to Read the book and approach the exercises (difficulty, mathematical complexity etc).


If you have to ask yourself this question. No, It is not.

I read TAOCP out of curiosity long time ago. Has it been useful for me? Very much. But the usefulness came from abstract connections that clicked later in my life almost randomly. I am not a common individual.

If you expect a direct practical application of what you read, I believe you better choose other material that suits your needs better.


nobody is gonna mention TeX? so i will.

yes, i have TAOCP on the shelf and yes guilty of "one day i'll ready it".

but every now and then i open them up and just flip thru that magnificent typography.

Knuth has not just written up all these things, he has developed an entire typesetting system (complete with fonts) to bring technical publishing screaming and kicking into the 20th century (when other software thought kerning and hypenation were creatures from space). it's the only program deserving a version number approximating PI.


It depends on the user and the usage. In some cases reading these books doesn't yield the best return for the investment of your time. Consulting the literature that Knuth refers to, rereading and working through the examples, writing your own code could give you so much more.

Volume 2 is within arms reach on my desk. After reading the book and additional literature over the course of several years, the book is filled with sticker notes. Over time my usage shifted from reading TAOCP to rereading (and changing) my notes.

> Would you still recommend this book, and if yes, in what circumstances?

Everyone in math/cs should at least know about the existence of theses books. A rough, cursory knowledge of the content is essential, so that one can look up things if there is the need to really understand a specific topic. TOACP should be considered as one of the best and most important/influential science books of the last century.


I recommend this book to anyone starting CS and wondering "why do I have to learn all this stuff?" "What good is data structures and algorithms?"

Read from a certain perspective, TAOCP is less a book about how to do things better (the focus of many CS courses), and more about "wow, who would have imagined ...". I think CS students might find their curricula more engaging if, rather than having a set of concepts to master, there were actually interesting puzzles to solve. Thus, I suggest reading TAOCP not as a text book, but more as a novel. Take a look at the parts that interest you (how many ways of sorting 10 numbers can you think of just using paper and pencil?). CS is a lot easier to learn as a process of surprising discovery, than as a set of barriers that must be overcome.


My advisor used to say:

"One of the CS bestsellers which is seldom read. That puts it right next to Bible"

From the practical element, mostly No. People won't use this as a first book - or the second or third. But if you wanted a reference on something deeper, which missed any of the standard textbooks, this is the book for you.

As an illustrative layman example, lets say you are modeling server usage with a mathematical model, most textbooks will tell you "hey, the pings and handshakes closely follow Poisson distribution". You pick from there and build on that information or mental model. But why Poisson model happens to be that way, why not something else - and how closely the server model cross-correlates will be realm of where TAOCP starts building the problem. It goes a layer or two deeper of yak shaving IMHO


    If all you want to do is write CRUD apps, the book is not for you. That being said...

    To quote directly fromt a quote in the book, "Now I see with eye serene, the very pulse of the machine." I finally jumped the gun and started reading it during the summer after my acquiring my bachelors. I finished the first volume during that summer. Here are some notes:

    The Mathematical premliminaries reads like a novel if you have some mathematical maturity, you don't dwell too much on the sections that are over your head, and you treat the exercises as optional. This is because the book can be read from a HS level all the way through the Post-Grad level. Choose your depth.

    The MMIX approach was eye-opening because it opened my eyes to see computers as the 'computing' machines that they are. I indirectly gained a fundamental understanding how everything starts with the CPU. I started seeing all the high-level languages (C++, Python, Rust, Java, etc...) as the abstractions that they are. It is liberating if you have ever felt the uneasiness when you are programming in a high level language when you are nagged by the question "how is the computer doing this?"

    Algorithms and Data Structures( a.k.a Information Structures in vol 1): It is like going to grammar school for algorithms! Have you ever asked yourself "Why are we learning all of these abstract things about algorithms and data structures?" I got my answer from the book. What I mean by this is that after studying Knuth's approach to algorithms, you gain a firm, fundamental, and concrete understanding of their necessity, their role. Afterwards ANYTHING I would encounter on Hackerank or Leetcode became digestible. You gain a "first principles" understanding of algorithms and information structures.

    I believe that the book stands the test of time largely because of these points.

    If you have programmed to the point where you've been exposed to some sorting algorithms and some data structures and you want to continue the road to becoming an expert programmer on a firm footing, it is worth the time and effort. The caveat that I would add is do not wait until you finish all the books before you enter the job market or write software out of your own interest.


Funny, my first ever submission to HN was this identically-worded question, with a lot of ensuing interesting discussion:

https://news.ycombinator.com/item?id=10897460


I bought a copy after graduating, having seen the serious-looking tomes on the shelves of many TAs and serious people. I was not a very good programmer and I thought possession of this thing would confer a magic aura that would make me look smart and maybe within its pages were incantations I could learn that made me deserve the appearance. I ventured probably past the first 25 pages. This volume and books like it are not for me, I just want to know how to do things, and a deeper understanding for me comes from doing and breaking stuff many times, over time.


Is Charles Dickens worth it? It's a matter of time, taste and purpose (in reading).

I've enjoyed dipping in. Sometimes found it directly useful to work. But the best of his works I read (in full) was the MMIX book that is a literate program of the interpreter. Second to that was the typography book, a total fascinating delight.

In reality there's probably almost no one who reads any volume of TAOCP in full.

The only way to answer your question is probably to start reading and see if you enjoy it. But expect it take some effort.


Much of what you'll read and learn in the books won't have any immediate impact on you or your career. If there is an impact, it will be subtle.

Does that mean you shouldn't read it? Maybe. Honestly, it doesn't matter if you read it or not. Do it if you want to, but do it without the expectation of getting something out of it.

If you want a return on investment for your time, you're better of building something with the knowledge you already have. Put the books away and create something.


I read the first three volumes and at least attempted all of the exercises. I got the easy ones, made some progress on the harder ones, and at least understood the "impossible" ones. These books are written on the assumption that you're going to work all of the exercises - I don't think that if you just read them you'll get a lot out of them.

I really enjoyed working through them - I keep meaning to get around to volume 4, but can't seem to find the time to do so.


I recommend it because it gives an excellent overview of the fundamentals of the field. Well, honestly, because the series of books were formative for me so I'm a bit biased. I can't overstate the value I got from them. I still use them as reference works to this day. However, I can't say whether or not anyone else would get the same value.

I didn't think of studying them as "effort", though, because they were (and are) extremely interesting to me.


There is no universal answer. Try it. Read a few random sections from the first three books. If you cannot put it down, this series is for you. Otherwise, I would skip it.

Personally, I think the TeX (that Knuth designed because he could not typeset the TAOCP to his liking) is a much more valuable side effect of Knuth writing the TAOCP than the books themselves, but this is not a popular opinion, at least within my Stanford-educated friends.


if you use "sort" in your daily life, you don't need TAOCP. A good tutorial will do. If the "sort" provided by the libs you use doesn't work for you and you feel you need to taylor it to your needs, then start by TAOCP (an dbe sure to have time to do so, it's not an easy read when it comes to details).


"Computer Science is no more about computers than astronomy is about telescopes." -- Edsger Dijkstra


I find reading the historic academic papers and technical reports on a specific subject that is covered by TAOCP more rewarding. I see TAOCP as a survey of the different fields and read the original sources for more depth.


The fact that it's not called The Art of Software Engineering is interesting. If computer programmer was a good enough title for Knuth and Thompson then it's good enough for me.


I like the (overlapping) definition of “software engineering” from Titus Winters (https://abseil.io/resources/swe-book) and Russ Cox (https://research.swtch.com/vgo-eng) respectively, that it is “programming integrated over time” and that “Software engineering is what happens to programming when you add time and other programmers”. In that sense, TAOCP is at something like the extreme of computer programming that is not software engineering: it's about programs for specific problems (don't need to be continuously changed with time), and by a programmer who works alone, does not involve other programmers or reuse their code/libraries.


I think software engineering encompasses a lot more than implementing algorithms. It’s a very different practice to computer science, which is probably what Knuth would call himself these days.


had he wrote it in 2 years in 1968, it would be long forgotten by now.


Probably because two years would let you produce a skinny inconsequential dip into a topic that would quickly be surpassed or barely distinguished from introductory texts on programming.


exactly my point ;)


It's definitely a good reference, but I'll never read them cover to cover. Not unless I'm having trouble sleeping. :-)


No, most of this is made trivial and redundant today.

Understanding your std library performance is lot more important


I thought the books where a kind of reference material, rather than a guided study.


It's a guided study, but because doing the guided study is daunting and would take years, in practice many people (among those who read it at all) tend to use it as a reference, dipping in to specific topics here and there.


If you're looking for a shortcut, read Peter Norvig's "Teach Yourself Programming in Ten Years".

https://norvig.com/21-days.html


Here's the reason why this is totally wrong advice:

1) Time is important, there's ageism in tech and you need to plan your career wisely - no one is going to do it for you. The best way to learn is to get hired ASAP and get professional experience.

2) College degree is very important. In this market, people are struggling to get jobs with experience & degrees. You just have much less likelier chance of success without a degree. This appears all the time, even actual prodigies like George Hotz are overcompensating and always appear eager to prove that they have the fundamentals down like a CS grad.

3) Moving fast into building things which generate hype is the best course of action for young people

I know a friend that got to interview stage with a FAANG they told him the position was for university degree holders. This is an old essay and I'd always take with a grain of salt what people say versus how the reality is. It's very easy to get wrong advice that's no longer applicable.


Let's imagine there was a 20 year old who dropped out of CS and spent the next 2 years reading Knuth and entered the work force at 22. Can you see any way that student would not be succesful?

I know someone who did something very similar (but in math) and jumped into a grad math program at age 17.


They would be at a significant disadvantage over students that completed their degree.

Reading TAOCP is the least efficient way to learn about algorithms. Just because something is harder doesn't mean it's better. Can't even put TAOCP in resume without appearing cringe.

>I know someone who did something very similar (but in math) and jumped into a grad math program at age 17.

Can you share more about this, even IMO medalists go to undergrad first.


> Reading TAOCP is the least efficient way to learn about algorithms

"Learning the algorithms" isn't a binary checkbox. It's a gradient of thinking and math skills, ranging up to a researcher in the field. TAOCP is the only book I know if that will give you that depth.

Learning those skills is not for everyone, but can be extremely valuable, and open a lot more doors than graduating with a class of 10,000 other CS students.

> Can you share more about this, even IMO medalists go to undergrad first.

He was an "undergrad" whose first math class was graduate real analysis.


I am asking you how will you tell the HR person overseeing your application that you have "mastered" Knuth's TAOCP. On paper, you have candidate A with university degree and 2 internships and B that sat in his basement and did TAOCP.

Which would you think they'd choose?

>He was an "undergrad" whose first math class was graduate real analysis.

This is unlikely, he'd still have to pass undergrad Math exams. I'd wager there are plenty of those for a 4 year degree.

> but can be extremely valuable, and open a lot more doors than graduating with a class of 10,000 other CS students.

You need to convince the hiring manager that the skills are extremely valuable. No one is going to take the word for it. Deep theoretical CS doesn't always translate to industry success. The flaw in your reasoning is that you'd try to impress some hardcore CS guy from super-duper company with TAOCP. But they already get 10s of thousands of applications. It won't even get the resume read without a BsC. All the companies which talk like broken records that they don't care about degrees, actually do care about them a *lot* and in your first intro call they'd ask about it if you even get that far.

Keep in mind during interview you have to implement the algorithm fast in a common language Java, C++ or Python. Just being theoretical about it isn't enough.


[flagged]


Please don't post shallow dismissals. If another comment is wrong, it's great to explain why (as long as you do it without swipes etc. - see https://news.ycombinator.com/newsguidelines.html). But if you don't explain why, the comment doesn't contain any information that people can learn from, and is just a putdown. We're trying to avoid that here.


yeah, i'll fix it sorry


Appreciated!


Not to continue learning to program for at least 10 years?


It's both. The texts are comprehensive and go into great detail on the covered material. There are also a large number of exercises throughout. It's suitable as both a reference text and a study text.


TAOCP for computers. Baby Rudin for math. The Feynman Lectures for physics.


I regularly reference Seminumerical Algorithms and Combinatorial Algorithms.


Is Charles Dickens worth a read. Depends on your time, purpose and taste.


Absolutely, this is required reading.


What do you mean by “worth?”


Yes.




Consider applying for YC's Spring batch! Applications are open till Feb 11.

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: