
A Comparison of Four Algorithms Textbooks (2016) - kerneldeveloper
https://porgionesanke.wordpress.com/2016/07/11/a-comparison-of-four-algorithms-textbooks/
======
brudgers
The criticism of Knuth's use of MIX seems a bit off because:

\+ Assembly language level instruction execution is a good way of talking
about the running time of algorithms at a finer level of detail than Big O
notation. Finer grain than Big O is helpful when analyzing and optimizing
programs.

\+ MIX is a good abstraction of the Von Neumann architecture and that's where
the analysis of actual programs occurs.

\+ MIX programs are still accessible to a general auidence 50 years after they
were written. No actual programming language available in 1967 is still
similarly accessible to such a degree.

\+ MMIX is the successor of MIX...but in so far as learning MIX is an issue,
it's more complex child probably is not an improvement.

The art of MIX is that it keeps the reader's focus on the fact that computer
programs ultimately load registers, read memory, execute mathematical
primatives, and branch no matter what object-oriented, functional, or type
safe abstractions a high level language might offer.

~~~
hodgesrm
This seems to be a point where reasonable people may differ. I generally go to
Knuth when I'm really stuck--not just how to do something but sometimes how to
frame the problem so it can be solved. In that sense MIX does not make much
difference since it's the discussion and backing mathematics that are most
interesting.

On the other hand outside of OS and embedded systems people very few people
regularly look at assembler, so for many people it's no longer an effective
lingua franca for sample implementations. It's a bit like reading Principia
Mathematica in the original Latin--sure, some people can do it but it's not
the best way to transfer information to a broad audience. Personally I would
be happy with some variant of C--the syntax is very broadly understood and the
language is close enough to the hardware that implementation concepts come
across clearly.

~~~
brudgers
I've got nothing against C. C was not around when Knuth published the first
(1968), second (1969), and third (1973) volumes. [1] By inventing a
programming language, Knuth was able to avoid syntactic changes between
versions (e.g. K&R C looks odd for someone who started with one of the ANSI
C's).

At the time, 'best practice' might have been Algol 60 or maybe Algol 68
because they were what it meant to be a standard language in those days even
though implementations varied widely.

[1]: Whether C would have been a reasonable choice the time of the fourth
(2011) volume is another matter.

~~~
todd8
Neither Algol 60 nor Algol 68 were very good choices, but it was basically
them or FORTRAN which were used back then in published algorithms.

Algol 60 was the forbearer of most modern programming languages. However,
implementations of Algol 60 were all a bit different and even after the the
revised report came out a few years later there remained serious ambiguities
in the meaning of common Algol 60 constructs, see for example [1]. Other
aspects of Algol 60, for example the use of call-by-name argument passing
semantics, made it not entirely suitable as a means of communicating
algorithms.

Algol 68, had, for many years, essentially no conforming implementations. The
language was remarkably orthogonal, every aspect seemed to be fully
generalized and usable everywhere in the language, but it was too complicated.
At least, it had a concept of I/O, which Algol 60 lacked. Some algorithms were
published in Algol 68, but I didn't see many of them.

In this environment, Knuth's books were quite valuable. The algorithms, while
not structured or written as we see them now were clear and unambiguous and
were written for a time when compilers didn't perform advanced code-
optimizations; Knuth's careful designs included optimizations that we would
leave to compilers today when talking about algorithms.

Eventually, Pascal came along and it became possible to publish algorithms in
an easy to read and unambiguous way. (It was still uncommon to see code
formatted in any consistent way. See Wirth's own books!) Pascal wasn't
perfect, but it was easy to understand and implementations were portable and
widely available.

[1] Knuth, The remaining trouble spots in Algol 60*,
[https://atlas.cs.virginia.edu/~asb/teaching/cs415-fall05/doc...](https://atlas.cs.virginia.edu/~asb/teaching/cs415-fall05/docs/algol-3.pdf)

------
dsacco
I’m surprised, but I like this (I typically don’t like or agree with textbook
comparisons, but I think this tour is mostly right). The Sedgewick text,
_Algorithms_ should be in there too but the author apparently didn’t read
that.

My own experience basically agrees: I’ve read and enjoy Skiena, it’s written
in clear style and it’s the “cover to cover” text for a working developer or
for interview practice. But I also have TAOCP and CLRS on my shelf, and I
haven’t read either of them. I’ve certainly _used_ both of them a lot, but I
simply haven’t read all of them because I use them more as reference texts.

Personally I find it bothersome that these textbooks are written with
idiosyncratic pseudocode. In my opinion, many of these authors lose their
grasp of common implementation difficulties if they don’t provide students
with working code that will compile and run. It’s easy to throw down the
theory and some okay-ish pseudocode while in effect saying, “...and the rest
is just a minor implementation detail, which I leave as an exercise to the
reader...”

~~~
nurblieh
The Sedgewick text is quite good but even better are his classes on Coursera!
Very accessible and built around practical exercises.

~~~
rashkov
I found the assignments to his class to be really weird and challenging. The
first one had you fill in a few functions of a half-complete percolation
simulation, but without having written the other part of it, it felt like
having to reverse engineer and guess at the solution. There was no real way to
measure your progress. Either it runs or it doesn't, and there's no
incremental progression towards a solution.

I'm curious if anyone else experienced this or could point out where I went
wrong. I really wanted to do the class and I liked the lectures, but i started
it after finishing Gardner's Stanford algorithms Coursera class which had some
of my favorite exercises ever. They had you write an algorithm in it's
entirety, gave you sample data sets to check against, and let you write it in
any language. Compared with that, Sedgwick's assignments were like coding with
one hand tied behind my back.

------
placebo
Honestly, I'm blown away that someone has actually read cover to cover Knuth,
CLRS, Dasgupta and Skiena. Is this a common thing? Has anyone here done
something similar? For years I've had the textbooks of CLRS and Skiena at home
(and a pdf of Dasgupta) but they are used only in the event I need to drill
down to understand a particular algorithm to solve a particular problem. I
feel that the most effective use of my time is to have a birds view of the
landscape (i.e understand the categories of algorithms and the problems they
solve) and dive down only when I actually need to know the nitty gritty to
implement a solution for a real problem. I can understand the joy of learning
them all just for fun, but who has the time? I wonder if he did all the
exercises too... :-)

~~~
sigjuice
Where does it say that he has read all these books cover to cover?

~~~
mathgeek
Generally when someone writes a review this in-depth, this is assumed. Not
always accurate, of course.

------
hackermailman
Many grad level algorithm courses still use Dexter C. Kozen's book The Design
and Analysis of Algorithms, Springer-Verlag, 1992. It's a timeless book with
clear analysis.

There's also this excellent free draft on analysis of common undergrad
algorithms for parallelism [http://www.parallel-algorithms-
book.com/](http://www.parallel-algorithms-book.com/)

TAOCP is more than an algorithms book, Knuth even have strategies for writing
lengthy programs from scratch, how to build test libraries, how to optimize a
program to make it cache memory friendly ect.

------
gregfjohnson
I became curious about quantum computing a few years ago. Googling around, I
came across the QC chapter in Dasgupta. I found their description of the topic
to be excellent, exactly what I was searching for. They provide a substantive
presentation of QC, as contrasted with many "pop sci" hand-wavy
impressionistic descriptions of QC. They tactfully avoid a few of the
extremely difficult proofs, but they do give a solid mathematical underpinning
for the subject. And, they convey in prose what the point of the whole thing
is. Also, they forthrightly express the mysterious nature of what seems to be
going on here. (If you have an N-qubit quantum computer, the universe seems to
somehow maintain and manipulate a vector of 2^N complex elements as a quantum
computation proceeds.)

They give a brilliant presentation of Shor's factorization algorithm, and how
quantum computers offer an amazingly natural way to do FFT's (a key aspect of
the Shor algorithm).

I would not contest the OP's questioning of the choice of a chapter on QC in
an undergrad algorithms textbook. I would, however, offer the standard "your
mileage may vary" caveat to the OP's very negative characterization. I had no
idea what QC was about, and this chapter provided me with a great "on-ramp" to
understanding what QC is all about.

------
dfan
It's odd to me that the author describes CLRS as graduate-level. It (back when
it was CLR and in its first edition) was the text in my undergrad Introduction
to Algorithms class (taught by L!).

I agree that Sedgewick belongs in this comparison, but I can't fault the
author for not having read it. I think it's the book I would most easily
recommend to others; it's quite clear and has lots of very nice
visualizations. I do think that Skiena deserves a special mention for
explicitly being about how to come up with your own fundamental algorithms and
data structures rather than just plug an existing one into your program.

~~~
arthurjj
CLRS was also the undergraduate textbook at my school, RIT. I didn't think it
did a very good job of explaining algorithms if you didn't already have a good
grasp of them.

Skiena on the other hand does a nice job of both describing the algorithm in a
straightforward manner and also getting you into the algorithm headspace.

------
chipkey
Kleinberg/Tardos book that I am reading right now needs trimming for brevity.
A good math text editor could, probably, easily lop off 1/3 of it without
affecting its content. Otherwise I prefer it to most other algo books because
its proofs are closest to how mathematicians approach their proofs. By
contrast, I am confused as to why CLRS contains proofs at all. At the
beginning of the book, the authors say something to the effect of actual
proofs being too messy/hard so they simply wave their hands through them. But
if most any intro discrete math books can do these proofs, why can't CLRS? To
me it's a turn-off.

------
AlexCoventry
Which book/website has the best exercises for developing algorithmic problem-
solving skill? I've started working through Skiena's exercises, but haven't
really looked at much else.

~~~
dsacco
If you're actually interested in drilling algorithm exercises - in which you
map a problem statement to a particular algorithm and time/space complexity,
then implement the solution - competitive programming websites are the best
way to practice. There's a good number of them now, including TopCoder,
LeetCode, CodeForces, SPOJ, HackerRank, etc. You can typically sort the
problems by difficulty or solution acceptance rate. Working through these
problems requires:

1\. Breaking down a problem statement into a heuristic pattern,

2\. Analyzing the best, average and worst case complexities of possible
solutions,

3\. Mapping the heuristic(s) to the data structures and algorithms with the
most optimal complexity,

4\. Implementing the code and having it successfully run, with runtime
performance feedback.

I think the aforementioned competition websites are better than things like
Project Euler because solving the problem requires running actual code instead
of just giving a correct answer. That makes them much more interactive (and
harder, in my opinion), because you might have to obey particular performance
or complexity constraints, and you can receive feedback about how efficient
your solution is.

I wouldn't say that practicing these problems will make you a better software
_developer_ , in the sense that you can develop maintainable software to solve
business problems in a team setting with a large codebase, and I make no
comment on whether these sorts of problems are optimal filters for tech
interviews (that's a dead horse). But much like mathematics, programming (and
specifically algorithm analysis) is not a _spectator sport._ You can't
efficiently learn the material just by reading it, you have to do it, just as
mathematical maturity comes about by solving many mathematical problems in
different domains. To that particular end, I would say that competitive
programming is about as perfect a formulation of practice as you can get for
improving algorithmic problem-solving skills.

~~~
AlexCoventry
> I wouldn't say that practicing these problems will make you a better
> software developer

I know. Apart from it being fun, I'm mostly interested for the social-proof
value in interviews. I don't want to beat the dead horse either, but it's
definitely a socially useful skill.

Thanks for the suggestions.

------
asnyc
Skiena is very good - Algorithms are lucidly explained - Actual code is
provided - Its relatively newer but highly recommended

Have read only parts of Knuth / CLRS - They are good, but for real problems
analysis , would prefer Sienna

------
vidoc
I'd actually add one thing: coding style.

Shame on me for admitting this! In fact, despite having personal CS favors
like everyone, the painfully obvious subjectivity of the whole matter has
always striked me as entirely futile to be taken into account for basically
everything. I even worked for years of uninterrupted peace with people who
would, for example, prototype pointer arguments of c/c++ functions gluing the
wildcard to the type, then space then boom variable name. I've always been
cool with this, even reading things like glibc's code.

This liberalism however has two exceptions: my own code obviously, a
dictatorship where merciless enforcement of inflexible and rigorous coding
style is accepted. But unfortunately, in algorithm books too! I know it is
completely idiotic but I'd be in denial not to admit how much of a total turn
off the coding style of code in algorithm books can be to me. Segdewick for
example, such a wonderful book, the prose is excellent, the content really is
outstanding (probably one of the most comprehensive I own), the editions I
have even has a superb typeface and paper quality, unfortunately the C coding
style of this book has this effect on me which makes me want to close the book
immediately. I feel the same deep pain every time I have to look at it (which
does happen a lot since it _is_ a great book really!). I'm actually jealous of
those more advanced human beings who are able to make an abstraction of that
when reading a coding book!

------
uptownfunk
I've used dasgupta in my ug algorithms class at Berkeley, which was actually
taught by papaD one of the coauthors. I love the text because it is so
concise. A great book to get started on algorithms. A good exercise is to
implement them in the language of your choice.

------
twblalock
As a self-taught developer, I prefer the Skiena book. The topics are ordered
in a way that make sense to me, and the examples are practical.

However, the other books are probably a better match for a standard university
algorithm course curriculum.

------
rosstex
Can we get a comparison of Algorithms undergrad courses? :)

------
jimmcslim
I have Knuth's series on my bookshelf. It looks great!... especially since I
haven't broken the bindings by actually reading any of it...

------
rootusrootus
This is an interesting read because Dasgupta is what the current OMSCS
algorithms class is using.

------
antonvs
I didn't realize there were that many textbooks that just covered four
algorithms.

