

Mathematically analyzing sorting algorithms - eigenschinkpete
http://www.peter-eigenschink.at/blog/sorting-analysis-bubble-quick/

======
pash
Like many self-taught programmers without any formal education in computer
science, I've always been a bit mystified by analysis of algorithms. Anything
beyond counting the iterations of loops seems to turn into a morass of
notation and obscure analytic techniques—ignorance of which has never kept me
from writing code that runs, well, fast enough.

This article, though, was a nice walk-through that reminded me that this stuff
is probably quite straight-forward if I would just sit down and give it a
shot. It got me to read the Wikipedia page on the Master theorem [1], which
seems easy to apply and genuinely useful for figuring out what's going on in
complex recursive code. I think I'll finally click "buy" on Carmen's book [2],
which has been sitting in my Amazon shopping cart for years.

1\. <http://en.wikipedia.org/wiki/Master_theorem>

2\.
[http://mitpress.mit.edu/catalog/item/default.asp?ttype=2&...](http://mitpress.mit.edu/catalog/item/default.asp?ttype=2&tid=11866)

~~~
psykotic
The master theorem is a blight on the teaching of analysis of algorithms. The
main reason is that few students understand what makes it work and end up
treating it as a piece of plug-and-chug equipment. The second reason is that
it is not very general.

The alternative? Learn to love recursion trees. It addresses both issues. It's
intuitive and it's general. As a side effect, the master theorem becomes an
almost obvious fact; the three different cases correspond to geometric sums
with increasing, decreasing and constant terms, with the caveat that since we
are working asymptotically, increasing/decreasing/constant must be interpreted
in the looser asymptotic sense. A geometric series with increasing
(decreasing) terms is dominated by its last (first) term, corresponding in the
recursion to the last (first) level of the tree. That makes it trivial to
estimate the sum: just estimate the cost at the appropriate level and you're
done. Only in the constant term case do all the terms play a role, but then
you get your estimate by just multiplying the number of terms (the tree depth)
by the constant value the terms have in common.

~~~
adrianN
All but the very simplest recursions require heavy machinery to find closed
forms. If you can't apply the Master Theorem often your best bet is to guess
the answer and try an induction, otherwise you quickly end up with generating
functions and differential equations.

------
eigenschinkpete
With this (my first) article I intend to give a little introduction into the
analysis of algorithms, to show why it is necessary and how it basically
works. There are many algorithms out there to give a solution to the same
problem and every programmer should know why they use which one in a
particular situation. As this was my first article, it should also be an
opportunity for the readers to tell me which aspects of the analysis should be
highlighted and to give me some feedback. Thank you anyway for the comment,
the next post will be concerned with a more specific topic.

------
crntaylor
I've got to admit, I don't understand why this article exists. If you're a
beginner, the math is too complicated. If you're anything more than a beginner
then you already know that bubble sort is O(n^2) and quick sort is O(n log n)
and you probably know how to derive it. Who is this aimed at?

~~~
agumonkey
I know the complexity, at the intuition level, but can't derive them, even
after reading this article.

------
buchgrab
As a fellow Austrian I don't intend to criticize the author too much (;. It's
an okay article, however nothing special and tbh it kinda feels like the
introductory section of the first chapter of an algorithms book.

What's bugging me tough, is that lately quite a few of those "okay but not
Hacker News material" articles made it to the top.

It seems that esp. the words mathematics and algorithm are enough for an
article to appear in the top ten. I wonder why that is ...

------
tikhonj
This article missed the last and most interesting bit: we know that Quicksort
is O(nlogn) in the best case and O(n^2) in the worst case; what is it like on
average? That is, what is the performance averaged over all possible inputs?

I can see why it was left out: the proof that Quicksort is O(nlogn) on average
is much more complicated than anything in the article. (So, of course, my
algorithms professor thought it would be a _great_ homework problem. Cute.)

That said, the fact that it _is_ O(nlogn) on average is actually well know--
it's just the step from where the article left off to the conclusion that's
missing. Wikipedia has an outline of the proof, if anybody is interested:
<http://en.wikipedia.org/wiki/Quicksort#Average_complexity>

------
mjcohenw
The bubble sort has an off-by-one error: The outer loop has i go from 1 to
A.length and then references A[i+1].

