
Sorting Algorithm Cheat Sheet - gameguy43
https://www.interviewcake.com/sorting-algorithm-cheat-sheet
======
swiftcoder
> Radix sort looks fast, with its O(n)O(n) worst-case time complexity. But, if
> you're using it to sort binary numbers, then there's a hidden constant
> factor that's usually 32 or 64 (depending on how many bits your numbers
> are). That's often way bigger than O(\lg(n))O(lg(n)), meaning radix sort
> tends to be slow in practice.

The constant factor in radix sort isn't the number of bits, it's the number of
digits. Since one typically sorts on a computer using 8-bit 'digits', that's a
constant factor of 4 or 8 (not 32 or 64).

~~~
ggggtez
It's the number of bits. You can change to bytes, but you are just hiding
another constant factor of 8 by doing that.

~~~
swiftcoder
If you have a theoretical computer with single-bit registers, sure. Quicksort
is also quite slow on such an computer.

~~~
kjeetgill
And this is why O() notation drops constants. 0(bits) or O(bits/8) aka bytes
are the same thing. It's also worth pointing out that in standard comparison
sorts, the comparison itself is _technically_ linear to radix too, but is
treated as constant. I get why it's dropped but it's worth knowing.

~~~
ggggtez
Exactly. You can try to compare two bytes, but really you are comparing 8
bits, if you thought about it algorithmically. Maybe those steps are 100%
parallel. But it's irrelevant to the big O. You are measuring number of steps,
and it's intended to be hardware independent.

------
ComputerGuru
Of course in the real world for problems where sorting is actually the
bottleneck and not just something you want not to kill your app's performance,
you end up with things like timsort that destroy all these in practice.

~~~
bjoli
I'd say it's the other way around: you use whatever your language provides as
default (which is probably something like Timsort) until it becomes a
bottleneck and then you analyse the data and write something that is specific
to your use case that will blow Timsort out of the water.

Timsort is your first choice.

~~~
btilly
In practice this boils down to, "you use whatever your language provides".

If sorting became a bottleneck, I'd literally look at every possible approach
to speed it up that I could find before considering improving the sorting
algorithm. Starting with trying to figure out how to throw around less data
while trying to solve the problem.

~~~
bjoli
For a problem i faced I ended up being able to produce very small chunks of
already sorted data and using the merge part of the stable sort algorithm (it
was provided).

Not only that, I ended up being able to do it lazily making the user
experience much better.

~~~
btilly
And I have been forced to write sorting algorithms from scratch as well. But
it was truly a last resort. In my case I had to work with data that literally
did not fit in uncompressed form on the computer that I had to work with. So I
had to write a disk sort where data as kept in compressed form. (I used merge
sort for this.)

However the fact that sometimes we are forced to our last resort doesn't
change the fact that we should be aware that it really is a last resort.

------
saagarjha
I think it's important to mention that counting sort and radix sort are not
"true" comparison-based sorts: they have additional requirements on the input
data.

~~~
why-el
Indeed. Usually if you get question such as "you have an array of 1 through
n", the immediate thought should be "why is it just up to n and not arbitrary
numbers"? From there, you know you can get a linear sort by leveraging the
mutual relationships between these numbers.

------
northisup
Who is still asking candidates to talk about sorting algorithms?

It is just trivia and has little to no bearing on if the candidate can
actually do the job (unless the job relates to the runtime of sorting
algorithms, of course).

~~~
saagarjha
Sorting is such a foundational problem in computer science that pretty much
every job will end up relying on the runtime of sorting algorithms. Not every
job requires being able to write a bulletproof dual-partition QuickSort, of
course, but knowing complexity bounds is important, even if you’re just
calling your standard library’s implementation: it places fundamental bounds
on what things you can improve.

~~~
CzechTech
Pretty much every job? I'd say about 10%. With all the web and web-adjacent
jobs around, maybe even less. Most stdlibs are already written in a sensible
way, so calling "sort" usually means calling quicksort. Most people just do
not care, they have tickets and bosses to worry about.

~~~
nkozyra
Absolutely. While every job will depend on sort, so many of them will have
negligible benefit by changing the least efficient algorithm to the most.

And most of them just use a native language sort that does something
relatively smart out of the box.

It's good to write all of these at don't point so you understand why things
are inefficient.

------
gameguy43
Original author here, happy to answer questions about data structures,
algorithms, and coding interviews!

~~~
strainer
It might be notable that on very small lists (up to around 50, maybe 100
depending on cpu/cache) insertion sort is fastest of all, even on difficult
input distributions. Its so simple that cpu can skip through it rapidly. I
noticed this myself and have also read other developers mention it in sort
design discussions. A popular general purpose sort called 'Timsort' defaults
to it for small sequences. Its possible to tweak the insert algorithm to sum
distance of elements moved so far and escape to a more substantial algorithm
when it gets excessive. If little rearrangement is required insertion sort is
about as fast as scanning through memory can be.

I think in practice the answer to the question "which should _I_ use" (rather
than "which should _ideally_ be used") is the best developed hybrid sort
library for the platform. Beyond the suitable case for insertion sort, its a
really substantial job for an individual to develop, test and optimize a high
performance sort library that can handle a range of distribution types.

~~~
btilly
One of the big factors behind this is that the cost of pipeline stalls in a
theoretically good algorithm can significantly exceed the cost of extra
comparisons.

Whether that is true depends on a combination of how much data you have, what
kind of data you have, and your programming language's low-level
representation.

For ints in C++ and a list of 20 elements, it will be reliably true. For
strings in Perl with a list of 500 elements, it reliably won't be true.

------
gameguy43
Curious: did people notice that you can click on each algorithm and click the
blue button to get a detailed write up of how it works? (Or is that too hard
to find?)

~~~
strainer
The links to decent writeups with example code ( excellent to have) are a nice
surprise when a row is expanded.

On my firefox browser the paragraphs and elements in the table and writeups
are somewhat excessively spaced vertically.

------
GorgeRonde
There is also sorting networks. If you have to sort short list of items, it's
probably the best solution around.

[https://en.wikipedia.org/wiki/Sorting_network](https://en.wikipedia.org/wiki/Sorting_network)

------
hopscotch
The best real sorting algorithms are often hybrid.

Would be nice to see mention of parallel sorting algorithms. These have
basically linear speedup and you can do them on GPUs.

Obvious disclaimer that you should never care about your sorting algorithm
choice until you need to. Performance is more often IO or memory related than
compute. Tech interviews are daft, etc.

------
marvinjames
Best case for heapsort is actually O(n). Build heap always takes O(n). Then
e.g. when all keys are the same, the max-heapify call will take O(1) instead
of O(logn).

~~~
gameguy43
Hmmmm good point!

------
CobrastanJorji
feature request: make O(k) bright red, like O(n^2). As it stands, it looks
nice and pleasant like O(1) but it's honkin' terrifying.

~~~
gameguy43
Oh yeah, looks like we messed that up. That space complexity also doesn't even
agree with the final space complexity we describe in the write up.

I'm gonna actually change the space complexity in the table for both counting
sort and radix sort to O(n)--that at least matches the amount of hand-waviness
we used for the time costs for those two in the table.

