

A beautiful algorithm that only a very few know of: In-Place Merge in O(n) time - dhruvbird
http://delivery.acm.org/10.1145/50000/42403/p348-huang.pdf?ip=129.49.21.121&acc=ACTIVE%20SERVICE&CFID=67964827&CFTOKEN=45973902&__acm__=1330311114_8fad21799284de31f9cd95120989f262

======
sblom
This link got me the paper in PDF form with absolutely no hassle:
[http://web.eecs.utk.edu/~langston/courses/cs594-fall2003/HL....](http://web.eecs.utk.edu/~langston/courses/cs594-fall2003/HL.pdf)

(Appears to be the faculty web site of one of the paper's authors.)

~~~
yason
Thanks for the link. For me, the article really didn't even exist since it
wasn't freely available from the ACM site.

------
forrestthewoods
Perhaps very few know of it because it's hidden behind a ridiculous pay wall.

~~~
dhruvbird
Sorry about the pay-wall link guys :-( I'm in university so I didn't realize
it was this way.

I've posted 3 other links which should allow free access to the paper. I think
these things should be open (and free) to start off with.

~~~
PaulHoule
This is precisely what is wrong with the Paywall -- people in the Uni don't
know it is there.

Donald Knuth embarassed himself a few years ago when he told people that the
library at the Uni was worthless because he can just sit as his computer and
view all the literature for free.

Academics generally don't realize that the library at their Uni cuts a million
dollar check to Elsevier every year, in addition to a number of other
publishers. As a result, they don't see that there is any problem.

~~~
bunderbunder
The other problem is that it cuts academics off from the rest of the world.
Because paywalls effectively prevent non-academics from seeing what academics
are doing, they end up being underappreciated, and their contributions end up
being underutilized. And, since we might as well bring it back to money, I
suspect it ultimately has a downward influence on the amount of money
legislatures earmark for universities and research.

~~~
PaulHoule
you took the words right out of my mouth!

unfortunately when academics are indifferent about marketing themselves to the
wider world, they are the losers in the long term.

------
bhickey
I came across this paper about five years back. At the time I sent an e-mail
to Professor Langston thanking him for his insights and clarity in explaining
them. Here's his reply:

    
    
        Thanks for your kind note Brendan.  It's been over
        twenty years since we wrote that paper.  One never
        knows which works will survive the test of time.

------
pwg
Some source - with a lot of comments - that claims to implement the algorithm:

[http://www.keithschwarz.com/interesting/code/?dir=inplace-
me...](http://www.keithschwarz.com/interesting/code/?dir=inplace-merge)

~~~
jemfinch
That's the same guy that did the _fantastic_ writeup of Dijkstra's Smoothsort:
<http://www.keithschwarz.com/smoothsort/>

------
ggchappell
There's an issue here that I _really_ don't understand: why is this never
used?

For example, the original C++ STL generally did a heck of a good job at
exposing an interface to the 1997 state of the art in general-purpose
algorithms. So, the spec for std::stable_sort was aimed at Mergesort: O(n log
n) comparisons and O(n) additional space due to the space required by the
Stable Merge. _HOWEVER_ , if limited space is available, then it is allowed to
use O(n log n log n) comparisons, since a slower in-place algorithm is used.
[1]

Why? With the algorithm from this article, we can do an O(n log n) in-place
Mergesort. I imagine it's not as fast as the standard algorithm that requires
O(n) additional space -- although the abstract of this article does call the
in-place merge algorithm "reasonably competitive". But it certainly sounds
like it would be a better fall-back option.

Similarly, lots of languages have moved toward some version of Mergesort for
their standard sorting algorithm: Perl, Python ("Timsort" is a Mergesort
variant). Even some versions of the "C" Standard Library's qsort are
implemented with Mergesort these days. I never hear about any of these being
in-place. But the work in this article isn't exactly a secret.

So: I don't get it.

[1] EDIT: I checked the C++11 spec. It is the same. std::stable_sort does O(n
log n) comparisons if sufficient memory is available, and O(n log n log n) if
not.

~~~
jemfinch
"For the sake of speed and simplicity, our algorithm is not stable." There's
your answer.

In their defense, the authors do include a footnote to another of their papers
adding stability and preserving the asymptotic properties, but admit that it's
significantly slower. The answer then becomes, "Because mandating the lower
complexity bound would increase the real world costs." It's the same reason
the standard only requires nth_element to be average case linear, rather than
worst-case linear; there exist worst-case linear algorithms (like CLR's median
of medians algorithm) but their real world performance is far worse than
quickselect.

~~~
ggchappell
Ah, I didn't catch the lack of stability. Thanks.

OTOH, that does not _completely_ address my difficulties. Because there are
fall-back algorithms to consider. And so it seems to me that the slower
version of Mergesort that is both stable and in-place, still has practical
value.

You mention that fast selection typically uses Quickselect, since the known
linear-time selection algorithms have much worse average-case performance.
That's true; however, e.g., in Musser's 1997 paper where he introduces
Introsort (log-linear-time Quicksort variant), he also recommends doing
selection by starting with Quickselect, then switching to a linear-time
algorithm if the recursion depth exceeds some threshold. The result is a
linear-time algorithm with the same average-case performance as Quickselect.

As far as I know, Musser's idea is the standard selection algorithm in use
today. Or if it isn't, then it probably should be. [1]

Similarly, I have no problem with C++ std::stable_sort using linear additional
space. But if it can't, then why not fall back to another log-linear-time
algorithm, instead of the O(n log n log n) that is allowed for in the
standard? This stuff has been known for decades. It wasn't taken into account
in the 1998 standard, and it still wasn't taken into account in the 2011
standard. Why not?

I kinda feel like I'm beating a dead horse here ... but in any case, there are
still unanswered questions.

[1] EDIT: Checked the C++11 standard again. The spec for std::nth_element has
_not_ been updated to reflect Musser's idea; it's still "Complexity: Linear on
average." OTOH, the spec for std::sort _has_ been updated. In 1998 it was
"Approximately N log N ... comparisons on the average," while in 2011 it was
simply "O(N log (N)) ... comparisons."

------
chmike
Please do not submit articles behing pay walls.

------
petercooper
Being behind a paywall isn't helping with its publicity. On the plus side,
this reminds me I really do need to sign up for the ACM! :)

~~~
icebraining
Searching the title returns this:
[http://akira.ruc.dk/~keld/teaching/algoritmedesign_f04/Artik...](http://akira.ruc.dk/~keld/teaching/algoritmedesign_f04/Artikler/04/Huang88.pdf)

~~~
sold
I get 404 at this URL but google cache is available.

~~~
dhruvbird
Google Cache Link:
[https://docs.google.com/viewer?a=v&q=cache:n-YwxuJHeAoJ:...](https://docs.google.com/viewer?a=v&q=cache:n-YwxuJHeAoJ:akira.ruc.dk/~keld/teaching/algoritmedesign_f04/Artikler/04/Huang88.pdf+&hl=en&gl=us&pid=bl&srcid=ADGEESiDsi1Ltl_rtQp5cG61XBYEycPP8BeHj4dwZBfcD0619C1dA-
ZsHL67j3l7by6tUVwr8ePc3NYX7G9dkQMXAwHN1GhSieFJvoEHE-rzLGuhClGvkOQVtE8saAY-
OMjmNjSUfpGN&sig=AHIEtbRfUaTf_oNJcc1KHP0KWLHm67xArg)

------
mdonahoe
Can someone writeup a gist of the algorithm?

~~~
k_l_s
Given two sorted lists A and B concatenated as a single list AB:

Divide the list up into sqrt(n) pieces, each of size sqrt(n). Create one such
piece consisting of the sqrt(n) largest elements of the combined list and move
this piece to the left-most position (call this the buffer piece), while the
remaining pieces are simply formed from the remaining elements. Note that the
remaining pieces will retain their ordering from the fact that A and B were
originally sorted (as long as care is taken not to cross the boundary of A/B),
while the left-most piece need not be sorted at this point. Stable? [I didn't
see it in the paper, but I think it's necessary] sort the sqrt(n)-1 non-
leftmost pieces in non-decreasing order by their tail element. Finally, merge
(details left out as to when to stop the merge) the longest sequence of pieces
constructable from the beginning of the un-merged list so that the tail
element of any piece is not greater than the head element of it's downstream
piece with this sequence's downstream piece, using the left-most piece as the
buffer and swapping the min element of the two lists into the min buffer
position instead of overwriting the buffer element. After a merge, the buffer
piece will remain as a single piece and may be re-used as the buffer for the
next merge. In the end, simply sort the buffer piece. Notice that we may use
an O(n^2) sort for the two sorts that we need to perform without breaking the
merge's linear time property since in both cases we're sorting sqrt(n) items.

The min element of the remaining pieces will always make its way into the min
buffer position a sorted lists' pieces will not be re-ordered by the tail-
element sort.

Anyhow, many details left off, but you asked for a gist and I believe that
this qualifies as a gist.

