
Mystery Math Whiz and Novelist Advance Permutation Problem - beefman
https://www.quantamagazine.org/sci-fi-writer-greg-egan-and-anonymous-math-whiz-advance-permutation-problem-20181105/
======
robinhouston
I’m one of the people who has worked on this problem, and is quoted in the
article. Anyone who is seriously interested in this problem may join the
group[0] where most of the interested people hang out.

We’re making progress pretty rapidly at the moment. The lower bound has
recently been improved (by one), and Greg Egan’s construction has been shown
to be the best possible under certain conditions.

I think it’s reasonably likely – though definitely not certain – that we will
have a complete solution within weeks.

0\.
[https://groups.google.com/forum/?nomobile=true#!forum/superp...](https://groups.google.com/forum/?nomobile=true#!forum/superpermutators)

~~~
aidos
Cool!

In the article they say _" The factorial rule predicts that to watch six
episodes in every possible order should require 873 episodes, but Houston
found a way to do it in 872. And since there is a simple way to convert a
short superpermutation on n symbols into a short superpermutation on n + 1
symbols, Houston’s example meant that the factorial rule fails for every value
of n above 6 too."_

I don't quite get this. Is there an inductive proof you can use as a
construction to reduce the length for subsequent N? Would love to see a birds-
eye view of the intuition of this.

~~~
robinhouston
Yes, there’s a recursive construction that converts a superpermutation of
length l on n symbols to a superpermutation of length l + (n+1)! on (n+1)
symbols.

The construction is described in various places online. I like Greg Egan’s
description here:
[http://www.gregegan.net/SCIENCE/Superpermutations/Superpermu...](http://www.gregegan.net/SCIENCE/Superpermutations/Superpermutations.html#ALG1)

Sorry for the delayed reply. I was stymied by my aggressive noprocrast
setting.

~~~
aidos
That's my lunch time reading sorted, thanks!

------
daenz
The article doesn't focus much on the 4chan aspect of the story, but I think
it's interesting to realize that your only real currency on anonymous message
boards is the quality and content of your post at that moment in time. In the
real world, or anywhere online where you have a reputation, that reputation
can give you credibility when credibility is not deserved. It also attaches an
ego dynamic to everything you say, and every time you're called out. With
anonymity, that's all gone. If your ideas stand, it has nothing to do with
your reputation. And if you're shown to be wrong, it's much easier to change
your position and adjust your worldview.

~~~
YorkshireSeason

       real currency on anonymous 
       message boards is the quality 
       and content of your ...
    

I suggest to generalise this great insight: anonymity, and the requirement
that one's arguments must be valid _independent_ from any reputation
associated with the speaker, is the core essence of the scientific method. The
edge case is _mathematical proof_ : nobody would accept a mathematical theorem
because it was from <famous person>. Instead it must be accompanied by a
proof, a proof so detailed that everyone can verify it. Over 2000 years ago,
Plato (in his Meno) use the _slave boy_ as an example for "everyone", while
today, we are even more stringent: even a machine (e.g. Isabelle/HOL, Agda,
Coq, Lean), the prototypical mechanised slave, must understand a proof.

Scholarly tools like _anonymous_ peer review, double blind reviews, double
blind experiemnts, anonymous grading etc all work precisely because of
anonymity.

A key mechanisim behind the positive qualities of anonymity is that the
anonymous author no longer must fear punishment for the anonmous speech. The
key invention here was the invention of _writing_ , which enabled the
separation (in space and time) between author and audience. This was amplified
by the invention of the printing press (which amphified the audience), and
later the internet.

I'm not a historian of writing, but I suggest the following hypothesis:
_contemporary arguments against online anonymity (e.g. leads to trolling etc)
are structurally somewhat similar to (but not idential with ) historical
arguments against writing and the printing press_.

~~~
whatshisface
> _today, we are even more stringent: even a machine (e.g. Isabelle /HOL,
> Agda, Coq, Lean), the prototypical mechanised slave, must understand a
> proof._

This is not quite true yet. An absolutely minuscule fraction of math papers
published today involve machine proofs.

~~~
MrEldritch
And indeed, this will likely always be the case, as the true purpose of
proving a mathematical result is almost never to actually get the boolean
true/false output, but to illustrate how to solve the problem in the first
place and what methods were helpful for that.

Which means that the proof doesn't _have_ to be machine-checkable; in fact,
it'd be _less_ effective at doing its job if it included every trivial step.
It's not code. It's pseudocode; the purpose is to communicate to humans.

~~~
IngoBlechschmid
> And indeed, this will likely always be the case, as the true purpose of
> proving a mathematical result is almost never to actually get the boolean
> true/false output, but to illustrate how to solve the problem in the first
> place and what methods were helpful for that.

As a working mathematician, I wholeheartedly agree with the second part of the
quoted sentence! But I think that the first part isn't quite right: The
function of proofs to determine truth is extremely important, especially
considering that, especially in pure mathematics, we often can't check our
results in any other manner (like conducting physical experiments or checking
many cases with the computer).

Bad experiences with proofs which later turned out to be faulty led Voevodsky
to initiate his _univalent foundations_ program (also called _homotopy type
theory_ ), where all results should be checked by the computer. (For the
particular area he was working on, no existing theorem prover turned out to be
sufficient. He made a breakthrough in finding a new foundation to base theorem
provers on, which made it possible to formalize results in his area of
mathematics not only in principle, but also in practice.)

The hope is also that with computer-checked proofs, we can advance to new
areas of mathematics which are currently inprenetable to our puny human minds.
I'm very much looking forward to this future!

You can learn more about Voevodsky's vision in slides of him and in a recorded
lecture:
[https://www.math.ias.edu/~vladimir/Site3/Univalent_Foundatio...](https://www.math.ias.edu/~vladimir/Site3/Univalent_Foundations_files/2014_IAS.pdf)
[https://www.youtube.com/watch?v=O45LaFsaqMA](https://www.youtube.com/watch?v=O45LaFsaqMA)

~~~
yesenadam
What do you think of Lamport's 'structured proofs'?

[https://www.microsoft.com/en-
us/research/uploads/prod/2016/1...](https://www.microsoft.com/en-
us/research/uploads/prod/2016/12/How-to-Write-a-Proof.pdf)

~~~
IngoBlechschmid
I love them! Done well, they can be extremely pleasant to read and understand.

A good friend of mine creates those as a hobby:
[https://pwacker.com/](https://pwacker.com/)

------
Nition
Here are the archived original 4chan posts:
[https://warosu.org/sci/thread/S3751105#p3751197](https://warosu.org/sci/thread/S3751105#p3751197)

There are also a bunch of rather unfortunate new comments at the end of the
archive from the last few days, when the link has obviously gained some public
attention. However, in the midst of all the new nonsense, the recent one that
begins "I think I solved it" is pretty interesting:

    
    
      --Anonymous Sun Oct 28 14:10:06 2018--
    
      I think I solved it.
    
      First calculate the max string which is the number of permutations multiplied by n.
      Then divide the max string by n - 1
      Then add to the answer the fibonacci number located at 2(n-3)
      
      int max_string = n! * n;
      int answer = max_string / (n - 1);
      answer += fib(2(n-3));
      
      Example for n = 5;
      
      max_string = 5! * 5 = 600
      answer = 600 / 4 = 150
      answer += fib(2(5-3)) = fib(4) = 3
      answer = 153
      
      Here is the first 14 using this method:
      
      2 -> 3
      3 -> 9
      4 -> 33
      5 -> 153
      6 -> 872
      7 -> 5901
      8 -> 46135
      9 -> 408384
      10 -> 4032377
      11 -> 43909467
      12 -> 522549784
      13 -> 6745945965
      14 -> 93884331311

~~~
kamaal
I wonder if some day, some one will anonymously post solution to the P vs NP
problem, without fully understanding the scope and effect of the solution on
the world.

Some times not knowing the history and hardness of a tasks helps in a novice
attempting an impossible task, and arriving at the solution, completely
ignorant of the history of the problem.

~~~
yes_man
Out of curiosity, if someone proved P to not equal NP, what kind of scope and
effect would it bring to the world outside academia? P == NP would obviously
be a world-transforming proof

~~~
MrEldritch
The actual result of "P =/= NP" wouldn't actually be very interesting or
informative - after all, we're all pretty sure of that anyway.

What would be _very_ interesting - and likely give pretty deep insights into
the very nature of computation - would be _how_ exactly they proved it.

~~~
drostie
Right. The important thing about the Clay Mathematics Millenium Prize is that
it is a set of well-chosen SMART-ish goals. They want to incentivize work on
really hard fields of mathematics but they have tried to do this by choosing
results that are each not too intimidating. The Navier-Stokes prize, for
example, allows you to assume all of the nicest features of Navier-Stokes
problems in practice -- basically all of the fluid flows are well below
supersonic and are occurring in highly homogeneous, nice fluids -- and ask the
most basic mathematical question: does a smooth solution to these equations
always exist? That question by itself is not so important, but it's specific
and measurable. But it's a bit of a "reach" \-- you would have to have some
cutting insight about turbulence in order to answer this question one way or
the other. That cutting insight is what they're trying to incentivize.

P vs NP is the same: if solutions to a problem are easy to check, is there
always some better way to analyze the mechanics of the checker to make those
solutions easy to find, so that we aren't stuck with brute-forcing it? Whether
the answer goes one way or the other, the point is that solving the problem
would have to provide some insight to the effect of "here is a periodic table
of elements for all of the 'easy' algorithms -- and here are the properties of
all the 'molecules' made by combining those building blocks." And only once
someone advances our understanding with those cutting insights can we say "yes
we can always reach into the verification mechanism to understand it well
enough to build a better-than-brute-force algorithm" or "no, here is such a
thing that algorithms cannot do, it's not just that I am not smart enough to
find a way for them to do this -- they are fundamentally incapable of doing it
faster."

------
hiisukun
If anyone is interested in a practical attack that relates to this problem,
you could check out Samy Kamkar's open sesame [1]. It leverages De Bruijn
(explained in the post) to create an efficient sequence containing the
required permutations.

The short description is: a project using a child's radio-toy to open a garage
door, without knowing the correct code.

[1] [http://samy.pl/opensesame/](http://samy.pl/opensesame/)

~~~
IshKebab
I've done this on a fire alarm panel too. It checked the last digits so even
though it required 3 digits you could check all combinations with 1002 button
pressed rather than 3000.

These are slightly different problems though because you need 111 in this case
but not in the one this article is about. It's much easier (according to my
mathmo friend who solved it).

------
koolba
I’ve seen a couple articles about this problem but never quite understood what
the difference was between it and simply the number of permutations. This
paragraph from the article does a great job (finally!) show casing it in an
example:

> If a television series has just three episodes, there are six possible
> orders in which to view them: 123, 132, 213, 231, 312 and 321. You could
> string these six sequences together to give a list of 18 episodes that
> includes every ordering, but there’s a much more efficient way to do it:
> 123121321.

Rephrasing it a bit, the problem is generating the shortest string of tokens
(with repetition of the N tokens) in which every possible permutation of those
N tokens is a substring.

------
rollulus
I highly recommend to read his novel that's also mentioned in the article:
"Permutation City" [1]. Apart from a great story, it provides a lot of food
for thought.

[1]:
[https://en.wikipedia.org/wiki/Permutation_City](https://en.wikipedia.org/wiki/Permutation_City)

~~~
IngoBlechschmid
Agreed. It's a must-read for anyone interested in a thrilling story mixing
ethics of simulated realities, uploading of consciousness, self-modification
and finite automata.

Simulated beings are a repeated topic in Egan's stories. Some of them are
freely available online, for instance
[https://www.gregegan.net/PLANCK/Complete/Planck.html](https://www.gregegan.net/PLANCK/Complete/Planck.html)
(where simulated beings create clones of themselves which are to fall into a
black hole, hoping to gain insights on fundamental physics) and
[https://www.gregegan.net/INCANDESCENCE/00/Crocodile.html](https://www.gregegan.net/INCANDESCENCE/00/Crocodile.html)
(where a couple, after having lived for more than ten thousand years, decides
to embark on a final adventure, leaving all loved ones behind because of
differences in time perception).

------
nyc111
I'm happy to notice that mathematics is no longer confined to academic
journals. Anyone can contribute at any level. I wish a similar destruction of
academic walls to other fields, especially physics.

~~~
anonytrary
I'd imagine it being the possible for theoretical physics, as it is in math.
Otherwise, one problem is when you start needing advanced equipment and
laboratories. The barrier to contributing is higher.

~~~
n4r9
This case is exceptional in that the problem and solution can both be
explained in fairly elementary terms. Combinatorics is really the only field
of mathematics with many such opportunities available.

------
gpderetta
> math whiz and novelist

Somehow I suspected it was Greg Egan before clicking on that link.

~~~
a_bonobo
I follow him on Twitter - I expected a lot of SF related posts, but it's a LOT
of maths and physics including complex simulations:

[https://twitter.com/gregeganSF](https://twitter.com/gregeganSF)

~~~
IngoBlechschmid
Greg Egan also is a frequent contributor to _The n-category café_ , a forum
for the particular branch of mathematics I'm working in. I think it's terrific
to meet your favorite science fiction author in a place like this.

------
mabbo
It reminds me a bit of the quest to find 'God's Number' for the minimum number
of moves such that you can always solve a Rubik's Cube. In that, the upper and
lower bounds were pushed closer and closer by proofs until finally they met-
meaning the number 20 was exactly the answer. (Well, the lower bound was 20
for years, and the upper bound was pushed down over time).

> Within a day or two, he had come up with a new upper bound on the length of
> the shortest superpermutation for n symbols: n! + (n – 1)! + (n – 2)! + (n –
> 3)! + n – 3

> The anonymous 4chan poster’s lower bound, meanwhile, was tantalizingly close
> to the new upper bound: It works out to n! + (n – 1)! + (n – 2)! + n – 3

IE: The upper bound has been lowered to

    
    
      n! + (n – 1)! + (n – 2)! + (n – 3)! + n – 3
    

The lower bound has been raised to

    
    
      n! + (n – 1)! + (n – 2)!            + n – 3
    

If the mathematicians can get either number closer by (n-3)!, then they have
it.

~~~
ctchocula
This is a common proof strategy in math. If you want to show two values a and
b are equal, proving (1) a cannot be greater than b and (2) a cannot be less
than b is often easier by proving (1) and (2) separately rather than directly
proving they are equal.

------
0xFFFE
Apologies if this sounds dumb. But isn't P(n) = n! In this case, n=14
(episodes), which translates to factorial(14) = 87178291200. Which covers
every possible arrangement of the 14 episodes? Where is this 93,884,313,611
coming from? Or am I making a fool of myself here?

~~~
FabHK
Yes, so there are n! = 87,178,291,200 different permutations of those 14
sequences. You want to watch the 14 episodes in every possible order, ie in
each of those permutations. You could just watch all of those permutations
consecutively, for n x n! episodes in sequence, which would be
1,220,496,076,800 episodes (or, well, 14 episodes per permutation).

However, you can do much better, exploiting overlap, using only 93,884,313,611
episodes, which is really quite remarkable, because it is just 1.08 episodes
per permutation.

Example with 3 (from the article):

3! = 6 permutations

Bad way to watch all: concatenate all permutations

A) 123 132 213 231 312 321 (= 3*6 = 18 episodes, or 3 episodes per
permutation)

Good way to watch all permutations:

B) 123121321 (= 9 episodes, or 1.5 episodes per permutation)

Note that each of the permutations above in A is contained in the string at
B).

~~~
0xFFFE
Ah, I see it now. Thank you for explaining it.

------
chime
For those interested in some background info, here's a Numberphile video
explaining Superpermutations:
[https://www.youtube.com/watch?v=wJGE4aEWc28](https://www.youtube.com/watch?v=wJGE4aEWc28)

------
taeric
I'm curious how this relates to some of the things talked about in Vol 4 of
Knuth's work. In particular, this sounds directly related Algorithm C
(Permutation generation by cyclic shifts) of 7.2.1.2.

------
rofo1
Check the original thread here:

[http://4watch.org/superstring/](http://4watch.org/superstring/)

------
lanius
I can think of at least one prominent mathematician and anime fan who might've
written that anonymous proof.

------
option_greek
Wait till this appears on LeetCode and gets included in the FAANG whiteboard
interviews to be solved in 15 minutes.

~~~
robinhouston
I know you’re joking, but apparently it _was_ given as a programming problem
in a contest on at least one occasion.

See here: [http://www.njohnston.ca/2014/08/obvious-does-not-imply-
true-...](http://www.njohnston.ca/2014/08/obvious-does-not-imply-true-the-
minimal-superpermutation-conjecture-is-false/)

~~~
bootsz
It's not a joke: [https://leetcode.com/problems/cracking-the-
safe/](https://leetcode.com/problems/cracking-the-safe/)

It has been asked in real interviews

~~~
robinhouston
That’s a different (rather easier) problem: note that the safe combination can
contain the same digit more than once.

The solution to this problem is given by de Bruijn sequences:
[https://en.wikipedia.org/wiki/De_Bruijn_sequence](https://en.wikipedia.org/wiki/De_Bruijn_sequence)

~~~
bootsz
Ah, I see. Thanks

------
allcentury
Real life Will Hunting...?

------
berti
Previous discussion:
[https://news.ycombinator.com/item?id=18292061](https://news.ycombinator.com/item?id=18292061)

~~~
dang
Since that discussion was mostly about whether/how to cite the finding, maybe
we can have another discussion about the more interesting points: what the
finding was, and how it was found.

(The submission title is by far the biggest influence on what comments get
posted.)

