
Users of my iOS Game Teach Me a Lesson MIT Didn’t - aaroniba
http://blog.aaroniba.net/2011/07/06/a-lesson-from-my-ios-users-they-dont-teach-at-mit/
======
trentfowler
The author is one cocky guy. This article mostly annoyed me.

>I figured that I, the author of the game, with my degree in mathematics from
MIT, and years of puzzle-solving experience, would be much better at solving
my own puzzles than random users downloading my app would be.

>I thought there was a mistake because some users were solving puzzles faster
than I was! Were they cheating?

~~~
pc
Anyone who knows Aaron will tell you that he's actually pretty humble. I think
it's easy to imagine the thought process without it being especially cocky: "I
_built_ this game and have spent more time solving these puzzles than any of
these guys. How are they so much faster?"

~~~
_debug_
OK, Aaron.

------
geuis
tldr: MIT doesn't teach everything. Author discovers rapid iteration, the
value of throwing away code, and possibly source version control because users
of his game solved it faster than he did with superior "logical" reasoning.

~~~
apike
Don't take this so lightly. I've seen a lot of brilliant engineers, especially
new CS grads, get stuck in this hole. They have the intelligence and knowledge
to "correctly" solve problems, and are driven to always to get them right the
first time.

These grads can have a hard time "just fucking doing it", especially when it
comes to getting started on very hard problems. It can take a few years of
real world experience for them to get comfortable with doing something that's
known to be wrong and moving from there.

~~~
palish
I don't understand. They know how to do things, yet they can't do things?

~~~
tuukkah
When theory meets practice, it's expected that the application doesn't come
immediately.

------
zeteo
What he calls "Iterate and Repair" is basically a greedy algorithm. Greedy
algorithms are great where they work, but there are also situations where they
break spectacularly. I guess the algorithms course is no longer required at
MIT?!

~~~
aaroniba
Most greedy algorithms I've seen do not have a repair step. But whether I call
it "iterate-and-repair" or "greedy" doesn't change the message, does it? Or
are you saying that I shouldn't be surprised that iterate-and-repair works so
well on this puzzle?

~~~
zeteo
What I'm saying is that you shouldn't draw such general conclusions from
seeing a simple algorithm work in a particular case. Your problem-game turned
out to be algorithmically simple. This probably makes it more appealing as a
game. But most things in life will not be as simple, and this includes large
coding projects.

What you call the repair step I see as running the greedy algorithm again,
from a different node in the search graph.

~~~
vecter
Can you define "algorithmically simple"? This looks like hamiltonian cycle
with an extra constraint that there is one bounded region. I wouldn't be
surprised if this problem was NP-hard.

~~~
zeteo
You're right, there are probably instances of this problem for which greedy
algorithms are quite slow.

------
_pius
Mentioning MIT makes this a nice, low-brow analog of "Worse is Better" by
Richard Gabriel:

<http://en.wikipedia.org/wiki/Worse_is_better>

------
dpcan
Every programmer could benefit from taking creative writing courses in
college.

You are forced to purge your mind of every disorganized thought onto paper,
and then go back through as often as necessary until it's right.

This may not be a great "programming" strategy, but it certainly makes the
design process more fun and productive.

~~~
blahedo
One of my students (CS) once took a creative writing class, and her professor,
a man well-known in the creative writing community, told her he thought
computer programmers make better writers. She was pretty sure this had a lot
to do with understanding revision and outside input. When you write a program,
the compiler (or possibly your unit tests) tells you "it's wrong! Fix it!" and
doesn't always give a lot of guidance as to how, but you figure it out and
come out the other side with a better product. A lot of the other students in
the class took the critical comments of the professor (and peers) as almost
personal attacks on their perfect creation.

All of which is to say that the Computer Science/Creative Writing benefits may
go both ways.

~~~
MaysonL
Writing is 90% editing: so is programming.

------
tmgrhm
As a designer (and soon-to-be programmer) I totally agree that iteration is
key. Iteration is so important to producing a polished product with the best
possible aesthetics and experience.

However, I really do not agree that the right approach is to not think about
the best possible design beforehand. Ramming your fingers in there and
fumbling around blindly is a terrible idea. Think beforehand about the best
possible solution and then repair and iterate on that.

------
sc68cal
Yes - this is exactly the advice that BWK gives. Rule #2 specifically.

    
    
        Keep it simple.
        Build it in stages.
        Let someone else do the hard part.
    

Sourced from: <http://rixstep.com/2/2/20080125,00.shtml>

~~~
pyre

      > Let someone else do the hard part.
    

Obviously not everyone can follow this strategy or none of the hard parts
would get done.

~~~
sc68cal
You're missing the meaning. You should be using libraries and other small
tools, to assemble more complex and powerful tools. Not reinventing the wheel.
That is the way of Unix.

------
coryl
As someone who has used trial and error processes my entire life to figure out
problems, am I missing out by not being a deliberate planner?

------
da5e
When I was a kid I had the top score in Galaga at my local arcade. I held the
spot for months. Then one day I came in to find my score doubled by someone.
The arcade owner told me that it was a guy who went around to each arcade and
ran up the scores. Like a game playing gunslinger.

Now on the web any time I find a fun game and play it for awhile and improve,
I come online to find that the high score has been run up by some
aspieoverlord to unwinnable levels.

So how do game designers overcome that demotivating unwinnable high score
phenomena?

~~~
monkeypizza
There should be 3 levels of high score lists should be 1. personal 2. friends
3. global

this actually applies everywhere you are showing group merged data related to
a particular person.

i.e. your height, your friends heights, and global heights

imagine a news article that has interactive graphs that query g+ or fb and use
the info to put a friends layer next to the global info.

~~~
chopsueyar
Instead of social circles, how about geographical locations?

1\. local (city or neighborhood)

2\. statewide

3\. country

4\. earth

It is more analogous to the parent comment.

------
tylerneylon
This reminds me of the famous Knuth quote: "We should forget about small
efficiencies, say about 97% of the time: premature optimization is the root of
all evil."

In this case, I'd say the premature optimization was simply trying to prove
the correctness of each move before it was made. In the bigger picture,
programmers have a meta-algorithm for how they solve problems. It's tempting
to sit down and figure out an O(1) or maybe O(n) algorithm - whatever is the
fastest possible - and only then start to program. In reality, many of these
optimizations, especially on a life-scale, can be counter-productive, because
the cost of designing, implementing, and maintaining them far outweighs the
benefits. One of the hardest parts of good engineering is true simplicity.

So I'd say Knuth's suggested philosophy applies to problem-solving at large,
and this is one perspective on the author's epiphany.

~~~
shaggyfrog
Eliminating invalid moves is not a "premature optimization", but is at the
heart of heuristic search. By eliminating impossible/invalid states, you
eliminate them from the search space, which not only speeds things up, but
prevents the discovery of an invalid "solution". Put another way, discarding
invalid moves reduces the branching factor of the problem -- a fundamental
optimization.

------
ericflo
This is exactly how I solve everything, and it's why I'm so bad at programming
on a whiteboard in interviews.

------
alanh
Reminds me of this strategy for finding Hamiltonian circuits in graph theory.
<http://alanhogan.com/asu/hamiltonian-circuit/>

------
mcphilip
Good thing he didn't make a sudoku app or he wouldn't have learned the same
lesson... good luck taking an iterative approach to that.

Seriously though, sometimes it's good to approach a problem using that advice
from the chess teacher in Searching for Bobby Fisher: "Don't move until you
see it"

[http://www.youtube.com/watch?v=V36gt6_uJy4&t=7m34s](http://www.youtube.com/watch?v=V36gt6_uJy4&t=7m34s)

~~~
ebiester
Oh, iterative approaches often are used when stuck in sudoku. deduce
everything you can, but once you can't, guess where there are the least
options remaining. erase if you can't get there from here.

~~~
mcphilip
Sure, you can guess at Sudoku if you want (though it's often considered poor
form to design a puzzle that requires a guess), but the (intended) point was
that you don't jump right in and start iterating from the beginning.

------
monkeypizza
This game is similar to Mortal Coil (flash frontend to a programmable backend)

<http://hacker.org/coil/>

But that allows automated solutions. As far as I know, all of the best solvers
there use a greedy algo, although there are some cases where you can prune
massively by using logical deductions as well.

------
caf
It sounds a bit like simulated annealing.

~~~
shaggyfrog
Simulated annealing is a probabilistic algorithm.

The state space for this game is not stochastic. It's deterministic.

Standard single-agent search techniques apply here (i.e. A* and its variants)
and essentially encapsulate this "iterate and repair" idea (especially IDA*).

~~~
caf
It doesn't sound like the users mentioned are using a completely deterministic
algorithm to determine the next solution to try, though - it sounds more like
they're making random tweaks. Albeit the tweaks are guided by intuition, which
makes it somewhere in between the two - neither completely deterministic nor
completely unguided.

~~~
shaggyfrog
Sounds like MIT could have taught him a lesson about selecting the correct
tool for a job, then -- there's no real sense to use a non-deterministic
algorithm to solve this class of problem. Beyond the spirit of just
experimenting, of course.

~~~
jules
Except the non deterministic algorithm worked better. Did you read the blog
post at all?

~~~
shaggyfrog
It's a false conclusion. His non-determinisitic algorithm is decidedly less
efficient than A*: it's essentially following a similar algorithm except it's
randomly choosing what node to expand next, sometimes repeating expanding the
same node.

~~~
jules
This problem cannot be solved by A*. This is a combinatorial problem which is
likely even NP-complete.

Also, we're talking about algorithms executed by humans.

~~~
shaggyfrog
The problem can most certainly be solved with A* or one of its variants.

Just because a problem is NP-complete doesn't preclude the use of
deterministic algorithms. For example, you can solve small instances of TSPs
with standard search techniques.

I'm not sure what distinction you're trying to make with your last statement.

~~~
jules
Can you explain how you'd solve this using A* efficiently? You will see what I
mean. A* finds the shortest path, this problem requires a Hamiltonian cycle,
two entirely different problems.

Finding a Hamiltonian cycle has much more in common with solving a sudoku
puzzle than with finding the shortest path in a graph; both are problems for
which no polynomial time algorithm is known. Now, this problem is about
finding a Hamiltonian cycle in a restricted class of graphs, so there is a
chance that you could solve it efficiently, but it is non obvious. If you know
how, perhaps you should publish a paper on it: that might be a major
breakthrough.

~~~
shaggyfrog
A-star is not limited to performing just pathfinding; consider it more a graph
traversal algorithm (which fits well with this state space). I would use one
variant in particular, IDA-star, for solving this game. If we consider one or
more nodes connected together as a "cluster", starting with an "empty
cluster", the start state is a set of clusters, and the end state is one
single cluster that represents a cycle. A "move" would connect two clusters
together.

There is nothing particularly publish-worthy although it would make an
interesting state space for a graduate-level AI class assignment.

(Another state space -- game -- that we can use A-star to solve are sliding
tile games, which have nothing outwardly to do with pathfinding per se.)

~~~
jules
What you are describing is exhaustive search on the state space and is orders
of magnitude slower than the randomized algorithm, especially when executed by
humans (which is why I added the qualifier "efficiently"). Also, A* is for
finding the shortest path, and the length of the path in this case is
irrelevant. In fact the length of all paths is the same.

------
sev
The used method reminds me of randomized sorting algorithms and how they're
often more efficient than deterministic methods.

------
danking00
Forgive me for I am a student of HtDP and not SICP, but when this author was
attending MIT wasn't 6.001 still in place? The Scheme REPL, which I imagine
was introduced to students of 6.001, encourages an iterative approach to
problem solving.

Perhaps _this author_ didn't learn about iteration from MIT, but implying that
MIT is entirely ignorant of this seems unfair.

edit: formatting

~~~
Legion
Or possibly he wasn't being so pedantically literal.

------
Apocryphon
I suppose MIT students don't learn guess and check.

