
Using Hilbert Curves to 100% Zelda - Merovius
https://blog.merovius.de/2017/07/22/using-hilbert-curves-to-100-zelda.html
======
decafb
For reference: The generic name what op does is "spatial indexing". There is
an algorithm for efficiently working with such data called Hilbert R-tree
([https://en.wikipedia.org/wiki/Hilbert_R-
tree](https://en.wikipedia.org/wiki/Hilbert_R-tree)). But there are
alternatives (see
[https://en.wikipedia.org/wiki/Spatial_database](https://en.wikipedia.org/wiki/Spatial_database)).

Many databases nowadays contain functions to these operations (e.g.
[https://dev.mysql.com/doc/refman/5.7/en/spatial-analysis-
fun...](https://dev.mysql.com/doc/refman/5.7/en/spatial-analysis-
functions.html))

~~~
brians
They can even be explained to a surprisingly low tech operator, with pre-
calculation at a high tech level:
[http://www2.isye.gatech.edu/~jjb/research/mow/mow.pdf](http://www2.isye.gatech.edu/~jjb/research/mow/mow.pdf)

------
ufo
> There might also be better approaches than Hilbert Curves. For example, we
> could view it as an instance of the Traveling Salesman Problem with a couple
> of hundred points; it should be possible to have a good heuristic solution
> for that. On the other hand, a TSP solution doesn't necessarily only have
> short jumps, so it might not be that good?

TSP is actually very amenable to heuristics and state of the art branch-and-
bound algorithms can often find optimal solutions even for instances with
thousands of points.

Does anyone here know if there is a good open-source solver that we could
throw this problem instance at?

~~~
modalduality
The issue isn't about approximating a TSP instance, but ensuring that points
aren't too far apart from each other, even at the cost of increasing total
length.

I can't immediately think of a reduction that would factor this in so the best
thing may be to just reduce this to the appropriate ILP instance and use a
mixed ILP approximator like GLPK (or Gurobi is free for students too).

Incidentally, achieving a polynomially-sized ILP formulation for TSP isn't
quite obvious. Wikipedia has a good explanation of how to do this:
[https://en.wikipedia.org/wiki/Travelling_salesman_problem#In...](https://en.wikipedia.org/wiki/Travelling_salesman_problem#Integer_linear_programming_formulation).
I'm not sure if the metric TSP has a simpler formulation.

Edit: Now that I think about it, it may be difficult to express the constraint
that edges should be balanced out in a linear way. I happened to write about a
similar problem a while ago ([https://modalduality.org/posts/optimizing-color-
coding/](https://modalduality.org/posts/optimizing-color-coding/)), I ended up
giving up on finding a linear formulation and went for sequential least
squares instead.

~~~
Retric
If you want to minimize the long distance walks add an exponential weight for
distance then solve. There are many ways to go about this stuff, but the goal
is to map your preferences to the weight function.

~~~
modalduality
Right, but then the problem is no longer linear and approximating is more
difficult. I'm not exactly sure what you mean by "exponential weight" in a
linear program, do you have an example?

~~~
Spivak
So say you have a graph on distance, d, that you want to weight based on a
preference, p, from 0-5 going from least preferred to most preferred.

You might make your weight function as w(d, p) = d * (2 ^ p)

~~~
modalduality
That's fine, but `p` is a variable, not a constant, in the original post's
case, right?

~~~
tgb
That doesn't seem necessary. Picking a value of p just selects how much you
prefer short hope to shorter total distance.

------
sdenton4
(I experimented with using space filling curves in City Skylines (a sim city
clone) a while ago; here's awriteup for any who might be interested:
[https://inventingsituations.net/2015/11/28/space-filling-
cur...](https://inventingsituations.net/2015/11/28/space-filling-curves-in-
simulated-cities/) )

~~~
Merovius
This is really interesting, thanks for linking :)

------
twotwotwo
> So I started on the onerous task of finding the last 17 locations.

A French guy (Xalikah) who did the first, manually planned 100% speedrun of
the game had a similar problem; he spent a few hours with a couple folks
helping him check his map for obscure place names he was missing, and when he
was at the last one, someone joked "99.81% speedrun," and people were
suggesting he do a slow systematic scroll over the map so they could look for
missing placenames. He wound up sleeping five hours or so and in the morning
remembered that he'd skipped using some bridge somewhere. 49 hours!

It's kinda wild that game worlds are now large enough that you can reasonably
use algorithms not only to write games but to get 100% completion playing them
as well.

~~~
skrebbel
> _It 's kinda wild that game worlds are now large enough that _

I haven't played Breath of the Wild yet, but eg Chrono Trigger has an enormous
game world as well. Did they really get much bigger since?

~~~
patio11
It is hard to make an apples to apples comparison against Chrono Trigger, but
you'd not be grossly wrong if you described BoW as having a 100X bigger map.
It is also much, much more populated with things to do, enemies to beat,
mushrooms to pick, etc.

~~~
twotwotwo
> mushrooms to pick

And durian. Delicious, delicious durian.

------
bhickey
2-opt can probably improve on the Hilbert solution
([https://en.wikipedia.org/wiki/2-opt](https://en.wikipedia.org/wiki/2-opt))

Is the world planar? There's an epsilon approximation scheme for planar TSP
that's linear in the graph size (but exponential in 1/ε)

------
eeZah7Ux
I'm confused by the grammar - is "100%" being used as a verb? Is that correct
in English?

~~~
wodenokoto
Strictly speaking you can use any noun as a verb in English. The grammar
allows for it, though most nouns will not have a meaningful verb counterpart,
semantically, or it will not have a usage that most people will understand
(i.e. it won't be colloquial)

So in a speed running context, I would definitely say it is acceptable, but it
might not be in the greater population. But the grammar nazi have nothing on
this one.

------
nwjtkjn
Why is the Hilbert Curve any better in this situation than say lexicographical
order on (x,y)?

~~~
cbhl
It's pretty easy to imagine a counter-example that would have you traverse the
length of the map, when the best route would be to detour on an existing
traversal. Consider:

    
    
        1...2...3...4...5
        7...............6
        8.......9........
        13..12.....11..10
    

versus

    
    
        1...2...3...4...5
        13..............6
        12......9........
        11..10......8...7

~~~
nwjtkjn
Good point, though I'm not convinced Hilbert really avoids this. For example,
look at n=3 here:

[http://www.texample.net/media/tikz/examples/PNG/hilbert-
curv...](http://www.texample.net/media/tikz/examples/PNG/hilbert-curve.png)

This might lead to say

. . . . . . . .

1 . . . . . . 2

4 . . . . . . 3

. . . . . . . .

when

. . . . . . . .

1 . . . . . . 4

2 . . . . . . 3

. . . . . . . .

is more efficient.

Generally speaking, it seems this is basically TSP.

~~~
Merovius
I tried describing here
[https://www.reddit.com/r/programming/comments/6oxra8/using_h...](https://www.reddit.com/r/programming/comments/6oxra8/using_hilbert_curves_to_100_zelda/dkn3gbr/)
why I think that this is theoretically a TSP problem, I don't consider that
necessarily the _best_ way to treat it. tl;dr is, that at least to me, at the
time, for this problem, the speedup of an optimal solution wouldn't have
outweighed the additional thinking time to get to that optimal solution (and,
most importantly, to _actually_ make it useful; after all, I wasn't actually
drawing a path, I was outputting a list of names).

You can even see that in the argument I added to the post; while a zig-zag
line might've been, in theory, a stupider, easier way to solve the problem, in
practice it would've meant spending some time thinking about the right
discretization. With a Hilbert-curve, I could just choose some n that is
definitely large enough and be done with it and the additional cost of the
more complicated curve doesn't really factor in, as I could just copy-paste it
anyway.

But yes. The theoretical problem is a TSP and with the right set of tools, I
could've added some efficiency to the search by viewing it as such.

------
nitrogen
_The returned file turns out to not actually be JSON (that 'd be too easy, I
guess) but some kind of javascript-code which is then probably eventually
eval'd to get the data:_

    
    
        /**/jQuery3110644358575
        2152035_1500757689075(
        /* json-data */)
    

Is that just JSONP?

~~~
xxbondsxx
Yes this is just a standard JSONP response as served through jQuery; not sure
why this wasn't clarified.

~~~
cbhl
My guess is author has never heard of JSONP before. Obligatory:
[https://xkcd.com/1053/](https://xkcd.com/1053/)

~~~
Merovius
I have seen JSONP. But I'm not a frontend dev and in my experience the web
suffers from a persistent overspecialization of terms. I didn't mention the
term JSONP, because I didn't want to end up calling something JSONP just to
have someone point out, that this isn't really JSONP, but floobleworp, which
is when you use jQuery and add the comment-symbols to make it safe against XSS
or something in that regard.

Also, I still consider JSONP an incredibly gross idea :)

------
fmsf
Spoilers! Hacker news bot posting the map image into a facebook post was a
good way to make sure that I know now where all south shrines are :(
[https://www.facebook.com/hnbot/posts/1427803187274958](https://www.facebook.com/hnbot/posts/1427803187274958)

------
rileytg
excellent! i wonder if there are more algorithms that can be used in this
game. binary search is an algorithm that can be easily done by a person
(doesn't need a ton of iterations to locate result).

~~~
evincarofautumn
Binary search would make a good bar bet. “Pick a number between 1 and 1000.
I’ll make guesses, and you just tell me whether they’re too low or too high.
I’ll bet you $10 that I can guess the number in only 10 tries or less.”

------
thesmallestcat
A Zelda game that depends on Google Maps. I'm getting old.

~~~
j05huaNathaniel
The game doesn't. He references the zeldadungeon.net map which uses google
maps javascript api. Nintendo would be full of morons if they ever did
something like that. Just read the fine print.

------
j05huaNathaniel
Except that this doesn't take into account elevation.

~~~
Merovius
As this was only about efficiently enumerating the missing locations, not
walking them, that's fine. I.e. I had to find the ~5 needles in the haystack
of hundreds that I needed to visit. Given that sparseness, it was most
efficient to just jump to the nearest warp point and directly go there.

Elevation and the like would be more interesting to figure out what's the most
efficient way to collect all Korok Seeds, for example, when the interesting
point cloud isn't as sparse. But then you'd definitely want to view it as a
TSP problem anyway.

