
Mapping the internet with Hilbert curves - randomdrake
https://blog.benjojo.co.uk/post/scan-ping-the-internet-hilbert-curve
======
elvinyung
Nice! Perhaps a Gibson quote is appropriate here:

> Program a map to display frequency of data exchange, every thousand
> megabytes a single pixel on a very large screen. Manhattan and Atlanta burn
> solid white. Then they start to pulse, the rate of traffic threatening to
> overload your simulation. Your map is about to go nova...

------
anigbrowl
I don't think straight hilbert curves:IP ranges are the best mapping, and
would prefer something built around ping times or hop check routings in 3
dimensions. Or integrate it with geolocation data. But! It's still super
useful. Or integrate it with geolocation data.

 _On top of all of this, I also did a bonus scan of a few APNIC IP blocks
every 30 mins for 24 hours. The data from that allows you to see the internet
“breathe” as clients come online in the morning and offline at night_

Really, I'm surprised there isn't a distributed/crowdsourced system to do this
all the time and allow people to study the 'weather' in the datasphere.

~~~
vanderZwan
H-curves have better locality, might be interesting to use those:

[0] [http://www.akt.tu-berlin.de/fileadmin/fg34/publications-
akt/...](http://www.akt.tu-berlin.de/fileadmin/fg34/publications-
akt/fct97.pdf)

[1] [http://hint.fm/papers/158-wattenberg-
final3.pdf](http://hint.fm/papers/158-wattenberg-final3.pdf)

------
walrus01
Also for fun, ipv6 exhaustion counter.

[https://samsclass.info/ipv6/exhaustion-2016.htm](https://samsclass.info/ipv6/exhaustion-2016.htm)

~~~
hexane360
Note: A power law is definitely not the same thing as an exponential fit.
Using the two interchangeably is disingenuous.

------
no_identd
I wish this used a good color mapping, like Viridis or cubehelix, or at least
used HSLuv or HPLuv to map the parameters to colors. I bet we could see a lot
more patterns in this then.

Edit: I made a github issue for this:

[https://github.com/measurement-
factory/ipv4-heatmap/issues/2](https://github.com/measurement-
factory/ipv4-heatmap/issues/2)

------
zokier
I did something similar few years back, mapping ipv4 address space owners.

[http://zokier.net/stuff/map_of_the_internet.png](http://zokier.net/stuff/map_of_the_internet.png)

------
swirepe
You can scan the whole internet in about an hour. I had luck using AWS and
zmap.

[https://github.com/zmap/zmap](https://github.com/zmap/zmap)

~~~
andai
>ZMap can scan the IPv4 address space in under 5 minutes.

~~~
hk__2
That’s misleading, because you’re quoting half of a sentence. Full quote:

> On a typical desktop computer with a gigabit Ethernet connection, ZMap is
> capable scanning the entire public IPv4 address space in under 45 minutes.
> With a 10gigE connection and PF_RING, ZMap can scan the IPv4 address space
> in under 5 minutes.

------
xvilka
According to [1] IPv6 adoption is slowed down significantly, so we stick to
NAT for a decade at least I think.

[1]
[https://www.google.com/intl/ru/ipv6/statistics.html](https://www.google.com/intl/ru/ipv6/statistics.html)

------
LeoPanthera
The 9MB PNG is unoptimized. By passing it through optipng and advdef I managed
to losslessly squish it down to 7MB.

Also, I would be remiss if I did not point out that this:

cat ping.txt | pcregrep -o1 ': (\d+\\.\d+\\.\d+\\.\d+)'

is a Useless Use Of Cat.[1]

It should be rewritten:

pcregrep -o1 ': (\d+\\.\d+\\.\d+\\.\d+)' <ping.txt

[1]
[http://porkmail.org/era/unix/award.html](http://porkmail.org/era/unix/award.html)

~~~
syrrim
>I managed to squish it down to 7 MB

wow, what a stellar compression ratio

>Useless Use Of Cat

Oh My God No One Cares

~~~
LeoPanthera
> wow, what a stellar compression ratio

It's pretty good when compared to uncompressed RGB of the same size, which
would be 48M.

------
toolslive
You can do the same with LBA's of a block device. It's interesting to see
where different file systems place the (meta) data.

------
bawana
How do the number of internet connections relate to the number of nodes?
Building fat pipes is not the answer just as more highways is not the answer
to more destinations. The increase in traffic will consume more resources
exponentially (factorially?)faster than the increase of address space

------
iod
IPv6 Active Webhosts Hilbert also exist based on Akamai data as I found this
d3 block by Vasco Asturiano:

[https://bl.ocks.org/vasturiano/0c0f60cf193fa3a04b5d414aed6f5...](https://bl.ocks.org/vasturiano/0c0f60cf193fa3a04b5d414aed6f5834)

The author also has some other cool d3 visualizations of IPv6 Routes, AS, as
well as IPv4 allocations.

------
infinity0
Surprised that he missed out 192.168.0.0/16

~~~
signa11
why ? these are non routable...

~~~
rocqua
It is missing from the table of reserved ip addresses.

He mentions:

0.0.0.0/8 Local System 10.0.0.0/8 Local LAN 127.0.0.0/8 Loopback
169.254.0.0/16 “Link Local” 172.16.0.0/12 Local LAN 224.0.0.0/4 Multicast
240.0.0.0/4 “Future use”

------
inetknght
It strikes me that we've "run out" of IPv4 address space but there's entire
large blocks of space allocated to entities that don't appear to be using
them.

------
eps
Aye :)

[https://xkcd.com/195/](https://xkcd.com/195/)

