
Ask HN: Can you help me monitor the Internet? - adrienjarthon
Hi, I want to build a service monitoring the Internet and I need your help to build the best tool possible.<p>Below is a rough idea of what I have in mind, please comment with any idea or opinion to refine my draft. How would you do it? What kind of tech&#x2F;tools do you recommend? What did I miss? Any help will be greatly appreciated ;)<p>Draft:
- A couple of servers would run various checks and store the results in a central database, a public page would be available at internet.rip and display the status of the Internet with details on ongoing outages and their estimated impact.
- Check most DNS servers (TLDs, ISPs, Public), public routers status (not sure how to do this, looking glass?), public BGP routes for instabilities or missing routes, Internet Exchanges status, route, latency and packet loss to most IPs (best would be to find the smallest set of IPv4 to monitor passing through the highest number of different routers to be able to draw a map of network paths and their status)
- Keep the infrastructure simple: don&#x27;t want to need a server on each AS so it would best to monitor from the outside as much as possible, otherwise this could be a different approach with an open-source deamon people can install on their servers to contribute, but that&#x27;s much more complicated.
- Make internet.rip (the website) as resilient to network issues as possible and for this I&#x27;m thinking CDN, multiple DNS, very long caching of the public page, service workers and multiple API endpoints to fetch the data reachable through JS directly using their IP so it could even work without any DNS if you have the site in cache.
- Keep internet.rip free by limiting the cost, the infrastructure will be supported by me. Maybe add a discrete donation button but no ads.<p>You can also reach me at adrien@updown.io if you have anything to discuss privately.
Feel free to register on the landing page to know when it launches (http:&#x2F;&#x2F;internet.rip)
======
jonny_storm
This is a hard problem. Internet tomography aside (an even harder problem),
route-servers and looking glasses everywhere expressly forbid automation, and
any network administrator, virtually anywhere, can break your reachability
testing. Add to this the myriad complications of BGP selectivity (Why do
people insist that "net neutrality" is in danger when it never existed as
stated?), and network operator participation becomes almost unavoidable.

While I understand that tools such as
[http://www.internettrafficreport.com](http://www.internettrafficreport.com)
and [http://internetpulse.keynote.com](http://internetpulse.keynote.com) are
not enough, it isn't at all clear to me how any single actor could provide
better monitoring. I admit this sounds pessimistic, but I've thought deeply
about this and similar topics for many years, and I'm all too familiar with
the obstacles to (and poor architecture underlying) Internet polling.

However, I do see a clear need for aggregated monitoring data--many sources,
re-published succinctly in one easy-to-find place--and I would certainly
donate to such a cause.

~~~
adrienjarthon
Yes indeed, that's what I noticed too and if I can do it with existing
sources, just presenting them in a cleaner way that's definitely what I will
do. I don't want to re-invent the wheel, just do as much as I can to make the
information available.

About the routing part other people mentioned
[https://atlas.ripe.net/](https://atlas.ripe.net/) which seems to be doing a
very good job at that thanks to their 4000 probes distributed across the
world, If I can use their data it'll already solve a good part of the
equation.

Thanks for your feedback jonny_storm!

------
kirankn
You may want to check out ripe.net

~~~
adrienjarthon
Yes, I already had a quick look and there's a lot of interesting stuff here,
I'll definitely need it. Thanks!

~~~
benzimmer
You especially might want to have a look at
[https://atlas.ripe.net/](https://atlas.ripe.net/).

Here you can find a podcast interview with Internet Hall-of-famer Daniel
Karrenberg[1] about RIPE and the ATLAS project (don't worry the page being in
german, the interview is in english language):
[https://requestforcomments.de/archives/280](https://requestforcomments.de/archives/280)

[1]([http://internethalloffame.org/inductees/daniel-
karrenberg](http://internethalloffame.org/inductees/daniel-karrenberg))

~~~
kirankn
A good explanation of ripe atlas can be found at Cisco ipj journal
[http://ipj.dreamhosters.com/wp-
content/uploads/2015/10/ipj18...](http://ipj.dreamhosters.com/wp-
content/uploads/2015/10/ipj18.3.pdf)

~~~
adrienjarthon
Oh sweet, thanks a lot benzimmer & kirankn!

