
Taking control of all .io domains with a targeted registration - koenrh
https://thehackerblog.com/the-io-error-taking-control-of-all-io-domains-with-a-targeted-registration/
======
walrus01
This is a huge screwup on the part of the people who run the 'root' of .IO,
and their entire operation should be severely scrutinized by ICANN.

In my opinion almost all of the 'weird' TLDs which are country codes that are
actually operated by a third party commercial service are 95% spam and junk
registrations. .TV is a good example.

Technical screwups aside, the existence of .IO and the fact that it "belongs"
to the UK government is morally questionable, since the entire country code
only exists because the British and American militaries forcibly removed the
original inhabitants of islands such as Diego Garcia so that they could use
the area as naval and air force bases.

[https://en.wikipedia.org/wiki/British_Indian_Ocean_Territory](https://en.wikipedia.org/wiki/British_Indian_Ocean_Territory)

[https://en.wikipedia.org/wiki/Diego_Garcia](https://en.wikipedia.org/wiki/Diego_Garcia)

If I had been able to successfully register these names and get live traffic
going to a BIND9 instance, my first instinct would not be to alert the company
through its first tier customer support levels, but to immediately post a
summary of the problem to ARIN, RIPE, APNIC and ICANN mailing lists. This
would get the issue in front of people who immediately understand how serious
the problem is, and hopefully one of them would be able to contact the
principals of .IO directly.

~~~
CydeWeys
I'm in the TLD space (we run a fair number of gTLDs). If a gTLD operator
screwed up like this then there could be consequences. A ccTLD, however, runs
with very few restrictions. I don't see much of consequence happening to it as
a result of this.

I will, however, say that gTLDs are generally more secure and well-run than
smaller ccTLDs, and are worth preferring for that reason. It's a weird
historical quirk that .io randomly became popular in the developer community,
but there are better options. And, as you point out, it's morally suspect,
which is why we don't use it for new domain names.

~~~
bad_user
Besides the old .org, what better options are there for software projects?

~~~
CydeWeys
We have .dev and .foo, which seem like they'd be good options, but neither are
available for open registration. Sorry :(

~~~
corobo
> .dev

I really hope that one never becomes available..

------
rmoriz
Originally, .io, .sh and .tm were operated by ICB PLC, later ICB LLC (owned by
Paul Kane[1]). Just a couple of weeks ago .io and .sh changed hands to Afilias
which swapped out all turning parts over a weekend. They really screwed up a
lot of things including losing my balance, manipulating domain information and
extortion attemps:

I was a reseller with ICB and have a contract. Without a cancellation period
(which as defined in the contract) they wanted to take over the contract,
introduce new contract details and probably a new pricing scheme. Around 20
pages of legal stuff and a window of <10 days to "decide"/comply.

I'm done with them (Afilias). I'll never ever spend a dime and advise anyone
not doing any business with them.

(But also the old technology backed (which still runs .tm) is far from secure
and reliable. At least it supports IPv6 whereas the Afilias infrastructure
still is not IPv6 ready…)

[1]
[https://en.wikipedia.org/wiki/Paul_Kane_(entrepreneur)](https://en.wikipedia.org/wiki/Paul_Kane_\(entrepreneur\))

------
mreithub
Wow, I don't think I would've even considered such an attack...

DNSSEC, HSTS and Certificate Pinning would've made it more difficult to abuse
this, but I guess it would've been pretty easy to get valid SSL certificates
for all your favourite .io domains.

Let's try to play malicious party here:

Phase A: First set up a simple DNS forwarder playing by the rules and
answering requests as we should (as to not get any unwanted attention). Gather
usage statistics.

Phase B: Crawl the list of most-used domains to see if there are any valuable
targets without HTTPS (port 443 is closed). Alternatively/additionally see if
there are API subdomains used by software other than browsers (of which a few
won't have annoying features like Cert Pinning - golang's DNS resolver for
example afaik doesn't do DNSSEC). Pick some medium to high level targets where
the attack might go undetected for at least some time.

Phase C: MitM time! Get certificates for the target domain(s) of your choice
and get to work. Start with only a few percent of the requests to not draw too
much attention (and to avoid the majority of their traffic coming from a
single IP (range) all of a sudden) Obfuscate the attack by acting like a third
party app or something simply doing requests for their users.

Congratulations on finding the vulnerability (and thanks for looking for that
kinda stuff in the first place).

~~~
LinuxBender
If you control the root DNS servers for .io, you can simply not answer the
DNSSEC queries. Many resolvers will fail open.

HSTS requires the site is HTTPS with a valid cert. If you own all .io, you can
use LetsEncrypt to get that for free. They now even support Wildcard Certs!
:-) That said, you would have to choose your targets carefully and/or load
balance your requests to LetsEncrypt. There is a rate limit. There are browser
plugins that can tell you if a cert just changed, assuming you have been to
that site prior.

Then there is Public Key Pinning. This would be great, but I suspect the
number of big companies implementing this are low. I don't have numbers, but
you can test your favorite sites in Qualys[1] or using testssl.sh[2] that only
depends on openssl and bash.

You could proxy all requests to the real root servers for .io and only become
authoritative for the ones you wish to target.

Given the small number of zones, I think a modest server could keep up, or you
could balance the load on a bunch of VM's. It may take a while for anyone to
notice. I am curious actually, how many fellow geeks have nagios/sensu alerts
that would tell them if the root server IP's changed.

All of this said, there are BGP attacks you can do that accomplish the same
thing for any TLD and the IP's wouldn't even have to change. Only more
advanced monitoring tools that keep an eye on route path might notice, but
probably would not alert anyone.

[1]
[https://www.ssllabs.com/ssltest/index.html](https://www.ssllabs.com/ssltest/index.html)
[2]
[https://github.com/drwetter/testssl.sh](https://github.com/drwetter/testssl.sh)

~~~
ConfucianNardin
LetsEncrypt don't support wildcards _yet_. They will, starting January next
year.

~~~
LinuxBender
Ah, I was not aware it would be January. Then for now the targets would have
to be specific DNS records.

------
justboxing
I read this, but still don't fully understand the implications or impact for
.IO TLD services and domain owners.

From the OP's "Impact" section.

> Given the fact that we were able to take over four of the seven
> authoritative nameservers for the .io TLD we would be able to
> poison/redirect the DNS for all .io domain names registered. Not only that,
> but since we have control over a majority of the nameservers it’s actually
> more likely that clients will randomly select our hijacked nameservers over
> any of the legitimate nameservers even before employing tricks like long TTL
> responses, etc to further tilt the odds in our favor.

What does this mean? Is the "poisoning" / "redirecting" of traffic for ALL .IO
domains possible through this "hack" because of some flaw at the .IO
registrar, or at 101domains.com or at Nameservers that .IO registrars are
using, or something else?

How can this be mitigated? Or am I over-reacting since I don't fully
understand this story?

~~~
billyhoffman
Yes. It does.

Someone noticed that the 4 of the 7 hostnames that were assigned for
authoritative names servers for .IO were available for registration. They
registered them, and started receiving DNS lookups for .IO hosts from what
appears to be actual internet users. Since the user had 4 or the 7, its
possible that the majority of DNS lookups for .IO hosts would be sent and
answer by the author's systems.

The author could have started replying with malicious lookups. "Oh some-sexy-
saas.io? yeah, that's [evil IP]." "oh, billing.otherapp.io? what a surprise
that site is _also_ available at the same [evil IP]!"

How could .io sites have avoiding getting spoofed? HTTPS + HSTS would prevent
the author from spoofing the DNS of that those sites and sending them to a
server over HTTP thus avoiding the certificate errors.

update: I overlooked that sites would also need to leverage Public Key Pinning
to be protected, since getting a valid DV cert for a spoofed cite when you
control DNS would be likely.

[https://en.wikipedia.org/wiki/HTTP_Public_Key_Pinning](https://en.wikipedia.org/wiki/HTTP_Public_Key_Pinning)

~~~
finnn
>HTTPS + HSTS would prevent the author from spoofing the DNS of that those
sites and sending them to a server over HTTP thus avoiding the certificate
errors.

I'm confused, how would that help? Could the attacker (the author, in this
case) not get a valid https certificate for these domains, returning spoofed
DNS responses when the CA goes to validate it?

~~~
billyhoffman
Depends, but for DV certificates, most likely. Certificate Transparency
could/would help alert the original site if that was the case.

~~~
finnn
yes, but HTTPS + HSTS would not enforce a certain validation level, eg you
can't enforce EV certs only in HSTS (as far as I know), so a DV cert would be
sufficient

------
mdellabitta
Just to add some color to this: There was an attack last Friday on a few geo
TLDs that ended up hijacking a bunch of traffic for a few hours, including
.la, .es, and .jp:

[https://news.gandi.net/en/2017/07/report-on-
july-7-2017-inci...](https://news.gandi.net/en/2017/07/report-on-
july-7-2017-incident/)

~~~
sirn
I'm one of the unlucky few that were affected by this (I own .ch).

One of my site user's reported that the website is inaccessible on Friday. I
went to check and observed the DNS changing. Then went to check Route 53
status page[2] in which I learn that this is not specific to my site. The
behavior is exactly the same as what is described in the SWITCH report[1].
Luckily that I have HSTS on my site, so the damage is limited (users not
getting redirected), and Gandi seems to fixed this quick enough (I was in the
middle of commuting back home by the time of the attack.)

[1]: [https://securityblog.switch.ch/2017/07/07/94-ch-li-domain-
na...](https://securityblog.switch.ch/2017/07/07/94-ch-li-domain-names-
hijacked-and-used-for-drive-by/)

[2]: [http://status.aws.amazon.com/](http://status.aws.amazon.com/) (The blue
icon for Amazon Route 53 Domain Registration)

~~~
jontro
We were also affected by this on a major e-commerce site. It was a .se domain.
Their post mortem isn't really convincing (
[https://news.gandi.net/en/2017/07/report-on-
july-7-2017-inci...](https://news.gandi.net/en/2017/07/report-on-
july-7-2017-incident/) ) since they do not state what really happened and how
it can be prevented again.

I issued a support ticket to aws today to see what measures can be taken,
otherwise we might need to change registrar.

~~~
vbernat
There is a more detailed followup today:
[https://news.gandi.net/en/2017/07/detailed-incident-
report/](https://news.gandi.net/en/2017/07/detailed-incident-report/)

~~~
vertex-four
> These credentials were likewise not obtained by a breach of our systems and
> we strongly suspect they were obtained from an insecure connection to our
> technical partner’s web portal (the web platform in question allows access
> via http).

This makes no sense - how did the attacker get between gandi.net and their
technical partner in order to MITM them? MITMs aren't magic - simply sending
an unencrypted password somewhere doesn't result in it becoming public
knowledge unless a router or switch in the path is malicious.

~~~
musicnarcoman
> This makes no sense - how did the attacker get between gandi.net and their
> technical partner in order to MITM them?

On the top of my head, bgp hijacking perhaps?

> MITMs aren't magic

No. But do not trust the network. Ever.

~~~
vertex-four
If it's BGP hijacking, there'll be evidence somewhere.

And no, don't trust the network, but "the network isn't trustworthy" is not a
diagnosis, only a potential risk factor. "X entity used BGP hijacking to
situate their router between me and Y" is a diagnosis.

------
michaelbuckbee
So, the real question is: "How much should we freak out about this?"

If you scroll back a few months to Cloudbleed/Cloudflare we sort of
collectively decided that because cache data containing sensitive info
(passwords, tokens, whatever) might be accessible for your site using
Cloudflare that everything should be revoked, force password resets, etc.

Now we have this vuln, which I'll dub "IOgate" because it's the cool thing to
name these. We don't know if this has ever happened before, there clearly were
not adequate safeguards in place, etc.

Should anyone operating a service using a ".io" TLD consider everything
potentially compromised?

~~~
lwansbrough
Well, if it's any consolation.. the domains weren't registered..

~~~
michaelbuckbee
Per the article, he got them registered and pointing at his own DNS test
server and received actual .io resolution requests to it.

~~~
softawre
Yes but [per the article] they weren't registered _before_ he bought them.
Obviously somebody could have done this before and stopped, or could control
some of the other servers.

------
belorn
As the article mention, this is a nice example where DNSSEC would had
prevented malicious activity for users which has DNSSEC validation enabled.
There are also countries like Sweden were almost all ISP has this, so a rather
large group of people in the world would likely have noticed if a majority of
.io nameservers was responding with unsigned data.

------
ChuckMcM
I am really super happy that the root domain serving the largest IOT
population on the planet wasn't co-opted by the MIRAI bot writers.

That could have been a net killing event.

~~~
gboudrias
There's an IOT population? I thought it was just a dumb trend everyone hates?

I'm not trying to diss anyone, I just thought the whole "IoT" concept was dead
in the water?

------
hk__2
Side note: Please don’t use such gray and thin fonts. I had to modify the CSS
to use black instead of #555 for the text color.

~~~
CrystalLangUser
I don't mean this pejoratively, but how old are you? I'm just curious as I had
no problems with the color / font weight. For my sites I usually use something
like #232323 instead of pure black.

~~~
narrowtux
I'm 25 and had to zoom to read and even then it wasn't comfortable. (not the
OP)

~~~
CrystalLangUser
Ah I see, upping the font-weight to 400 helped when viewing it on my laptop
screen. I didn't really have that issue on mobile, though.

------
tbarbugli
Not the first time .io TLD messes things pretty bad. A few months ago they had
a couple of name servers poisoning internet with false negatives, that made a
few hours very "interesting"

------
Mizza
NIC.io are a terrible, terrible registrar. I reported an account takeover
security vulnerability to them 6 years ago that they only fixed after 4ish
years.

------
inetknght
> After sending the email I immediately received a bounce message indicating
> that the adminstrator@nic.io was not an email address that existed at all

> This was not a strong vote of confidence that someone was going to see this
> notice.

Honestly though, this seems like common practice to me.

~~~
JorgeGT
I though ICANN was supposed to be getting serious against fake contact data on
domain registrations? The fact that even a TLD's NIC admin info is fake is
pretty spectacular.

------
_eht
This sounds an awful lot like what happened to the .sr ccTLD on 06/23/17\.
Part of my business runs on a .sr and we were down several days.

Any way to look up a history of such an event?

------
tannhaeuser
Good grief! I've always found using vanity domains for your project/company to
be tasteless at best. Now .io is even associated with deportation, dispute of
territory, and security screwups. Using an .io domain only serves to
demonstrate that you care about pretending to be a 2010-ish startup at the
expense of everything else at this point.

------
7ewis
I had a similar issue with the .IM domain three months ago. One of the four NS
for the domain was not responding.

Two of the guys at Cloudflare diagnosed it for me:
[https://twitter.com/xxdesmus/status/855858441289572353](https://twitter.com/xxdesmus/status/855858441289572353)

~~~
addedlovely
One Cofounder of cloudflare, pretty impressed they picked up tweets directly.

~~~
jgrahamc
Many people at Cloudflare (including myself, Matthew, Justin and others)
monitor Twitter, HN and other forums carefully and reply quickly to folks.

Personally, I run a script that emails me for every HN comment that mentions
Cloudflare and use Tweetdeck to monitor @cloudflare/cloudflare mentions.

------
chmike
Not even a "thank you" message from the TLD managers ? That is the most
shameful behavior.

------
davisonio
Wow this is quite shocking. A very nice find, this is all the more concerning
considering many high profile startups including financial ones are using .io
domains (I'm using one too).

------
mgalka
Wow! That is one scary vulnerability. Glad we have smart people like this to
find such problems before the bad guys do. Well done!

------
mpounsett
While it's definitely an error on the part of the backend registry operator
for .io, this is not the major security issue the author describes. He
couldn't have hijacked any DNS traffic this way.

I've written in detail about why this is the case at
[https://mpounsett.blogspot.ca/2017/07/the-io-error-
problem-w...](https://mpounsett.blogspot.ca/2017/07/the-io-error-problem-with-
bad-optics.html)

~~~
mandatory
Author here, responding here like I did on Twitter. DNS resolver
implementations matter here greatly. I received so many DNS queries (without
me actually responding to any of them) that I quickly filled up my VPS with
gigabytes of data from IP addresses of DNS resolvers across the Internet.

Saying "this is not the major security issue the author describes. He couldn't
have hijacked any DNS traffic this way." seems a bit dishonest. You're saying
that you have personally vetting all the DNS implementations of various DNS
resolvers and have verified all of them take the resolution steps you've
described exactly? If this is the case why did I receive so many queries (such
as A, AAAA for the NS hostnames - which I assumed/assume was to cached these
IP addresses for future resolution of the TLD's IPs). The way dig resolves
things is different from how many production resolvers would do so, etc.

I can certainly see that some resolvers may take different steps for
resolution which would make them unaffected by this issue (I'd have to think
on it some more). A big issue here is of course that I didn't actually attempt
to poison a bunch of the DNS resolvers which were hitting my server because I
didn't want to affect any actual users. "Proving the point" in this case
would've been dangerous and probably illegal as well.

That being said, mapping out how various DNS resolvers would perform their
full resolution is an interesting side project and I've added it to my TODO
list :)

------
superjisan
I need like a dumbed-down version of what went wrong here.

------
cnkk
seems to be a bad idea to get domains from strange small countries just
because they look cool.

------
jshelly
Complicated by the fact that the .io domain has recently become popular for
startups

------
runnr_az
That's amazing! Nice find, man!

~~~
andai
Trying to figure out why you were downvoted. Was it the positivity?

The exclamation marks? The lack of pretending to contribute to the
conversation?

~~~
chriswarbo
I downvoted because it doesn't raise any questions or add any information to
the discussion.

BTW, neither does commenting about votes ;)

As per
[https://news.ycombinator.com/newsguidelines.html](https://news.ycombinator.com/newsguidelines.html)

"Please resist commenting about being downvoted. It never does any good, and
it makes boring reading."

~~~
andai
I was merely puzzled that my favorite comment was downvoted to the bottom of
the thread!

I am sure the author (who appeared in the discussion here) would have
appreciated the positive feedback. Are you suggesting it would have been
better to contact the author directly?

------
felipeerias
Who would have thought that _adminstrator@nic.io_ registered in a small
overseas territory in the middle of the Indian Ocean might not be entirely
reliable?

------
redthrowaway
Considering there are a grand total of 2500 people in the BIOT, all of whom
are British or American military personnel, it _might_ not be a great idea to
route a good chunk of the world's tech traffic through them. The disparity
between how important the .io TLD is for the Internet and how few resources
must go to running it is pretty appalling.

~~~
mino
"route"? That's not how DNS works.

~~~
andai
How does DNS work?

~~~
walrus01
when one packet loves another packet very much....

