
Internet traffic for Google, Facebook, Apple was briefly rerouted to Russia - couchand
https://bgpmon.net/popular-destinations-rerouted-to-russia/
======
gtrubetskoy
This is a well known "feature" (because it is not a bug) of BGP and every ISP
worth their salt ought to be doing route filtering to prevent this sort of
thing. If they're not, then the traffic can be diverted to Russia or the
proverbial bit bucket rather trivially.

I understand that less network savvy readers think this story is about Russia,
but it really is about ISP's and the old issue of lazy network admins who do
not filter routes and what happens. You can replace "Russia" with "Canada" and
it wouldn't change the meaning. The country attribution is based on the AS
(Autonomous System) registration. If these were really the malicious state
actors, they'd probably use an AS registered in another country to cover their
tracks. This is either an honest mistake or possibly a "let's see what would
happen" prank.

~~~
OldSchoolJohnny
I understand ISP's are to blame here, but how is it _not_ about Russia, the
article specifically states that only certain entities were targeted in a
suspicious manner. Seems like nefarious purposes at work here no?

~~~
gtrubetskoy
> how is it not about Russia

It could be, the key missing piece of information is the physical location of
the actual router that is the ultimate source of this. It is only known by
their peers or upstream and is not easily traceable (in the US it's usually
under NDA and you'd need court order to find out "officially"). Some network
admin somewhere probably knows. May be he/she reads HN. The second question is
whether that device is hacked and is doing it unbeknown to its admin.

But without this information, the fact that this is an AS registered with a
bogus .ru domain doesn't mean anything, this could be coming from
Liechtenstein or Bolivia.

------
riffic
The open trusting nature of BGP is kind of problematic isn’t it?

[https://en.wikipedia.org/wiki/BGP_hijacking](https://en.wikipedia.org/wiki/BGP_hijacking)

~~~
maltalex
Yes, but to be honest, it's not just BGP. One of the things that BGPMon won't
tell you about since they're in the business of monitoring BGP is that the
same attack is often performed by changing forwarding tables i.e. in network's
data plane.

An attacker compromises a router (there are known vulnerabilities and back-
doors) or gains leverage over a technician authorized to configure routers in
some network. The attacker then manually changes the router's local routing
table to forward packets to specific destinations through a different path. In
such cases, the BGP path and announcements look absolutely fine and you'd need
to look at the actual path at the data plane level to detect that something is
wrong.

This is often done in big exchange points where many networks interconnect.
These are places in which routers from major western countries can sit
basically side-by-side with routers from Russia and China.

This isn't a paranoid fantasy by the way, I work for a company that monitors
these kinds of attacks for a living.

~~~
skj
> This isn't a paranoid fantasy by the way, I work for a company that monitors
> these kinds of attacks for a living.

Then you'll certainly be familiar with some citation or documentation of
evidence? I'm curious to see some.

~~~
maltalex
I've asked around the company and unfortunately I can't talk about attacks
(and/or suspicious "configuration errors") we've detected. There are legal
issues involved, plus some of these cases are currently being investigated by
law enforcement. Moreover, I don't know of any other company or person that
deals with such attacks, so I have no one to quote. The only thing I can do is
to offer some hand waving:

1\. It's known publicly that nation states as well as criminal organizations
are deflecting Internet routes. There are numerous reports of such cases;
these cases aren't that hard to find since BGP information is (mostly) public.
A few published examples:

\- Russian network "Rostelecom" hijacks sites of financial services:
[https://arstechnica.com/information-
technology/2017/04/russi...](https://arstechnica.com/information-
technology/2017/04/russian-controlled-telecom-hijacks-financial-services-
internet-traffic/)

\- Global Large scale BGP hijacks in 2013, some through "Rostelecom":
[https://arstechnica.com/information-
technology/2013/11/repea...](https://arstechnica.com/information-
technology/2013/11/repeated-attacks-hijack-huge-chunks-of-internet-traffic-
researchers-warn/)

\- Spammers use BGP to announce fake IP addresses to spam yahoo mail users:
[https://ripe72.ripe.net/presentations/45-Invisible_Hijacking...](https://ripe72.ripe.net/presentations/45-Invisible_Hijacking.pdf)

\- Traffic to UK organization that deals with nuclear weapons hijacked using
BGP to the Ukraine: [http://hub.dyn.com/dyn-research/uk-traffic-diverted-
through-...](http://hub.dyn.com/dyn-research/uk-traffic-diverted-through-
ukraine)

\- BGP hijack of bitcoin miners: [https://bgpmon.net/the-canadian-bitcoin-
hijack/](https://bgpmon.net/the-canadian-bitcoin-hijack/)

\- "Hacking Team" (Italian company selling spyware to law enforcement) helps
Italian police perform BGP hijack: [https://arstechnica.com/information-
technology/2015/07/hacki...](https://arstechnica.com/information-
technology/2015/07/hacking-team-orchestrated-brazen-bgp-hack-to-hijack-ips-it-
didnt-own/)

\- Chinese hijack of US military an governmental networks (2010):
[http://www.theregister.co.uk/2010/11/17/bgp_hijacking_report...](http://www.theregister.co.uk/2010/11/17/bgp_hijacking_report/)

2\. There are known attacks against routing protocols which aren't BGP (e.g.
OSPF, Black Hat 2011) and against routers (see CVEs for CISCO IOS).

3\. Routing changes that happen within an autonomous system leave a much
smaller footprint compared to BGP.

4\. Nation states are known to have an interest in direct access into routers:

\- Suspected NSA example:
[http://www.theregister.co.uk/2015/09/15/compromised_cisco_ro...](http://www.theregister.co.uk/2015/09/15/compromised_cisco_routers/)

\- Suspected Chinese example:
[http://www.abovetopsecret.com/forum/thread350381/pg1](http://www.abovetopsecret.com/forum/thread350381/pg1)

5\. Network connect to each other (mainly) in central exchange points. In
these exchanges networks can pay for a direct point-to-point connection from
network to network, or use open peering through a layer 2 switch that
aggregates connections from dozens or hundreds of different networks at once
(Example: [https://www.de-cix.net/en/de-cix-service-
world/globepeer](https://www.de-cix.net/en/de-cix-service-world/globepeer)).
Hundreds of networks use these switches with very little visibility. Everyone
can talk to everyone else on the same switch without leaving a trace. If a
router connected to such a switch is locally configured to forward traffic to
some other router on the same switch, it can do so regardless of BGP routing
or any common sense.

------
vectorEQ
'normally' it gets routed to US (even if they have servers elsewhere to handle
same shit...), which for most countries is actually more worrying...

~~~
futurix
If you believe that Russia is less worrying than US, then... err... I don't
believe you exist actually.

~~~
pixl97
Well, if you are a security researcher, Russia is less apt to redirect your
flight mid ocean and arrest you in a country you weren't even going to. So I
guess the US _just might_ be more worrying to particular people based upon
their occupation.

~~~
kbhn
Security researcher is a pretty loose and fear mongering title to describe
Snowden's actual role - Taking sensitive government secrets to a hostile
foreign power in the name of whistleblowing.

But please, don't let my clarification get in the way of your whataboutism.

~~~
otakucode
'to a hostile foreign power'? You mean the public? The US government is why
Snowden is in Russia, not Snowden. He didn't go to Russia. His flight was
forced to land there when the US revoked his passport mid-flight.

~~~
kbhn
Yes, to a hostile foreign power. The public isn't a place you can reside.
Russia is, however.

> The US government is why Snowden is in Russia, not Snowden

The US government is not why Snowden is in Russia, Snowden is why Snowden is
in Russia. He _went_ to Russia intentionally. He wasn't kidnapped, he chose
Russia as a destination specifically due to their reluctance to cooperate with
the US.

> He didn't go to Russia. His flight was forced to land there...

That's not what happened, please reread his flight history because the facts
are well known at this point. He told his superiors at the NSA that he was
headed to the mainland USA for medical treatment, but instead flew direct to
Hong Kong where he lived for over a month. 30+ days after arriving in HK he
fled to Russia, with the intent of fleeing to Cuba the next day.

> ...when the US revoked his passport mid-flight.

His flight wasn't forced to land mid-flight because Snowden never even boarded
the plane that day. His passport was revoked before the Cuba trip, leaving him
stranded in the Russian airport.

If you're going to shill for someone who is unwilling to come back to the US
to be held accountable (rightly or wrongly) for his actions, please be
factually correct about it.

~~~
boomboomsubban
>The US government is not why Snowden is in Russia, Snowden is why Snowden is
in Russia. He went to Russia intentionally. He wasn't kidnapped, he chose
Russia as a destination specifically due to their reluctance to cooperate with
the US.

He chose Russia as a layover location, Hong Kong has a limited number of
outgoing flights and that likely was the one the US would be least able to
grab him. However, the US is why he is in Russia now. You mention that he had
a flight to Cuba booked, and his ultimate goal was Ecuador.

There is no proof that he took "state secrets" to Russia. By his account he
had destroyed any of his remaining copies before leaving Hong Kong. This is a
far more egregious error than confusing a layover stop for being forced to
land.

------
partycoder
In 1998, a hacker collective known as the L0pht testified in Congress that
they could take down the Internet in 30 minutes.

It is believed BGP black holes could be one way of achieving this.

Basically tell every node "I've got the lowest routing cost to every node,
send me all your traffic", and then drop all the traffic.

~~~
aaomidi
This is much better nowadays but bgp is still problematic.

~~~
maltalex
It's not really better today. Possibly even worse.

In 1998 there were fewer than 5000 different networks (Autonomous systems) on
the Internet. Today that number is over 80K. Network operators used to
personally know each other and their clients.

So the automatic filtering is better today than it was in 1998, but it can
still be easily bypassed.

I know for a fact that even as a tiny network, all you need to do to get the
some of the biggest networks in the world to accept your (real _or fake_ ) BGP
announcement is to email their support center.

~~~
dredmorbius
Aren't ASNs capped at 2^15 (65k and change)? Or has the IPv6 expansion already
started happening?

~~~
dsp1234
ASN's switched over to 32 bits around 2007

~~~
dredmorbius
Thanks.

------
baxtr
Could someone explain in laymen terms what happened and what that means?

~~~
ag_47
Basically, a router somewhere in Russia claimed to be the owner of some ip
addresses belonging to Google, Facebook, etc. Other neighbouring routers began
forwarding packets from actual users to this router. The packets contain the
HTTPS requests people were making to these sites.

~~~
VVayneTracker
Bit late to the party but what could be done with the information that they
got their hands on?

------
emmelaich
If this is not a reason to punish (blackhole?) that AS I don't know what is.

~~~
maltalex
Sure, it's a reason to blackhole the isp, but this is the Internet we're
talking about. So say its upstream providers will backhole it, say even the
tier 1 providers do the same.

That won't stop it from connecting to smaller networks and announcing the same
fake routes there. And even if the small networks blackhole it, the attackers
can just register a new AS.

Also, there are cases in which attackers are performed from large networks,
with millions of users. Are you going to blackhole such a network? They'll
apologize and claim that it was a configuration error.

Note that these types of attacks are generally not performed by amateurs.
They're performed by states, intelligence agencies, criminal organizations and
the like.

------
yeukhon
Is it fair to say BGP in general relies on “good will” trust model? I wonder
if there exists an authoritative mechanism similar to browser punishing rouge
or incompetent CA. I supposed the integrity and the authentication is achieved
by implementing and deploying RPKI, but what about punishment?

~~~
maltalex
> Is it fair to say BGP in general relies on “good will” trust model?

Yes.

> I wonder if there exists an authoritative mechanism similar to browser
> punishing rouge or incompetent CA

Not really. Although if you get a reputation for being notoriously
problematic, the "big boys" \- the large networks who are the responsible
adults in this game might deny you service or stop accepting your
announcements.

> I supposed the integrity and the authentication is achieved by implementing
> and deploying RPKI

RPKI doesn't really solve the issue. It validates only the initial announcer,
so BGP hijacks can still occur. It would however help reduce hijacks due to
mistakes in configuration.

------
alex_g
What is a prefix (in context), and what does it mean to ‘announce’ one?

~~~
jlgaddis
A prefix is an IP subnet, basically.

Most here are familiar with Google's public DNS resolver, 8.8.8.8, so let's
use that as an example.

Google has been assigned the autonomous system number (ASN) 15169. The
8.0.0.0/8 IP address space was allocated to Level3 and they have reallocated a
small part of it, 8.8.8.0/24, to Google.

Google, when speaking to their BGP peers, "announces" routes, including an
announcement for the 8.8.8.0/24 prefix.

An announcement is basically Google's router telling, for example, Level3's
router, "Hey, I know how to get to 8.8.8.0/24\. If you have traffic going
there, you can send it to me."

Level3 then passes that along to other peers, like me. Level3 says, "If you
have traffic for 8.8.8.0/24, you can send it to me and I'll send it on towards
its origin". The announcements continue to be passed along to other peers.

A withdrawal is the opposite of an announcement: "Hey, I don't have a route to
8.8.8.0/24 anymore" or, in many cases, "the route to 8.8.8.0/24 has changed,
here's the new one".

Here's a (slimmed down) example from one of my routers:

    
    
      # sh ip bgp 8.8.8.0/24 | b 3356
        3356 15169, (received & used)
          4.69.248.15 from 4.69.248.15 (4.69.180.167)
            Origin IGP, metric 0, localpref 100, valid, external, best
            Community: 3356:3 3356:86 3356:575 3356:666 3356:2042
    

This shows the "AS path" (the path through the various ASNs back to the
origin), "3356 15169". AS3356 is Level3. We can see that AS15169 (Google)
originated the prefix (8.8.8.0/24), and announced it to their BGP peer,
Level3. Level3 then passed that announcement along to my router.

Here's the same thing for 66.232.224.0/24, mentioned in the article:

    
    
      # sh ip bgp 66.232.224.0/24
      BGP routing table entry for 66.232.224.0/24, version 1125190207
      Paths: (1 available, best #1, table Default-IP-Routing-Table)
        Advertised to update-groups:
              4
        3356 209 40839, (received & used)
          4.69.248.15 from 4.69.248.15 (4.69.180.167)
            Origin IGP, metric 0, localpref 100, valid, external, best
            Community: 209:209 209:13070 3356:3 3356:22 3356:86 3356:575 3356:666 3356:2042
    

The AS path here is "3356 209 40839" which shows that traffic from me to
66.232.224.0/24 will first go to Level3 (AS3356), then to Qwest/CenturyLink
(AS209), and finally to AS40839 (Kohl's Department Stores).

With regard to the hijack described in the article, this means that another
organization announced the prefixes into BGP, effectively saying, "So, all of
that traffic you have that's going to Apple/Facebook/Google/etc., just go
ahead and start sending that to AS39523 now". Instead of ending up at
Apple/Facebook/Google, it'll instead be redirected to this unknown
organization in Russia.

(N.B.: Before anyone chimes in to "correct" something, this is a bit
simplified. There's much more to BGP than this and there are a number of
different factors which determine which route is chosen. A longer prefix --
mentioned in the article -- is one way to "defeat" a better route.)

~~~
digi_owl
One curious thing about 8.8.8.8 (and 8.8.4.4) is that it gets redirected to
the closest Google CDN.

~~~
jsjohnst
> One curious thing

DNS is something that’s commonly Anycasted. In laymen’s terms, basically what
that means is rather than 8.8.8.8 pointing to one server (or a load balanced
cluster in one DC) as you might expect, it rather points to potentially
thousands of servers all over the globe and the closest one (BGP route wise)
is the one picked for you.

~~~
sp332
8.8.8.8 is already an IP address so DNS is not involved. The anycast is
handled by just BGP.

~~~
jsjohnst
Kelnos was correct, the service that 8.8.8.8 provides is DNS, hence the
relevance when talking about 8.8.8.8 being anycasted.

~~~
sp332
Man, I must have been asleep this afternoon because I could not figure out
what a bare IP address had to do with DNS. I get it now.

------
bandrami
I've often said it's just as well that the public doesn't understand how BGP
actually works.

~~~
maltalex
It's not just the public, it's professionals too.

There are probably more brain surgeons on earth than people who understand how
BGP works.

------
bawana
basically, a more specific routing address is used to direct packets through a
specific route. In the earlier instance( the second example) Japan was the
target. It is easy to see how the routing system in japan could have been
crashed through such an overload (basically a ddos). Who would do that to
Japan? China? Korea?

In the more recent instance, all the traffic from google, facebook, apple, and
other major players was routed through russia. Is this china making russia
look evil? after all, if the traffic is going through russia, all russia has
to do is turn off the power for that router to momentarily stop all traffic.

or is this the NSA trying to #uck w russia and at the same time trying to
create an incident so the fcc kills net neutrality to provide 'security'.

This is state of the art espionage

------
codedokode
This is not the first time when BGP is misused this way. I think governments
should regularly perform live tests to check whether their countries' AS's can
be hijacked.

~~~
maltalex
> I think governments should regularly perform live tests to check whether
> their countries' AS's can be hijacked.

That's a waste of money and effort. They should just write "yes" on a piece of
paper, there's really no reason to check.

------
baybal2
That AS was registered to dv-hyperlink.ru which is not in whois database

Looks suspicious. How did they register an AS to a non-existing company to
begin with?

~~~
topranks
It was assigned by RIPE to "Vasilyev Ivan Ivanovich, 8 Pionerskaya st., office
10, Nekrasovka, Khabarovsk region Russia".

ALFA TELECOM s.r.o., in the Czech Republic seem to have acted as sponsoring
LIR for the allocation.

RIPE rules require that the sponsoring LIR submit documents to them showing
the legal standing of the organisation that will use the resources. So in
theory that should have happened here.

------
ryanmarsh
I always wonder how Russian people, either Russian Americans or Russian
citizens, feel about all this Russia hysteria.

~~~
nasredin
Putin's approval rating is in the 80s.

That should be enough to tell you what they think of you without looking at
polls.

(IIRC they were trying to shut down the one independent poll firm recently)

------
lexxed
Isn't that what https is for ? So no one can read the traffic ?

~~~
zaarn
Even with encrypted traffic, an attacker can learn things, especially if you
get DNS and SNI data

I suspect (but cannot prove) that this might have been a leak from Russia's
internal internet into the wider global net. Since Russia isn't well known for
it's privacy respecting nature, it might have been a traffic scanner to see if
people are being good citizens. However, that is just speculation and I hope
it's wrong.

~~~
cwilkes
Yep, all you need to know is the IP addresses of certain domains (say
Facebook) and then look for user IP transferring a lot of data to it meaning
they are probably uploading a photo. Now tie that IP to an ISP and maybe a
user and you can find out who might be posting derogatory memes about Putin.

~~~
codedokode
But you don't have to change routing for that - you can do that with just
passive monitoring. And by the way, the law [1] that requires ISPs to store up
to 6 months worth of traffic is coming into effect next year. So even
monitoring won't be necessary.

Maybe they were testing effective ways to block foreign sites?

[1]
[https://en.wikipedia.org/wiki/Yarovaya_law](https://en.wikipedia.org/wiki/Yarovaya_law)

------
manugarri
hopefully Comcast and AT&T will fix this soon

------
euske
Someone should make a blockchain-based routing system.

~~~
cat199
It's called UUCP.

Transactions take 3-5 days to process, are sometimes lost, but hey, it's the
next great thing, so worth every expensive penny.

