I was affected by that attack. After lunch here in Barcelona several sites started to time out on Firefox: GitHub, Twitter, Hacker News, Canva, DuckDuckGo. While others like Reddit, Google, YouTube kept working.
Found via Twitter that others had already tried changing DNS servers, and then did a tracepath and found that I couldn't reach the resolved IP's. Thought it would have been a misconfiguration of Orange. Then on Twitter (accessed via Orange mobile, which funnily worked fine -- probably a different network?) I found a thread of the people in Spain complaining about it, where someone later replied with links to the RIPE account take-over tweet.
Took about 2-4 hours for the service to be fixed. Haven't fixed any other issues so far. One of the articles pointed that it could have been due to someone that was not using 2FA, but there were no sources in that article.
For those like myself that are somewhat unfamiliar with what RIPE is:
RIPE is the European equivalent of ARIN (the North American regional authority of ISP addressing matters). They are sort of like the postal/zoning agents of the internet in that they oversee the distribution and management of IP address blocks. ISPs like Orange Spain have a RIPE account that they use manage their IP allocations, which is what was compromised.
The technical term for these is RIRs. Regional Internet Registries. There are five of them, ARIN (North America), APNIC (Asia Pacific), RIPE (Europe), AFRINIC (Africa) and LACNIC (Latin America).
The RIRs are delegated the numbers from IANA, and they in turn delegate numbers to LIRs, Local Internet Registries, such as an ISP or a maybe a large organisation which has vertically incorporated that functionality.
The RIRs represent, and are paid for by, the LIRs in their region and so to some extent they reflect local consensus, local culture, etc. This means in some sense RIPE's security policies reflect what European ISPs and similar wanted, for good or ill.
> This means in some sense RIPE's security policies reflect what European ISPs and similar wanted, for good or ill.
Traditional routing has always been very, very open. Like a club where anyone is welcome and if new folks mess stuff up it just gets fixed.
Several IRRs used to allow RPSL changes by email to their systems with only maintainer auth and no verification of route ownership (e.g. you register with the IRR, then publish Google routes).
It’s all getting hardened nowadays with things like no more email, layers of verification, RPKI, etc.
It’s not necessarily what folks want but how this stuff had traditionally been handled. Thankfully it’s all changing because of events like this.
> ... Orange Spain's RIPE password was "ripeadmin"
if it's a thing at Orange Spain and they're reusing that excellent scheme for several services (the excellent scheme being adding "admin" after the name), they're in for a world of hurt...
A company I worked for had “servername_db” as that DB server’s internal domain, then the credentials for the server were “servername_db_user” and “servername_db_password”. The argument was that it was behind a VPN and bastion server. My argument was that it was almost as easy to have a bitwarden account with 2fa and generate passwords that would never be cracked, and it would look really bad if the company was broken into and we couldn’t spend two extra seconds per login to secure it better and lose customer data.
if it's true, it's ridiculous BUT the cost is just the cost of downtime. is anyone under the impression that they'd actually be out these IP addresses permanently? obviously RIPE would fix the issue and get the IPs returned
If this really lead to an entire country losing access to GitHub, Twitter, Canva, etc for four hours, I wager that's a pretty substantial economic loss. Of course Orange doesn't suffer this cost directly, but the downtime and the surrounding revelations/rumours about their security posture might cost them a lot of business in the medium term (even outside Spain).
I mean, it didn't affect the _entire_ country. I am in Spain and didn't notice (neither did anyone I know) until I saw the news. Logically it should only affect Orange customers, maybe transiting traffic from other ISPs?
Yes and no, while RIPE can help they can only up to a point. For example, the assets are transferred to someone , up to that point RIPE can help but if another transaction has followed then sorry but they can't do much
Talking from experience in such a case, malicious actor transfers IP range from entity A to intermediate entity B and then entity B sells that range to entity C.
Entity C obtained the asset on a perfectly legal way as they don't know and care how entity B got the asset. Nothing entity A can do then when they discover it, they can't get assets back plus RIPE also can't do anything since they only offer the tool for tracking such transactions they don't take part on them . If you want follow any legal process you want to be reimbursed from entity B and the malicious actor but you are not getting your IP range back
I don't know the legal system in Spain or the wider EU, but in the US for example stolen goods are stolen goods no matter how many hands they pass through. Washing them through a fence doesn't suddenly make them legitimate
I didn't mention cost for that reason. It's just silly to put so much into assets and then not even put basic protections on them being accessed by others instead.
Maliciously changing a RIPE allocation would be like going into a pub and moving Geoff’s drink to one side so you and your mate can take his and Frank’s spot at the bar. While you may physically succeed at this, you will also find you don’t get served and you do get thrown out and barred.
I’ve always liked the professional level of collusion that makes the Internet work. One day it will all go wrong and we’ll have to set up something democratic, but until then the technocracy works well.
This is one of several attacks that can be used to generate valid TLS certificates for domains you don't own. There are mitigations but they can all be defeated. This will persist until there is reform among internet standards groups, but they are controlled by the browser market, and they aren't responsible for generating the certs, so it's one of those "nobody is responsible so it will never be fixed" deals.
The problem here appears to be that the victim didn't do even a halfway decent job of protecting their RIPE account.
RIPE offers TOTP. TOTP is hardly the best possible security, because it can be phished, but assuming the people using it are competent (and why are your networking team incompetent?) that ought to be adequate. It seems as though Orange didn't bother to enable TOTP and indeed didn't even set a decent password.
Nobody does this, but in theory dns-sec combined with a CAA record set to nothing would mitigate this in a way that can't be defeated, right?
In principle key pinning would be the counter measure, but that was so problematic it was removed from browsers. I think chrome still does static key pinning for their own domain, and many mobile apps do key pinning, so its not entirely dead, just mostly.
Potentially yes: Site owners could set a CAA record like…
nocerts.example.com CAA 0 issue ";"
… telling CAs that they shouldn't issue certs for `nocerts.example.com` at all.
That being said, it would make things difficult when it's time to do a cert renewal: You'd have to update the CAA record, wait for it to propagate, do the renewal, and then set it back. And that's a window that could be exploited.
What could work is CAA, along with RFC 8657: Certification Authority Authorization (CAA) Record Extensions for Account URI and Automatic Certificate Management Environment (ACME) Method Binding (https://datatracker.ietf.org/doc/html/rfc8657). RFC 8657 extends the CAA record to say "Only this specific CA account may request certs for this domain" and/or "Only this specific validation method may be used when requesting certs for this domain".
Well the first problem with that scenario is it doesn't work for most of the internet. It requires CAs and domain owners to use DNSSEC, and most do not. It requires strict validation of DNSSEC, and most dns resolvers do not. It requires the use of CAA records, which most domain owners do not. And it requires CAs to properly implement CAA (it was found in 2017 that many did not, and Let's Encrypt messed up 3 million certs due to CAA in 2020). So, a single person with their own domain could do all these things, but the rest of the internet won't be doing it, so most of the internet will be insecure. That's not tenable, because that means the rest of the internet can be used in attacks that appear legitimate. Even most of the top 100,000 websites on the internet don't use DNSSEC and CAA, and if you asked them to, they'd ignore you.
But the second problem is it doesn't really solve the problem. Eventually you have to renew the cert, and when that happens, an attacker can spoof the IP of the validating source and have a valid cert issued. CA validation itself is flawed by design.
The title was confusing at first, with "was breaking into" I thought the attacker didn't succeed, but they did. Can the title be changed to something like "Someone hacked Orange Spain's RIPE account and broke their /12"?
We were affected by the downtime, suddenly our VPNs went down and we started troubleshooting if it was a switch sudden reboot, a physical problem, anything. Odd for some of our VPNs to randomly shut down rejecting all traffic but just at the endpoint. Then we started having some very specific problems with just some FTP addresses and couldn't figure out what was happening.
Until our CIO went to the bathroom and came back saying "Hey, these websites aren't working from our WIFI network but they're working fine from mobile data? Could it be a navigation problem from Orange?"
Some years back, Russia hijacked a BGP belonging to a major transit provider. For a few hours, international traffic was rerouted thru Russian networks where it could be cloned (like the NSA does in the states) and examined.
They could have been after something specific in the traffic but my unqualified guess is that it was a test or they were showing off.
> Surely OpenNet/ClassNet/NIPRNet/SIPRNet/GWAN/JWICS aren't accepting BGP updates, where they even touch public network infrastructure.
If i understand, the article is not so much a typical bgp hijack but the attacker gaining control of their account that controls routing. Seems like private networks would be just as vulnerable to that.
As they stated, European gov networks (plus military and universities OFC) had and still have their private backbone. In Spain, RedIris was for scientific research, universities and such. IDK about the rest, but for sure they'd have a similar approach.
Generally yes, but there is a slight mitigating factor that certificate transparency makes it very public who you are attacking.
In theory, if it was just your site ip that was attacked and not your dns provider, CAA record can protect you. Probably one of the few cases where dns-sec is actually useful. All that though is an edge case that probably doesn't apply 99% of the time.
Hijacking the IP would indeed allow you to respond to ACME HTTP-01 validations. That can be protected against by using RFC 8657: Certification Authority Authorization (CAA) Record Extensions for Account URI and Automatic Certificate Management Environment (ACME) Method Binding (https://datatracker.ietf.org/doc/html/rfc8657). RFC 8657 extends the CAA record to say "Only this specific CA account may request certs for this domain" and/or "Only this specific validation method may be used when requesting certs for this domain".
Well, the new certificate shows up, but so what? CT logs aren't like "Barry Shitpeas got a certificate for hugecorp.example" they just tell you what was issued, not why, not who in any useful sense was issued it, and they take up to 24 hours to do that.
What is the overlap between the set of people who think "pass1234" is a good password and the set of people who have great oversight of their cert issuance and would flag unexpected issuances ? I'd expect it to be approximately empty.
> … they just tell you what was issued, not why, not who in any useful sense was issued it …
They give you the certificate, which contains the issuer's CPS. If it's an issuer that you (the domain owner) don't recognize, you have at least a starting point for reaching out.
> … and they take up to 24 hours to do that.
Indeed, the Maximum Merge Delay is 24 hours. But in practice, by monitoring multiple CT logs, it takes just a minute from successful issuance to having the certificate show up in at least one CT log (see https://utcc.utoronto.ca/~cks/space/blog/web/WebProbeSpeedNe...).
Found via Twitter that others had already tried changing DNS servers, and then did a tracepath and found that I couldn't reach the resolved IP's. Thought it would have been a misconfiguration of Orange. Then on Twitter (accessed via Orange mobile, which funnily worked fine -- probably a different network?) I found a thread of the people in Spain complaining about it, where someone later replied with links to the RIPE account take-over tweet.
Took about 2-4 hours for the service to be fixed. Haven't fixed any other issues so far. One of the articles pointed that it could have been due to someone that was not using 2FA, but there were no sources in that article.
EDIT: the article mentioned above https://bandaancha.eu/articulos/secuestran-cuenta-ripe-orang...