Hacker News new | past | comments | ask | show | jobs | submit login
AT&T updates firmware to block access to (dslreports.com)
999 points by antoinefink on May 2, 2018 | hide | past | favorite | 367 comments

I'd say there is a 98% chance this is a bug in some firmware and a 2% chance AT&T is intentionally trying to block Cloudflare DNS.

I get why people are paranoid about ISPs blocking content and net neutrality, but let's not cry wolf prematurely. The technical details here strongly suggest a bug rather than intentional blocking of DNS traffic.

Then the odds appear to not be in our favor.

CF CEO tweets that is also blocked.


Others have confirmed that the ipv6 address belonging to CF appears to be blocked.

Do you have a reference for the ipv6 address being blocked? That would be a much bigger smoking gun


> For IPv6, we have chosen 2606:4700:4700::1111 and 2606:4700:4700::1001 for our service. It’s not as easy to get cool IPv6 addresses; however, we’ve picked an address that only uses digits.

For me up in Canada, ping works. But

    ping6 2606:4700:4700::1111
    ping6 2606:4700:4700::1001
shows "connect: Network is unreachable". Am I using ping6 wrong?

We also need to confirm IPV6 works outside AT&T's network.

Edit: Just tried Google's DNS. works, but their IPv6 doesn't, so I guess this was a bad test.

Edit2: Learned about nslookup, but it does not seem to work with either Google or CloudFlare's DNS.

    nslookup reddit.com                       # Works
    nslookup reddit.com               # Works
    nslookup reddit.com               # Works
    nslookup reddit.com 2606:4700:4700::1111  # Does not
    nslookup reddit.com               # Works
    nslookup reddit.com 2001:4860:4860::8888         # Does not
    nslookup reddit.com 2001:4860:4860:0:0:0:0:8888  # Does not
Edit3: Apparently my ISP doesn't support IPv6 yet.

You're using the IPV6 address correctly, does https://test-ipv6.com report everything's dandy for you? If it does maybe they're blocking traffic or there's something else going on.

> No IPv6 address detected. Connections to IPv6-only sites are timing out. Any web site that is IPv6 only, will appear to be down to you.

Okay, guess my PC/LAN/ISP doesn't support IPv6 yet.

If you're in Ontario, Rogers doesn't support IPv6 yet. If you want IPv6, then your only option is Bell (or a reseller, like Teksavvy).

I'm using Bell in Ontario. It could be either my Router doesn't support it, the Apartment isn't wired up to support it (if that's required?), my ISP doesn't support it in my area, or my Bell internet plan doesn't cover IPv6...

I'll ask them about it when they ring me up next time asking for more money.

Hmm... looked at this again and it looks like Rogers may have rolled out IPv6 last year.

I recall on Teksavvy I had to pay extra for a "static IP" to get IPv6. Not sure if you're with Bell directly, though.

Everyone in Ontario on TekSavvy should have IPv6 now without having to pay for a static IP address but at least for me it's still wonky at best.

I tested out both addresses via my phone's web browser just now.

Connecting to WiFi (Time Warner), I got a 403 from cloudflare (presumably there just isn't a web server set up on that address).

Using mobile data (AT&T), I got ERR_ADDRESS_UNREACHABLE. However, actually works on AT&T cellular, so I'm not sure what to think.

most modern linux distros regular `ping` will work for ipv6.

(US based) frontier, vz, and spectrum all can ping that ipv6 address (though all have way over 10ms latency)

fwiw, I am an AT&T customer in Atlanta on their fiber service.

the nslookup reddit.com does not return for me, if I connect to work via VPN it does. and do work without VPN. while the AT&T modem shows IPV6 I did not test.

System Information Type Value Manufacturer Pace Plc Model 5268AC

ipv6 supports traceroute too

Missing brackets probably:

  ping6 '[2606:4700:4700::1111]'

> Am I using ping6 wrong?

I'm pretty sure the other end has to be running `pingd` to get a response from ping. Some do, some don't.

I might be wrong but that's always been my understanding.

You are definitely wrong. No daemons have to be running, ping operates using standard ICMP echo messages that are a part of any complete IP stack. Any meaningful OS will respond to pings unless prevented from receiving them by a firewall. It wouldn't surprise me to find that some embedded implementations skip that part for size reasons, but even in that category most devices I have available to me still respond. It's a basic network connectivity diagnostic tool.

What is unfortunately common though is people blocking ICMP at their firewall, either at the host level itself or further upstream. Sometimes they just block echo requests, but often they block ICMP entirely which breaks things in very weird ways from time to time.

Blocking ICMP in any way is generally to be considered harmful. It's not 1997 anymore, the "ping of death" is not a thing on any OS you should actually be connecting to the internet.

I have AT&T internet, and the BGW-210 gateway with the latest firmware. And my area was upgraded to native dual stack ipv6 about a year ago. So I tested it out and the ipv6 CloudFlare DNS (2606:4700:4700::1111 , 2606:4700:4700::1001) works perfectly fine. https://imgur.com/a/grUzeDD Its only the ipv4 that dose not. And AT&T made a statement why that is.

""With the recent launch of Cloudflare's DNS service, we have discovered an unintentional gateway IP address conflict with 1 of their 4 usable IPs and are working to resolve the issue,"


A few of you will be disappointed to know its not a evil attempt to block you from using it. Same way they have literally never blocked the ability to use any other DNS service before.It's simply a bug caused by the way the BGW-210, and Pace 5268AC operate and make use of internally in some way and it will be fixed with a firmware update.

Just curious - can cloudflare blackhole all of Att traffic?

Could they physically? Yes. But they'd be screwing over their own customers who rely on that traffic.

Isn’t AT&T screwing their own customers by blocking as well ?

AT&T isn’t blocking, just tested it on my uverse connection. As much as I hate AT&T their internet is pretty solid with the exception of datacaps

A more interesting use case though it would have its dangers is them showing a message to AT&T users that their ISP is doing things to damage the internet and that they should call and complain. People got mad at the idea of CloudFlare slowing down network requests by FCC members in protest of their shenanigans.


Not sure how that makes any sense whatsoever...?

Shameless plug. was working for me on AT&T after Cloudflare released, then shortly after that it ceased working.

Maybe the firmware update has a bug, but it's very suspiciously timed. Notice that the OP is dated April 2, while was released April 1.

This is what happened to me as well. It worked for a day or so and then stopped.

I have ATT U-verse internet service and use their Arris BGW210-700 gateway

One interesting thing is that if I go to the gateway management page, and use their diagnostic tools, I'm able to ping / traceroute the address - but I can't from any devices connected to the gateway

From gateway diag page:

PING ( 56 data bytes 64 bytes from seq=0 ttl=64 time=0.568 ms 64 bytes from seq=1 ttl=64 time=0.156 ms 64 bytes from seq=2 ttl=64 time=0.164 ms 64 bytes from seq=3 ttl=64 time=0.144 ms

--- ping statistics --- 4 packets transmitted, 4 packets received, 0% packet loss round-trip min/avg/max = 0.144/0.258/0.568 ms

traceroute to (, 30 hops max, 38 byte packets 1 1dot1dot1dot1.cloudflare-dns.com ( 0.285 ms 0.177 ms 0.090 ms

The times on the pings make it look like its hitting a loopback address instead. Pings to from the diagnostics page take about 23 ms. No way is completing in under 1ms haha

Yes, is in use on your Arris device, the same issue with the 5268AC since day one.

A possible explanation is that the traffic from active use of caused some backend service to get overloaded with traffic due to a faulty assumption that the address would never be used by customers. Anyone keep traceroutes while before the patch to see if there were errant stops or delays?

They had the choice of "fix the whole backend" or "block 1.x on the user end".

Guess we know which one was easier. If all this wild speculation is true, maybe they're working on a fix to the root cause and will roll back the patch when complete.

This would make the situation both due to incompetence and intentional. is well known (based on the announcement from cloudflare anyway) to have tons of random traffic. That's part of the reason it wasn't implemented by others as a valid address for anything. Could the fact that they're simply allowing traffic at that address cause additional stress on AT&T's network?

I ask because I don't know. I figure any traffic headed that direction would go anyway it just wouldn't get routed very far with no valid destination.

Yeah. And there's also a lot of traffic going in Facebook's direction, for example. Hey, let's blackhole that too - and alleviate the stress on our network that comes from people using it. (In non-sarcastic tone: that doesn't make any sense.)

Based on what I understand, the amount of traffic headed to is much more significant. I agree with you though, that wouldn’t be justification to block it. It looks like they’re also blocking and the relevant ipv6 addresses which shouldn’t have the same traffic issue.

I doubt it's all that significant, it's a really small portion of traffic compared to a web page, javascript, css or images... and with caching even less of an impact.

The problem isn’t DNS traffic. The problem is that for years people have been using in the configuration of software and devices when they didn’t have an up address to configure. The result is that when becomes routable all that additional traffic flows there and AT&T along with other provides carries that traffic. I was wrong that AT&T was blocking it for honorable reasons but this is a still a significant amount of traffic.

If they were so determined to block it, why would they do it in firmware and not upstream? I think people are reading too much into this.

It's cheaper to do it on the mobile?

I was using with AT&T Fiber and it stopped working. I didn't really question it, I figured maybe something went down at Cloudflare so I just switched my Mac back to using the defaults again. It never even occurred to me that AT&T might be blocking it.

Maybe stupid question, but why would AT&T block it?

A few others have mentioned this already, but has become a colloquial private address, used either as a blackhole or as a destination for internal traffic. Sort of like how 555-5555 technically isn't reserved (only 555-01xx is, according to Wikipedia), but practically, it's not really a workable number and phone companies don't hand it out.

According to the announcement post, part of the reason that Cloudflare was allocated the address is that they were ready and willing to handle the expected inundation of all kinds of bizarre traffic.

It seems that one of those "off-label" uses of is an internal / network control interface on [some?] AT&T networks. I'm just speculating, but it's definitely possible that suddenly becoming publicly routable and pointed to a real thing caused some problems. "Patch it out" may be an acceptable emergency response depending on the breakages, but not really acceptable long-term.

Not an acceptable thing to do silently though, in any term.

and the reports of

Same thing happened to me using at&t fiber.

They want you using their DNS for traffic snooping?

Pretty sure they don't block though.

They can snoop on your DNS anyways.

Not with DNS over TLS. EDIT: Which CF supports.

So does Google DNS (using DNS-over-HTTPS), yet they haven't been blocked.

You're absolutely right about this. This is almost certainly just there to block people who mistakenly paste in an example configuration somewhere.

Back in 2010 there were problems that came up when IANA started allocating out of (e.g. [1]). Things that were once assumed to be unused started being used, leading to strange issues.

Also, why on earth would AT&T block and not Google DNS and OpenDNS?

[1] https://bgpmon.net/issues-with-allocating-from-1-0-0-08/

According to the thread the timing on this looks pretty bad since those DNS IPs were previously working on the earlier firmware.

How would it make sense to block it only on a small fraction of their entire network? It wouldn't accomplish anything.

when was first announced a few weeks ago, many people pointed out at the time that it was already blocked because so many people had effectively polluted it by over-using it for demo examples and testing traffic. CF announced they knew this and intended to do a project analyzing the data. Perhaps this done, whether conveniently or not, with the same intention. We'll see if they reverse it.

Can someone link to the firmware? It shouldn't be hard to binwalk this and figure out wtf is happening.

Also- If this was intentional- I'm betting they'd filter it for the mobile network as well. This has got to be a fuck-up.

Having it seem like a bug would be an effective way to block it intentionally. The timing of such an unusual regression is suspicious. The fact that is also blocked is also suspicious.

A conspiratorial Hanlon's corollary: The most effective malice is that which can be ascribed to incompetence.

Have you seen the Underhanded C Contest? http://www.underhanded-c.org/_page_id_5.html

"Bugs are worth more points if, once discovered, they are plausibly deniable as an innocent programming error."

Genuinely brilliant.

> Having it seem like a bug would be an effective way to block it intentionally.

Just like how only the true messiah denies his divinity, it doesn't give innocent bugs much of a chance.

In fact, now we can show that all bugs are suspicious, with apologies to the interesting number paradox:

The least intentional looking bug is the most effectively hidden, and therefore should probably be suspected of being intentional. Since it's now suspect, it's longer the least intentional looking bug, so the next least suspicious bug suddenly deserves a bit more scrutiny, and so on.

That's not my point. It's suspicious because of its specificity and timing; its bug-like presentation isn't evidence either way.

This is an unrelated yet related question. I am trying to access apple support, I use at&t. When I go to support.apple.com I get an error message stating : Access Denied. You do not have permission to access "http://support.apple.com" on this server. And gives me a long reference hash. This is at&t denying me access?

A university I went to used for its WiFi loginpage

I thiught thats what early cisco (after they bought air-something) used c. 2005 or so

It's either malicious or a major fuck up. Either way it's worth shouting about.

It's the latter. Here is the CEO of Cloudflare tweeting about it: https://twitter.com/eastdakota/status/991718955021623296


Where do you see that this was a mistake? That tweet thread doesn't have anything definitive on it.

Blocking -> 98% chance it is a bug

Blocking and -> what are the odds here?

everything from to would encompass those IPs so who knows what but my guess would be some routing or other strange internal usage of some of those subnets

Only one may be coincidental. Two is enemy action.

No. Block due to internal use. That block includes both of those addresses.

the problem with this sentiment is what "one" and "two" imply. it's possible to refer/block/whatever multiple addresses with "one" action.

Anyone work at AT&T who could give us the inside scoop on these firmware changes? Snapping a photo of the blocking code would be a valuable public service.

Remember to scrub EXIF data!

More than likely:

- If the action was malicious, the people involved in writing this code are likely okay with it and not likely to leak details of it.

- If the issue is a bug, the people involved in writing this code are probably working to fix it, and not likely to leak details of it.

- People not involved with making it would likely leave an internal access trail (independent of EXIF data) when they access that code.

Which is to say, expecting an Ed Snowden every time a company does something unethical is kinda silly, otherwise we'd have Google's search algorithm by now.

1. Probably.

2. True.

3. Unlikely; it's likely in a big repo that's synched all at once.

Alternatively, we can just obtain the firmware from a device and diff it against the last-known-working version, to see how the routing is failing.

My router stopped working a few weeks after using Weird things are happening with it.

I really hope that's the case, was allocated before CF acquired it? Was also blocked? was reserved by APNIC for research use.

Lots of cisco example config use for router internal identifier / DHCP server / OSPF dummy network .

Not suprised if it break anything.

I didn’t know this and it sure seems to corroborate what others suggested about it being intentional, but for completely different reasons.

Makes sense. Thanks!

What's that saying about not attributing to malice, what is more easily explained as stupidity or incompetence or whatever? (Occom's Razon and all that).

AT&T routers also don't let you use a 10.x address at home (possibly to prepare for carrier grade NAT, although there is an official 100.x address reserved for that; so fuck you ATT).

I'm so sick of my AT&T router/modem for various other reasons. I hate how you are required to use it for many of their offerings (including Fiber to the home).

There are a number of tools out there for putting their router behind your Linux box. Most of them configure ebtables or use scripts to forward the 802.1q authentication packets to/from the router.

> What's that saying about not attributing to malice, what is more easily explained as stupidity or incompetence or whatever?

Hanlon's razor: https://en.wikipedia.org/wiki/Hanlon%27s_razor

Wouldn't it be possible to use your own router and treat the AT&T router essentially like a modem? I ask because I'm about to move to an address that can get AT&T fiber.

Sort-of. It has a DMZPlus mode, but all it does is assign the public IP to an specific internal device and uses NAT, as well as forwarding all ports, to make it look like that device is onthe public Internet (even though the modem has the same public IP). You can still plug in other devices and they get private IPv4s or parts of your IPv6 prefix and it NATs (the IPv4) those as well (it's to support their VoIP phones and TV service).

It's a shitty hack and it adds a weird layer of indirection that's kinda buggy and doesn't always flow traffic the way you think it's being flowed. The IPv6 stuff gets confusing as well because the modem is still dishing out public IPv6 address, so if you want to advertise them as well, you've got to start slicing up your prefix.

Yes: https://www.att.com/Common/storefront/resources/pdf/att_brid...

However, I still can't ping

You could swap out those two odds and you would be just as right...

They could swap those odds with anything but 100% to 0% and they would still be just as "right" once the answer comes out.

exactly... why pull out random numbers out of your ass?

I wonder if anyone has considered some sort of legislation whereby internet service providers are not allowed to block or disrupt service to certain parts of the internet in order to promote their own business model.

The argument I've made is that if they're blocking certain parts of the internet, then they shouldn't be allowed to call themselves an Internet Service Provider.

I think ISPs would be welcoming to that change. They'd market as "WWW-Providers" or "Social media providers" and most people would be happy.

But hey, if you have advanced needs, no problem, let me refer you too our Gaming Provider and Streaming Provider subsidiaries.

Oh you need actual technical access to the internet because you write your own software? Tricky, but I'm sure our Business Technology Services Provider subsidiary will have the service you need. (You do have a business, right?)

> I think ISPs would be welcoming to that change. They'd market as "WWW-Providers" or "Social media providers" and most people would be happy.

They'd also become unreliable and untrustworthy.

"Mom, I'm going over to Timmy's house tonight. They have _good_ Internet"

"Mom, we need to move downtown, where there are two competing shady ISPs and not just the one we've got here, so we can buy different packages from both to get 95% of the Internet we need."

"Mom, I'm going over to Timmy's house tonight. They have more internet, not just Facebook!"

"Hold on... they have what?! I'll talk with Timmy's mother - and you don't go anywhere. The nerve of her to her own child roam around unsecured just like that. What if you'd hit one of those pedophile sites?"

(Meanwhile this whole exchange is probably already obsolete because who visits their people's houses when you have phones?)

No doubt. That is absolutely how it would work out.

I've made this argument before (and it does make some sense), but I also doubt that enough people will understand this nuance for it to really matter.

Maybe not-really-ISPs should be made ineligible for certain privileges / rights given to real ISPs. Like not-really-doctors can't do everything that real doctors can (grasping for a better analogy).

Other entities could punish them by revoking peering agreements. Or if CloudFlare wanted to play hardball, they could deny access to their CDN from AT&T IP ranges. That would be punishing AT&T customers further, but it would get their attention quickly and they'd complain to their ISP.

It would also be punishing CloudFlare customers quite a lot.

Taking a moral stand is honorable, but using your customers to do it isn't.

> but I also doubt that enough people will understand this nuance for it to really matter.

Certainly that's the first step.

There's options for the second step. But advertising seems like it would be the most powerful.

"Why use us over AT&T? Because you're not getting the Internet. You're getting what AT&T decides you should look at."

"We don't block Netflix or Hulu or a whole host of other streaming services, unlike AT&T"

Who's advertising this when AT&T has a legal monopoly or duopoly in your town?

The arguments for anti-net-neutrality has basically come down to "let the free market sort it out." I don't agree with that, but if we can't have net neutrality, at least define to the customers what the "internet" means.

And in that case, the town just lost it's internet. What makes you think the residents won't remember this come election day?

The problem with the "let the market decide" is that there is no free market for Internet access in the US!

In most areas there is effectively a government imposed monopoly on who can provide you access. So there is no "market" to normalise things. You simply cannot vote with your feet.

In Europe, where the regulatory framework is different, people would just switch ISPs if one started acting in bad faith.

>In most areas there is effectively a government imposed monopoly on who can provide you access.

And that government is elected by the people, right? Which means they could make this an election issue and vote candidates that don't support monopolies, right?

I don't understand what part of my statement you're arguing with.

This doesnt work well in practice though.

Most people don't have the grasp on the technicalities to even be able to make the decision to vote for a specific candidates because their internet access is sub-par

Not to mention if you vote for someone you also get all the other things that candidate aligns with, not just better internet.

(not super sure how voting on city/state level works in the us, but it should be accurate enough)

That would be less of a problem if most US elections weren't duopolies as well.

Except they haven't, really. They can still turn on their phone and login onto Facebook and watch stuff on YouTube. Someone telling them they no longer have Internet will just sound silly.

Oh god I really hate that that sounds so accurate in so many cases.

How do we provide a kiddie day care service level for people who won't or don't want to care, and a full service level for the rest of us?

Or do I owe the Internet an apology?

A court might.

but they'd just call themselves a "networking communications service provider" or something, or call themselves nothing, and people will still just use them.

You can call yourself whatever you want, but these are the regulations.

In the US, the internet is hardly regulated at all. ISPs can legally get away with anything.

Great point. Like at some point Hershey was on the verge to lose ability to call it's chocolate 'milk chocolate' because it's contents didn't have enough of it and cocoa.

I really love your idea.

You know what happens then . . .

"Last year, a number of industry groups lobbied for a change to the FDA’s definition of chocolate — a change that would have allowed cocoa butter to be replaced with vegetable oil. At the time, Hershey’s spokesman Kirk Saville told the Harrisburg Patriot-News that “there are high-quality oils available which are equal to or better than cocoa butter in taste, nutrition, texture and function, and are preferred by consumers.”"


"Internet-like product"

Made From Reconstituted Internet

In many parts of southeast Asia you can find plenty of "web access" providers that literally give you a private IP behind a NAT in their "LAN", and they are much cheaper than "real Internet". Free WiFi is almost always a similar thing. They are sometimes called InterNAT instead of Internet service.

Do you think anyone will notice or care if Spectrum or Verizon stops using the term ISP to describe themselves?

It does seem like quite a few ISPs are little more than WWW providers with partial email functionality.

Organic, fair trade, vegan, gluten free internet service provider

I'm fine with this. It's fraud otherwise.

NN seems like probably a good idea, but it's crazy to me how the whole internet went crazy over something with at-most marginal effects, but barely a peep over FOSTA which has already taken out vast swathes of valuable websites, craigslist personals perhaps most notably.

It's very unfortunate that people are simply fatigued of fighting this fight.

Also see the UK as well for an example of how previously unregulated speech has become regulated because the authorities have pushed over and over again, backing off every time there's a loud enough protest, but trying again after a short time.

All the stuff in the UK is voluntary (except the traffic analysis snooping stuff, but that's centralised and the Americans were doing that to their own citizens when it was theoretically illegal, so, meh). All the big famous ISPs you see advertising on TV have decided to volunteer to censor, but it's not a law. Smaller specialist ISPs just say "No". Mine even had a thing saying look at this great endorsement and it was a link to Hansard (the official parliamentary record) where a Peer was moaning that bad people can get uncensored Internet service from that ISP and the law doesn't stop them.

The digital economy act 2017 requires porn with "insufficient" age verification to be blocked. Required by law.

So exactly what parent said, happened.

Nope. It's fascinating how many people believe this, but it isn't what that law says, and so sure enough such sites are accessible via my ISP. The ISP is required by law to provide some means by which consumers can choose not to be able to access "adult" content. It does this during sign up, if you pick "Yes, block adult content" it informs you that they choose not to do business with you and suggest you use a different ISP.

>Nope. It's fascinating how many people believe this, but it isn't what that law says

They do because it's true and that's exactly what the law says.

Digital Economy Act 2017 14 (1):

>A person contravenes this subsection if the person makes pornographic material available on the internet to persons in the United Kingdom on a commercial basis other than in a way that secures that, at any given time, the material is not normally accessible by persons under the age of 18.

Section 23: Regulator’s power to require internet service providers to block access to material

(1) Where the age-verification regulator considers that a person (“the non-complying person”) is—

(a)contravening section 14(1), or

Like its predecessor, the Digital Economy Act 2017 has a huge amount of text that's basically predicated on the relevant Minister pushing the button. And of course this text is a huge mess (which is why it doesn't take effect immediately, the intent is you can come back and fix it before pushing the button) and so in reality nobody pushes the button. Section 23 is one of those parts. The hypothetical regulator doesn't exist, the infrastructure for all this doesn't exist. None of this is actually law.

Go read the "commencement" section - it's actually eye-opening to do this for other laws you've heard are supposed to have drastic effects.

This is almost funny. We have the exact opposite problem in Sweden, it was just in the news today. One ISP has been convicted for allowing access to facebook even though the user has reached it's data limit for the month. This is unfair competition since the local swedish newspapers are still blocked when you reach your limit.

That's exactly the problem: Facebook holds a special position on that ISP. Imagine a new social network trying to compete. If users can access Facebook when they can't access the new social network it's yet another reason to avoid switching.

Look at it the other way around. They're blocking everything except Facebook. It's the same, but instead of 1 IP, they're blocking almost everything.

Yes, that is what I'm trying to say. They have been battling it out in court for two years and lost so far. Still free bandwidth for facebook though.

This is so-called zero rating. EU net neutrality regs are usually interpreted as banning it, at least on fixed line connections (mobile is more sketchy). Enforcement by country varies wildly, though, as is often the problem with EU regs.

Certain mobile ISPs are advertising this as a feature in the UK.

Source of this news, please?

In swedish: https://www.svd.se/telia-fortsatter-att-ge-facebook-fordelar

(note that its not an article but a debate post)

Are they blocking Why do you think they're blocking

They were blocking on some firmwares long before cloudflare's dns service started. From what I've read, the routers use it on some internal interface.

It's likely incompetence, not malice. If they didn't want people using other DNS, and were willing to fuck with ip addresses they don't own to accomplish that, they'd be blackholing google's and opendns's public caching nameservers too.

It might even have been a conscious decision. Even though it's horrible and the people involved in developing the firmware need re-education. The decision probably went like this: we need an internal address to do something. We can't use 10, 172.16, or 192.168 ranges because those might conflict with internal LANs. 1.x is safe because we all know nobody uses them. The correct decision obviously would have been to get at&t corporate to commit to never using some tiny corner of their address space, and use that. Or 127.a.b.c if that works on the OS. Those options are only needed if they really need an extra IP address. They might not need one after all if they designed their firmware better.

Whenever I've needed IP ranges for similar purposes (i.e., default IPs for container or VM internal / private networks) I've used ranges from RFC 5737 (,, and These are for reserved for documentation purposes, so it is highly unlikely that a customer would have these going in their own internal network. Not the best solution, but better than tying up a public /24 that we own.

We used to use RFC1918 (172.16/12 IIRC) addresses for the communication between internal nodes in a cluster-in-box system that I worked on, which worked great until we had a subnet collision on a customer's network. Leaves me wondering if link-local (169.254/16, fe80::/10) would have been a better option - while technically the customer could decide to make the external (customer-facing) network have a link-local interface, the chances of that configuration actually happening are pretty slim.

I'm still not entirely sure what the best option is there. Maybe some clever use of network namespaces, with a named pipe to bridge between the "internal" and "external" universes? Just typing up that idea makes me cringe though.

BTW, for those wondering what this particular failure scenario is: Let's use Docker's default subnet as an example. So your docker host has iptable DNAT rules that routes a given "external" IP address ( to a given docker container ( That works great, unless you have a workstation on a subnet such as When that workstation sends a packet to, that packet gets routed to the destination container That container goes to reply, but the reply packet never makes it out to the original workstation because the container host thinks it is bound for something else on its version of the 172.17 subnet.

One workaround to this is to have the container host also put in an SNAT rule, so that anything that it forwards to a container would have the source IP address re-written to appear to come from the container host's IP, or the docker0 bridge IP (

On a similar note, Docker for Mac assigns (or used to?) the IP address to the VM that runs Docker. One day I was working in a coffee shop and got really confused as to why I couldn’t connect to my application, even though the server was running. Then I realised the coffee shop WiFi was using for client IPs.

IPv6 Unique Local Addressing is made for this scenario, with a low chance of collisions. fd: + 40 random bits, becomes your new /48.


I can't wait till the world comes around to the true advantages of IPv6. It's not just about adding more global addresses...nodes participate in multiple first class networks now (one of those networks is often the global internet). I'd be much more comfortable with smart devices in my home if they're on a universal local network with a public internet federation service for things like software updates. IPv6 makes this possible.

In a cluster-in-a-box scenario, you could modify the OS's network scripts to have the cluster-specific private interface start after the general LAN interface is up. Check both 10/8 and 172.16/12 to see if they're used by the public interface, and use whichever one isn't for the cluster network.

That only works if the host is on the conflicting network. But if the conflict is a couple hops away, Docker won't detect it.

The correct solution is to choose a small part of their allocated public range, and reserve it for this purpose.

This is also the correct solution for your cluster.

It’s weird at&t is in such poor technical shape that they can’t control a single ip address, and then just use that.

This was an organization that sustained five mines of uptime for decades.

Crazy to see a fallen (or broken up) titan struggle with basic stuff. I mean, basic compared to their heyday.

If you look at the history, the current AT&T is actually Southern Bell. It got eaten by one of its children.

You would be amazed at how antiquated telecom companies are. They port numbers manually in many scenarios.

I like to use for that stuff since I don't believe any of those IPs are available on the open internet.

Which is the exact problem that we're seeing, here. "Oh, I know, I'll just use a segment allocated to somebody else; it's not like they use it!" Aaand...whoops, they do.

I can't see anything about the range being reserved

https://en.wikipedia.org/wiki/Reserved_IP_addresses belongs to the Department of Defense. https://whois.arin.net/rest/net/NET-33-0-0-0-1/pft


It's allocated to "DLA Systems Automation Center," a branch of the US military. The addresses are probably used on NIPRNet/SIPRNet, but not publically routed. (Much like

The OP better not have anything juicy on his network. The Russian and Chinese are gonna be on you like a wife in a Finnish wife-carrying competition.

Don't use Kaspersky!

My personal favorites are, the explicitly unallocated test network for amateur packet radio to internet gateways, and, the address range for bidirectional carrier grade NAT.

Until they are and then everything breaks, which is basically what happened for

Of course they don't want to use their own address space or commit to anything.

Curious to know if they block Google’s DNS servers as well. That 1.x space was a RIPE research segment, so it’s possible that some internal AT&T group was using it with the assumption it would not be publicly routable and got bit. I was enjoying the shorthand ping of 1.1 for my router at home until Cloudflare took it over. Needless to say, if that was the case for AT&T, their ‘fix’ is not at all acceptable.

Because users were able to connect and after the firmware update they are not? And also because they didn't even let you change this setting to begin with.

There is not enough data to attribute this to malice yet, but it does not look good (see CloudFlare's tweet).

And they singled out this one instead of Google’s, which has been around since well before NN existed and is far more well-known, because...? I remember seeing talk about this on dslreports a couple weeks ago, IIRC it’s not a deliberate block, they were using this IP or a range internally.

I think they'll block if the anger for blocking isn't too loud.

I think they're blocking because customers are now using DNS that isn't them, which deprives them of valuable data on which domain names their customers go to, which they can sell to advertisers. Yes, there's other ways to get that information but the DNS server is an easy one.

> I think they'll block if the anger for blocking isn't too loud.

On what basis? Google started Google Public DNS in 2009 and, as far as I know, it was never intentionally blocked by any ISPs. The issue with is a lot of hardware treats it as though it was reserved for private networks. For instance, I can't access right now since I'm connected to a Cisco router. So this could very well be a technical issue.

But even if is taking off more than did, your assuming the DNS queries people are sending are secure anyway. I'll admit I'm not completely up-to-date on the whole "DNS over TLS" thing but I haven't noticed any support for it on my fully-updated Windows machine or Android phone. I'd love for someone to correct me, but I don't believe any major electronics ship with secure DNS by default. If people are sending DNS queries unencrypted the ISPs can just sniff them.

> On what basis? Google started Google Public DNS in 2009 and, as far as I know, it was never intentionally blocked by any ISPs.

Net Neutrality wasn't considered much of an issue back then, it was just taken for granted (and the administration at the time was attempting to enforce it as vigorously as possible).

Forcing independent internet technical infrastructure off the internet and through their own proprietary infrastructure would be the opening shot you would expect if they wanted to open that battle. After all, you gotta boil the frog slowly, and nobody but a tiny minority of technical users would really care about not being able to use third-party DNS servers.

> I can't access right now since I'm connected to a Cisco router.

I've never seen or heard of a Cisco router doing anything that would interfere with access to

Their wireless LAN controllers on the other hand, use as the default (but entirely configurable) Virtual IP to use as an anchor for the captive portal.

If you can't access behind a Cisco router it's likely because someone set it up incorrectly.

> I've never seen or heard of a Cisco router doing anything that would interfere with access to

I have news for you...

"After very little research we quickly came across Cisco mis-using, a quick search for “cisco” brought up numerous articles where Cisco are squatting on for their Wireless LAN Controllers (WLC). It’s unclear if Cisco officially regards as bogon space, but there are lots of examples that can be found on their community websites giving example bogon lists that include the /8. It mostly seems to be used for captive portal when authenticating to the wireless access point, often found in hotels, cafés and other public WiFi hotspot locations."

from: https://blog.cloudflare.com/fixing-reachability-to-1-1-1-1-g...

As I already mentioned, their wireless LAN controller uses it as a configurable default. The Cisco Wireless LAN controller is not a Cisco "router".

> I've never seen or heard of a Cisco router doing anything that would interfere with access to

Well, now you have.

> If you can't access behind a Cisco router it's likely because someone set it up incorrectly.

That’s kinda the point.

> Well, now you have.

Allow me to rephrase, I've never heard of a Cisco router doing that from a reliable source.

> That’s kinda the point.

Then it has nothing to do with Cisco and everything to do with the person who configured it.

> As far as I know, it was never intentionally blocked by any ISPs.

My Spanish ISP (Vodafone ES) doesn't block external DNS at the ISP level. However, the router they give you:

1) Blocks outgoing DNS requests from the internal network by default. This can be disabled.

2) Doesn't let you specify any other than Vodafone's DNS servers on the DHCP Server configuration. This cannot be changed.

I'll let you decide whether this is blocking or not...

I'm guessing they aren't blocking, but internally routing that ip that does not go where it should. Many cisco/Airspace wireless network gear would put the sign in network on

>in order to promote their own business model.

What's the theory exactly? What would be the benefit for AT&T to block a new 3rd party DNS? Did they do similar things in the past for other 3rd party DNSs such as OpenDNS, Quad9 or Google's? Seems odd to target this one service in particular.

I would think that being able to see what people are looking up would be quite valuable to an ISP; would help with customer profiling and selling ads.

The ship may have sailed on blocking at this point; some things _hard-code_ it.

> I would think that being able to see what people are looking up would be quite valuable to an ISP

Definitely. So if this truly was their strategy, why are they blocking instead of pointing it at their own DNS? It would be less immediately obvious what’s happening versus outright blockage. I really think people are prematurely attributing this to nefariousness.

Then take your business elsewhere. There are options.

There really aren't for most people.

Isn't that what net neutrality is all about?

That's the joke

Maybe something about being neutral on the internet.

Net neutrality started disappearing long before it was even called "net neutrality" --- a lot of residential ISPs won't even let others send packets to the full 64K port range of TCP/UDP to the IP it gives you, blocking some of them for "security reasons", throttling/cutting off certain protocols like BitTorrent, censoring "malicious" sites, etc. If we want true Internet connections we're going to have to fight a lot harder...

I would guess it has something to do with cisco asking them to help alleviate issues with their squatting on a bunch of devices. I tested it when it came out, and if I set my DNS to, then logged into a hotel wireless network (that I knew was running those devices), as soon as a request was made, I was logged out of the captive portal.

I would have expected to already be blocked if anyone filters on bogon-space (or has dealt with i

Is there a database of who blocks what? I searched but didn't find a collection anywhere.

Unless we are looking at port 25 and whatnot. Yes, it is not allowing you to use a (not technically)-arbitrary port, but most would agree that the internet is better off for that.

1/8 hasn't been "bogus" since 1/2010. ( http://www.iana.org/assignments/ipv4-address-space/ipv4-addr... )

Using unallocated IPs for "internal" or bogus purposes is sketchy, continuing to use them after they are allocated is something else. Especially so nearly a decade on.

The wheels of technological change in the Telecom space turn very, VERY slowly.

Not upgrading equipment and configs for 10 years is nothing in the ISP world.

You'd be scarily surprised just how much telecommunications runs on Perl5 ranging around the ~150GB level.

I had my stint at an ISP that worked with around 40 state level and national orgs. I saw the underbelly of how things work, and its frankly scary.

Nothing wrong with Perl5 though.

There is when much of the code was "write once, read never". There's more than a a few dozen MB blobs of dense perl5 code that we had no clue what it actually did, and was told not to touch it, lest many things break.

I had to end up touching one of them, because of things breaking with that subsystem and the new ticketing system that was being implemented. It had the wonderful line

     database_user = root
     database_password = [current mysql root password]
Lest to say, I no longer work there.

Every time I write some crap code at work, someone on HN tells a story about such horrors that I no longer feel bad. Thanks for making my day better :).

The most referred to bogon list is Team Cymru:


This team provide a great side service - you can setup BGP with them using an internal AS. It's one of the few ways you can get practical experience setting up BGP in the home with a third party. I'm running it right now.

For anyone else wondering:

> A bogon prefix is a route that should never appear in the Internet routing table. A packet routed over the public Internet (not including over VPNs or other tunnels) should never have a source address in a bogon range. These are commonly found as the source addresses of DDoS attacks.

With CGNat, you're lucky if you even get a routeable IP address anymore. ISPs have actually gotten substantially worse over the past ten years in this regard.

You can't be too mad about the full port range. Residential ISPs blocking port 25 outbound (spam malware) and inbound (people installing mailer services as an open relay by default) contributed to tonnes of unwanted traffic.

I know there was an amount of collateral damage, but if you think about it, it's been many years since malware would get in user desktops and just send spam, largely due to this.

It's the internet, blocking ports without explicit reason is totally unacceptable. It's also in most cases since people will just tunnel their traffic over ports used by other applications, such as 80.

The right response is to contact the owners of the servers/services they're running and tell them to configure them correctly - if they continue to abuse them or don't show the technical skills, then that's another matter.

Blocking things like Windows file sharing ports by default is fine, as long as you have the option to turn that off. Other ports, including mail, should be open.

I had one provider interfering with war thunder traffic somehow. packet loss always in the 20%+, which disappeared immediately if tunneled trough a vpn. switched provider and while war thunder now works, I can't play anymore dwarf fortress remote on my ipad.

even diagnosing the issue and finding someone on the other side that understand the topic is hard. I'm no network engineer and definitely neither are the support guys.

it's just a roulette. you have to change until you find one that works. and it sucks.

At&t routers are complete garbage anyways. I've literally had the following conversation with a csr:

Me: hi can you open up some ports on my router? CSR: sure which port? Me: all of them

But at the same time doung nothing to prevent IP spoofing.

You mean like net neutrality?

How is this not illegal?

Because as it stands right now, AT&T sells you access to their network. What happens on their network is for AT&T to decide. With the FCC striking down net neutrality [1], AT&T is probably testing out the waters.

[1] According to google, it's defined as:

"the principle that Internet service providers should enable access to all content and applications regardless of the source, and without favoring or blocking particular products or websites."

They've done this before, during, and after net neutrality. AT&T regularly blocks entire ranges at the IP level because they are "suspected of cyberattacks." I've frequently had issues with web hosts who are blocked only on AT&T, and this was the case in October-November (before the FCC vote) while I was launching a new site.

If you can construe some horizontal where Cloudflare and AT&T are competitors it could of course still be illegal for AT&T to block the others services simply under antitrust law.

I'm sure ATT has cdn or similar services.

NN hasn't actually been struck off yet though, it's still on the books isn't it? Pai needs to sign something and for some reason he hasn't. It was in the news a few days back.

> How is this not illegal?

Well, since the formal repeal of net neutrality has been delayed, I think it technically is a violation of the no-blocking rule.

OTOH, it's not like the FCC is enforcing net neutrality while delaying the official effect of its repeal.

The routers in question are the only ones I’ve encountered that are incompatible with my home router.

Clearly, they’re discriminating against certain client devices, and were under the Obama administration too.

However, the documentation says it should work, and AT&T won’t provide support.

They’ve been getting away with this for years, so I guess plausible deniability (it is “just a bug”) can work wonders in this space.

That’s what I want to know. I’ll save the soapbox speech and just leave it at isn’t this why monopoly laws exist?

The issue is that the cable companies have monopolies set in law already. There are numerous regulations designed to stop any new last-mile telecom companies from starting up, which literally guarantees a monopoly for the few companies that already exist in the vast majority of the US. As good as Net Neutrality sounds in theory, all we really need to do is drop the regulations and allow new players to enter the game and the market will fix it for itself.

Unenforced laws might as well not exist.

See: Monopoly/duopoly of telecom in the USA, net neutrality protections, and the speed limit signs on 280.

> Unenforced laws might as well not exist.

until they are selectively enforced.

Let's give them the benefit of the doubt, it's probably incompetence ;)

Incompetence or intent, if there was a penalty for this kind of thing you'd see it happen a lot less.

Low in the thread but is this a possibility?

Republicans showed they wouldn't enforce net neutrality regulations so there isn't fear of Govt action.

The guy which himself banned a site from his service? Surely if he has the right to block others do to. After all, it's a free market and private companies are allowed to do what they want, you don't like it, go to someone else. Remember, only the government can censorship.

Cloudflare are not a monopoly or duopoly. Cloudflare isn't a critical link in the chain between consumers and the wider internet. Being a Cloudflare customer isn't a necessary part of internet access.

This isn't malice. AT&T has an internal IP they assigned to because it was unused and they used it as an image caching proxy so it browsing the internet would feel faster on early phones. I've seen it when I was reverse engineering on Android a while back.

If not malice, it is incredibly bad engineering. Every IP address on a foreign, public network has to be considered as "in use".

This is actually the reason that gets so much traffic. People just assume it's not in use and can be abused a bit. Once it's available on the internet then all that excess traffic that was going nowhere gets transferred there.

Still, it looks more like malice since there are other addresses besides that are also blocked.

Was this on AT&T's wireless network or wireline DSL/fiber? This problem is about the wireline network/CPE.


So it's not just malice but doubly so: they used an IP they didn't have the rights to and they're now blocking proper users of it.

Let's not act like using a "probably not in-use IPv4 but we can't really be sure" is a crime against humanity. If you're designing any kind of large scale system over the internet you end up hitting the problem sooner or later (like how some VPN solutions started using 5.x.y.z to be sure not to clash with LAN IPs for instance). The real solution of course would be to switch to IPv6 where any vendor can claim some private address anywhere without any realistic risk of collision but we all know that we're not ready for that yet.

By their own admission CF receives a ridiculous amount of garbage traffic at this IP, it was not absurd for AT&T engineers in the past to thing "well, we need an IP that we can be reasonably sure nobody is going to use and is never going to conflict with anything on any network, seems reasonable". Seeing everybody in this thread jumping into conspiracy theories instead of the much more likely configuration issue is a bit disappointing for a community that's supposed to understand technology.

> it was not absurd for AT&T engineers in the past to thing

That is an utterly unreasonable conclusion.

The same logic resulted in Y2K, which was generally a huge waste of time, money, and resources.

The same logic has resulted in the anemic adoption of IPv6, which is NOT a correct solution, because it doesn't work properly for large swaths of the public.

The correct answer always was, and will continue to be, to use internal routes for internal routing, and external routes for external. Clashes with your LAN? Too fucking bad.

This sort of pushing out of externalizes onto the customer results in the same exact outcome anyway: customer gets screwed.

The customer always gets screwed. Don't rationalize the incompetence of engineers who should know better, and corporate execs who don't give a fuck.

Would you feel the same way if AT&T started blasting static on "unused" frequencies they don't have the right to?

Do that and the FCC will come down on you like a tonne of bricks, and I feel that is absolutely justified.

It's not reasonable. We have RFCs for a reason, which define which subnets can be used for public use, and which can be used internally. This has been written down for a long time and anyone working at ATT that can make these kind of decisions should know better.

"Never attribute to malice that which is adequately explained by stupidity"

Hanlon's razor is dangerous. "Playing stupid" and its adult cousin "plausible deniability" trivially exploit the razor under adversarial conditions.

That said, we agree on the probable cause of this particular issue.

Except when it is AT&T. Then you can just assume it was more malicious than you even originally thought

Equal parts malice and stupidity.

It's stupidity, then malice as a cover-up.

No, that's not what malice means.

Unless you're actually trying to say that AT&T has a grudge against Cloudflare and are only doing this to harm their company.

This is something more like negligence or gross negligence.

> only doing this to harm their company

That's not wholly unthinkable as far as AT&T is concerned. They're historically a bad player.

AT&T regularly assigns my phone an IP in 10/8, instead of using 100.64/16 as they should [1]. IIRC, they used to even have the gall to use 172.16/12, which is crazy when you consider the amount of corporate networks using those addresses.

This caused issues where my phone would try to get on wifi, but the DHCPACK would be sent along on the existing interface rather than the one coming up. So the wifi icon was continually bouncing back and forth. My only solution was to go into airplane mode and bring the cellular down before bringing the wifi up. I don't think Android ever addressed this issue, and I had to switch around the entire subnet to avoid the conflicts.

If I knew enough about how Android worked, I'd write a patch to have all android interfaces in their own linux netns, with the dhcp client exec'd in that netns, that way you'd never have to worry about this sort of conflict.

[1]: https://tools.ietf.org/html/rfc6598

A carrier deploying NAT might not be using CGNAT, and hence, in compliance with the spec, decide not to use the 100.64/16 space.

One requirement of CGNAT, for instance, requires that the carrier's router be able to handle "address crashes". [1]

[1] https://tools.ietf.org/html/rfc6598#section-4

I think it says that you can use it even if you're not doing CGNAT, provided that you are a service provider.

> In particular, Shared Address Space can only be used in Service Provider networks OR on routing equipment that is able to do address translation across router interfaces when the addresses are identical on two different interfaces.

(edit: no, very clearly, "Devices MUST be capable of performing address translation when identical Shared Address Space ranges are used on two different interfaces." )

Also, is it wrong to assume that cellular networks are able to handle "address crashes" due to the inherent centralization that comes as a result of having clients maintain the same IP (and same connections) as the device hops from tower to tower? Maybe I don't understand the topics at play here...

They used an IP that was originally reserved for what reserved IP's are used for. Now that Cloudflare convinced to be released, I'm sure AT&T wants service continuity and had to make this decision, which is well within their rights as an ISP. I dislike AT&T so if this was entirely opinion-based, I would be against them here. But this is a knee-jerk reaction to a well justified decision.

Except that it wasn't classified as a private IP address. They should've use something like,, or

The range was owned by IANA from _1981_ up until 2010, when it was transfered to APNIC. (The range was also owned by IANA until 2010, thentransfered to RIPE NCC).

If you want to get technical, use of the space could be construed as theft.

As for the continuity issue, it was stated that it was an old device, so they have no responsibility to continue supporting it, and considering the age of the device in question, it may not be able to connect to the existing network.

They should be using 100.64/10


It was not ever a reserved IP address in the proper sense. They should have used something from RFC1918.

Any sane gateway would block any access to anything from RFC1918, and using a private IP address for a public service is a bad idea in general.

Fair enough, if it was traversing public (non-ATT) networks without a tunnel.

It would be reasonable for AT&T equipment to block any private IP traffic outside a customer's private network by default. That's ignoring that AT&T's network is public, and it wouldn't make sense to use a private IP for a service they provide.

Reversing that would likely require AT&T to make a firewall change to literally every piece of equipment they operate, and that's assuming that they don't use the blocks internally. That, and I can guarantee that some customer, somewhere, would be using whatever IP they chose.

> They used an IP that was originally reserved for what reserved IP's are used for.

I'm eagerly awaiting your evidence for this statement, which to my knowledge is entirely incorrect.

So far's I know, existing uses of have almost universally been illegitimate (e.g. captive portals, internal-ish services, …)

I'd be more inclined to agree with you if AT&T were to come out and say what the problem is along with assurances that they are working on rectifying the situation and expect to have available in X days.

and theres nothing we can do about it.

If you own a device that is supported by a custom ROM such as Lineage OS then you can flash that and not worry about this change.

Otherwise, you can purchase a different device, preferably a Nexus/Pixel, or at least one that's unlocked. If that's impossible for you then, yes, you're stuck with AT&T's "best efforts."

then it's stupidity and malice.

stupid to think using the IP was a good idea

malice to break my device in order to paper over their stupidity.

And from a telecom no less - of all people, they should know better.

You are wrong, that is obviously not the cause.

Shanghai. One of the largest Chinese data-centers with direct peering to all major national networks. I'm inside, testing a new colocation unit we just put there. Pinging in 4.2ms, wow! Putting it in resolv.conf. Nothing works. WTF? Turns out they route across the whole DC to one of their internal services "for engineers' convenience". Not gonna change. TIC.

From https://en.wikipedia.org/wiki/ :

Technological websites noted that by using as the IP address for their service, Cloudflare created problems with existing setups. While was not a reserved IP address, it was and is used by many existing routers (mostly those sold by Cisco Systems) and companies for hosting login pages to private networks, exit pages or other purposes, rendering the use of as a manually configured DNS server impossible on those systems. Additionally, is blocked on many networks and by multiple ISPs because the simplicity of the address means that it was previously often used for testing purposes and not legitimate use. These previous uses has lead to a huge influx of "garbage" data to Cloudflare's servers.

What kind of demented person uses, a routable public address since 2010, for internal addresses. What's wrong with or 192.168/16?

I'm gonna guess they valued the aesthetics over the problems / conditions.

You'd think in that case they'd have gone with Silly people.

That’s intentional, from what I remember. All non-DNS traffic is analyzed for research purposes (not by Cloudflare though).

A wake-up call for all those (ab)users of public address space is also desperately needed. All IPv4 addresses will soon be allocated. Failure to use only private address spaces will cause problems, very soon.

That's not a valid criticism; that's an excuse.

CloudFlare likely did this on purpose, because so many people can't get their heads out of their own asses and follow spec. Now there's a big spotlight on the people purposefully breaking the network. And it will be fixed, eventually, whereas previously, AT&T would have just said "take a hike".

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact