Hacker News new | past | comments | ask | show | jobs | submit login
Ask HN: Is there something wrong with internet right now?
166 points by leesalminen on Nov 6, 2017 | hide | past | web | favorite | 73 comments
Down Detector (http://downdetector.com/) is showing a huge spike in reports over the past 30 minutes. Many of our users are having trouble reaching our LB in Northern Virginia.

Anyone else having this issue or any insight as to what's going on?




I'm a Comcast user in Hayward (Bay Area), CA, USA.

Some sites are easily available: Google, Yahoo.

Some sites are partially available: Twitter, Hacker News.

Many sites are completely unavailable: Down Detector.


I was having maddening issues with Comcast Business in Portland, OR this morning.

Switched over to my VPN and haven’t had any problems since.

My initial pokes at the issues I was seeing were DNS related, but even switching the DNS hosts to Google or OpenDNS didn’t completely fix it. Maybe they’ve got some deep packet inspection issues?


I'm in the province of Quebec (Eastern Canada), and we're having huge problems with most websites. Strangely enough, LTE network is working on the three providers I've tried.


Same here. I was going crazy trying to diagnose potential WiFi issues. Since LTE worked fine on my phone I assumed the culprit had to be the WiFi link.

Its really weird.


Everything's back online now. It's been like 5 minutes.


Southern Ontario, everything is slow as all hell


Oh no, the Internet Monopolies are instituting "Rolling Blackouts" ahead of the FCC decision on Net Neutrality!!


Also Southern Ontario (Toronto), I've noticed nothing. Been or reddit, HN, many random programming websites, starcraft, etc.

I've checked all the urls people said they had problems with so far (spotify, cloudflare, downdetector) and haven't had any.


Are you on Rogers or Bell? I'm on Bell and it's slow. Also slow at Starbucks which is Bell...


everything's back up for me


Rogers


everything's back up for me


Yep, in Mississauga (just west of Toronto) using Bell, most sites are slow or don't load at all. Can't see a pattern yet though.


everything's back up for me


IT worker here... Everything is on fire and my phone is ringing like crazy.


https://cloudflare.com is down for me. Comcast in California. HN is really slow.


Same here. Slack is working great though.


And giphy!


Lots of similar reports on the outages mailing list: https://puck.nether.net/pipermail/outages/2017-November/0108...

"Anyone having packet loss, please send traceroutes to the list. There is clearly something happening, but not much useful data yet. Some traceroutes might help narrow down the cause of the problem."


I'm having significant issues as well -- it happened quite suddenly an hour ago (9:50 Pacific Time, northern California). Thought it was a local issue at first and spent the last hour debugging my network. I think my apartment is on Comcast, although I'm not 100% certain (one of my roommates is in charge of it).

Spamming `date` at an affected SSH connection suggests it's entirely a download problem, since all of the timestamps are very close, but they're received multiple seconds apart.

Like some other commenters here, my mobile connectivity is just fine.

EDIT: Certain sites (including Hacker News) are now qualitatively more accessible, and I'm now hitting expected speeds at http://testmy.net/download. SSH to my DigitalOcean is also better. Here's hoping everything else smoothes out soon!


Same here on Comcast in Menlo Park (CA). T-Mobile is zippy, though, so I guess I'll switch to mobile hotspot for a while.


Related thread: https://news.ycombinator.com/item?id=15637412

There are I think three threads talking about this right now. This one won the front page and topical title fight.


I've read anecdotal reports of an Level3 outage that makes Comcast have poor connectivity outside its network.

I have websites that are up right now according to monitoring tools, but they are inaccessible to me from home.


Comcast Business in Nashville is terrible right now for anything that's not local (Facebook, Reddit, etc.). Some are worse than others (HN is just slow; Reddit's very slow but loads eventually; Down Detector won't load at all on our work connection now).

AT&T (in Nashville) doesn't seem to be impacted; I'm RDP'd into one of my home machines with Uverse and everything's working great there.



Comcast says it's been resolved for most customers now: https://twitter.com/comcastcares/status/927636388517838848


I was just thinking, shouldn't there/couldn't there be a site that aggregates actual statistics (as opposed to the crowd-sourced DownDetector) regarding internet connectivity/health globally to provide an accurate snapshot of what might actually be happening?

Edit:

Sorta like this[0] I suppose.

[0]https://www.akamai.com/us/en/solutions/intelligent-platform/...


For many years there was a site called the Internet Health Report which would show packet loss and problems between multiple carriers. It was pretty useful until Dynatrace deep-sixed it this year.

I haven't found a suitable replacement, would be interested to hear any suggestions.


Yeah, I just headed there for the first time in a while.

DISAPPOINTED.


Same here! I seriously miss that site.


This is how WW3 will start


Last night I had trouble with Google, Netflix, Reddit redirect links, and some other sites. I was trying to think of places an explosion could happen that would cause my issues.

This morning I've narrowed the explosion's epicenter down to the Wifi card in my PC. :/


I contacted Comcast Network Operations Center. They confirmed they're aware of the issue and working on fixing it, but didn't provide any more details.


Sweden here. Facebook and other select sites down.


I’m in the SF area and Comcast is between slow and not working. Cellular data is working fine though


I'm in Northern California and also having issues.

One thing I noticed is that it is only Downloads having issues. My uploads are as fast as ever.

I then connected to my VPN, and everything is working great. I think it is their routing that is having the issues, not the connection itself.


Maybe. I am getting a weird issue when trying to open a page for research I am doing for paid work. I can't get to the page and it offers to open as a PDF or something and then fails to open anything. Just more error messages follow.


Malaysian here. I'm surprising a Comcast issue would reach all the way here.

I noticed a lot of sites were working really badly, thought it was some problem with AWS or some damaged internet artery cable.


It appears to be a Comcast issue: https://news.ycombinator.com/item?id=15637412


Dang I can't even access downdetector. I'm in California in the central valley. Everything is very sluggish and the ol' router/modem reboot did nothing. :/


Second Life is reporting major problems. [1]

[1] https://status.secondlifegrid.net/


The situation here started to improve within the last two minutes.


I thought it was just me. I’m in Colorado. Bandwidth seems to be reporting as normal, however I’m having issues loading Facebook, imgur, some advertisements on reddit, etc.


It may have something to do with a large, prolonged power outage in downtown Denver. I've seen at least one ISP tweet about possible disruption to services due to this, and it may be impacting others. http://www.9news.com/news/local/power-outage-in-downtown-den...


I'm in Colorado as well! Glad to know it's not just me. Are you on CenturyLink or Comcast?


Comcast


Yes, 4.2.2.1 DNS is down. Probably your problem.


Might as well be a symptom.


Symptom of it being a primary cause would be no loading, then all of a sudden loading of site. However, I've had intermittent connection with HN, so not sure that is necessarily the primary. Edit: clearly spell check is down.


Thanks for the tip. I'll pass this along to our users.


Google provides alternatives at 8.8.8.8 and 8.8.4.4


My local network has those as the default nameservers, and my laptop has them set manually as well. Still having problems.

Normally, ISP name resolution is the first thing I look at, but this seems atypical.


Issues here in San Jose area - it appears to be Comcast from what I can tell. Same as others, cellular via ATT is fine.


I can’t connect to VPN (on AWS), but AWS CLI is working fine. Weird. But, I’ll just do something else for a while ...


Salem, Oregon here. Everything is zippy. Seems like it's location dependent? Just checked reddit and facebook.


Everything loading blazing fast (Romania).


No problems reaching any site from the UK


None of the sites mentioned in this or other threads are down from the vantagepoint of Cox in Arizona.

Sounds like an ISP issue.


More likely BGP or similar border gateway routing things. In some cases, gateways can become overloaded because of bad destination routes that other gateways are routing correctly. Not that I'm an expert at this stuff. I'm not.



I thought it was just me - was about to head to a cafe. On comcast here in Berkeley FWIW.


Also Comcast (in Portland, OR) also having major issues, resolved by tethering.


Gizmodo is reporting that it is a DDoS attack.

https://gizmodo.com/this-is-probably-why-half-the-internet-s...

[edit] Dang, I thought I had checked the date. Sorry everyone!


That's a year old article, FYI.


Goes to show you how incredible slow this attack has made the Internet!


That article is from 2016.


That's from the outage on 10/21/16.


That’s from 2016?


yes, lots of sites offline for me. I'm in Atlanta GA.


downdetector.com now seems to be down. never a good sign.


Spotify has been down for me since ~1:00 EST in Toronto.


There should be a service sysadmins can run that sends controlled pings, download/upload requests, etc, to specific hosts. The idea being that the results are shared publicly.

--

Copypasting https://www.wired.com/story/how-a-tiny-error-shut-off-the-in... (dated today):

--------------

A YEAR AGO, a DDoS attack caused internet outages around the US by targeting the internet-infrastructure company Dyn, which provides Domain Name System services to look up web servers. Monday saw a nationwide series of outages as well, but with a more pedestrian cause: a misconfiguration at Level 3, an internet backbone company—and enterprise ISP—that underpins other big networks. Network analysts say that the misconfiguration was a routing issue that created a ripple effect, causing problems for companies like Comcast, Spectrum, Verizon, Cox, and RCN across the country. Level 3, whose acquisition by CenturyLink closed recently, said in a statement to WIRED that it resolved the issue in about 90 minutes. "Our network experienced a service disruption affecting some customers with IP-based services," the company said. "The disruption was caused by a configuration error." Comcast users started reporting internet outages around the time of the Level 3 outages on Monday, but the company said that it was monitoring "an external network issue" and not a problem with its own infrastructure. RCN confirmed that it had some network problems on Monday because of Level 3. The company said it had restored RCN service by rerouting traffic to a different backbone.

The misconfiguration was a "route leak," according to Roland Dobbins, a principal engineer at the DDoS and network-security firm Arbor Networks, which monitors global internet operations. ISPs use "Autonomous Systems," also known as ASes, to keep track of what IP addresses are on which networks, and route packets of data between them. They use the Border Gateway Protocol (BGP) to establish and communicate routes. For example, packets can route between networks A and B, but network A can also route packets to network C through network B, and so on. This is how internet service providers interoperate to let you browse the whole internet, not just the IP addresses on their own networks.

In a "route leak," an AS, or multiple ASes, issue incorrect information about the IP addresses on their network, which causes inefficient routing and failures for both the originating ISP and other ISPs trying to route traffic through. Think of it like a series of street signs that help keep traffic flowing in the right directions. If some of them are mislabeled or point the wrong way, assorted chaos can ensue.

Route leaks can be malicious, sometimes called "route hijacks" or "BGP hijacks," but Monday's incident seems to have been caused by a simple mistake that ballooned to have national impact. Large outages caused by accidental route leaks have cropped up before. "Folks are looking to tweak routing policies, and make mistakes," Arbor Networks' Dobbins says. The problem could have come as CenturyLink works to integrate the Level 3 network, or could have stemmed from typical traffic engineering and efficiency work.

Internet outages of all sizes caused by route leaks have occurred occasionally, but consistently, for decades. ISPs attempt to minimize them using "route filters" that check the IP routes their peers and customers intend to use to send and receive packets, and attempt to catch any problematic plans. But these filters are difficult to maintain on the scale of the modern internet, and can have their own mistakes. Monday's outages reinforce how precarious connectivity really is, and how certain aspects of the internet's architecture—offering flexibility and ease-of-use—can introduce instability into what has become a vital service.


facebook, discord, reddit; all unusable

what the hell?




Registration is open for Startup School 2019. Classes start July 22nd.

Guidelines | FAQ | Support | API | Security | Lists | Bookmarklet | Legal | Apply to YC | Contact

Search: