I've been tracking the performance with PingPlotter, if you're curious how bad it is right now here's the last 10 minutes: https://i.imgur.com/AnUqv3j.png (red lines are packet loss) Pretty interesting how current circumstances are pushing even tried and tested infrastructure to their limits.
Here's a nice blog post about the subject:
There is, however, a high chance of encountering buffer bloat if countermeasures are not taken at the chokepoint:
Modern cable modems, for example, are required to implement such countermeasures. My ISP is at over 90% capacity and round trip times are still mostly reasonable. (Bandwidth is atrocious, of course.)
If you have more control over or knowledge of your load, you can safely go higher than 80%.
Eg when I was working at Google we carefully tagged our RPC calls by how 'sheddable' they were. More sheddable load gets dropped first. Or, from the opposite perspective: when important load is safely under 100%, which it is almost all the time in a well-designed system, we can also handle more optional, more sheddable load.
As a further aside, parts of the financial system work on similar principles:
If you have a flow of income over time, like from a portfolio of offices you are renting out, you can take the first 80% of dollars that come in on average every month and sell that very steady stream of income off for a high price.
The rest of income is much choppier. Sometimes you fail to rent everything. Sometimes occupants fall behind on rent. Sometimes a building burns down.
So you sell the rest off as cheaper equity. It's more risky, but also has more upside potential.
The more stable and diversified your business, the bigger proportion you can sell off as expensive fixed income.
Not only did the rep freely share the utilization numbers with me (80% during the day and 90% at night), he also mentioned that things would not get better until end of the year when they would do a node split.
As consolation, they offered me 10x the download speed for half the price. I'm not really sure how that would help congestion...
In fiber connections is actually not that expensive to split a fiber after a CTO, you can actually sort of daisy chain it, but you want to keep everything as standard as possible.
Some EU relatives of mine keep their phone plans living here because it's cheaper with the overseas rate than paying Canadian plan rates (!!!)
Adding to that, even clueful places may be held back by one or more vendor or provider, all of which need to have working v6 support before you yourself can deploy it.
That is, it was my understanding that there was no real blocker to supporting it in the interim, except for the lack of any immediate benefit. Though I'm also not clear on whether supportinf both introduces any significant complexity
Suppose an ISP wants to provide IPv6 besides v4. What does that ISP need? Well, first, v6 from the upstreams, that's simple, and v6-capable name servers, routers, that's simple too nowadays.
But there's more. Suppose that the ISP has some homegrown scripts connected to its monitoring or accounting, written by a ninny years ago, uncommented, and some of those assume IPv4, and noone wants to touch them.
Suppose that ISP outsources its support, and the outsourcing company promises to do the needful regarding IPv6 support but never actually does it.
Suppose that that ISP is in a country where ISPs have to answer automated requests from the police or courts, and one of the software packages involved in that has a v6-related bug. Or the ISP worries that it's poorly tested and the ISP's lawyer advises that if there are any bugs, the ISP will be criminally liable.
And so on. Enabling IPv6 may need a fair number of ducks lined up.
The mind boggles. These people maintain our infrastructure.
I understand that you don't need an electrical engineering degree to be an electrician, but still, these are some fairly basic concepts in the electric power industry, especially the safety aspects, so you'd think someone working on live wires would know better.
Honestly, any halfway-intelligent person who travels internationally should know that Europe runs at 240VAC/60Hz, because this is really important if you want to use your American electronics there without a transformer. (When I went to Europe last, I brought my laptop, and an adapter which does not convert voltage, only the prongs, but that's OK because the laptop's power brick says it works on everything from 100VAC to 240VAC, as do a lot of electronics these days. But you have to check this first, you can't assume! Plugging a 120V-only device into this adapter could cause a fire.)
Luckily, the 50/60Hz stuff really doesn't matter these days except maybe for some digital clocks on appliances.
Not that there's anything wrong with that, everyone was entry-level at some point, but engineers who do capacity planning and traffic engineering they are emphatically not.
They'll be happy to deal with the last mile segment, but anything beyond that is murky and most companies I know aren't going to share much. Helps to have friends on the inside leak some graphs, though.
MRTG graph, ISP circa 1995. Colorized.
See a flat line? that's congestion. Now figure out where it is coming from. Sorry, we have been doing this for thirty years so I'm kind of cranky. It is not a rocket science.
Network congestion issues shouldn't be handed off to field techs to check local loop (last mile) and CPE (Customer Premises Equipment.
I guess I need to remember the ISP I work for here in Australia (front line tech support, and then network operations physical security and infrastructure) was widely recognised as the best ISP in Australia multiple years running, so I shouldn't use it as a baseline expectation.
Yeah, was a good place to work. I was in their Adelaide data centres when iiNet acquired the company.
I’m not sure what the incentives are for an ISP to try to get the provider to fix issues, or even if they would e.g. https://company.chorus.co.nz/what-we-do is notoriously bad for service and the copper network is being deprecated. Locally https://www.enable.net.nz/about-enable/ are doing a good job of service, because they are well subsidised by the government and seem to be effectively operated.
On some days my connection resets 5 times within an hour, which is quite annoying since retraining the connection takes a minute or two. When I call support about it they have zero monitoring in place that would let them know about the recent history of the connection quality, they can only do spot tests of SNR on demand, which of course doesn't show any transient events. According support forum posts of other users they'd have to explicitly enable "long term monitoring" based on user input to get that information.
Of course SNR line quality is an issue separate from congestion, but still, automatic monitoring appears to be limited.
More recent versions of DOCSIS have moved away from that layer of backwards compatibility, so you would probably need some specialised equipment, if it is possible at all (I don't know at what layer exactly encryption happens).
My connection: https://www.thinkbroadband.com/broadband/monitoring/quality/...
In case anyone is shopping for broadband in the UK, I only have great things to say about Zen pictured above. It's so good I just called to upgrade my 80 Mb to a 300 Mb just for fun, meanwhile my quarantined Italian friends are suffering awful internet now that everybody's at home streaming Netflix.
I used to have Virgin fibre and my average ping was 80ms with a ton of jitter. The plot above is my internet while downloading at about 2MB/s average over the past 24 hours, and surprisingly stays the same even at peak download.
Don't make decisions about the European infrastructure based on American problems.
I haven't had any problems with my internet (I do have fiber straight into my house, wired network on my laptop, fast.com reports 600 Mb/s), but Skype, which we use for meetings, has been pretty shit in terms of sound quality.
But mostly I'm amazed how well the internet is working given the circumstances.
I've written more about this problem here .
I wonder how TCP BBR would react here. If I understand it right, it wouldn't need RED to back off: the increased latency of buffers filling up would do that automatically. But BBR also wouldn't let the occasional dropped packet make it back off.
TCP BBR would still rely on RED/WRED to compute the connection rate estimate initially, then it would attempt to send below that rate to avoid packet loss. If packet loss is detected it would recompute the estimated connection rate.
I found this page  useful, especially the graphs.
Doesn't sound ideal for distancing.
Yes, the ICMP response packets could still be skewed, and the effect you mention is definitely real, but on a good connection, usually there should not be much to drop at all, neither TCP/UDP traffic nor ICMP packets.
Doesn't matter what it uses (though by default MTR does use regular old ICMP Echo - you have to specify -u or -t to get it in UDP or TCP mode). When TTL expires it still requires an ICMP TTL Exceeded be sent, regardless of whether or not you were sending ICMP through it.
Traceroute implementations in general are probably telling most everyone in this thread a lot less than they think, even without icmp deprioritization being taken into account.
https://archive.nanog.org/meetings/nanog47/presentations/Sun... is worth a read for most anyone that's ever attempted to use traceroute to troubleshoot networking, because they're almost certainly doing it wrong.
Is your own connection idle though? Pings are also affected by the congestion on your own router†, especially if you don't have good AQM (such as CAKE). Dumb queues will just drop all packets equally, smart queues will do flow isolation and penalize the bulk flows first while keeping the trickle ones (ping, ssh, voip, ...) untouched.
† and anything else along the path to your ping target
I am sure its affecting you internet speed, what sorts of tasks are you generally doing now that the entire is state is pretty much on lockdown?
Here in Alberta, although we are told be socially distant, there is no full lockdown and I want to know what kind of issues would I be expecting to run into in the up coming weeks/months?
I suspect its the services that relay on super low prices and don't have excess capacity Talk Talk etc that are really going to feel the pressure in the UK
Last ISP I worked at would have email and SMS notifications going to On Call staff.
In some parts of the world running links at 95 percent is okay because look 5 percent left (totally ignorant of buffers or microbursts etc.).