Rogers is the top ISP in the set, but according to some official statements, their systems inadvertently began severe DPI shaping on encrypted and P2P traffic since September 2010 around when Netflix was rolled out to Canada. This issue still remains today.
I have a 50 megabit plan with Rogers and all my SSL traffic throttles the entire line with an exponential dropoff to 25 Kilobytes a second. The packet loss is severe and persists for all packets even HTTP. It takes a long time sometimes an hour for the connection to recover. You can imagine how many applications this would impact (syncs with large files, P2P, Dropbox make the Internet virtually unusable).
I don't speculate on the matter, but here is some more information.
There are a few elements to it that one could use to dispute its value — for example, Netflix has no way of knowing what level of service a customer has paid for. They also don't appear to differentiate between different types of service from a single ISP (e.g., they lump together AT&T's old ADSL customers in with ADSL2/U-Verse, and, presumably Verizon's DSL with FiOS). I would hope that they are also filtering for sessions where a higher bitrate is requested, which Netflix's Silverlight app doesn't do unless the window/screen is large enough to benefit from the extra resolution.
Of course, one could argue that those differentiators don't ultimately matter, and that if its customers are opting for older or slower service levels, that can tell you something about the ISP's pricing competitiveness.
With respect to the service tiers of the various networks, a much more useful graph would include the mean, median, and standard deviation of the observed throughput. It should then be really easy to determine if DSL vs. FiOS is responsible for the skew towards cable companies presented in these graphs.
"they lump together AT&T's old ADSL customers in with ADSL2/U-Verse, and, presumably Verizon's DSL with FiOS"
I was wondering why both AT&T and Verizon scored lower on the graph than I expected, though it's a pleasant surprise to see my local cable operator (Charter) parked at the top should I ever need to switch.
Perhaps Netflix can't easily separate the two? I know the hostname attached to the IP I get from Verizon reveals that I'm using FiOS, but the IP address itself seems to be assigned out of the same blocks that Verizon DSL customers use. I suspect Netflix isn't--at least for an external graph like this--putting in the computing resources needed to reverse look-up all the connecting IPs and parsing them for the individual provider offerings.
I'm sorry but I don't see how this is a fantastic idea at all.
First, this is next to useless from a consumer perspective. Because anyone whose actually tested a single provider can tell you speeds vary by geography. Time Warner might be middle of the pack overall but the fastest in your particular area.
Second, they're really p#ss#ng off the ISPs here. If you want faster bandwidth out of the ISPs its better to privately talk to them than publicly embarrass them.
Finally this reinforces the reputation Netflix is already getting among content producers which is the company doesn't play well with others. There's a reason why the head of HBO and Time Warner are taking hard lines against Netflix.
Bottom line: Netflix is part of a content delivering ecosystem. If they offend every other part of that ecosystem it will be much harder for them to survive in the future
I respectfully disagree. This is a fantastic move by NetFlix.
NetFlix knows that the ISPs are already pissed at them and are moving to throttle them, so they fire back by showing consumers which of the ISPs are a better fit for their NetFlix viewing habits. NetFlix is flexing its muscle a bit here to show it can redirect customers to choose different ISPs and affect their bottom line.
> NetFlix is flexing its muscle a bit here to show it can redirect customers to choose different ISPs and affect their bottom line.
I, too, like that Netflix is giving out this kind of information, but let's remember that in the USA many people have two or even just one choice. I have my choice of Comcast cable or AT&T DSL for broadband. That's it. And I'm not 100% sure that AT&T DSL would actually work in my building, and it's top speed is 6Mb/s compared to Comcast's 12Mb/s (or maybe 15Mb/s, I forget). Plus, AT&T: the only company I can think of with worse customer service than Comcast.
So while I like that they're giving out this information I'm not sure it's going to be a good idea to piss off ISPs with this near monopoly situation in many parts of the USA.
A BIG fight is brewing, and NetFlix is summoning allies to its cause. If they don't, the ISPs will eventually have their way anyway. So, while I agree, I think NetFlix has almost no choice but to go down this path.
I, too, like that Netflix is giving out this kind of information, but let's remember that in the USA many people have two or even just one choice.
It's even worse: I live in Tucson, and I have two "choices:" Qwest DSL, with a max download speed of 1.5 mb/sec, or Comcast cable, which offers 1.5, 7, or 12. In other words, I have one real choice.
But see they have no muscle. I'll happily eat my words if this leads to some kind of mass exodus but I don't see it happening. Because in my admittedly anecdotal experience most people already don't like their ISP but are forced into the relationship because of various circumstances.
So while this information would have been worth its weight in gold as a threat to the ISPs its now released where it can do all the damage its going to do. If the impact is found to be negligible than Netflix has lost what little leverage it has over the ISPs.
Their leverage doesn't come from an increased threat of defections, which is obviously meaningless to monopoly operators. It comes from the promise of painful regulatory intervention in a monopolists operations, backed by the kind of massive and incontrovertible evidence no individual screwee could ever amass, let alone get in front of a court or the FCC.
Oh god how I wish there would be some regulatory action to stop all this horse trading that forms these local monopolies. I was stunned when I learned that cable operators actually trade customers and regions rather than competing.
Verizon FIOS looks like the best chance at breaking the monopolies' backs, but it seems to be slow going - I haven't had it available in any of the last four places I've lived in in very dense NYC and Bay Area, yet my parents in rural Maryland who live next to cows have had it for at least four years.
@TomOfTTB - How do you know Netflix hasn't already tried the private route? And if you think your "go along to get along" outlook had any merit in this arena, why do you think major entertainment companies spent the last decade trying to undo the internet?
Remember that "the others" with whom "Netflix does not play well" include members of the RIAA, the MPAA, and primary backers of ACTA. I'm sure you're also aware that cable operators have a similarly anti-customer track record in Washington DC.
These not-minor details matter. Why? Because the audience is also part of "the entertainment ecosystem". Suing them, DRMing them, and bribing their elected representatives in Congress all count as fundamentally offensive - and all that was going on well before Netflix could get anyone to return its calls.
I'm not saying that your advice isn't good. I'm just saying that Netflix isn't the first company to which it should be directed.
"Netflix has no way of knowing what level of service a customer has paid for"
I think that partially explains the clump of RBOCs in the 1,800 - 2,000 band.
I manage a network that has several hundred subscribers wholesaled off of AT&T's legacy DSL network in the midwest.
Many of our users can't get speeds beyond that range because of line errors. The copper plant is degrading. Those RBOCs primarily use ADSL. Their speeds will probably degrade until someone overbuilds with FTTH.
Yes. The only thing I found useful about this graph was the variability over time by provider. I'm a (surprisingly) happy WindStream customer, and I routinely get the 6Mbit down that I pay for. If I wanted to pay more, I could get 12Mbit, but I really don't need it. Conversely, if I wanted to save money, I could buy only 1.5Mbit down. If many Windstream customers only pay for 1.5Mbit but still stream Netflix, then the Windstream average will look crappy even though Windstream is delivering as promised. I wonder how Netflix could reliably collect up/down SLAs and report metrics compared to them.
I'm glad someone is showing average bandwidth rates for this many ISPs; I'm not sure I've ever seen any tech site test that many.
I gotta nitpick the graph though, it's incredibly hard to figure out which line is which ISP (too many lines and similar colors). A regular bar chart with the ISP names under each one would have worked fine (since the avg. bandwidth didn't change much over the course of the study).
This community could make some very interesting visualizations if Netflix would simply provide the raw data used to make the chart. This would be even better if the data were broken down by zip code. C'mon Netflix, let us have at it!
I am on their fastest tested network in Canada, Rogers. Just few months ago, Rogers upped our prices and dropped our monthly quota to 60 gb/month. That means I can use up my monthly quota by watching Netflix for about 3-4 days.
Yeah, we are getting raped either way. It's pretty much impossible to win this argument as a consumer, even if Netflix wins their with the ISPs.
I used to be with Teksavvy where 200 gb/ month was the norm. I just might have to convince my g/f and get her to switch.
Just to compare, Rogers is $59 for 60 gb/ month, Teksavvy is $29.99 for 200 gb.
Teksavvy won't be able to offer 200GB for $29 anymore now that UBB is a reality (at least for their DSL offering, not sure about their cable offering).
Offtopic, but do you find this especially offensive? (The GP's quote was "Yeah, we are getting raped either way").
If "fucked" was substituted for "raped", would you still be offended? What about "violated"? "shat on"? "murdered"?
I'm trying to determine whether it's the content of the metaphor, the language, or something else which is offensive. (FWIW, my friends use "raped" a lot, and I sense that some are offended by it, so I'm trying to distinguish why.)
I'd like to comment on this too because I've always disliked the term "raped" in common discourse.
I don't dislike it because its impolite, or because its offensive to others.
Rather, its because it conjures up an image in my mind that is too strong for the context it is used. Lost a game of foosball? You were "raped". Getting a raw deal from your ISP? You were "raped".
Unlike murder, 'rape' is more commonly used non-euphemistically. At least at my university, everytime you would go to the bathroom you'd see an anti-rape poster. When you got back from summer vacation, you'd be handed a 'rape whistle'. We also had patrols specifically to prevent 'rape'--not mugging, not murder, but rape.
Because of that when someone says "raped" I think of the actual graphic act. However the term 'murdered' in my mind is far disassociated from the actual act.
Question, and perhaps unrelated, why do highschool football teams have pep-talks where the coach tells them to "murder" the other team. Why aren't they extorted to "rape" the other team instead?
I think 'shat on' is the best one. Shafted is second. People who have experienced or come close to the other things tend to experience the longest and the worse post traumatic stress syndromes from them.
You* shouldn't trivialize a term and crime like "rape" - and doing it makes you look like a clueless, insensitive asshole, because there might be a rape victim who reads and hears you.
Feel free to say "fuck", "shit", etc. for all I care.
Surely if I'm getting "fucked" against my will, I'm getting "raped".
Personally, I think the suggestion that using words in alternative contexts somehow diminishes their power in a negative way to be very silly. Maybe I'm wrong though, and Disney should stop making pirate movies because victims of piracy might be watching.
I agree with your argument as far as "fuck" is concerned, but not "rape"; "shit" is analogous to "crap" and "excrement", and yet it is deemed more offensive and strong, because, let's say, its "power". That doesn't mean it should be - in a decade or two, it will probably not raise many eyebrows, but by then, new terms will come up, because people want to express themselves in the way the people who initially started using the word shit did.
"Rape" isn't irksome because of its power, but its meaning; if there were or are synonyms, they would be just as bad (assuming they weren't euphemistic to the point of complete obfuscation).
To see American news channels use terms like the "F-word", "N-word", "C-word", etc., when they are reporting on what someone said, it becomes farcical. It reminds me of the Fry and Laurie sketch in a courtroom where Hugh Laurie quotes a heated exchange, but has to replace all the profanities with words that aren't offensive to the court, which obviously distorts the picture and thereby completely destroys the purpose of it.
They already say "rape" outright when reporting on it - which they should - because that's more of a quantitative word. The word "rape" shouldn't be banned or anything like that, but people shouldn't become oblivious to the meaning and ramifications of the word.
Thanks! I want to know what you think of "murder" though. Why isn't it the same (or is it)? I feel like someone whose dad was murdered would be offended by the same principle.
> I feel like someone whose dad was murdered would be offended by the same principle.
Actually, my mom was murdered gruesomely. I have better things to do than getting offended too much over people's word choices when I know that they're just venting and not actually trying to insult me personally.
That said, there's plenty of awkwardness when people assume that a person your age should have living parents and you always end up having to explain what happened when you get to know new people.
No problem. I personally don't take offence to people using "rape" like that (since rape hasn't been a part of my life, fortunately), but it irks me when other use it, and I just mention it as a piece of advice, because people might get themselves into awkward situations without knowing it. It's inconsiderate, and that part does tick me off a little.
I come from a country with perhaps the most open mind on free speech, so there probably isn't going to be anything that gets under my skin - I probably use profanity too much as a result.
I only see "rape" on the internet and don't know how prevalent it is in real-life vernacular in English-speaking countries, but it's something I'm glad I don't see that much (outside of StarCraft 2 discussions).
I don't really know that many contexts a vernacular "murder" is used in, so I don't know about that.
Regardless, the victim isn't going to be able to take any offence.
They really should have labeled the lines themselves on the right side instead of just using the default legend in Excel. I can't really tell who the hell is on the top since there are like three different greens (let alone the colorblind).
Wow, interesting, but a textbook example of how not to make a chart. I had to break out the DigitalColor Meter utility in OS X to tell which was which. Here's the list, fastest to slowest based on the leftmost point of each line:
1)Charter 2)Comcast 3)Cox 4)Time Warner 5)Suddenlink 6)Cablevision 7)Cable One 8)Verizon 9)ATT 10)Bellsouth 11)Windstream 12)Embarq 13)CenturyTel 14)Frontier 15)Clearwire
edit: Ah well, I guess that will do, can't make them stand up in a column.
For all the shit I give Comcast, I must take my hat off to them for being at the top of the heap. And I'm not surprised – my Comcast service here in the Bay Area has been awesome. I get every last bit of my 20 Mbit pipe, Watch Instantly starts almost instantly and I always get the HD version of my programming when available.
I hasten to add that part of this must be a function of competition here. I can choose from Comcast, DSL or AT&T's U-verse thing, so they have some incentive to provide a compelling bandwidth offering. Still – at least where data service is concerned, Comcast delivers.
comcast's end product is not bad at all, compared to what is available in america.
their problem is customer service and their corporate policies. i avoid calling them at all costs and only do so once i know for sure that my problem is unfixable on my side.
Did you know they have a direct support forum at BroadBandReports.com? I used it a couple days ago and it is useful for smaller issues that you don't want to call in. Must be a subscriber and only you and Comcast can read your posts in the direct forum:
I'm with Charter, 30Mbit down 5Mbit up at a not too outrageous price. I typically see 32-33Mbit down in actual use (sustained). 60Mbit down is available but is still too expensive for my use. They don't cap your bandwidth at this time either. Their customer service really isn't that bad in my experience. They haven't hesitated to send out techs to look into potential line problems. They've swapped out modems (leased) on request. They proactively send out hardware when upgrading (DOCSIS 2.0 to 3.0 for example) so that consumers are ready ahead of time etc. They certainly aren't perfect but my other option is AT&T which is atrocious in this area and has trouble providing their top speed U-Verse. My only complaints with Charter are with their television packages which stink and are overpriced. I'm pretty sure that's only partially their fault.
It's things like this that make me love this Netflix. By opening up and showing their customers the courtesy of their inner workings, they invite them to become more than customers.
By showing that they care ALOT about what their users care about, content delivery performance, they demonstrate credibility, a key component of trust.
Consistent trust building exercises such as these does more than a temporary sale ever could for long term customer loyalty, even if it can’t be perfectly measured.
I would like to see more companies try the same, even if they can’t prove the ROI on a spreadsheet before hand.
Why use a spreadsheet when you have a perfectly good gut you can check to see if it is the right thing to do?
Welcome to the opening salvos of the Age Of Preferred Internet. By publishing the streaming performance metrics of consumer ISPs, Netflix is making sure providers know they can influence the ISP choice of their customers. This as a warning, pure and simple.
They should also filter for geographies that providers serve. e.g. If one provider is serving predominantly rural areas, most of their trunks will be using directional wireless, leaving them at a severe disadvantage which is no fault of theirs.
If we were then to rank ISP's on throughput, even though a rural ISP is serving a critical and under-served part of our community, they would be rewarded with negative perception.
I'll second this. When I lived in Mitchell, SD (15k people - look it up) I had 3 ISPs to choose from (in city limits). I had pretty good throughput from 2 of them - one was wireless.
This definitely fits with my experience using Qwest DSL for my business in Seattle and Comcast cable at home. Comcast has consistently high throughput, low latency to my Seattle data center, fewer hops to the local DC, and for some reason Comcast routing seems to be smarter about routing via the higher quality of our 4 X 1 gigabit providers when I'm connecting to our local DC.
My ISP (cox) is near the top, and dealing with their customer service has been a breath of fresh air compared with any other ISP I've dealt with.
On an absolute scale, it's so-so customer service, but on a relative scale it's amaizing how many things they don't do wrong compared to other ISPs I've worked with (Comcast in particular)
There's a major flaw with this chart: companies that offer more than one connection service have all their services lumped together. The reason Comcast, Cox and Charter are at the top is because they're cable-only systems. Most of the other companies on that chart offer a mix of DSL, wireless and FTTH services.
All the traffic seems to follow the same curve, with the same dips and spikes. The Canada curve is different than the US curve, but there is still a curve there too. I'm curious why that is.
I think people are misunderstanding this report, which is surely their intent. This isn't a chart of how well each ISP performs in general, or even how well they perform in relation to each other.
As a baseline, all of the ISPs on this list that I've used (ftth, coax and dsl providers) have been able to push to 90%-100% of quoted line speed 99% of the time.
What this chart shows is netflix performance, which is quite different than general network performance. What goes into netflix performance? Total netflix service load, CDN pop load, CDN circuit size, SLA/QOS terms, the choices the netflix client makes about adaptive bitrates and shaping the provider applies to the traffic.
Take a look at the US chart and note the correlations in spikes and dips across all providers at certain times - 10/21 peak, 10/23 valley, 11/13 valley etc. These are indications of aggregate stream load netflix wide and are an indication that netflix manages their peak throughput to be pretty close to their max throughput (as well they should). But it's also a strong indication that if 10% of netflix customers switched to a competitor overnight average stream performance for netflix would jump noticeably, perhaps even 10%.
How their adaptive streaming works is another huge factor. First they measure your current performance to the CDN POP and then explicitly chooses to use only 60% of that[1]. So add 40% to all the ISP numbers out of the gate. TCP+HTTP overhead is another ~4%. Downloading something else while you stream? If the streaming sees more than a few headroom faults (ie 1000ms worth of data takes more than 600ms) the system will quickly adapt down in bitrate trying to do its best to never hard fault (buffering). It likely is considerable more conservative in moving the stream speed back up. Customer using 802.11b? Max throughput there is ~5mbps, so netflix will use a max of 3mbps. In an urban area on a congested channel? You may only see 1-2mbps at times. Microwave on? etc.
Lastly, with the news that netflix is 20% of ISP traffic at night the network providers have every incentive to manage that traffic down. Look at the best US performer at ~2700kbits. The fastest HD stream is 4800kbits - Nearly twice the speed. Worst case if an ISP allowed full rate streaming they might increase their peak network load by 10% or more.
Who was it that announced netflix is now 20% of night time traffic? Our old friend sandvine, provider of the DPI gear made famous by comcast using it to throttle bittorrent. This tells us two things - 1) sandvine is on net at at least a few major broadband providers and 2) by announcing that number they are implicitly telling network managers that they can help reduce that traffic.
Netflix makes it very easy to do fine grained control on bitrates. If you are watching the HTTP headers you know exactly which customers are on which stream rate without having to count the throughput. Most users won't notice a QOE change from 4800kbits to 3200kbits. Want to cut your network traffic? Use DPI to identify the higher rates and label them bulk-toofast. Then either explicitly rate limit them, give them a low queue priority or simply route the acks through a big buffer under performing router or indirect route. Once the adaptive bitrate slows down to a rate you like switch the label. The more subtle of these methods are much, much harder to detect than the comcast RST's were, and wouldn't technically violate any net neutrality rules in place.
This is not a very useful graph to assign blame. :) I've always wondered why I don't always get movies in streamed in HD even for movies available in HD.
Almost anytime of the day, I can saturate my DSL line's download bandwidth from fast sites such as Amazon S3. But Netflix doesn't always stream in HD even though my link is faster than what's required (4800 Kbps).
The bottleneck is most likely not at Netflix's end. They use a number of CDNs, such as Akamai and Level 3. These CDNs usually co-locate a box right at your ISP. So, it is entirely up to your ISP how much of that bandwidth makes it through to you. I can assure you, Akamai content is fully capable of saturating your tiny broadband connection. I regularly download large files hosted on Akamai (commercial software) and achieve download rates of 2MB/sec on my 20 megabit Cablevision line.
However, ISPs do "manage" Netflix traffic. How they choose to manage it varies by ISP, but it might be something as simple as giving streaming traffic a different priority over web traffic, which tends to be more latency sensitive. Joe sixpack tends to complain when his browser takes longer than a few seconds to load a page, which ties up a customer service line. If your Netflix stream automatically switches from HD to non-HD, you might not even notice because they've made their software pretty good at adapting to changes in available bandwidth.
Are you using a dns provider like opendns or google? I am and it drastically affects streaming rates because you get sent to the wrong place in the cdn. I changed my streaming box's dns server to my isp's and got much better service from Netflix.
My ISP says I get 15 Mbps download and 2 Mbps upload, and speed tests confirm I do. For sustained flows, I get a solid 15 Mbps when downloading software from Microsoft.
So the 15 Mbps is way above anything in the Netflix graph.
So, the ISP can't always be the bottleneck keeping data rates below 3 Mbps.
Netflix (apparently) can't measure anything above 4.8 Mbps, so 15 counts as 4.8 and the average is brought down by people who are on the el cheapo 1.5 Mbps plan.
I have a 50 megabit plan with Rogers and all my SSL traffic throttles the entire line with an exponential dropoff to 25 Kilobytes a second. The packet loss is severe and persists for all packets even HTTP. It takes a long time sometimes an hour for the connection to recover. You can imagine how many applications this would impact (syncs with large files, P2P, Dropbox make the Internet virtually unusable).
I don't speculate on the matter, but here is some more information.
http://www.christopher-parsons.com/blog/isps/rogers-network-...
http://torrentfreak.com/rogers-bittorrent-throttling-experim...