Wow, I'm surprised at the level of assumptions being made in this thread.
Guys, some networking 101:
* The route your traffic takes to get from point a to point b depends on your network/ISP/etc
* The CDN you use when accessing YouTube, et. al. depends on the route you take. The first/nearest CDN to you is (usually, depending on the CDN owner's configuration) the one that will be used.
* The fact that a video loads quickly on one ISP and slowly on another means absolutely, completely, totally NOTHING in and of itself.
To find out if the ISP is to blame or not, you must attempt to access the same CDN server from two different ISPs and see if you get the same problem. The latency will be different, but unless there is a massive bandwidth or latency bottleneck between two hops along either route, the overall bandwidth (for a large enough file) should be sufficient to deduce whether or not the problem is with your ISP or the CDN servers corresponding to the route your ISP is taking to contact Google's servers (the results need to be statistically significant taking into account margin of error and network conditions).
If the CDN is the problem, unless the CDN is actually owned by your ISP, your ISP is not to blame.
In fact, for traditional non-net-neutral throttling, it does not matter which/how many CDN IPs you block. Your ISP should (if they're doing it right) detect your connection to YouTube's subnet and throttle your data rates regardless of which CDN you use. The CDNs in the original article belong to Google/YouTube, not TW. As such, TW would throttle your connection on the way to Google's subnet, not at Google's subnet. They have no control over Google's subnet. The hops past TW's (or whatever ISP you use) servers are not under their control, cannot be bandwidth-throttled by them, and have nothing to do with net neutrality.
The real explanation is most likely poorly-balanced CDN servers. i.e. the traffic going to the CDNs is unfairly skewed towards one or more CDN servers, causing them to serve content to all users of all networks more slowly. By explicitly avoiding said CDNs which are slow on Google's end, you will use a different, less-pounded CDN that can serve your content faster.
Note that I am not even a TW user (Comcast here), but this lynch mob is getting out of control. I expect a higher understanding of basic network principles when I browse HN, and "I can't load YouTube quickly so this means my ISP is shaping my bandwidth, and I need not look for actual evidence to support this claim" does not qualify as such.
That said, yes, it is possible for a cunning ISP to shape your traffic by purposely mis-directing CDN selection, for example, making it so that all their users end up at the same exit (slow) node when contacting a YouTube IP as such effectively YouTube into serving all their content to all the ISP's users from the same CDN node(s), resulting in poor connection. The way to test this would be to map out the routes for packets sent all over, and search for statistically-significant routing anomalies when attempting to pass packets on to Google's network from within a certain ISP.
The CDN you use is often selected off a DNS response for many networks. An easy way to select a different CDN (that may adversely affect your browsing speed due to geo-origination!) would be to use a different DNS server (make sure to flush the DNS cache in your OS and in your browser). This is why it's not advised to use non-ISP DNS such as Google DNS, OpenDNS, etc) unless they're both a) anycast (basically CDN for DNS, your DNS query will go to the nearest geographic location to you) and b) have enough servers distributed around the country so that your anycast DNS request will be resolved near you, so that the CDN based off of DNS will also be physically near you. You can use namebench  by Google to query the fastest DNS servers, typically faster means closer as hops then physical distance are the biggest factors in DNS speed, though a shitty DNS server will obviously skew those results.
> The CDN you use when accessing YouTube, et. al. depends on the route you take. The first/nearest CDN to you is (usually, depending on the CDN owner's configuration) the one that will be used.
Just load a few videos in youtube with the chrome network tab from dev tools open, you'll very commonly see a CDN request returning a redirect to another CDN, 4-5 redirects isn't uncommon before the video plays.
Assuming that CDN selection is purely geographical or route based is just as poor an assumption as your accusing others of.
> Assuming that CDN selection is purely geographical or route based is just as poor an assumption as your accusing others of.
As you yourself quoted, I used the word "usually." That directly and explicitly implies "not purely."
The CDN you use depends on many factors. Nearness to your route/physical location is a big factor. The relative power/bandwidth of each CDN server is another factor (weighted distribution). Internal network factors also contribute to the decision, as does maintenance, BGP peering, random load distribution, and current network health/status.
Thanks for the detailed comment above. As I said earlier, I'm no networking guy! However, before posting I did test on a friend's Verizon FiOS connection, my own Verizon Mobile Hotspot, and also the various message boards I read complaining about speed appeared to be overwhelmingly TWC. I do understand that's not ideal proof. This was just intended to be a share of a quick fix.
Any suggestion to make my blog post more technically sound, I'm all ears! I can't seem to find a conclusion in all the HN comments except that "it could be a bunch of stuff going wrong", and to be honest, I don't have the expertise to find said conclusion.
The criticism was not leveled at you in particular, thank you for your willingness to revisit your assumptions. You need to test the connection to the same CDN node from a different ISP. Get the final URL for the video and try it on the other PC, and time the download for comparison. Make sure the CDN does not force a redirect to a different node.
You can get that kind of info from Chrome's "Resources" tab in the dev console (or the equivalent in whatever browser you are using). You should run the test several times to avoid warm/cold cache along the way. Should the results significantly differ, you should also note the traceroute/tracert results to the final hop.
These are probably off-net cache servers for Google, which means that to save money on transit costs, Time Warner has installed special servers and Youtube references them directly for Comcast subscribers (lot's of things give this away, but I wouldn't be surprised if it was DNS based and they know based upon your resolver. Try switching over to a global but not eDNS based resolver and see what happens).
If that is the case, it's not unlikely that the off-net boxes are simply overloaded for people in this guy's usage area.
I'd like to correct your last paragraph. Most CDNs use anycast to send traffic to the closest POP they have to you. However, most implementations use DNS responses for the front-facing servers. Not many companies outside of CDNs have the need or capital to use anycast for normal web/end-user traffic.
There are several other scenarios where it can be your ISP or can cause significantly different timings between ISPs. A lot of ISPs offer a network internal forward caching system (CDN-like) so your traffic will appear to go directly from your ISP to the CDN. This is typically done with servers inside your ISPs network but with anycast routing so requests to the CDN outside of their network stay within it. I don't know if Google/YouTube does this, but I do know that Netflix does.
Edit: this is independent of the post, which shows nothing but a fundamental misunderstanding of what's actually going on and why. I'd be more interested to see if this was flash or html5 video, whether it was using rtmp(s) for flash or http streaming, what IPs it ended up hitting instead of those two blocks and whether a traceroute for those two blocks went through the same or a different peer from where it finally connected.
I'm also on Time Warner, and my graph shows wild inconsistencies - sometimes it spikes up to much faster than others, sometimes it's way below the average speed, and it also shows that Time Warner is the slowest provider out of all of the NYC area ISPs.
I wish it was easier to switch to another provider out here.
Thanks for the my_speed link. When I run the test video contained at 720p, I consistently get outstanding video with no streaming pauses.
However, if I switch to another video (say a Google IO 2012 presentation), I get long pauses within the video constantly. 5-10 seconds of video, with 3-5 second pauses. In short, the test is hardly representative of my normal YouTube results.
I'm on FiOS, and my graph also shows wild inconsistencies, sometimes spiking up much faster than others, sometimes way below every other line in the graph for multiple days. I'm leaning towards this not being an ISP problem at all until there's more evidence.
I'm also on Fios and have 75 down package I just got it. It was so bad I couldn't even watch youtube videos 720 or 1080. Pausing them did nothing. I installed a script called YousableTubeFix for Chrome and now I download at 60-70. I don't know what the problem was but having 75 down and not being able to watch a video was the most frustrating thing in the world. Finally did some research and someone with fios recommended this.
I'm also on time warner NYC. The funny thing is my 30 mbps wideband is consistently ~30 mbps from almost everywhere. Torrents and downloads consistently max out my bandwidth (even during peak hours) and most sites load just as quickly as my work's speedy connection. Youtube is the only site I have issues with.
Interestingly enough, the video embedded in that page doesn't load at all when I have the two subnets listed in the article blocked. Everything else on YT loads fine, and that particular video loads fine if the subnets are unblocked. I'm guessing this is a unique experience based on the other comments.
I have been screaming at YouTube for being crap, especially lately. Five seconds of a video would play, then I'd get like a thirty-second pause -- with NO pre-load. I thought this was Google being even cheaper than they have been (doing away with total pre-loads) but now I find out it's been TWC! My god! They should be fined into non-existence.
The odd thing is, on my primary desktop PC, I am rarely served ads. If I use a Win 7 notebook via WiFi, every YouTube video starts with an ad -- even if it's a video I just watched on the desktop that had no ad. Go figure...
Can you SSH into your router and do an `iptables --list` ? I'm thinking that implementation uses the L7 chain. If that's the case then it may be a hair bit quicker using my implementation. Also, the REJECT rule is a bit cleaner than just dropping the packet, which I also suspect to be the case with "Access Restriction" implementation.
I can do iptables under the firewall settings, but ... it took me a while to find my way through this setting and I know it works, I tested by streaming YouTube content on AppleTV. I am going to leave it as it is.
If this workaround gets popular, I wonder if my provider (Comcast) and others will catch up and throttle youtube directly.
I ran iftop and checked IPs with arin.net. I found a couple in the 184.108.40.206/16 range that didn't point to google or youtube. However, I originally had a handful of /24's, but the OP pointed out the entire /16 -- so I edited as such.
I also didn't find 220.127.116.11/24 in my testing. It's also a Google IP address, so I doubt it's causing any issues -- in fact, it may be the cause of some playback issues my wife is now having. (I think it may be their video ads, but have nothing to go on aside from getting "Playback Error" when an ad usually plays)
Is there any evidence TW is throttling versus these CDN servers you're being routed to simply being slow or overloaded?
I've noticed a change in YouTube buffering on FiOS the past few weeks as well. Other commenters here see it on Comcast. That may point to YouTube leaning more heavily on a CDN that can't handle the traffic (or some hop along the network to that CDN) rather than throttling by any one ISP.
Prior to blocking these subnets, I would have significant pauses every few seconds using 360p. I just tested a few 1080p videos with no issues and without buffering, which is about as big of difference as possible.
I have TW... and I just tried what the guy suggested.
One hell of a difference. I'm pretty shocked that they're doing this. I always thought it was Youtube that was being slow. http://www.youtube.com/watch?v=CB8UADuVM5A is an accurate depiction of what I just experienced right now.
That blocking the CDN improved your loading speed doesn't provide evidence of throttling. It's equally plausible that the CDN is slow for everyone, whether you're a Time Warner subscriber or not. The way to verify this would be to connect to that CDN from different ISPs with similar routes and see if there appears to be a hard cap in one connection but not the other.
I have Verizon, as does a buddy of mine, and he and I have both been trading horror stories about how much YouTube sucks for the past few months.
I also applied the trick and now, magically, everything is quite a bit faster.
That said, while I'm not a network guy, this suggests that there's an issue with the YouTube CDN, not anything in particular with Time Warner or Verizon, unless they're doing something horrible relating to net neutrality.
My guess is just that there are a couple of CDN blocks that are preferred by the load balancers and they're saturated as a result, and blocking these IP ranges sends you to a less balanced block of CDNs.
I also don't know if there is any evidence. The cause can be many things. However, I can now verify without a doubt that this works. I turned the firewall rule on and off and tried multiple videos. When the rule is on, I can actually buffer an entire video like I could in the old days. With the rule off, it works, but is not so great.
I can only refer to anecdotal evidence I have seen on forums, Reddit, and from friends. That being said, it seems to be a stronger correlation with TWC rather than Verizon, but YMMV. Let me know if you uncover more details!
Measuring YouTube Quality of Experience for Users in Residential ISPs*
Deep Medhi email@example.com University of Missouri–Kansas City
*Joint work with Parikshit Juluri (UMKC), Louis Plissonneau
(Orange Labs, France), and Yong Zeng (UMKC)
Weird, I could have used this thread four days ago. I cancelled my newly purchased 18mb/s uverse account because youtube (and only youtube) was taking 20+ minutes to download 5 minute videos. I switched over the Brighthouse (division of time warner I believe) because I still had an account and everything was fine. When I contacted AT&T, all they did was tell me that youtube was to blame and that I could purchase a tech support contract. I declined :). There's a massive thread on the topic over here: http://forums.att.com/t5/Features-and-How-To/Hey-At-amp-T-St...
My YouTube experience with uverse has been very frustrating of late. I routinely get streaming freezing, even on lower resolution streams. I have been considering a switch to Time Warner, but perhaps that's not the best solution.
I checked the speed link given by the ex-youtube engineer and it showed that Brighthouse was actually faster. Those are the exact symptoms I was having (freezing -- basically zero download speed, not that the rendering stop taking place) Not surprisingly enough, I think that it is very location dependent on what ISP provides the best connectivity to a given service (e.g.: the internet is very, very complicated network and it is hard to sum of the performance of any point to point connection with one data point)
I've had a non-cable DSL connection for years through the same independent provider and noticed about 6-12mos ago that YT was doing the partial-buffer thing. I haven't looked into this too much aside from reading this article and comments, but I'm inclined to think that maybe the problem is with the CDNs rather than TWC or any ISP.
This actually helped a lot, as soon as videos started, the grey buffering meter (that sits behind the red play meter) would shoot over to the other side ... I was actually astonished.
Alas there was one horrible side effect. Videos took forever to start playing, and thumbnails for other videos on the screen (in the related videos section and other image assets) would not load until afte the video started playing.
My immediate thoughts exactly. The other competitor in my area currently can't go beyond 12 mbps down while TWC offers 20 mbps down and more.
There was a young upstart I had high hopes for, but I was out of their area. They are http://www.widerangebroadband.net/ and their customer service was great, but unfortunately I'm just outside their area at the moment.
20mbps from TWC is fine for YouTube for me. I don't seem to have the filtering problems the others have. I also stream Netflix fine. Sometimes the HTML5 player glitches, but that's about all I can report. Another part I didn't mention above is that the 20mbps plan bumps me to a higher up-speed plan, making remote work less laggy, etc.
Yes, typically cable companies have been granted monopolies within a municipality. In municipalities where cable companies have refused to upgrade infrastructure, and municipalities have attempted to build out fiber infrastructure to homes, cable companies have used the courts to delay movement by the municipality. Cable companies have also been pushing through laws on the state level to prevent municipalities from ever building out their own data infrastructure.
Any new entrant has to build out a cable plant to reach subscribers and capture enough subscribers at a high enough density to make it worthwhile. The only company that's even trying at this point is Google.
1. I presume that the capital costs of infrastructure in creating a new ISP are significant.
2. Most of the larger markets are in densely populated cities, which makes laying infrastructure even more difficult (there's a reason that Verizon rolled out FiOS to suburban areas rather than urban areas).
3. If your 'value added' is something that the ISPs can easier undercut to drive you out of business (e.g. expanding bandwidth caps, increasing speeds), then your business proposition becomes even shakier.
I personally avoid cable ISPs precisely because of this kind of junk. Other option is DSL (Verizon), but it's unfortunately 3 Mbit/sec max in many cases. If you are lucky you get 7 or even 15, but it's not common (only if you are very close to the Verizon station). Some rare places have fiber optics, but it seems they stopped expanding it. In general the situation in US is horrible - I still don't understand why there is no competition. Are there any artificial barriers?
My Verizon DSL is peaking at about 1 mbit/s: http://imgur.com/a/2yXOT . Our plan should be providing 1.1-15. Given that per youtube.com/my_speed, most people on my ISP are getting much better speeds I'm wondering if it's due to my setup. I can't imagine what could be wrong though: it's a Linksys WRT54G running latest version of Tomato firmware, I'm on a MacBook Air.
This could actually be due to DNS, rather than anything particular to that edge-cache/CDN. Resolvers that aren't actually close to you (network-wise) could cause problems like this, or TWC partitioning their network in a way that causes geographically diverse locations to get lumped together, etc. In essence you'd be talking to edge-caches on the wrong edge, so to speak.
To test: try a different DNS resolver, and see if you start getting faster video loads. If you're using TWC's local resolver, try 18.104.22.168 (run by Google); if you're using Google's already try the local resolver and/or OpenDNS (22.214.171.124).
If that does work as well, I'd recommend it over the heavy-handed approach of blocking arbitrary netblocks. Especially since that second netblock listed isn't owned by Google, and so you could be collaterally blocking other content that you still want.
(Disclaimer: I work for Google, but in Ads, not anything network or YouTube related).
I'm a TWC customer, and I used to have consistently horrible YouTube speeds. I eventually got sick of it and read somewhere to try different DNS, as you suggested. I switched from TWC's DNS to OpenDNS probably around a year ago, and YouTube did become faster. It's less consistent now, sometimes loading nicely and sometimes slowly. youtube.com/my_speed claims I'm averaging 7.6 Mbps vs. the average 5.2 for my ISP and 6.4 for the region. I see occasional traffic from 126.96.36.199/16, but it seems to be coming largely from 188.8.131.52/16 and 184.108.40.206/16.
Based on the discussion here at HN, I've updated the title in my blog post to call out "ISPs" generically and not TWC specifically. I felt I did my due diligence sufficiently before crafting my title, but as we know, a sample of one is not statistically significant!
Fios user here. I'm travelling right now so I can't test. But youtube has been consistently poor for months now. Even 240p is unusable in many cases.
Interestingly, I get consistently the best performance from 360p and 1080 with the rest giving absurdly unusable streaming speeds. Not great, but servicable if i go get some coffee while buffers. I wonder if the different resolutions are being served up from different CDNs?
Anybody try this on fios and find improvements?
(btw my wife gets better streaming performance than i do on youtube from various video streaming sites in asia and i'd bet a dollar i live within 10 miles of a google data center. The google forums are on fire with complaints)
I have noticed some serious throttling on from Comcast for Youtube as of late.
I can tell because directly viewing youtube gets throttled about 2min in (so they pick up on it about 2min in, regardless of where I start watching the video). If I VPN to work I can watch the same video no throttle, and still have bandwidth to spare.
It's the same bottle neck, Comcast, and work is down the road from me on their own dedicated fiber, it's adding 4 hops and it actually runs faster. (and I checked the IPs, I'm connecting to the same youtube servers as far as I can tell)
interesting that using google's dns server isn't going to make things faster. I also noticed youtube was slower for me recently, and I do use google's dns. Let me switch back to my isp's dns and see if it makes any difference.
Another anecdotal "yes" seems to be working much faster for me. Versus skipping, slow load times, random freezes, and, impossible to play any youtubes in HD quality at all. Now, I can stream them. I've had many friends around me complaining about YouTube quality as of late, so interesting thread/discussion and quick-fixes.
I tried adding the ip ranges in article and it rendered youtube inoperable for me. No matter what quality setting I used nothing would stream. Removed the rules and youtube streamed again, albeit slow as ever (still cant stream 720 & 1080)
I can't stand this ISP and I have no alternative available to me here. Talk about living in misery.
I had a similar problem, it looked like it wouldn't stream, but it actually just took a long time to start, once it did, it buffered very very quickly. but that delay was a killer. had to delete the rules after a couple of videos
Given how big an ISP Time Warner is, wouldn't the CDN and/or Google eventually notice the slow responses from their end and start playing a game of cat-and-mouse rotating IP addresses in and out of service as proxies?
The gentleman in the video is Laurier. I sent him a message on FB thanking him for the videos, he is extremely friendly. There is actually a neat story behind the videos. The current playlist is slightly different than the original set. A while back someone paid him to take the original series offline for six months (under the theory that they were too helpful). Laurier took him up on his offer and when he put them back online the playlist was slightly different but still remarkably helpful. I am an order of magnitude faster now when I rock a room. The biggest change for me was using the concrete trowel instead of the normal 4/6/12 inch mud knife. That being said I did pick up a number of tips from all of the videos. The only thing I cannot do is freehand cuts with a tape measure and a blade, I still use the T.
I would suggest watching the entire playlist once before your next project starts and then watch the relevant video before each step as a refresher. I used to dread any project that involved drywall. When I do "habitat for humantiy"-esque projects now I try my hardest to be on the rocking team.
If you are skeptical about watching all the videos my big ah-ha moment came when I understood "mud control." This is a nice example:
I'm on FiOS and have also been suffering through terribly slow speeds with YouTube and Twitch.tv. I found that discussion on Reddit about a week ago, and it did temporarily fix my problem. However, after a day or so I noticed my speeds went back down to a crawl.
I also experienced this temporary speed increase (from barely streaming 240p to getting smooth 1080p) if I switched locales (adding "&gl=CA" to the end of the URL) or switched to HTTPS.
How do I get Time Warner to start working in the first place? I bought a FreedomPop device just to get 4G wifi for the last month waiting for Time Warner to just fix our Internet (already had 3 separate appointments..). But glad to hear I'm going to have YouTube issues regardless..
Noticed this from TWC somewhat recently as well when I realized that I was taking out my phone to watch YouTube videos more frequently (on 3G/4G). Now, I just keep my computers on a VPN as much as possible and there's a number of sites that just feel faster.
This is going to be dependent on your location within the network. I'd never assume advice like this will work just anywhere on the planet, let alone within a large country. It certainly does provide a data point for the problem and a localized solution that can be generalized by those with more expertise. This author admits limited expertise with "Other people can dive into the complexity much better than I ever could, but that’s the overall theme."
The DMCA modernized copyright law for the digital age, creating protections for network providers against infringement by their users, and making illegal circumvention of new digital copyright enforcement schemes. What part of this act do you think applies to throttling network connections, if that's what's happening here?