
Why YouTube buffers: The secret deals that make and break online video - abraham
http://arstechnica.com/information-technology/2013/07/why-youtube-buffers-the-secret-deals-that-make-and-break-online-video/
======
revelation
This is just one of the many fights surrounding net neutrality.

We desperately need to break up ISP monopolies and enact strict net neutrality
regulation and bandwidth (even latency/routing) guarantees. The idea that a
single ISP is in a position to charge both YouTube _and_ its customers is
bizarre. By any basic reasoning, an ISP that doesn't enable access to YouTube
should be out of customers within a few months. That this doesn't happen is
testament to the broken market surrounding internet access, and a bad sign for
the fights that await us.

(Google needs to start naming and shaming the ISPs that do not want to peer
with them on reasonable conditions or with sufficient capacity when they
detect bad performance on YouTube. They have been doing this locally, e.g.
they delisted french news sites from Google News when these lobbied for
protectionist laws, and they have been stating the parties that forbid them
from showing copyrighted content, e.g. the GEMA in Germany.)

~~~
randomfool
Your ISP wanting money to install caching servers for Netflix and YouTube is
not something which has been covered ny net neutrality in the past. It would
be great if there was a fixed colo fee for caching servers.

That said, I pay my ISP for bandwidth and service. It's bullshit if they
refuse free installation of caching servers which improves my service and
frees up upstream bandwidth for others.

~~~
jlgaddis
FWIW, if Netflix agrees to install one of their appliances in your network,
they do it completely free of charge.

If you don't meet their requirements (which are based upon average aggregate
traffic levels), you can't even _pay_ them to get one.

If your ISP meets the requirements and is refusing, well, it's their own damn
fault -- they're simply wasting money.

Personally, I am happy to peer with _anyone_ with whom I share a presence,
even if they're an "eyeball network" like me and we'd only be passing a few
megabits of traffic.

~~~
toomuchtodo
Looking forward to Netflix Preferred(TM) ISP recommendations.

~~~
Bosence
[http://ispspeedindex.netflix.com/](http://ispspeedindex.netflix.com/)

------
epa
While it may be true that there are some outstanding ISP issues that lead to
slow internet, I think the more pressing issue is the behaviour of the
new(ish) youtube player acting erratic. For instance, you cannot fully load a
video anymore, it will only load a portion as you continue watching. If you
wish to rewind the video, for some reason youtube will want you to rebuffer
the entire from where you have rewinded to. This is terrible when dealing with
slow connections. I have found that since youtube has implemented three
specific things, 1) new player, 2) changing of related videos, 3) google
account requirements, the user experience has gone way down hill.

~~~
Spittie
ViewTube:
[http://userscripts.org/scripts/show/87011](http://userscripts.org/scripts/show/87011)

It replaces the YouTube player with the standard html5 video player. This
alone made my experience on YouTube way better. My connection is slow, so many
times I'll just open a YouTube video in a background tab, and leave it loading
for a while. With this, I can just preload the whole video. And seeking
doesn't make the video reload.

A side bonus, also, is that every video will use the html5 player, so making
flash useless on YouTube (one of the few sites that make me keep flash
installed). It will also remove the ads, so it might not be your thing for
moral/ethical reasons.

~~~
zanny
This thing is amazing, thanks for the script link!

My slow 1.5mbps Internet has had awful youtube experience since the player
change. I have been downloading most of the videos directly to avoid using it.

~~~
chii
how do you download the video? is it a browser plugin? I use chrome, and the
old youtube downloader extension no longer works after their UI revamp (and
the extension is no longer updated as far as i know).

~~~
voltagex_
The best, non-scumware script is: [http://rg3.github.io/youtube-
dl/](http://rg3.github.io/youtube-dl/)

------
rossjudson
Step one here is transparency. Before we impose new mechanisms by legislating,
we should allow detailed study of the ones that are in place. To do that we
may need legislation that states simply: All such peering agreements,
including all of the technical and financial terms, shall be filed as public
documents with the FCC at least 7 days prior to taking effect.

This accomplishes two things: It allows third-party analysis, and it provides
early warning of change to affected parties.

------
x0054
If you have a Mac, try a free program called YouView. Google YouView Mac. It
gives you YouTube experience, but uses the MP4 files YouTube hosts for the
mobile platforms. I never have a problem with YouView. Some times it's slow,
but at least you can download the entire movie before watching. Also, it uses
a lot less resources, 5-10% CPU vs 30%+ if played via browser. This means
better battery life.

I think something like that must exist for windows and Linux, but I could not
find anything free.

~~~
homeomorphic
youtube-dl ([http://rg3.github.io/youtube-dl/](http://rg3.github.io/youtube-
dl/)) is a nice cross-platform solution. It also works with vimeo and a bunch
of other services.

------
Qantourisc
The internet used to be ran by engineers. Now it's ran by suits ...

The former does what needs to be done, and all pitch in to move traffic. Suits
on the other hand only see $$$.

As for the suits: either your are overselling your bandwidth and you don't
have enough money to make the necessary changes REGARDLESS of where the
traffic is going/coming from. Or your just being greedy !

To fight this, I think it's best to get review ISP by their real speed. Then
getting this information out to the public. So people can select the best ISP,
and natural selection of the fittest(fairest ISP) will occur.

~~~
jaynos
Not sure if that last part will work. I can definitely see Comcast allowing
speed test to work to it's fullest and then buffering youtube and other sites
where performance matters.

~~~
estebank
What stops speed tests being done that use real sites for their tests?

I could envision using a random YouTube/Netflix/Hulu video for a test to be
done across several providers at the same time. Add some legal torrent and FTP
servers as you're at it.

That way you get results that can be compared directly. For end user results,
you really don't need to be that thorough when comparing results (after all,
if you're doing at different times on different you are already adding
unhandled variables) you just use _any_ random video from one of those
providers.

------
300bps
The reality is that major ISPs like Verizon and Comcast want Netflix to DIE
because it is a major competitor of theirs. How many people have cut their
cable TV service because Netflix is "good enough?"

So Verizon and Comcast can compete with Netflix, or they can attempt to cut
off their knees.

"Nice streaming service you got there. Be a shame if it was so slow your users
got frustrated and cancelled their service."

This is why Verizon and Comcast don't take Netflix up on their offer to
provide caching servers, create peering agreements and take other actions that
actually decrease the ISP's cost. Because _Verizon and Comcast want Netflix
DEAD_.

~~~
recursive
Good enough? It's better. There are no ads.

~~~
300bps
...yet

~~~
recursive
There are no ads now, which is when I'm using it. When they add them, I'm out.

------
adrr
Content is King. Youtube should be copying the model of local TV stations.
Charge the cable companies for the privilege of accessing content. It
shouldn't be reversed.

~~~
aidenn0
Except it doesn't work that way because nobody who uses Comcast knows that
Comcast charges content providers for peering. If you use Comcast, then you
aren't the customer, you're the product.

------
nivla
So its basically ISPs acting as CDNs, infact the best one money can buy.

Does this mean no one will every be a true competitor to Youtube or match its
performance unless they pay all the local ISPs?

~~~
Qantourisc
ISP's have always delivered content/bytes. They just stopped treating all
bytes equal.

------
shoxty
This peering issue has affected me a lot. I watch a lot of gaming streams
online and during peek hours Twitch.tv has becomes unusable. During a major
tournament this is very unfortunate. Another example of this peering dispute
affecting me is with the game League of Legends.

Almost everyone who is on Comcast (myself included) during peek times
experiences lag that makes the game unplayable. Your character "skates" along
the map and usually when it buffers and catches up to you you'll find yourself
dead. My friends and I got together to play over the weekend and ended up
having to stop due to the lag.

Heres a thread where the guys at Riot are trying to identify the issue. It
wasn't until 7/22 that they identified the problem everyone has been having
was due to a peering dispute between Verizon and other vendors. To fix the
problem they are having to make changes with their provider to route traffic
to bypass problematic junctions.

[http://na.leagueoflegends.com/board/showthread.php?t=3521364](http://na.leagueoflegends.com/board/showthread.php?t=3521364)

Its been very eye opening to me. I have a 50mb connection and I always thought
that as long as I had a faster connection I wouldn't have lag issues. It was
disappointing to learn that regardless of your connection you can still be
throttled by ISPs and their vendors.

~~~
cpncrunch
Can you not change to a better ISP? It's ridiculous to be paying good money
for high speed internet if your ISP can't handle the load. I've been using
shaw.ca for years with zero problems since they upgraded their network to
fibre, and we have two other pretty decent providers to choose from as well.

I'm sure you must have a decent ISP somewhere down in the states. If not,
perhaps you just need to wait for google to do it.

~~~
shoxty
The way I understand it is that my internet traffic is like a little dog that
wants to get from my yard to Netflix's yard to play. To do that, the poor pup
has to cross 6 other yards to get there. Some of the yards are nice and
friendly with pretty flowers to sniff along the way.

Other yards are owned by mean people that don't want to let the dog through
unless it's owner (your ISP) pays them more money. If the owner doesn't pay
more money, the mean yard owners will make the pup sit there and wait for a
long time until they say it's okay to go (LAG).

And sometimes, the mean owner have so many dogs waiting in his yard, he
decides to throw them into the street, where they are never seen nor heard
from again (packet loss).

Most people could definitely change ISPs and avoid the troublesome vendors
their ISP is currently peered with but I live in a town (low population) and
unfortunately Comcast is about as good as it gets.

One way to get around it is to route my traffic through a proxy to bypass the
troublesome vendors but that would degrade my bitrate which is frustrating
since what I really want is to just get what I am paying for 50/mb and wish
the internet providers would just work it out.

~~~
cpncrunch
No, that's not (always) true. My ISP's traffic goes directly from their
network to Google.

According to the article Comcast are trying to charge Google for letting
Comcast's customers watch youtube, which is a weird and evil way to do
business. I think it basically boils down to Comcast being a really shitty ISP
and you (unfortunately) not having much choice.

The irony here is that Comcast is a 10x larger company than Shaw, so perhaps
it's just a case of Comcast throwing their weight around and basically
resorting to extortion.

~~~
shoxty
:(

------
jimmaswell
I prefer to download videos with an extension and watch the file as it
downloads in VLC instead, for longer videos.

------
leke
I thought this idea was actually quite good, although sometimes the buffering
issue is there. About 50% of videos I've watched, I've tuned of within 1
minute because it wasn't what I expected. The old system would have buffered
about 10 minutes worth by then. The new system only buffers about 15 seconds.
This is a huge bandwidth saving when you think of it on a mass scale.

------
incision
Just the other day I had a non-technical friend ask me if upgrading his 50Mb
connection to 75 as suggested by Verizon would get rid of his YouTube
buffering problems.

I wonder how prevalent and successful such fraudulent upselling is?

------
GhotiFish
Youtube has a very poor connection around my area in Canada. Video connections
are unreliable, unexpectedly dropping or just being slow. Slower than the Kbps
of the video I'm trying to watch.

For youtube's client side player this is completely intolerable. Which is why
you don't depend on youtube's clientside player for anything. Use SMplayers
downloader, or youtube-dl, or videodownloadhelper. These make use of more
dependable protocols for downloading the video, and they will download 100% of
the video.

You can buffer several videos up and watch effectively.

My user experience is so much better than what people experience through
official channels. How can that be? I hacked my user experience together with
bits of string and duct tape. Why should my experience be so miserable on
official channels when so many manhours have supposedly went into making that
experience the best?

Should you really be losing out to bits of string and duct tape?

What the ISP's are doing to youtube isn't fair, but the ISP's are clearly not
the only ones at fault.

~~~
gregsadetsky
What area? What ISP? Let's start naming and shaming.. ;)

I had a pathetic experience with an ISP "reselling" Bell's DSL service. The
ISP's name is Distributel. YouTube would not, at "peak times" (6pm - 9pm on
weekdays, most times on weekends), work at 1080. The download speed forced you
to wait for the entire clip to pre-buffer before it would be possible to play
it.

What was frustrating was that opening an SSH proxy to a US server (hosted at
Linode) and going through that proxy would allow you to watch 1080 clips with
plenty of pre-buffering headroom. I tried to explain this to tech support
(with bandwidth measurements, etc) to no avail.

I finally switched over to TekSavvy who have always had their shit down. They
truly rock (the tech support is amazing).

~~~
GhotiFish
Shaw. lower mainland. Also I can't get out of using shaw, Telus isn't
competing with them, and there are no other lines into our complex. I am stuck
with them, I have no choice. I want to leave.

I wish I had TekSavvy. I really do. Next time I move I'm checking which
telecoms I have access to. If it's only shaw or telus, I'm demanding
discounts.

------
baddox
You should try to stream live content from twitch.tv. Pretty much everyone
complains about how bad their service is, and twitch has been very clear that
they're doing everything they can but that it mostly comes down to these deals
with the big Internet infrastructure guys.

------
bane
Youtube was almost unwatchable on my 50/30 FiOS till I did this
[http://mitchribar.com/2013/02/how-to-stop-youtube-sucking-
wi...](http://mitchribar.com/2013/02/how-to-stop-youtube-sucking-windows-
guide/). This completely transformed my youtube experience to one that's fine
compared to waiting 10 minutes for a 1 minute 240p video to buffer enough I
could watch it all the way through.

Now the only problems I have are with the semi-broken new player, fast
forwards and rewinds are often broken, full screen doesn't always full screen
etc.

------
IgorPartola
As a temporary workaround I access YouTube over an IPv6 tunnel through
Hurricane Electric. Since I set it up almost two years ago I have had no
issues with YouTube. The only downside has been the IRS's quarterly tax site
which seems to be misconfigured for IPv6 access but works if r over IPv4
(/etc/hosts FTW).

~~~
drbawb
Interesting, thanks for the idea. I've just finished setting up an IPv6tunnel
w/ sixxs.net, so I'll have to see if IPv6 connectivity provides a better
YouTube experience on my network.

YouTube usually works quite well on my network (TimeWarner Cable in SE
Wisconsin, US), but occasionally I'll be completely unable to receive HD
streams of certain content, so I have to select 480/320p to buffer the video
at all.

------
osth
Sounds like mirroring and ftp servers (or even bittorrent) would work just
fine for distributing copyright-cleared video. Indeed that's how I remember it
being done before YouTube and Netflix existed.

Today, with the explosion of online video, the copyright-clearance step could
be administered by companies (as it already is, e.g., YouTube), but the
servers providing distribution to the users at the network edge do not have to
be run by companies.

1\. Recall that storage is quite inexpensive and users are today quite capable
of providing their own at home or on-the-go storage for terabytes or gigabytes
of video.

2\. Recall the "content-centric" networking idea Van Jacobsen has presented to
Googlers. Does it really matter where the user gets the content? No. What is
important is that it is authentic (and copyright-cleared).

------
codecool
Here I was cursing my ISP for ridiculous connection but I should have been
cursing them for entirely different reasons! What is the use of taking costly
high speed plans when the sites that require high speed (youtube, other video
streaming) are still going to be slow?

------
umsm
There was a similar discussion on HN a few weeks ago regarding a similar issue
about ISPs buffering the CDN (I think).

I dug up a link that I saved:

[http://mitchribar.com/2013/02/time-warner-cable-sucks-for-
yo...](http://mitchribar.com/2013/02/time-warner-cable-sucks-for-youtube-
twitchtv/)

Here is the fix:

    
    
      sudo ipfw add reject src-ip 173.194.55.0/24 in
      sudo ipfw add reject src-ip 206.111.0.0/16 in

------
Oculus
This article just brought a whole new meaning to Google Fibre. I think Google
is seriously considering becoming an ISP which would make a lot of sense
seeing as how they make up 25% of North America's traffic already[1].

1 - [http://www.wired.com/wiredenterprise/2013/07/google-
internet...](http://www.wired.com/wiredenterprise/2013/07/google-internet-
traffic/)

------
marquis
I watch
[http://www.internethealthreport.com](http://www.internethealthreport.com) out
of interest - it shows just how clearly the peering is skewed for US tier 1s.

------
rgbrgb
Given this discussion, what's the best resource for choosing a new ISP? I'm
moving to SF in a week and I want to get a really nice connection for my
apartment?

------
Dewie
I remember the glory days when YouTube would buffer the whole video from where
you started, at a very uniform speed. If you wanted to jump to somewhere in
the video and it was buffered, there was no rebuffering.

ISP's or not, it is infuriating for me as a user to see a website get
_significantly worse_ as it gets bigger and bigger.

