
Stanford Researchers Launch Free TV Service to Improve Live Streaming Using AI - fomopop
https://thestreamable.com/guides/live-tv-streaming/news/exclusive-stanford-researchers-launch-free-streaming-service-to-improve-video-streaming-algorithms
======
keithwinstein
Hi folks -- study PI here. Happy to answer any questions.

We're working on a paper for submission to a research conference soon, so
every viewer gives us helpful data for the system to learn in-situ and improve
its performance over different kinds of Internet connections and congestion
situations. (We are randomizing connections to get either our in-development
algorithm or a bunch of competing ABR and congestion-control schemes so we can
properly evaluate, so if you get a stall or bad quality...... we'll blame
that.)

All the source code is at
[https://github.com/StanfordSNR/puffer](https://github.com/StanfordSNR/puffer)
and is or will be released with an open-source license. The study is led by my
doctoral student Francis Yan, along with Sadjad Fouladi, Hudson Ayers, Chenzhi
Zhu, and my colleague Philip Levis. The goal is to train video-streaming
algorithms (bitrate selection and congestion control) online, continually, in
situ, to minimize stalls and deliver the best picture quality across a diverse
range of real-world Internet connections and users. This is from some of the
same people who brought you Salsify (functional video compression for lower-
latency real-time video:
[https://snr.stanford.edu/salsify](https://snr.stanford.edu/salsify)), Lepton
(distributed/parallel JPEG compression:
[https://github.com/dropbox/lepton](https://github.com/dropbox/lepton)), Mosh
(functional terminal emulation for mobility:
[https://mosh.org](https://mosh.org)), Remy (machine learning for congestion
control; [http://mit.edu/remy](http://mit.edu/remy)), and the Pantheon of
Congestion Control
([https://pantheon.stanford.edu](https://pantheon.stanford.edu)).

There are some unusual details of the Puffer system to make experimentation
easier (and, we hope, performance better): the ABR algorithm is server-side,
and video is streamed continuously over a WebSocket and the client gives
asynchronous feedback, instead of using DASH HTTP request/replies. The
tcp_info struct (including the delivery rate estimate) from the congestion-
control layer is plumbed through to the ABR, and to the neural network that
tries to predict "how long will it take to send a video chunk of length y," so
it's weakly cross-layer. The "transmission-time predictor" is probabilistic
instead of simply producing a point estimate. We encode with libx264 and then
measure the SSIM of every encoded chunk to calculate a reward for each
possible stream (instead of making the assumption that more bitrate is better
-- the correlation is not so good when you consider different times in the
same video). And we encode 10 variants of each channel, so more than typical.
And everything is done at 60 fps. Of course some of these details mean these
algorithms are not going to be deployed any time soon on a production CDN or
streaming service, but the hope is to demonstrate the value of some of the
ideas in an academic setting for future commercial deployment. The use of
WebSockets, MSE, and Opus is why this doesn't work on Safari or on iOS.

FAQ here: [https://puffer.stanford.edu/faq/](https://puffer.stanford.edu/faq/)

Also I'm happy we were able to make some of the Grafana monitoring public:
[https://puffer.stanford.edu/monitoring](https://puffer.stanford.edu/monitoring)

~~~
daveevad
My apologies that this question is likely tangential to the main objective of
the research, but will this project be open sourcing the signal acquisition,
transcoding, and delivery portion of its research as well?

This part of this project alone is good enough to improve, dare I claim
disrupt, enormous monied interests in the global media landscape.

If this whole thing is legal (500 simultaneous peers), and can be reduced to a
docker image and some rabbit ears, you could probably improve the lives of
millions (60?) of Americans that do not have access to local stations.

Also, the claim of "not going to be deployed any time soon on a production CDN
or streaming service", I would like to make a friendly wager that you are
mistaken. I won't even demand favorable odds because of my disadvantaged
intellect. This is impressive and valuable work.

My applause.

~~~
keithwinstein
Thank you for your kind words! The students worked really hard to get all
those details right. And yes, all that stuff is at the GitHub repo
([https://github.com/StanfordSNR/puffer](https://github.com/StanfordSNR/puffer)).

------
abhiminator
Streaming it in another browser window as a part of my Mechanical Turk HIT
assignment as I type, and I have to say -- I'm blown away by the incredible
picture quality and buffer-free performance.

Puffer team has done a great job.

As a sidenote, I just realized how annoying cable TV ads are. The content to
advertising ratio on cable is insanely bad. This 'Puffer' experience just
validated my decision to stay away from cable TV.

~~~
ehsankia
Switching channel takes less than a quarter of a second, this is pretty
impressive.

~~~
ian0
Indeed - it's very well done.

I think what makes it so smooth is something quite simple. Its letting the
previous stream run while it buffers the new one and switches you over?

~~~
rasz
That would still make you wait for the switch. Switching channels on digital
TV is slow due to key frames - you need to wait until one full Intra-frame of
the new channel has been decoded. Here client can simply request Intra-frame
on demand and receive one immediately.

------
daveevad
Anyone with legal knowledge about how services like this and
[https://www.locast.org/](https://www.locast.org/) are able to operate?

It doesn't seem significantly different to me than the aereo tv situation.

Very cool project!

~~~
fomopop
In the case of Locast, part of the Aereo ruling was that it was a for profit
service. Locast is able to do this because they are not charging the end user.

------
phillipseamore
This is very impressive. FYI: Delaying the buffer spinner, not showing it
unless buffering has been (let's say) >300ms for instance, makes the switching
feel a lot faster. If you could also get rid of the black between switching
out the media source buffer, they would feel instant.

------
milankragujevic
I'm going to be experimenting with the source code, I really need some good
basis for live streaming TV over the internet (OTT). Unrelated, but important
for me, TV stations in my country (Serbia), forbid rebroadcasting but they are
available OTA, with a DVB-T2 receiver and an aerial/antenna. However, some
cable providers cannot show the programs, and if they do show them, they have
negotiated special licenses for that. How do I make a case that by being OTA
Free, and not DRM'd, I, by not changing the contents of the broadcast, should
be allowed to rebroadcast to people for no financial gain to myself or the TV
stations (other than the ads the TV stations put inside of the broadcast
content) (all to serve as an experiment)?

------
sand500
This is a pretty cool idea

Note, you still have to deal with TV ads and there is no scrubbing. I wonder
if you could make your own client to this and get DVR functionality? Actually
since they are just grabbing over-the-air signals, I wonder if there is a SDR
project for this.

------
bane
This is actually faster at channel switching than my home fiber TV service.

------
voltagex_
It's quite nice, but it's definitely going to get a DRM layer and geoblocking
applied to it at some stage.

~~~
wmf
More likely they'll never expand beyond 500 "experimental" accounts and shut
it down once their research is finished. I don't intend this as a criticism of
the researchers; the more academic and experimental you make something look
the less likely you will be threatened/sued into compliance.

------
homero
The quality and speed is incredible

------
voltagex_
Edit: bitrate and resolution varies - I've seen bitrates as high as 20
megabit/s.

------
masonic
Is closed captioning supported?

------
compsciphd
wondering what the odds are they get a takedown from the NFL during the super
bowl or preemptively won't show CBS during that time period?

