
JSMpeg – Decode It Like It's 1999 - phoboslab
http://jsmpeg.com/
======
cs2818
The "Why Use JSMPEG" section hits the nail on the head with respect to the
needs of some projects I have worked on.

Particularly, I once needed to stream video from a UAV to multiple tablets and
phones on a local network to allow collaborative annotations. Since it was a
disaster response system there was no depending on external services, which at
the time put WebRTC out of the picture (all of the easy to use implementations
required internet access to use existing signaling services).

We ended up using MJPEG and then later a JavaScript implementation of a MPEG-1
decoder. This library certainly would have made my life a little easier at the
time!

~~~
donatj
MJPEG is so freaking simple [1] and almost no one knows about it. It's one of
my favorite things to play around with and in my opinion way underutilized.

I built "drawing board" a while ago using an old school image map streaming a
motion jpeg. You can interact with the video without ANY JavaScript [2]. The
browser stays where you are on posting because the server returns an HTTP 204
- also highly underutilized in my opinion.

1\. [https://github.com/donatj/mjpeg-
php/blob/master/mjpeg.php](https://github.com/donatj/mjpeg-
php/blob/master/mjpeg.php)

2\. [https://github.com/donatj/imgboard](https://github.com/donatj/imgboard)

~~~
niftich
MJPEG in casual parlance is more of a "technique" than a format -- and
multipart/x-mixed-replace is a very loose definition of even something as
loosely defined as MJPEG.

Out of video containers, QuickTime introduced "Motion-JPEG" [1] with each
frame in the container being a JPEG, and there have been formal definitions of
how to communicate "JPEG-compressed video" with network protocols, like in RTP
[2].

Meanwhile mixed-replace was a Netscape trick [3] to do server push with HTTP
that never became a real force but was reimplemented by almost every browser
trying to stay Mozilla-compatible (except IE), and happily used by webcams for
dead-simple videostreaming.

[1]
[https://developer.apple.com/standards/qtff-2001.pdf](https://developer.apple.com/standards/qtff-2001.pdf)
[2] [https://tools.ietf.org/html/rfc2435](https://tools.ietf.org/html/rfc2435)
[3]
[https://web.archive.org/web/19981203153836/fishcam.netscape....](https://web.archive.org/web/19981203153836/fishcam.netscape.com/assist/net_sites/pushpull.html)

------
peterhajas
Video decoding in JS is very impressive - really highlights the speed of
modern interpreters. I especially love that a Björk track from the 90's is
featured.

I recently worked on a personal project which had to play back .webm files,
and I used a similar utility:

[https://github.com/brion/ogv.js/](https://github.com/brion/ogv.js/)

It decodes .webm files and plays them in the web. I believe it's also used by
Wikipedia to play bag Ogg files.

~~~
flavioribeiro
here is a nice talk about ogv.js:
[https://www.twitch.tv/videos/94956075](https://www.twitch.tv/videos/94956075)

------
stepik777
Yet another people who cannot correctly decode video. Videos are not in sRGB
color space and must be converted to it for displaying in browser. Also videos
do not use full range 0-255, instead they use narrow range 15-235 where color
(15, 15, 15) means black and (235, 235, 235) means white; this should be
converted to the full range which is used in web-canvas.

~~~
keyle
It's great that you bring more information to the table, (I didn't know about
the narrow range myself) but you don't have to be condescending about it.

~~~
brokenmachine
I'm impressed by someone being able to write their own decoder, wrong color
range or not.

------
Tade0
I remember toying with this idea when I was doing some web-based slot-machine
mobile games.

We had a ridiculous amount of assets(mostly animations) that had to be
compressed, because one of the sales representatives noticed, that it's
impossible to play any of the games if you're connected to a 2G network.

Eventually we didn't go with this solution, because it considerably reduced
battery life and made the devices heat up too much.

------
chadnickbok
This is really cool! But the compression quality of MPEG1 is really nowhere
close to h264.

If you're interested in this sorta thing, try taking a look at Broadway JS:
[https://github.com/mbebenita/Broadway](https://github.com/mbebenita/Broadway)

Here's a simple demo page they have setup:
[http://mbebenita.github.io/Broadway/foxDemo.html](http://mbebenita.github.io/Broadway/foxDemo.html)

Its entirely possible to use WebSockets to stream H264 to a browser and decode
using broadway, and the performance is pretty good, even on mobile.

~~~
wmf
Don't browsers have built-in native H.264 support?

~~~
colek42
Media source extensions don't give you the low-level control needed for low-
latency and streams need to be re-muxed into fMP4. Web-RTC is a good option,
but the infrastructure required can be a non-starter for many projects. For
example, native webRTC libraries for golang are currently lacking to support
needed to stream low-latency video.

------
Mizza
This has also opened up some awesome new browser experiments, such as playing
GTAV in the browser (on an iPhone!)

[http://phoboslab.org/log/2015/07/play-gta-v-in-your-
browser-...](http://phoboslab.org/log/2015/07/play-gta-v-in-your-browser-sort-
of)

------
mborch
Cisco's got a royalty-free codec that would let you decode like it's 2017.
[https://github.com/cisco/thor](https://github.com/cisco/thor)

~~~
DiabloD3
Actually, they don't.

Thor has been merged with Xiph's Dalaa into IETF's NETVC effort, and both
Cisco and Xiph are backing this.

NETVC is a next generation codec designed to replace H264, H265, and all
future MPEG codecs with a system that is not user- and developer-hostile wrt
licensing lock-in via (possibly invalid) patents.

------
yeureka
This is very cool! In 1999 my degree final year project was to implement an
mpeg decoder in software. My only source of information was the MPEG technical
reference manuals. It took me 3 months to be able to decode my first frame. It
ran at less than 10fps on an AMD K6, but I learn a lot about video and
compression.

~~~
w0utert
I did exactly the same, but instead purely for fun and learning. Picked a
random/blind clip from a naughty movie for extra motivation to get the first
frames to the screen :D

I also worked from just the reference book, with no prior knowledge of video
coding at all, which made it quite a puzzle to get something on the screen and
moving, but it was extremely satisfying when it all worked (to some extent,
the thing was horribly slow, broke after one group of predicted frames, and I
never implemented chroma, just luma)

------
rucas__
The talk about JSMpeg is an awesome learning experience.
[https://fronteers.nl/congres/2015/sessions/jsmpeg-by-
dominic...](https://fronteers.nl/congres/2015/sessions/jsmpeg-by-dominic-
szablewski)

------
shabbyrobe
Watch out! Some of this code may be GPL-encumbered. The decoder for JSMpeg is
based in part on "Java MPEG-1 Video Decoder and Player" [1], which is licensed
under the GPLv2. I am not a lawyer, but an argument could be made that the
sections of JSMpeg's decoder that are directly ported from that project are a
derivative work.

    
    
      [1]: https://sourceforge.net/projects/javampeg1video/

------
eriknstr
I notice that the demo video stops when I switch tabs. Is this by design or by
accident? Can I use JSMpeg but have it play in the background? Another thing;
can I have video controls?

PS: That video has a very late nineties, early double-ohs feel to it indeed.
Good choice of video :)

~~~
stilldavid
Yeah. From the docs:

> pauseWhenHidden – whether to pause playback when the tab is inactive.
> Default true. Note that browsers usually throttle JS in inactive tabs
> anyway.

~~~
tyingq
Supposedly, Chrome doesn't throttle background tabs where audio is playing.

~~~
koolba
JS hack idea! Play a silent sound to force Chrome to not throttle your page
when it's not primary.

~~~
nix0n
Please don't.

Also, Chrome puts a little speaker icon on the tab, so users might notice.

~~~
recursive
In the long run, the probability of this being exploited is approximately
100%. I'm surprised they left in such an obvious loophole.

~~~
daurnimator
I'm not sure how usurping this is an 'exploit'. It's used as a heuristic to
save battery life.

and if you're trying to run down a user's battery I can think of better ways.

~~~
recursive
The goal isn't to run down the users' battery. The goal is to do whatever
timer callback processing a tab is doing that got throttled in the first
place.

------
VeejayRampay
Combine this with webtorrent and you have yourself a real nice distribution
platform.

------
bagels
I don't know where they get the "5 seconds" of latency for dash videos.
There's definitely a startup latency, but it's not 5 seconds, and can be
avoided with some simple techniques.

~~~
londons_explore
They're thinking of live dash streams. Eg. how many seconds from video camera
to screen.

I can see it would probably be possible to get low latency, but without a
fancy server stalling connections till frames become available, flushing on
frame boundaries, etc, I can't see it working. Try doing such things with a
CDN like cloudflare to support lots of users...

Also, for true streaming, timing should be done by the _camera_. Ie. If the
cameras frame rate is 0.0001% slower than the 60fps advertised, the display
device should slow down to match.

MPEG-DASH has no ability to do this, and would lead to a "buffering" gap for a
second every few hours of playback time.

~~~
bagels
Segment length can be set to 1 second, if you're brave. 5 seconds for live
streams just happens to be the most commonly reached compromise.

------
flukus
Advertisers will use this to get around video blocking in 3...2...1...

------
dalanmiller
It is incredibly hard to stream video from say, a Raspberry Pi to the web in a
way/format that's easy to consume on multiple devices or using just the
browser. This is awesome.

------
Retr0spectrum
This begs the question, which codec is optimal in terms of "cpu-cycles-to-
decode per second" for a given image quality ( for some subjective measure of
quality)?

~~~
ryandrake
Offtopic grammar peeve: You probably mean "This _raises_ the question".
Begging the question is rather different.

~~~
aninhumer
"Begs the question" is widely understood to mean the same thing as "raises the
question" in common usage, and complaining about it is one of the sillier
prescriptivist hills to die on.

The older meaning is obscure, largely redundant, and doesn't really make any
sense etymologically, so it's not really surprising that the newer meaning
caught on.

(And since we're being pedantic, it's got nothing to do with grammar.)

~~~
drinkjuice
> In modern vernacular usage, "to beg the question" is frequently[citation
> needed] used to mean "to invite the question" (as in "This begs the question
> of whether...") or "to dodge a question"

That's just useless. And you know, saying "it once was A, now it's B, so it
can never be A again" is exactly as "prescriptivist".

Little things add up, and before you know it people are just stringing words
together as demonstrated on a million youtube videos.

> The older meaning is obscure, largely redundant

How so? How is the "new one" (which one? heh) not redundant? If you want to
say something raises a question, that's an easy way to put it right there. On
the other hand, I'm not even convinced that the bastardization of "begs the
question" into "raises the question" wasn't simply based on not even
understanding what assuming an initial point even could be, of just hearing
the phrase without understanding the context and using it as _another_ way to
say something or someone raises a question. I certainly don't hear it in
common usage, regardless of the phrasing used. You know, if all those other
people say they just say it because "most" people do, then none of them
actually do have a reason. A billion times zero is zero.

And don't even get me started on people suddenly calling low framerates "lag"
:P It just destroys information, and you can call it progress because the
hands on the clock moved a little, but I won't.

~~~
aninhumer
>saying "it once was A, now it's B, so it can never be A again" is exactly as
"prescriptivist".

Except I'm not telling you what you _can_ say, I'm telling you what other
people _do_ say, which is descriptivist. You can use meaning A if you want to,
but don't expect people to understand you.

>You know, if all those other people say they just say it because "most"
people do, then none of them actually do have a reason.

That's like saying no one in the US has a reason to speak English, they're
doing it just because everyone else does.

Language is about shared understanding, so most people around you using a
particular meaning is pretty much the _only_ reason for someone to use it.

>you can call it progress

I don't call it progress, just change.

------
colek42
You can actually do this with H264 as well. I did it for a hack-a-thon a
couple months back. We are actually looking at doing this in production for a
video streaming on an internal network now that NPAPI is deprecated.

[https://github.com/colek42/streamingDemo](https://github.com/colek42/streamingDemo)

~~~
Ezhik
Beware of patents.

------
tambourine_man
_A stack that you can understand from top to bottom._

Love it

------
faljse
Awesome work. I´m using it for an e-learning/robotics platform for low latency
live streaming where it does a great job.

I´d like to know how reverse playback could be achieved. Is there a way to
"undo" a P-Frame calculation, or is a second (reverse) mpeg file a possible
solution?

------
markwillis82
I used JSMPEG to process a live 360 camera feed and then a canvas to convert
the output to the format to work with a google cardboard.

I was pleasantly surprised with the quality of the output and speed of which
the devices could decode and render the new output.

------
edwinyzh
What does "Decode It Like It's 1999" mean? Thanks.

~~~
guyzero
It's a reference to the Prince song "1999" and the fact that it's using MPEG 1
video encoding that was available in 1999.

------
brian_herman
sounds like the internet archive would love something like this.

~~~
sp332
VLC is working on a JS port.
[https://news.ycombinator.com/item?id=13113529](https://news.ycombinator.com/item?id=13113529)

------
ilurkedhere
How power efficient is this? Just curious because iOS was mentioned as a
target.

~~~
wmf
Not at all.

------
z3t4
Imagine all cool stuff you can build with this. And it's JavaScript!!!

------
adrianN
This stutters a bit on my 2009 Macbook. If I weren't plugged in it'd probably
kill my battery really fast.

It's a nice hack to get this kind of video on the iPhone, but somehow I feel
that for decoding video JS is a bit high on the stack.

------
streamer45
main problem I see are bandwidth requirements to keep a decent quality. That
said, very interesting project and well executed.

