
JavaScript mp3 decoder allows Firefox to play mp3 without flash - phoboslab
http://jsmad.org/
======
RyanMcGreal
If imitation is the sincerest form of flattery, feature requests must be the
next-sincerest. With that out of the way, I'd love to see:

* Standard playback widgets: volume, position slider, etc.

* A way to link songs, e.g. <http://jsmad.org/play/114578>

Fantastic work!

~~~
romac
I just added the link song feature. The link above should now work ;) Enjoy!

~~~
nddrylliog
For those wondering: romac is too modest. He's this awesome guy I'm building a
startup with, and he's been kind enough to let me host the demo on our shared
VPS zip.cat, and he's been sysadmining really hard for the last 2+ hours to
keep our server from collapsing under the combined weight of the internets :)

------
kellysutton
As a wise Keanu Reeves once said, "Woah."

Is there any upper limit you've found with bitrates, etc.? Also, what was the
most difficult part in building this?

~~~
nddrylliog
Been playing 320kbps Stereo flawlessly afaik. The bitrate is really not much
of an issue.

Browsers have a much harder time dealing with binary XHRs, huge typed arrays,
huge strings, and a real fucking lot of arithmetic operations.

I think the most difficult part was the debugging. Jens Nockert, Matthias
Georgi & I finally used node.js and carefully diff'd the output from minimad.c
(the canonical libmad example) and jsmad.

However, it quickly became obvious that comparing exact values wasn't going to
cut it. Listening was also almost useless (try to debug when all you have is
SCRRRCHHHHHH vs CRRRRRRKKKKKK). So we plotted graphs. With gnuplot. It was a
matrix-like experience but quickly allowed us to see what was wrong.

Profiling the code via Firebug made us realize that most of the time (~28%)
was spent in the synthetizer (synth.js). The I/O code is particularly
expensive too, it's a mix of Uint8Arrays and javascript Strings, depending on
whether you're using the local file version (filestream.js) or the http
streaming version (ajaxstream.js)

One of the funniest part was when I had to code 36-bit precision integer
routines in Javascript such as shifting, OR-ing, AND-ing.. I had to separate
the numbers in a low word and a high word and then work with floats (which
have a 54-bit mantissa in JS).

Really quite an experience!

~~~
halostatue
I suspect that there's a fairly interesting blog post with samples of the
gnuplot output waiting to be written about this.

~~~
nddrylliog
That's a good idea. I think some of the graphs are still on the depo (in
experiments/node/ or something).

------
sandGorgon
Doesnt work for me (no audio): Chrome 12 on Ubuntu 10.10 64-bit Youtube is
playing fine.

Here's the Chrome debugger log:<http://pastebin.com/MwqfD1K5>

~~~
nddrylliog
Yes, as said in the README, the audio output doesn't work on Chrome. I have no
idea why - audiolib.js should support it. Maybe we're just using it wrong?

If the audiolib.js dev is here or anyone else has a clue, it'd be very
welcome.

~~~
kelnos
Really? It's working on Chrome 13 for me, though because of how Chrome slows
down JS execution when you switch to another tab, things can get... funky.

~~~
jensnockert
It's the same thing in Firefox,
<https://bugzilla.mozilla.org/show_bug.cgi?id=665000>

------
nddrylliog
jsmad lead developer here, shoot if you have any questions!

~~~
pja
Does anyone have a good handle on what the GPL means in the context of a
javascript library invoked by an html page rendered by a browser? I certainly
don't!

nb, you say: "libmad has commercial license. As for jsmad, we're in a sort of
grey legal area." I presume you mean that you would have problems with a
commercial licence? Your work is clearly a derivative work of libmad, so it
has to be GPLv2 licensed: any commercial license would have to be by agreement
with the libmad developers I think.

Regardless: very cool stuff!

~~~
jensnockert
Yes, anything under the GPLv2 is obviously fine. But we cannot relicense it
under a commercial/liberal license like libmad could (at least not without
their permission).

~~~
pja
That's my take too.

The remaining question then is just how far the reach of the GPL is in the
context of a javascript library. Just the javascript that actually calls the
code? Everything else called from the same html page? Everything else on the
same site?

It's a shame the history of libmpg123 is so clouded: they've made a good
effort to contact all the previous developers to get permission to LGPL the
library, but they haven't been able to reach everyone so there are always
going to be lingering doubts.

------
nddrylliog
Firefox 4.0+ AND Chrome 13.0+ are supported.

However, Web Audio API doesn't seem enabled in Chrome 13 by default, just go
to 'audio:flags', check "Web Audio", then click the "restart browser" button
and it should work.

~~~
cpearce
How did you find working with Firefox's AudioAPI and Chrome's Web Audio API,
how do they compare? What drawbacks and advantages did you find working with
each of them?

~~~
nddrylliog
Actually, we're using audiolib.js, which abstracts over the Audio Data API and
the Web Audio API, so your question would be better directed to the
audiolib.js author :)

------
hieronymusN
Technical question: You're using goto in a do/while loop here :

[https://github.com/nddrylliog/jsmad/blob/master/src/decoder....](https://github.com/nddrylliog/jsmad/blob/master/src/decoder.js#L86)

How are you pulling this off? I ask because while goto is a reserved word in
JavaScript I didn't think it did anything.

~~~
nddrylliog
Good question :) If you watch the resources/network tab under Chrome/Firebug
you'll notice that decoder.js isn't used in the demo. It's my unfinished
attempt to port libmad's decoder.c, so it still contains a bit of C code, I'm
not even sure it's valid Javascript at this point :)

In the demo, we're replicating some of its functionality in
[https://github.com/nddrylliog/jsmad/blob/master/src/player.j...](https://github.com/nddrylliog/jsmad/blob/master/src/player.js)
instead, although in a less robust way.

------
chops
If the site isn't loading for you (I was getting nginx errors for a bit before
it finally loaded for me), here's the github repo:
<https://github.com/nddrylliog/jsmad>

~~~
nddrylliog
Yes, we're running this on a node.js+nginx VPS and you guys are MURDERING it.

That's great news though! Bear with us as we try to handle the traffic :) And
reload from time to time until it works.

Edit: all static assets are now served by nginx, and the default track is
cached on our server instead of being proxied by node.js all the way. Wish we
had CORS at official.fm!

------
skue
I'm curious whether you came up with the logo after the fact, or whether you
named it after the mag?

I've been surprised at how blatantly some recent open source projects
reference pop culture in ways that are just begging for a takedown order -
cool.io is another. I suppose it may be worth it if you're primarily focused
on getting noticed in the short term (perhaps to get investment or a good job
offer), but long term it seems to guarantee that the project will need to go
through a rename with all the confusion that entails.

Edit: Forgot to say the technology really is amazing.

~~~
jensnockert
The library is named after libmad, which when ported to js became jsmad.

The page linked is just a demo for Music Hackday Barcelona, was not intended
to get _this_ much publicity.

~~~
skue
Ah, makes perfect sense. Thanks and good luck with the project!

~~~
nddrylliog
Yeah, this is not the official logo btw - @antoinem is the one to contact
about the Demo's visual look.

------
netnichols
It'd be neat to have a overview of the different 'modules' in play and what
they do (either in the README or a separate wiki page). Great stuff!

~~~
nddrylliog
We would've loved to have that too when porting the mad code.

Here's a quick overview:

bytestream is our general I/O abstract class - there's much room for
improvement because it works with Strings. substream allows to have a 'view'
of a slice of a bytestream.

filestream and ajaxstream are two implementations of bytestream, doing local
File I/O (with the upcoming W3C File API) and XMLHttpRequest streaming. On
Firefox 4.0+, ajaxstream takes advantage of mozBinaryResponse and uses
Uint8Array(s) to 1) compute the amount of data read, and 2) implement getU8
(which is used a lot).

bit.js is a straight port of libmad's bit manipulation routines. Bit-level I/O
is a must-have in an MP3 decoder because you have numbers stored on 7 bytes,
19 bytes, and other weird stuff like this (especially in the huffman decoding
part).

huffman.js, imdct_s.js, rq_table.js are all tables used somewhere else in the
process.

layer3.js is really the central part. it implements huffman decoding, the
various IMDCT transforms, reordering, requantizing... all sorts of magic
tricks I didn't fully grasp even while porting. Comparing the output was
really our best weapon. The libmad guys are really the reference guys for this
kind of question. We had to strip out all their fixed-point stuff because it's
really of no use in Javascript, since JS only has floating point numbers -
even if we can make them _behave_ almost as integers, it's still as costly as
FP arithmetic so we just went floating point all the way - hence, jsmad
doesn't have 24-bit output like libmad has. libmad is wayyyy better at
everything :)

synth.js is the second central part - it synthesizes the sound waves from the
decoded MPEG Layer-III data. It's the most expensive part of the project,
since it's just thousand meaningless arithmetic operations, something
JavaScript doesn't excel at.

mad.js contains mostly constants (errors), a few utility functions

decoder.js isn't used anywhere, it's my attempt to port decoder.c from libmad,
but it's unfinished.

player.js is our ill-fated attempt to play the decoded MP3 correctly - it's
really naive (no seeking, stutter in some cases, no buffering of decoded
frames, etc.)

~~~
romaniv
Maybe it's a silly question, but how did you output sound after decoding the
stream? I wasn't aware browser JavaScript had any APIs that work with sounds
directly. There is Audio, but IIRC it just streams the file from the server.

I've looked at the sources, but couldn't fine the part that would answer my
question.

~~~
nddrylliog
You're talking about the HTML5 <audio> tag.

We're using audiolib.js, which abstracts over Mozilla's Audio Data API and
Chrome's Web Audio API - both of which allow direct access to the sound device
:)

(And no, it's a really good question! More people should know about this
stuff!)

------
ZeroGravitas
When does mp3 playback fall out of patent protection? I know people keep
digging up new patents for popular standards but assuming Mozilla and like-
minded developers put some effort into patent avoidance you'd think they'd be
able to include native support for some subset of mp3 playback without too
much patent trouble by now.

------
jovrtn
Not sure if anyone has mentioned this yet, but I'm noticing speed/pitch
differences in playback when I load a file from my disk versus playing in
iTunes. Billy Joel sounds more like Janis Joplin.

Edit: Horrible bug report -- I'm in Chrome 12 with the Web Audio about:flag
enabled on Mac OS X 1.6.7

Edit 2: Just tested in Chrome Dev 14 and the issue is unchanged.

~~~
nddrylliog
Haven't noticed this myself. Can you upload an example mp3 file and file a
Github bug? <https://github.com/nddrylliog/jsmad/issues> (Also, I don't have a
'Web Audio' flag in my Chrome 12 - I have in Chrome Canary on Windows though).

------
jensnockert
Oh, and by the way. Please join #jsmad on freenode (IRC) if you want to help
us make the library awesome!

------
tocomment
How does this work? I didn't think JavaScript could read a file on a byte
level.

~~~
nddrylliog
Magnets heh. In JavaScript you have basically two options:

The legacy way is dealing with binary data as an ASCII String - effectively a
string of bytes. This works but it's hella slow - probably because browsers
didn't think we'd be using 67MB strings :)

The new way is with Typed Arrays: using an Uint8Array is pretty awesome, see
the [Mozilla Developer
Docs](<https://developer.mozilla.org/en/JavaScript_typed_arrays>).

Mozilla Firefox since version 4.0 allows to do XMLHttpRequests (xhr) to be
retrieved directly as an ArrayBuffer, which you can then create an Uint8Array
view on. Since Gecko 6.0 (Firefox 6.0), mozResponseByteArray is deprecated and
they've implemented the new standard way: see
[https://developer.mozilla.org/En/XMLHttpRequest/Using_XMLHtt...](https://developer.mozilla.org/En/XMLHttpRequest/Using_XMLHttpRequest#Receiving_binary_data_using_JavaScript_typed_arrays)
for more details.

As for reading local files, the [W3C File API
Draft](<http://dev.w3.org/2006/webapi/FileAPI/>) is implemented in Firefox 4.0
and Chrome 11+ as far as I can tell, and it also allows us to retrieve a
binary string.

------
mise
Awesome! But...

I'm not a JS guy. I can't contextualise this. How does this compare to HTML5
browser-based audio playing? Will this be a new way to make an audio button?
Or does it go much deeper than that for JS apps?

~~~
nddrylliog
From what I hear,the HTML5 <audio> tag implementation quality varies wildly.
It hasn't been used much in production for desktop-targeted web apps. It is,
however, quite popular on the iPlatforms, because you have no Flash there!

As far as I can tell, jsmad is not going to replace the HTML5 audio tag on
mobile devices. However, it does constitute a nice legal workaround for MP3 in
desktop browsers such as Firefox.

It gives access to the full decoded audio data, the code doing the playback
(player.js) is actually just a tiny little part of jsmad! The HTML5 audio tag
seems to also give access to the decoded data, but I haven't messed with it
enough to know what's the level of control.

What would be really interesting though, would be something like streaming
audio applications - completely flash-free, something that would be (imho)
hard to achieve using the HTML5 <audio> tag.

My opinion is that it's wrong to try to add _everything_ as browser built-ins.
If the Javascript are powerful enough, why not use it to decode
PDF/MP3/whatever? I'm guessing we won't see a video decoder for a few months
more, but who knows? Things are moving fast.

So to answer your original question: it goes much deeper. For an audio button
it would probably be overkill, but then again, why not?

~~~
seanalltogether
At 50% cpu utilization this is very impractical outside tech demos.

~~~
jensnockert
Yes, please help optimize it.

~~~
seanalltogether
The question is can it be optimized? My knowledge of this stuff is very
limited but I assume javascript object wrappers for number types and lack of
SSE/SSE2 instructions will be the biggest blockers to performance.

~~~
jensnockert
I doubt SSE2 would help much, at most 2x on arithmetic performance. Much time
is spent fiddling with arrays, and the newer version of firefox the better
that performance is. In addition there are many optimizations to do.

A Pentium 1 can decode MP3s in assembly, so in Javascript we probably need a
few times more power, but not insanely much more.

~~~
nddrylliog
Switching entirely from Strings to typed arrays would help a great deal imho.
Right now the code is hybrid.

------
aolnerd
Thanks to all contributed to this clever hack. MP3 on all browsers, without
flash, is a really worthy goal.

But I'm hearing sputtering, myself. Are you guys going to carry this to a
production-capable system?

~~~
jensnockert
Yes, it is stuttering right now. Especially when it is in the background. We
think this is a buffering issue, but yes. We hope this will be able to go into
production soon(tm).

~~~
aolnerd
FWIW, I'd love to see support for icecast or shoutcast streams.

~~~
nddrylliog
As long as they contain only MP3 files, that would be really really easy to
do. If someone does it on Github, please ping me - we'd love to host it on
jsmad.org

------
floppydisk
Couldn't get it to load in Firefox 4.x, it just says loading. Looks like a
neat idea though, I'll give it a second look once the Slashdot load clears
off.

------
tomrod
Please get on this Grooveshark. I run Linux and your client runs so slowly.

------
Apocryphon
First HTML5, now JS? Looks like everyone's trying to make Adobe obsolete.

~~~
nddrylliog
And I'm sure hoping we succeed! They recently pulled the plug on Adobe Air
Linux support ( [http://blogs.adobe.com/open/2011/06/focusing-on-the-next-
lin...](http://blogs.adobe.com/open/2011/06/focusing-on-the-next-linux-
client.html) ), which only makes them look worse in the eyes of devs like me.

I don't care if the year of the Linux desktop never comes - you just don't
pull the plug one something like that. Adobe is becoming incredibly irrelevant
to the consumer, while still strong to the content producer market - which I
want to know nothing about anyway. So, all the best :)

------
pistoriusp
Would have been great if it worked on mobile Safari!

~~~
jensnockert
When it supports the Webkit Audio APIs, it will work. But it might be too slow
on a phone. Even on a laptop it is a serious CPU hog.

------
evanrmurphy
Hmm... 502 Bad Gateway. Can't wait to see this!

~~~
jensnockert
Sorry, yes. nddrylliog and romac are doing heroic efforts to keep the servers
up. But we got slashdotted.

------
scythe
Can this be made to work in Opera?

~~~
jensnockert
I don't know, I would guess that some sort of Audio API might be keeping it
from running (like in Webkit).

------
bartl
Does nothing on Firefox 3.6.x

~~~
jensnockert
No, maybe we should add something to the page about not working on
oldfox/chrome/ie/etc.

