
Serve Your Music Collection Using HTML5, Backbone, and Flask - zx2c4
http://git.zx2c4.com/zmusic-ng/about/
======
graue
Wow, this is crazy because I'm working on a streaming music player of my own,
also using HTML5, Backbone and Flask. (As well as Flask-SQLAlchemy and
Mutagen, but those are obvious choices given Flask.) Congrats on shipping
first. How long have you been working on this?

I might actually use yours until my take on this is done. It certainly looks
superior to Subsonic in every way.

I'm still going to finish mine, because I have some different UI/design ideas.
For instance, I want a visually-oriented player with cover art very
prominently featured, whereas yours doesn't show art at all.

But on the technical side of things, the concept is rather identical. Have you
tested it on a smartphone browser and does it work/look good? I didn't use
Bootstrap like you, so I'll probably have to create a totally separate mobile
interface.

Anyway, well done!

~~~
mikevm
Wow, so many technologies. How do you people find out (and keep track) of
these frameworks?

~~~
zanny
Read a lot of HN, and as a general principle, whenever you are about to write
a novel application, google for it, because someone already did it and its on
github. Then read a lot of code and framework manuals.

------
blcArmadillo
If you're interested in a program that does this + video check out Subsonic:
<http://www.subsonic.org/>.

~~~
zobzu
I must say, its not nearly as slick as the article's linked, and it uses flash

~~~
laumars
To be fair, the screenshots do not do it justice. The default theme is a lot
more slick than the horrible blue themes in the screen shots.

You also have to bare in mind that Subsonic also has native players for a
whole bucket load of platforms (both mobile and desktop), supports video,
transcodes, is multi user and has a tone of other neat functions.

In many ways, Subsonic is more akin to Spotify.

~~~
zx2c4
From the site:

> Based on Java technology, ...

I stopped reading there. Bias? Maybe.

~~~
laumars
I'd say that was more "narrow minded prejudice" to dismiss an application
you've not researched nor ran simply because of a tired old myth about an
enterprise level language which is in wide scale production use on devices
ranging from 'dumb' feature phones through to enterprise servers.

But really, it's your loss.

~~~
zx2c4
Narrow minded prejudice is probably accurate, yea. Guilty as charged.

------
mqzaidi
Curious as to why you aren't hosting this on github? I would have loved to
star it and then come back later.

~~~
mcovey
your web browser might have a feature similar to the github "star", called a
bookmark.

~~~
mqzaidi
my web browser (and the device I am using changes several times a day - so I
must store my bookmarks in cloud). Of course, I could use a cloud bookmark
service, or cloned the project to github, but as I said, I was just curious :)

------
johnnymonster
Does anyone know anything that can do the same thing for movies? Specifically
the transcoding part. I use plex, but i hate the fact that they add in GA
tracking.

~~~
laumars
As already mentioned in these comments, Subsonic will do that.

------
stoplight
Pretty interesting. What does your uwsgi line look like? Mine keeps failing
with "ImportError: No module named zmusic.login". I'm running:

uwsgi --socket /tmp/uwsgi-zmusic.sock --file /home/user/zmusic-
ng/backend/zmusic -H /home/user/zmusic-ng/venv --callable app --processes 4

~~~
zx2c4
What you want is zmusic:app with the -w switch.

    
    
        -w zmusic:app --socket /tmp/uwsgi-zmusic.sock --chmod-socket=660
    

Using --chdir to change to the backend/ directory may be useful as well.

I've just updated the README with these instructions.

I've also added a section on using uwsgi in standalone mode, as the built-in
debugging server can't handle concurrent requests.

~~~
stoplight
Thanks, that seemed to do the trick!

------
hellerbarde
Amazing. Hat off to you, sir! I've stumbled through a few of such projects and
most fell flat on their collective noses, at the latest when it came to
seeking. zmusic-ng works phenomenally well. Next thing I want to try is over a
slow and laggy connection, but that has to wait for later.

One minor, minor gripe is that it uses a Makefile instead of the canonical
setup.py for building. (did I mention "minor"?) For me personally, it makes up
for this fact by using Flask. (Yes, i am unabashedly a fanboy of that
microframework. _squeee_ )

One question though: What is the [git-root]/server.cfg? it seems not to have
an influence? is it deprecated?

~~~
zx2c4
I'm glad you like the project!

Makefiles. I love Makefiles. I kind of dislike all these bloated new-fangled
build-systems - Rake, Jake, Pip, Pap, Nodifier, Grunt, whatever else is on the
street. A setup.py might be nice, though, for installing system-wide, but
generally speaking, standard GNU Make is wonderful.

server.cfg influences where 'make deploy' and 'make upload' deploy things to.
Check out the bottom section of the README for info.

------
onli
Sadly this link doesn't seem to work anymore. The about-page is broken?

But i had this open in a different browser some minutes ago and it seemed
quite impressive. Especially the encoding on the fly, as that is something I
experimented with on my own mini-project (<https://github.com/onli/music-
streamer>, ruby with sinatra, plain html5-audio). I'll have a look at your
code how you did it, but I wonder if that would work with a low-end homeserver
(i'm targeting a pogoplug). On what hardware did you test that?

~~~
zx2c4
Server was DoS'd. Back up now with a static cache.

I've got it working well on:

    
    
        * Highend Core i7 laptop
        * Atom 1U rackmount server
        * Linode VPS
        * Raspberry pi

~~~
onli
Raspberry pi should be comparable. So it could work. Good to know and thanks,
i'll have a try.

------
ejstronge
This is lovely - thanks a lot!

I listen to a lot of foreign music and have had a hard time populating my
zmusic database. There's a work-around (decode the relevant strings to
Unicode) that seems to have worked but I have to leave for work before I'll be
able to test this fully. I've never submitted a patch before and hope to use
this as an opportunity to learn how to make one.

I have a basic question, however: why did you decide to use tab characters in
your python? Most libs whose code I've read use spaces but I've never seen an
argument for one convention or the other.

~~~
zx2c4
> I listen to a lot of foreign music and have had a hard time populating my
> zmusic database

Oh noes! I too have lots of foreign music in my collection, and it seemed to
work fine. I'm using the tagging framework out of MusicBrainz' Picard
application, so things should be pretty flawless. I wonder what happened...
Maybe the encoding system-wide needs to be set to UTF-8 so Python picks this
up? If you email me more information, I can take a look.

> why did you decide to use tab characters in your python?

I'm partial to Linux-kernel C-style. I like tabs. That's what they're for.
Holy wars though. What else? I use VIM, KDE, and Gentoo. Death to Emacs,
Gnome, and Mandrake! Ahh, preferences...

~~~
dschulz
I ran into this problem as well. It seems that the filesystem paths should not
contain non-ASCII characters. In my case, my music is stored in ~/Música and
scan.py fails on every single audio file it finds.

> ProgrammingError: (ProgrammingError) You must not use 8-bit bytestrings
> unless you use a text_factory that can interpret 8-bit bytestrings (like
> text_factory = str). It is highly recommended that you instead just switch
> your application to Unicode strings. u'SELECT songs.filename AS
> songs_filename, songs.id AS songs_id, songs.title AS songs_title,
> songs.album AS songs_album, songs.artist AS songs_artist, songs.mimetype AS
> songs_mimetype, songs.year AS songs_year, songs.track AS songs_track,
> songs.disc AS songs_disc, songs.lastmodified AS songs_lastmodified,
> songs.filesize AS songs_filesize, songs.length AS songs_length \nFROM songs
> \nWHERE songs.filename = ?'
> ('/home/dschulz/M\xc3\xbasica/Michel_Petrucciani/Blue_Note/06-Trouble.mp3',)

~~~
zx2c4
Unicode issue fixed in the repo.

[http://git.zx2c4.com/zmusic-
ng/commit/?id=297514d6e6c42057ee...](http://git.zx2c4.com/zmusic-
ng/commit/?id=297514d6e6c42057ee769f19a3b9b4ed7127d503)

------
yatsyk
Have you considered to implement subsonic api to support native apps for
iphone and android? I run subsonic server on my home server and it's quite
resource hungry.

~~~
zx2c4
This would be pretty cool. I hadn't considered it. But you say application
support is widespread for their API. If so, it could be a worthwhile one to
implement.

> I run subsonic server on my home server and it's quite resource hungry.

Yea, that's the thing about subsonic -- big bloated app. Lots of features, but
it needs to start fresh and lean.

------
ghayes
Looks really interesting. Do you have a live demo, possibly with public-domain
music? Would love to try it out without yet building it on a server.

~~~
zx2c4
No live demo, unfortunately. I think archive.org has plenty of public-domain
music in ogg format. Want to put something together and I'll link to it in the
README? This would be quite nice indeed to have.

On the plus side though, it's super easy to try it out locally using the
built-in server:

Give backend/app.cfg the path of the music on your computer. Then run
backend/local_server.py. Browse to localhost:5000 in your web browser, and
you're set.

Info here: <http://git.zx2c4.com/zmusic-ng/about/#standalone>

------
feltnerm
Nice! I'm working on something very similar except I'm using Music Player
Daemon as my database. It seemed like I may as well let one of the best do the
work rather than deal with syncing the filesystem and mapping ID3 tags in the
database on my own.

I'm curious as to the performance of your implementation? Does it handle a
large library of music well?

~~~
zx2c4
MPD is indeed nice, but not the intended use case here. I don't want to be
running it on my server or have anything related to audio playback there. They
do indeed index metadata, though I'm not sure what their implementation is
like. If you finish your project, drop me a line though - would be curious.

ZX2C4 Music handles my 100 gigabyte collection without a hitch, and I'm just
using sqlite on my box. If I wanted something more intense, I could run this
with PostgreSQL or MySQL or MariaDB or something a little bit better primed
for performance (zx2c4 music supports multiple database types out of the box).
But, even with sqlite, searches are practically instantaneous.

------
jameswyse
I'm not sure how possible it would be, but I'd love to see something like this
with AirPlay support!

~~~
zx2c4
From the browser, it's a bit tricky, but it would be possible to ask the
server to stream audio to AirPlay speakers. This would likely limit it to the
subnet the server is on, but still could be useful.

Incidentally, I'm somewhat interested in the protocol:
<http://git.zx2c4.com/Airtunes2/about/>

------
grosskur
Nice work! I see it supports X-Accel-Redirect. Do you think it could proxy
music from a remote HTTP server, i.e., an S3 bucket?

[http://kovyrin.net/2010/07/24/nginx-fu-x-accel-redirect-
remo...](http://kovyrin.net/2010/07/24/nginx-fu-x-accel-redirect-remote/)

~~~
zx2c4
Absolutely. Lots of ways to set things up here. You could build support
directly into the app, which isn't very difficult, or even easier, just use
s3fs:

<https://code.google.com/p/s3fs/wiki/FuseOverAmazon>

------
ndesaulniers
Nice! I had written a audio serving app using ffmpeg to compensate for lack of
browser support for various codecs & containers:
<https://github.com/nickdesaulniers/audiostream>

------
nthitz
Great now let's see it for video!

~~~
maratd
This shouldn't be that much more difficult ... but it will probably require
more horsepower for transcoding on the fly.

------
blux
Really nice!

One thing that concerns me though is the CPU usage on the client machine.
chromium uses roughly ~30% CPU on a 6-core AMD Phenom II running only zmusic.
Anyone else seeing this too?

~~~
zx2c4
That's quite astounding, yikes. I have less than 1 percent of CPU usage with
my Chromium. (Using latest dev channel, built from scratch with -march=native
-O2 -fomit-frame-pointers -pipe.)

If there's high CPU usage, file a Chrome bug -- it's possibly do to some
nastiness in their audio decoder.

~~~
blux
Found out that it seems to be the drawing of the progress bar, as weird as
that may sound. When I switch to another tab, the CPU usage drops. I'm on
Linux using the open source ATI radeon driver, that might have something to do
with it.

~~~
zx2c4
Well that's quite interesting... Try and see if it still happens with fglrx.
The progress bar is a CSS3 animation, which means Chromium likely hardware
accelerates it, so I could imagine buggy drivers causing issues.

------
wooptoo
I've been using a very similar player named Zeya to stream music from home to
work: <http://web.psung.name/zeya/>

------
dewey
So basically it's subsonic without all the java ugliness. Sounds great, I'll
give it a shot.

If you are looking for feature requests: I'd love to see a last.fm scrobbler
integrated.

~~~
zx2c4
That's a cool idea. I'll make note of it.

------
prezjordan
How did you deal with multiple simultaneous requests? (two clients streaming
at the same time, two clients encoding at the same time, etc)

~~~
zx2c4
The Linux process scheduler deals with it, I guess. I suppose I could add some
kind of my own limiting mechanism, but since encoding ties up the application
process, I figure it's limited to how uwsgi works with this kind of thing
(multiple child processes, threads, event loops, whatever).

I really wish there was an nginx header like X-Accel-Redirect called X-Accel-
FD where I could pass it a file descriptor to send the data directly, with
SIGPIPE set up fine. That way I wouldn't have to buffer from the application
process.

Wait a second. Here's an idea -- maybe I can just pass a fifo file to X-Accel-
Redirect... I'll give that a whirl. One potential issue here is that it's not
going to behave nicely with the client terminating the connection prematurely
-- it'll leave a zombified process or a filled pipe or both. So maybe this
isn't best then. Any suggestions here? I'd like to not wind up rolling my own
nginx module for this.

That said, the current solution works very well, and in my tests works great
with several clients transcoding. My server is pretty underpowered, but it
doesn't seem to have much of an issue.

~~~
mbell
I wouldn't worry about it. At the scale I could see it being an issue I think
the right answer becomes to simply store all the transcoded versions on disk
and serve them directly. It would in all likelihood be more cost effective to
pay for the storage than to pay for the CPU time to encode everything on the
fly given the small size of audio files.

------
samrat
This looks really great. Silly question: is there any way to access localhost
from other devices on my WLAN?

~~~
zx2c4
No, not localhost. But you can make the built-in debugging server bind to all
IP addresses, including the one used on your WLAN.

In backend/local_server.py, change the line that says:

    
    
        app.run()
    

to instead read:

    
    
        app.run(host='0.0.0.0')
    

You should then be able to access the server at the WLAN IP address of your
machine, provided port 5000 is opened by your firewall.

Even better, however, would be to use uwsgi in standalone mode. I've updated
the README with instructions on this.

------
jamespollack
took a while to get everything up and running -- works nicely now though.

it'd be nice during setup if you used the non-minified includes for all the
scripts, with the minified paths commented out (or vice versa!)

i was trying to change the default search terms and they were stuck in
scripts.min.js

~~~
zx2c4
Yay! Glad you like it.

As mentioned in the setup guide, you can build unminifed by setting the DEBUG
environment variable to 1:

    
    
        zmusic-ng $ make clean
        zmusic-ng $ DEBUG=1 make
    

But, for changing the default search terms, just change the config in
frontend/js/app.js _before_ running make. I could add some fancy frontend
configuration system, but as there is only that one thing to configure at the
moment, this is easy enough.

For more details, all of this is in the README linked above.

Let me know if you have any other questions or feedback. I'm thrilled someone
is already making use of the project.

~~~
jamespollack
it's pretty neat -- with some nice styling and sort options, there's a lot of
potential. but it's a great base. maybe i'll send you a pull request sometime
-- edit to say running this on ec2 server with a shared dropbox as the music
folder could lead to a whole lot of awesome

------
jokull
I’ve done something like this with multi-user support and uploading + tag
edito. Django + NGiNX.

------
capttwinky
This is pretty great. Are you going to be @ pycon or anything? I'll buy you a
beer.

~~~
zx2c4
Thanks! Doubtful I'll be there, but I'll probably be at Hackito Ergo Sum.

------
codegeek
Looks great. Also, kudos for using Flask. I love that framework!!

~~~
zx2c4
Thanks! Glad you like it.

------
jamespollack
what's behind not being able to search while a song is playing or paused? is
that expected behavior?

~~~
zx2c4
You're using the built-in debugging server, which does not handle concurrent
requests. To use something standalone, that's a bit more robust, use uwsgi:

    
    
        zmusic-ng $ uwsgi --chdir backend/ -w zmusic:app --http-socket 0.0.0.0:5000
    

I've updated the README to reflect this suggestion:

<http://git.zx2c4.com/zmusic-ng/about/#standalone>

------
Buzaga
Congrats, this is awesome!

------
naturalethic
Needs upload support.

~~~
zx2c4
<http://en.wikipedia.org/wiki/Rsync>

