
Sad but true: Napster '99 still smokes Spotify 2012 - borism
http://www.theregister.co.uk/2012/03/17/parker_fanning_on_napster/
======
Aloisius
It is really hard to compare Napster and Spotify as they are entirely
different beasts. Let me just say that I built the Napster server, so I'm
biased. I have also been a Spotify user for several years.

For me, Spotify is a sub par experience because I don't know what I want and I
have different music tastes than my friends. The only thing that makes Spotify
useful for me is sites like <http://sharemyplaylists.com/>. I imagine for
people who know exactly what they want and just treat it like a giant jukebox,
it is a fantastic service. I am not one of those people.

Napster for me, was about finding people as much as music. I was part of a
community of people that were super stoked to talk about some new indie band.
I would search, notice who had music I was looking for, send them a message
and start talking.

I met a lot of people in real life through Napster often at shows. I know
others did as well since we received thank you letters and in a couple cases,
wedding announcements.

I think the community with its passion for music was what made Napster great,
not the massive catalog. All I had to do was enter one of many indie channels
and read for half a minute before I had three new things to listen to and a
bunch of people to talk to them about.

Not to discount the catalog, I will say that the catalog at Napster will
possibly never be duplicated. There were a lot of back catalog works, a lot of
pre-release works and a ton of bootlegs. Sure, the quality was sometimes poor,
but then I used Napster as a tasting service and then, you know, bought the
originals when I could.

My hope is Spotify continues to improve and eventually becomes more community
oriented or something else comes along and re-ignites the flame. Music is one
of those things that is central to a lot of people's lives and you couldn't
ask for a more passionate userbase.

~~~
Mizza
Woah, you wrote the server for Napster?! That's awesome!

Can you talk about that a bit? What was the underlying algorithm, what was the
stack and how much data were you pushing at your max?

~~~
Aloisius
Which underlying algorithm?

The search engine was built on a ternary tree with a custom merging algorithm.
I honestly don't know if the merging algorithm has a name as it was something
I came up with (literally) while sleeping one night. Because we mostly used
ID3 tags and file names, it was completely unnecessary to use a stemming
algorithm because if you typed in a misspelled search, given the size of our
index, there was probably someone who tagged their file using the same
misspelling.

The network biasing code used BGP data combined from a number of looking glass
servers to build a map of ip/prefix -> ASN number and ASN->ASN distances. It
was then used to reorder search results based on network distance to users so
they would bias towards their own networks and save ISPs money and speed
transfers on broadband connections.

Servers were linked through a fully meshed network. Each had presence
information about every user on the network so that they could route IMs
around. The chat system was semi-linked (fully linked on some servers, but we
couldn't fully link the whole thing because the client had no administrative
functions for chat). If we couldn't send the user back enough results for a
search, the query was simply passed around the backend.

The whole thing was written in C++. At its peak, there were about 2.3 million
users online at any given time (80 million total users growing by a million
every 4 days). The system would be indexing about 17.6 million files per
second (and de-indexing about the same amount). The whole system pushed out
about 2 Gbps of bandwidth in search results (which were tiny).

Napster was one of the very first services to push past 10K connections on a
Linux machine. At peak, I could get over 100K users on a single process
(though I'd run out of memory indexing files on the tiny 2 GB machines and
blow out the NIC sending search results). During normal operations, each
server process had around 40K users on it and between 7-12 million files
indexed.

There were a bunch of side infrastructure things no one saw. Court mandated
copyright filtering systems, recommender systems (most for play), load
balancing servers, bot detection and sequestration systems, analytics
reporting jobs, etc.

Nowadays, I could probably fit all of Napster on one big machine. Heh.

~~~
Mizza
Wow, that's incredibly fascinating! Thanks for posting such a great reply! You
built something that really changed the world, in my opinion, and you should
be very proud.

I'm particularly pleased by the use of ASN/distancing weights in your results!
None of the BT trackers I've hacked around on have had anything like that in
them.

Where is the code now? It should be in a museum.

~~~
Aloisius
No problem! It occurred to me that I've never really talked about the Napster
architecture or algorithms publicly.

I once hacked up Transmission to re-prioritize based on network distance. It
worked rather well when it discovered over the DHT. It is a lot easier when
you only have to calculate distance between you and other networks. Storing
the graph of distances between two arbitrary users is harder.

The code was part of the assets that were bought out of bankruptcy by Roxio
(who renamed themselves Napster). I doubt it is being used for anything.

~~~
tuna
you should definitely publish this kind of things somewhere. it was big way
before the whole big data stuff around.

cheers.

------
brianwhitman
I disagree with this -- i was a very heavy Napster user in 99 and am a very
heavy Spotify user (and, NB, they are a customer) today. Obviously it's a fun
angle for a SXSW talk, but:

\- The "depth of catalog" is actually incorrect -- Spotify has far far more
songs than Napster did (you can see in that screenshot even, most
logins/servers hit less than 1m songs totally available, and of course those
are just total indexed, in reality almost 50% of downloads would not be
available.) Yes, you could find niche stuff on Napster-- as a musician in 1999
I would make sure all my stuff was available, pre-release, demos, etc. But we
have Soundcloud for that now, so...

\- The discovery on Napster was non-existent. You could browse through a
users' collection, the same way you can browse through playlists today. But
the only entry into anything was a search box that only looked at ID3v1 data
and filenames. There's been amazing leaps and bounds in discovery since then,
and it's very clear by the #s that it's what people wanted -- a guided (radio,
playlist) experience over a wild west single song retrieval thing.

\- Napster was stupid bad at search. There obviously was no catalog resolution
and the quality of the results was abysmal. I am pretty sure it was a
substring match, for one, and then there was the bad metadata, fake songs,
later on a huge spam / "SEO" problem.

\- Not going to get into speed, because it wasn't Napster's fault, but even
back then it was far easier to get music elsewhere other than Napster if
possible. Ratio FTP sites, hotline, and of course bubblecruft startups
building customer bases by selling new CDs for $4. It was very clear at the
time even that the distributed nature of Napster was a liability, not a
promise, as all the single-server solutions were far more convenient and
reliable. But this was not their fault and of course their success inspired
everything after it.

~~~
Aloisius
\- _most logins/servers hit less than 1m songs totally available_

The backend servers were linked. If your request couldn't be fulfilled on one,
your search was forwarded to the next server. The total number of files on
Napster at its peek was over half a billion files. Further, that picture must
have been from a server that just started because the average server had
significantly more users/files on it.

Further, Napster users were ripping everything in sight. There were mp3
encodings of old _wax cylinders_ uploaded for goodness sakes.

\- _The discovery on Napster was non-existent. You could browse through a
users' collection, the same way you can browse through playlists today._

There was an entire curated music website dedicated to music discovery that
loaded into the client.

The chat and instant message system allowed people to talk about music which
created a massive music-focused community. It was wildly popular.

And don't underestimate browsing. People would search for the one song they
were interested in, notice who they were downloading from, browse the other
user and then start pulling down their music if they noticed several songs
they liked in it. They then could send a message to that user and add them to
a friend's list. That was not just music discovery, but friend discovery as
well.

\- _I am pretty sure it was a substring match_

The very first versions were substring match when there were maybe 10,000
users. Later version were not and allowed basic boolean queries like term
exclusion.

\- _Not going to get into speed, because it wasn't Napster's fault, but even
back then it was far easier to get music elsewhere other than Napster if
possible_

There was an algorithm on Napster that did network distance biasing.
Basically, if you were an AOL user, you'd first get AOL users back when doing
a search. If you were an Internet 2 or even @Home user however, your speeds
were epic.

* Note: I built and ran the Napster server.

~~~
palish
_Note: I built and ran the Napster server._

I just want to say: thank you for building one of the most important
inventions of our time. (Napster kickstarted the information sharing
revolution.)

~~~
corford
Usenet might wish to contest that claim :)

~~~
Splines
BBS's also functioned in a similar way before Usenet did.

That said, Napster made it all _really_ easy. They also came along when CD
drives were finally commonplace, and Winamp had seeded the ground by having
people build up a library of MP3s beforehand.

~~~
corford
I don't know, I remember Napster as being quite late to the party. Everyone I
knew (geeks and some non geeks) had been happily using ratio FTPs for years
(using usenet and irc to find them) and then later Audiogalaxy took care of
the discovery problem. I also remember CD drives being fairly common by 1996/7
(it was CD burners that took a few years longer to hit sensible price points).

But its been a while so maybe my rose tinted glasses have munged things a bit!
:)

And you're right, BBSes were doing similar things first (although I assume
good ones were harder to find than usenet groups or irc chans - don't know
because BBSes were before my time!) but of course the mp3 standard didn't
exist back when they were at their zenith. MP3s only started appearing en-mass
around 1997 I think (??). Before that, I remember every one used to share tiny
wav and midi sound file clips on their homepages (my first ever homepage was
made to share wav clips of simpsons and monty python dialogue - lol how
embarassingly quaint!)

~~~
Splines
You know, you're probably right, I think I'm off about 2 years.

Yeah, in my BBS days I remember downloading MOD files and demo videos. I still
remember being blown away by Future Crew.

------
xpose2000
Napster was cool because it had live music and bootlegs that are often times
very tough to find. A lot of napster was sifting through crap songs to find
the good versions.

Spotify and the others are cool because you can stream stuff while on your
phone. And getting to the music you want to hear was more seamless and
straightfoward.

So in summary, almost, but not quite.

Napster was also one of the first geek-tools to really hit mainstream, and boy
was that weird.

~~~
dasil003
You really nailed it with this comment. Napster is from another era when the
thrill of discovery was much greater than it is now. Objectively maybe Napster
isn't as great as all the nostalgia would have us believe. Maybe there are
actually more songs on Spotify (probably not, but humor me). The thing is,
back then, if you searched and didn't find today, it didn't mean you wouldn't
find tomorrow. You never knew when you would find a gold mine of a collection
that not only had what you were looking for but actually schooled you on a
bunch of new stuff.

Napster was a magical moment where the underground went mainstream for a split
second before being snuffed out. Of course these type of (smaller) communities
still exist online, and music fans are better off than they ever were, but
Napster embodied a sea change that could only happen once.

------
icebraining
_But the "evil" aspect of Napster wasn't that it was P2P: it was that it
didn't return any money to the creators._

No, it's because it forfeited the control the labels had over the music.
They've showed again and again that they don't mind losing money to maintain
that control.

~~~
psychotik
P2P is just technology. Making it work so that the right people are
compensated is hard. My startup is trying -
[http://www.gigaom.com/2012/03/05/audiogalaxy-
personalized-p2...](http://www.gigaom.com/2012/03/05/audiogalaxy-
personalized-p2p-radio/). This just launched a week or so ago - give it a
shot!

~~~
icebraining
Thanks for the suggestion and I hope you'll succeed, but frankly I can't;
assuming you're giving a cut of your income to the big labels, I'd be
contributing to companies that actively lobby to destroy things I hold dear.

Nowadays I only buy music that I'm reasonably sure it won't feed the RIAA.

------
rmc
It's interesting that it's more than a decade since Napster, and the music
industry hasn't gone bust yet. It's almost like P2P cannot destroy the main
music industry.

~~~
meatsock
no but the music industry can destroy the music industry, and they're doing a
bang-up job of it. maybe a different way to think of it is: it's been more
than a decade since napster and the music industry is just now beginning to
get a clue.

~~~
Samuel_Michon
The iTunes Music Store has been around since 2003. It has been three years
since all DRM has been removed and files are 256kbit/s AAC.

Granted, this happened _despite_ the music industry's efforts, it took a lot
of pressure from Apple to get there. But still, I very much prefer the current
situation over when Napster reigned. (Does noone remember the amount of
garbage on Napster? The malware, the fake files, the low quality files? Not to
speak of the terrible speeds -- many peers were still on dial-up...)

With iTunes, you can find new music easily, download high quality files, and
get the album art and liner notes. Downloading and purchasing is fast and
safe. You get to pay a fair price and know that the rights holders are being
compensated.

~~~
Aloisius
Actually, SNOCAP started selling DRM-free music on MySpace exclusively back
before DRM-free music was a glimmer in Apple's eye. We were mostly doing indie
music when we had all the major labels come to us and ask how they could sell
music on MySpace and eventually negotiated deals for it to all be DRM-free.

After that, it was relatively easy for Apple to go DRM-free (though oddly,
they still refused to do it for some time after).

* Note: Shawn Fanning and I founded SNOCAP after Napster blew up.

------
nitrogen
For me the coolest thing about Napster was all the remixes people were making.
You could search for "[anything] remix" (e.g. Tetris remix, Nintendo remix,
Sesame Street remix, ...) and find something awesome that would never see the
light of day in a label. Some of those remixes bled through into the later P2P
systems.

Napster was one of the first services to give the mainstream world a glimpse
at the digital information utopia made possible by the Internet. The
paternalistic distribution systems we have now are a mere shadow of what could
be without the strangling influence of the media industry.

~~~
ljf
they were excellent! but the remixes are still around (and new ones being
made), they've just moved to YouTube now.

------
nhangen
The fatal flaw of Spotify, Rdio, and even Last.fm is that finding good music
is just too much work. Too much social and too many clicks.

I wish Pandora could buy one of these services and combine their discovery
engine with the catalogues of the subscription services.

Until then, I think I'll just keep buying music.

~~~
psychotik
Hey, if you like discovery try Audiogalaxy Mixes - we just launched it a week
or so ago [1]. We think it's better than Pandora (much larger catalog, smarter
algos, more input types) - tell me if it's not. It's free on the web (no ads)
with a free trial on mobile, after which we charge a small fee to cover
licensing costs. If you email viraj@audiogalaxy.com with your account I'll
comp you a free month. I'd love to know what you think.

Here's a Mix I've made, for example:
<http://www.audiogalaxy.com/mix/87-Guitar%20Gods/>?

[1][http://www.geekwire.com/2012/audiogalaxy-music-service-
rebor...](http://www.geekwire.com/2012/audiogalaxy-music-service-reborn-
approach)

~~~
andybak
Ahhhh. Audiogalaxy. That brings back memories.

It was amazing back in it's (less legitimate) days. It took over from Napster
but improved on it in many ways.

~~~
psychotik
It's evolved and we're trying to re-create some of the same magic, with the
same team. First step: [http://gigaom.com/2012/03/05/audiogalaxy-
personalized-p2p-ra...](http://gigaom.com/2012/03/05/audiogalaxy-
personalized-p2p-radio/)

------
mds101
So just because Napster had built in chat and a wider illegal catalogue it
becomes better than Spotify eh? Also, why would I want to know the personality
of some ramdom guy on the other side of my p2p connection? This is mostly a
fluff piece with absolutely nothing new or interesting being said.

~~~
jchrisa
I thought browsing other folks libraries was pretty cool. As an artist, I'd
look myself up, and see what else those people listened to. Pretty neat and
I've yet to come across anything similar. I haven't had to the time to
investigate the other p2p music platforms since then, so probably others have
that feature...

~~~
_delirium
I like that aspect of Soulseek. When someone shows up in the search results
for some obscure track, you can see what else they listen to. The browsing
interface also keeps their music organization, so it can be a pretty
interesting way to browse if you run across a collector-type person with an
organized-into-genre filesystem. You can also chat with them, and there are in
addition IRC-style chat rooms for various genres and interests.

~~~
kreate
yes, soulseek was created by a former napster developer. it is very similar to
napster.

------
Hyena
The irony being that Spotify has a social strategy.

------
comex
> What a pity the large labels a decade ago didn't appreciate that Napster was
> a social network – just one built around music. Who knows, today it might be
> as big as you-know-who.

Ping?

------
ecocentrik
I like Spotify and find it infinitely more useful for music discovery than
Napster ever was. Sure it would be nice if they had a more comprehensive
catalog but finding complete high quality recordings on Napster was always
impossible. I dug around for weeks trying to complete an album at a decent
bitrate. By comparison Spotify has figured out how to serve complete high
quality music legally at a very reasonable price.

------
invisible
Absolutely a fluff piece. No mention of any other legal services that offer
exactly what the author is writing about (namely Grooveshark).

~~~
thematt
Grooveshark's legality is questionable at best.

~~~
invisible
Legality is following the letter of the law (e.g. DMCA), to which Grooveshark
does and exceeds. Being praised and loved by all labels is what Spotify is
doing (by paying out the ass in investor money).

------
gaoshan
I find more awesome music, quicker, with Spotify than I ever did with any
other music tool. I especially like the apps for this: Pitchfork, Guardian,
RollingStone, etc. This combined with checking out what my "friends" are
listening to really is a great way to find good stuff.

------
cantbecool
The author didn't even clearly state how Napster '99 smokes Spotify. The only
feature that I thought he stated was that Napster had a form of chat and how
it could have potentially evolved into a social network.

------
jwblackwell
I think it's a little unfair to make a comparison to something that is
essentially illegal against something that is not. I know Spotify has it's
flaws both in its business model and catalog but as someone who has used
virtually every illegal filesharing method going in the past and has now been
converted to Spotify I think they should be given a little more credit.

It's not the fault of Spotify that the collection is limited. At least they
are trying to move the music industry into the future - whether it will work
or not who knows but I'm sure we will all look back in hindsight and see
Spotify as the pioneer of whatever comes.

~~~
icebraining
I think you're reading too much into it. The article just gives you a fact -
Napster was in many ways better than Spotify is now. It doesn't blame the
Spotify devs for it.

------
bitterfounder0
My college used DC++ to internally share music, movies, tv shows, and porn.
You could browse someones entire media collection. It gave you an overall
sense for their taste in music and was a great way to find new bands. Reading
about Napster reminded me how much I miss this feature.

The article glosses over how slow Napster '99 was. I remember waiting a couple
hours to download an album. Fast downloads are more important than social
features. I do not miss Napster.

~~~
rmc
To be fair, this was the age of dial-up, not broadband.

~~~
keypusher
I disagree. 97-99 were the years of massive broadband cable rollout to
suburban America. This was the time that broadband arrived, and you see it
reflected in the software of that era particularly video games for example
Quake, Half-Life, Counter-Strike, Ultima Online, Everquest. It was the boom of
real time multiplayer, which required a broadband connection.

------
shingen
I hadn't bothered to look at Spotify, in terms of using the service (I was
familiar with the company et al). I've used iTunes with locally stored music
for a really long time.

So I went to spotify.com, read over their product and information. Checked out
their plans. Decided I'd see what they can do with the Unlimited $4.99 plan.

I went to sign up for Spotify. They required I use my Facebook account.

Nope.

------
funkah
Cloud music services are too limited, and all they get you is not having to
store your music on a device you carry around with you all the time anyway.

