
Online Monoculture and the End of the Niche - r7000
http://whimsley.typepad.com/whimsley/2009/03/online-monoculture-and-the-end-of-the-niche.html
======
icefox
"The set of 48 customers is divided into equal-sized communities, with
members&nbsp;chosen at random so they may not be close in taste.

The recommender function chooses an item by looking at what customers in the
same community have chosen. It recommends the one most popular among others in
the community."

So first we randomly assign you to a group and then suggest items that has
already been marked as popular. With an algorithm like that it is no wonder
why he ended up with the data he did. This is the model where we all go and
buy from the amazon top sellers list, not because they are good but because
everyone else is buying it. There is zero surprise that showing this list to
users would cause the items on the list to go up. This is also about the worst
recommendation algorithm I could think of. His algorithm is the one used
_before_ the internet by the general population. You don't listen to every
band, but see what your friends are listening too. You buy the same fridge as
your friend because they recommended it. You did things usually because other
people had, not because they were good. So according to his data what was
happening before the internet should never have happened.

A good recommendation algorithm will present me with choices that I actually
want to see. The key is that the choices presented me are only my choices and
not choices that everyone sees. So as time goes by the odds that I will see
the Nova episode on Rats from a few weeks ago will grow higher (really good
one btw). But I don't think tivo will suggest that to all tivo's.

In a real recommendation algorithm as time goes by good items will get viewed
more and bad items will not and the spread will grow larger. This is exactly
what Netflix people talk about when they say they have been shown all sorts of
great movies they never would have tried on their own. Netflix _hates_ ,
absolutely _hates_ monoculture because it means they have to buy 1 zillion of
the blockbuster dvd from the summer. Which is part of the reason they do
recommendations and sponsor its research. If they can find you something
better to watch it means they can spread the que of users out to not be a
monoculture. That is what recommendation algorithms do, they create niche's.

------
kmavm
"The result is what's important here, rather than the particular algorithm
used to generate this instance of it."

Ah, the "computer modeler's" rallying cry. Something that has always worried
me about performing "experiments" on computer models: software has bugs, and
the investigators building and using them have preconceptions. The two
interact to collectively bias results towards fitting the modelers'
preconceptions: you fix bugs until the output "looks plausible" and fits some
control data, and then stop.

~~~
joeyo
"All models are wrong, but some are useful" -- George Box

------
calambrac
Shorter version: if you're not in a box, you're probably going to see more
things, and if others aren't in a box, they're probably going to see more
things, too, but the odds that those things will overlap goes up.

Then the things that were in the box that aren't getting the same amount of
attention paid to them because people have more choice, those things get their
feelings hurt, or something.

Add dubious computer model and some inscrutable graphs.

