

Ask HN: Can Alexa ratings/rankings be trusted? - mootymoots

Hi All,<p>What do you all think about the validity of Alexa rankings? Is it your opinion that the stats that they produce are an accurate representation of users, or is it likely a more techie demographic?<p>Would you use the stats for talking to potential online advertisers?<p>Thanks!
======
tokenadult
Alexa users are a convenience sample, not a simple random sample, and they may
badly misrepresent your user base. Essentially, installing Alexa (which I did
once, but haven't done on my current browser) is something like participating
in a voluntary response poll. One professor of statistics, who is a co-author
of a highly regarded AP statistics textbook, has tried to popularize the
phrase that "voluntary response data are worthless" to go along with the
phrase "correlation does not imply causation." Other statistics teachers are
gradually picking up this phrase.

[quote=Paul Velleman]

\-----Original Message----- From: Paul Velleman [SMTPfv2@cornell.edu] Sent:
Wednesday, January 14, 1998 5:10 PM To: apstat-l@etc.bc.ca; Kim Robinson Cc:
mmbalach@mtu.edu Subject: Re: qualtiative study

Sorry Kim, but it just aint so. Voluntary response data are _worthless_. One
excellent example is the books by Shere Hite. She collected many responses
from biased lists with voluntary response and drew conclusions that are
roundly contradicted by all responsible studies. She claimed to be doing only
qualitative work, but what she got was just plain garbage. Another famous
example is the Literary Digest "poll". All you learn from voluntary response
is what is said by those who choose to respond. Unless the respondents are a
substantially large fraction of the population, they are very likely to be a
biased -- possibly a very biased -- subset. Anecdotes tell you nothing at all
about the state of the world. They can't be "used only as a description"
because they describe nothing but themselves.[/quote]

[http://mathforum.org/kb/thread.jspa?threadID=194473&tsta...](http://mathforum.org/kb/thread.jspa?threadID=194473&tstart=36420)

But to answer your underlying question, if I had good Alexa stats, I would
talk them up. You may as well talk about the most convincing data you have,
while acknowledging that the data may not be definitive.

------
bbuffone
I typically utilize Alexa and Compete or others as a good way to compare sites
of a similar demographic. Having access to google analytics for two different
sites; one being my blog and the other site is the company I work at.

I know that my blog gets about half the traffic of my companies site but the
corporate site is ranked 370,673 and my blog is ranked 395,487. I would think
there would be more of a difference in the two sites ranking.

But if I compare several of our companies competitors sites the numbers seem
to line up. The numbers also make sense as trend data for a single site. I
know from monitoring my blog when there is a spike in traffic; Compete and
Alexa show a spike in their rankings.

Because Alexa captures their information through a browser plugin you want to
make sure that the set of people that visit each site are equally likely to
have the plugin installed. If people that read hacker news are half as likely
to install the Alexa Plugin then people that read the new york times then
there would be variability in comparing those two site.

------
jacquesm
I think all 'free' online ranking sites give you data that you should take
with a large grain of salt. You can use some of them for relative comparisons
but not if the comparison is a close one.

Another use is to extrapolate from 'known' numbers to an unknown site (say a
competitor).

It helps to separate truth from bull, but it is definitely not an exact
measurement. If someone is off by a factor of 10 then you're on solid ground
to call bull, if it is a factor of 2 or less then it gets a lot harder.

------
geuis
No. Use Compete.com or Hitwise

~~~
timf
I see a lot of people here complain about compete.com stats:

[http://www.google.com/search?hl=en&q=site%3Anews.ycombin...](http://www.google.com/search?hl=en&q=site%3Anews.ycombinator.com+compete.com)

At least as far as the data goes, maybe it's better for making comparisons.

