

Ask HN: Where can I find Internet browsing data? - gregw134

Somebody posted a large dataset from Indiana University a while back, but it's several years old and only available to researchers.<p>http://cnets.indiana.edu/groups/nan/webtraffic/click-dataset<p>Does anyone know of another source of browsing data? Is it possible to get this data in real-time?
======
Irishsteve
To get it in real time you most likely have to pay someone who is already
collecting said data (Don't know who) or write software yourself and pay
people to install it.

Is there any particular type of data you are looking for? Or is it simply anon
user x went to page y on domain z

~~~
gregw134
>you most likely have to pay someone who is already collecting said data
(Don't know who) or write software yourself and pay people to install it.

Yeah that's what I was afraid of :(

I'm looking for basic, anonymized browsing history for users over a period of
time. The idea I'm trying to test is whether you can identify the users who
are "trendsetters"--people who visit sites or posts before everyone else--and
whether you can sniff out links that will be popular based on their behavior.

I've found a few more links, but most of them require you to be a researcher
to access the data:

<https://www.predict.org> <http://www.caida.org>
<http://crawdad.cs.dartmouth.edu/>

~~~
Irishsteve
Spotting trend setters on the web is going to be a little tough because of how
large and segregated it is.

Some datasets people use is Wiki based around comments / edits or who created
a page first which became popular in the end.

You could also use Facebook likes (facebook.com/me/likes) where each hlike is
time stamped.

