Ask HN: Extracting the Twitter Social Graph

[m0th87]

I have an idea for a webapp, but I would need a local copy of the Twitter social graph to make it. Any suggestions on the easiest method of getting it?

Originally, I thought of creating an app, or using 80legs, to crawl Twitter's social graph API. But even if I were whitelisted (which allows 20,000 requests per hour) it would take months to finish. So instead I was hoping someone has made a CSV or database dump of a copy of the Twitter social graph, but I haven't found anything online. Is anyone aware if such a thing exists?

Surely someone else has had this same need. How did they go about doing this?

[kellyjoseph]

We'd love to talk with you about what you're building, our scrape is the most complete outside of Twitter that we know of.

http://blog.infochimps.org/2008/12/29/massive-scrape-of-twit... It has since grown to 40M users.

joe@infochimps.org

[collint]

Hmmm. I don't have it. If you don't find it, get whatever sample of the graph you can now. If you can get people excited about the idea you can make it happen.

[jdrock]

Check out Infochimps - they have a big Twitter data dump.