Looking for a "firehose" feed from Hacker News

[davewiner]

[vyrotek]

A firehose doesn't exist. But, a few people have created API's which do the page scraping for you.

I recommend http://api.ihackernews.com

[ronnier → vyrotek]

Soon I'll also be providing the entire hn database for download. I have the db, but I have bandwidth concerns. Anybody willing to work with me who has bandwidth to share?

[davewiner → ronnier]

Why not distribute it via BitTorrent?

[ronnier → davewiner]

UPDATE: I'm now seeding the file. You can find the torrent at http://api.ihackernews.com/

[ronnier → davewiner]

That's probably what I'll end up doing. Id like to have it updated daily, seems like with BT there would be older copies floating around with newer copies unseeded.

[davewiner → davewiner]

BTW, I'm happy to help with bandwidth, if I can. :-)

[ronnier → davewiner]

Great! I'll email you when I get home. It's a 300 MB compressed XML file. 1 GB uncompressed. Contains comments, links, postedby, dates, parentid, and so on.

[templaedhel → ronnier]

What size is it?

[ronnier → templaedhel]

300 MB compressed.

[templaedhel → ronnier]

Thats not to bad, it would be a quick torrent. I was just reading about the 640.6gb torrent of all the 2010 world cup games, in HD, 12gb per game, and the geocites torrent which was also quite big.

[sophiebits]

There's http://news.ycombinator.com/newest and http://news.ycombinator.com/newcomments for new submissions and comments, respectively.

[davewiner]

Here's the feed.

http://static.scripting.com/hackernews/rss.xml

Converted from the API, very easy to do. :-)

[BarkMore]

I poked around and found this: http://hackernews.com/rss