

I made a ruby library for accessing hacker news - check it out! - random_guy
http://github.com/bolthar/ruby-hackernews

======
lukehasnoname
It might be fun to scrape and index all HN stories; and perhaps create a way
to download links/comments to a local machine automatically . There is a lot
of really great information provided via this place.

~~~
random_guy
The idea is interesting indeed - and easily doable as well: Entry.all gives
you all the links on the front page, you'd just have fetch the main page
between regular intervals of time (say, once every hour) and then save the
entry information to a database.

Could you elaborate on this? If you can provide better "specifications" I'll
implement this and add it to the library.

~~~
lukehasnoname
I imagine this:

-DOWNLOAD HN pages locally OR to another site

This is to make a permanent copy of the site and its discussions locally, so
you'll have the knowledge of the articles and discussion forever.

Either download the linked site itself, or just the link, folowed by comments.
Automatically grab comment updates once a day for a week, or whenever
discussion statistically tapers off.

In other words, here are the options:

-Submission titles, dates, and links to HN page

This is to provide a searchable database for HN links, and perhaps (if this
were done on a separate website) you could implement a "favorites" system for
people to login and bookmark those topics of discussion they liked most, ala
Server Fault.

Downside: I don't know how long HN articles persist. If they disappear, this
is largely useless.

-Linked sites themselves, URL to HN page, and comments

Much more permanent; a complete mirror of source sites and HN discussions,
searchable. Similar login/favorites iface could be put in place.

Downside: More information is cached, more complicated scraping (source sites
and HN).

