

Ask HN: Are there any HN mashups? - SudarshanP

A lot of precious jewels lie buried deep inside HN. A few tools like like Search YC, Ask HN, Hacker Monthly exist to bring that to the surface.<p>What other existing tools can you think of?<p>Are HN archives available anywhere for HN enthusiasts/arc hackers to slice and dice?<p>If someone wanted to create such tools, is it generally encouraged or discouraged?<p>Is there some particular tool you wished, was available to surface the content of HN?<p>Has PG ever discouraged or prevented HNers from creating such tools?
======
petercooper
A notable one: <http://hacker-newspaper.gilesb.com/>

_Has PG ever discouraged or prevented HNers from creating such tools?_

The only thing that I recall was discouraged was a Greasemonkey script that
resulted in HN getting hammered with lots of requests on every page (I forget
what it did but it did a lot of lookups on HN). People who used it had their
IPs banned for a day or so.

~~~
SudarshanP
<http://hacker-newspaper.gilesb.com/> is awsome!!!

I am planning to crawl HN using <http://scrapy.org> Strictly adhering to
<http://news.ycombinator.com/robots.txt> will severely limit the usefulness of
the data scraped.

User-Agent: * Disallow: /x? Disallow: /vote? Disallow: /reply? Disallow:
/submitted? Disallow: /threads?

So I hope to crawl HN /x,/threads etc. Of course I plan to do it as slowly and
unobtrusively as possible. If a database dump or archive is available, the
whole crawling could be skipped.

