I've developed an API for HN http://api.ihackernews.com. I need to throttle requests to news.ycombinator.com to a level that doesn't hurt HN or result in the banning of ihackernews. Do you know the rate?
Cache your data and hit google instead of HN. That's the best way to solve this. Google can handle the traffic with considerable ease, HN is only one server that has a lot better things to do than to serve up multitudes of automated requests.
Btw, I like your API idea, it would be neat if HN had an 'official' one.
PG is quite responsive in email, maybe you could ask him directly about the maximum rate? I believe the search.yc guys do it once every 15 minutes or so.