

Ask HN: Why does Hacker News throttle ec2? - kolodny

I'm trying to write a script to scrape data from Hacker News on my ec2 instance but I seem to consistently be throttled at around 5k/s<p><pre><code>    [ec2-user@ip-**-***-**-** ~]$ curl https://news.ycombinator.com/ &#62; /dev/null
      % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                     Dload  Upload   Total   Spent    Left  Speed
    100 22237    0 22237    0     0   5195      0 --:--:--  0:00:04 --:--:--  5242
</code></pre>
Can anyone else verify if this is happening to them or shed any light on this?
======
hollerith
I think PG wants people to use <http://www.hnsearch.com/api>

~~~
kolodny
It doesn't seem like that api has an easy way to just get a page be it's post
id, which is what I'm trying to do

