
Show HN: Get unthrottled access to Amazon product data - QueensGambit
https://github.com/ManiDoraisamy/AmazonUnthrottledAPI
======
QueensGambit
If you are a developer using Amazon product feed, you might be familiar with
this error message: "AWS Access Key ID: XXXXXXXXXXXXXXXX. You are submitting
requests too quickly. Please retry your requests at a slower rate."

This happens when your application is submitting requests to Amazon faster
than once per second. Amazon has a one-second rule i.e. if you are submitting
more than one request per second, it will throw RequestThrottled error. I
faced this quite often and built this shared cache to overcome this problem.
Sharing it here and looking forward to your feedback!

~~~
vlab_mech
How does this shared cache work? and how does it solve the throttling problem?

~~~
QueensGambit
The shared cache stores Amazon data in our database. Subsequently, when any
user makes a request, it will hit Amazon only if there is a cache miss.

Lets say, user 1 makes an API call to our server to fetch browse node: 1000
(Books). The first time such request is made, our server will make an API call
to Amazon using user 1's AWS key. Once we get the response back, we will cache
the result and send back the result to user 1. When user 2 makes the request
for the same browse node 1000 (Books), we will not hit Amazon server again.
Instead, we will get the cached data from our database and send it back to the
user 2 without using his/her AWS key. This way the number of API calls made
will be less than one per second for a user and avoids the throttling problem.

~~~
mooreds
How do you deal with stale data?

~~~
QueensGambit
I have set the cache expiry to 24 hours. If the cached data is more than 24
hours, it will fetch fresh data from Amazon.

