
Show HN: BrowseNodes – Get unthrottled access to Amazon Product API - BrowseNodes
https://browsenodes.com
======
BrowseNodes
When I was building a price comparison site using Amazon Product Advertising
API, I ran into this throttling error frequently: "You are submitting requests
too quickly. Please retry your requests at a slower rate."

So, I built a shared cache, so that developers can hit the cache instead of
hitting Amazon API every time. That's how I created BrowseNodes. Have you
faced the same problem like me? Do you think this will solve the throttling
problem for you?

Looking forward to your feedback!

------
visionerdy
According to the Amazon API docs, we shouldn't be sharing the API key. Do you
store our API key in the cache?

~~~
BrowseNodes
No. I don't store your API key (i.e. awsSecret). I only cache the product data
to serve other users: [https://browsenodes.com/how-to-solve-
requestthrottled.html](https://browsenodes.com/how-to-solve-
requestthrottled.html)

Hope that helps!

------
gitgud
If this just caches the requests, then couldn't you still be throttled?

This just moves the throttling problem to BrowserNode

~~~
BrowseNodes
Not really! Lets say, user 1 makes an API call to our server to fetch browse
node: 1000 (Books). The first time such request is made, our server will make
an API call to Amazon using user 1's AWS key. Once we get the response back,
we will cache the result and send back the result to user 1. When user 2 makes
the request for the same browse node 1000 (Books), we will not hit Amazon
server again. Instead, we will get the cached data from our database and send
it back to the user 2 without using his/her AWS key. This way the number of
API calls made will be less than one per second for a user.

~~~
chatmasta
This assumes all your users are searching for the same browse node.

I’m not familiar with the Amazon API, how many browse nodes are there?

~~~
BrowseNodes
Yes. Browse Nodes are essentially product categories. So, users search for the
same set of categories. There are ~40K browse nodes in each Amazon domain and
we have cached most of them already [1]. So, the cache miss is unlikely here.

But, product cache is little complex. We have already cached the top products
[2]. But, products are added every day. So, there will be some amount of cache
miss there.

[1] Cached browse nodes -
[https://browsenodes.com/amazon.com/explore.html](https://browsenodes.com/amazon.com/explore.html)

[2] Cached products -
[https://browsenodes.com/amazon.co.uk/itemSearch/83451031.htm...](https://browsenodes.com/amazon.co.uk/itemSearch/83451031.html?parent=Apparel)

~~~
gitgud
Ah I see now, so you guys have pre-fetched most of Amazon and are providing a
less restricted API to the cached content.

Is that permitted in the Terms and Service of the Amazon API?

Anyway great idea, would make a lot of developers lives easier!

~~~
BrowseNodes
Thanks! Amazon TOS allows caching. But, not sure if there is any specific
restriction on shared cache.

