
Ask HN: How do you cache “rate-limited” 3rd party APIs? - gitgud
Often in side projects I&#x27;m using 3rd party API&#x27;s where the data is rate limited e.g; Weather apis, Social media apis etc..<p>Is there any easy solution like a proxy server where you funnel API requests through it and it only hits the actual Api when the cache expires or something?<p>Preferably, I want a simple and general solution which I can easily plug into a new project. Docker container or easily deployable would be great.<p>Anyone have any ideas?
======
dylz
> Is there any easy solution like a proxy server where you funnel API requests
> through it and it only hits the actual Api when the cache expires or
> something?

My lazy solution is nginx or varnish. You can set both to locally cache, set a
burst r/s, etc. Listen on localhost or on your docker container, proxy_pass to
the underlying API.

You can set it to disregard caching headers and always cache for a specific
amount of time based on the request. If your URLs look like
/weather/94704?key=11942890238, you can use proxy_cache_key to disregard the
actual key and only cache against /weather/94704 for example (by dropping
$is_args$args).

A docker container where you inject a nginx conf, or set up variables that
overwrite like PROXY_PASS_TO_URL=, PROXY_AUTHORIZATION_BEARER=, ..., but the
actual cache rules might need some tweaking.

~~~
gitgud
Thanks for the detailed reply,

That looks like a great setup, I wasn't aware of proxying through nginx, I'll
definitely have look at that setup.

Cheers

