

Scaling to 2M API request per day - mttsn
http://stackshare.io/clearbit/scaling-clearbit-to-2m-api-requests-per-day

======
timchunght
Just finished the article. Great read! How are you guys able to retrieve data
instantaneously for the personal API? I understand that you can cache the
company API but how are you guys caching the Person API as there will be
billions of users online across all social networks.

~~~
mttsn
Quoting Harlow here:

Great question! When we don't have an email address (or company domain) in
cache we use our robust worker platform to doing live lookups. We leverage
Sidekiq Pro to process millions of Lookup Jobs per day. We can typically
finish a Person Lookup in under 3 seconds by leveraging the massive
concurrency that Sidekiq affords us.

~~~
timchunght
Great answer! Thank you for your response. I am a fanboy of Sidekiq and I
enjoy its speed and ease of use. What are some places you guys look for
personal info. Are they just APIs or do you do some social network scraping?
Thanks.

