
Ask HN: How do you keep application data synced with a CRM - charlesnodier
I work for an early stage startup (~8 people). Our business is high touch and we store a lot of the data we gather in our CRM (Salesforce). When our custom built applications need that data, they access it via the CRM&#x27;s API.<p>This has worked very well for us, but we&#x27;re starting to bump up against API limits. I&#x27;d like to move more of that data into our own DBs, but then we run into consistency issues: If a Sales Rep edits a customer&#x27;s name in Salesforce, how do we keep our database consistent with that? We could build services to propagate any DB edits to the CRM and vice versa, but I don&#x27;t think we have the person power to do that in a robust way.<p>I have to assume that many, many startups have run into the issue of needing to keep application data synced with CRM data. Any advice?
======
davismwfl
It has been awhile since I did SF integration work but they do have table like
triggers for updates/upserts etc (or at least have in the past) so you could
setup triggers and sync the data that way.

Another (dirty) option is query the updated records on a schedule and update
your local database. If you see on average 20 records change a day, then just
do syncs every 30 minutes so at worse you are out a short bit. Triggers are
obviously superior however. This all depends no what you are using the data
for and what is the impact of a stale record. Cause you could also do this
once a day but then anytime an action/decision is to be taken on a specific
record you hit SF database and confirm you have the latest. But for general
reporting it is rare that the count can't be slightly delayed, and where it
can't there are options.

Add a hook to the form itself to push the update to your system as well, again
it has been awhile, but I worked on a project where this was done by another
team, not for the same reason you are interested but it was possible.
Basically it was similar to using a trigger but was done at the form level
with a web hook, but sorry I don't know much about this methodology.

You might want to pay for an hour or two from a SF integration consultant to
help give you some ideas, there are lots of options usually for this and SF is
a huge product so might save you tons of time to get an expert
recommendations.

I am a huge fan of not leaving your data in SF only, there are so many reasons
it is better to have your data locally.

------
Avalaxy
It really depends on the SaaS products. Often we have to pull the data out of
the REST API, like you suggested. I have no experience with salesforce, but a
quick Google taught me that you can access their database. This is done using
SOQL, as illustrated here [https://developer.salesforce.com/docs/atlas.en-
us.soql_sosl....](https://developer.salesforce.com/docs/atlas.en-
us.soql_sosl.meta/soql_sosl/sforce_api_calls_soql.htm). In this case you just
pull the data out of salesforce when you need it, without caching it in your
own database.

I hope this helps you, let me know if it gets you any further.

~~~
charlesnodier
Thanks Avalaxy. That's exactly what we're doing now (pulling data out of
Salesforce using SOQL). The issue is that we're starting to do it so
frequently that we're bumping up against Salesforce's API call limits.

~~~
Avalaxy
Gotcha. With a high velocity of data changes this is usually done with
streaming data. Just did another quick google and I saw that SalesForce has a
Streaming API that lets you capture events:
[https://developer.salesforce.com/docs/atlas.en-
us.api_stream...](https://developer.salesforce.com/docs/atlas.en-
us.api_streaming.meta/api_streaming/intro_stream.htm)

Their description sounds like your use case:

> Applications That Poll Frequently Applications that have constant polling
> action against the Salesforce infrastructure, consuming unnecessary API
> calls and processing time, would benefit from Streaming API because it
> reduces the number of requests that return no data.

I hope this will help you.

