
Segment Sources – Load Salesforce, Zendesk, Stripe into Redshift and Postgres - TheHydroImpulse
https://segment.com/sources
======
samcheng
ETL-as-a-Service is a great idea, particularly one that is
visualization/analytics-tool-agnostic!

However, there are so many data sources, and they all require different
integrations with their different APIs or export mechanisms. A service isn't
really useful unless it can import the lion's share of services that a given
company uses...

~~~
pkrein
You’re right. There are a lot of sources out there. It’s a ton of work for
companies to build out their own pipelines and learn every new API. We want to
save them from that burden so that they can focus on the analysis. We’ll be
adding many more connections in the coming weeks and months, and also opening
up the platform for cloud services to add themselves. Stay tuned!

------
dan_ahmadi
I wonder if this makes BI companies freak out a little bit -- because pushing
this data into redshift and adding a visualization layer on top takes care of
most smaller scale BI needs...

~~~
pkrein
Actually, we don't have any interest in being a visualization tool, and are
super focused on building customer data infrastructure of the future.

This product release is in close partnership with our BI partners (Looker,
Mode, Wagon, Periscope, BIME and Chartio). One of the biggest problems our
mutual customers face is getting data into their warehouse so that they can
use the BI tool in the first place. This launch significantly expands the
possible audience for them.

Even better, all of our BI partners built out-of-the-box reporting and
dashboards based on Segment's schemas for these new third party sources. So
our mutual customers can get set up even faster.

~~~
vyrotek
Can someone use Segment to generate per-user reporting data?

So, instead of Total Company Sales this Month, I want Bob's Sales, Joe's
Sales, etc. It feels like this is just a filter on top of what you already
have. Almost like a parameterized query? (pass in @UserId for a where
statement)

I originally thought maybe this is the job for one of your visualization
partners but you really need to filter the results before you perform the
aggregate.

~~~
n2parko
hey vyrotek - you can definitely build that report using Segment sources + BI
tool.

Our salesforce source pulls in the Salesforce `Opportunities` and `Users`
table (your sales team members). So to get sales by sales rep, you can join
the `Opportunities` table to `Users` table, and then aggregate by sales rep.

Once you get the raw data into your data warehouse, you have a ton of
flexibility with how you aggregate and analyze it.

------
primeobsession
RJMetrics has a similar product (ETL as a service) with 10x the number of rows
for their free tier.
[https://rjmetrics.com/product/pipeline/](https://rjmetrics.com/product/pipeline/)

------
uberneo
What a coincidence, today only I came across another ETL as a service from
Pintrest -
[https://news.ycombinator.com/item?id=11438216](https://news.ycombinator.com/item?id=11438216)

------
vyrotek
Is this just running the SF SoQL statement for you and storing the aggregated
result in a Segment table?

Or does Segment provide a way to completely clone entire SF tables such as
Opportunities & Cases and then create the aggregate queries later in segment?

~~~
sperand_io
Great question! When you enable a source, we begin running a job on an
interval for you that pulls the data from the source, applies some light
normalizations and transformations, and sends the data to our Object API
(which is in charge of upserting the data into Segment and flushing it to your
warehouse).

In Salesforce's case, we issue bulk queries to pull the complete collection on
the first run, then modify the queries thereafter to request only data that's
changed since the last run.

We don't do any aggregation of the data. We load it into a data warehouse
(redshift or postgres) in its complete, raw form so that you can use SQL to
aggregate/join to your heart's content. Here's an example:
[https://help.segment.com/hc/en-
us/articles/208215583-Salesfo...](https://help.segment.com/hc/en-
us/articles/208215583-Salesforce-Calculate-close-rates-for-qualified-deals-
over-time)

~~~
npace12
You should make this available for trial. No way I'm paying $449 just to try
it.

I set it up with the developer account, but when I try to connect the
integration, it says it's not available for the developer plan. I was like
"ok, fine, i'll pay 50 bucks to try it" but then it says salesforce is only
available on the Growth plan, which is $449.

Closed the tab, and moved on with my life.

~~~
sperand_io
Sorry for the confusion there — our integration that sends data _to_
Salesforce is on the integrations growth tier, but the pricing for data
sources and warehouses is separate (and does indeed have a free trial :).

Your point prompted us to add a callout in the UI to prevent this sort of
confusion going forward (see here:
[https://www.dropbox.com/s/yoc6a3nvsoupesa/Screenshot%202016-...](https://www.dropbox.com/s/yoc6a3nvsoupesa/Screenshot%202016-04-06%2013.44.39.png?dl=0))...
and more generally we'll be working to make the distinction between data
sources and data destinations clearer from a UX perspective in the near
future.

------
pinaceae
just be careful with SFDC, bunch of harsh limits, killer one being 5k bulk api
requests per rolling 24h.

if you think that's a lot you haven't seen big orgs with a shit ton of
integrations and custom stuff on top of them.

------
josep2
Segment is always killing it.

~~~
grinich
They're totally doing great! (But we should use less violent language to
describe it...)

------
slachterman
Are there any plans for Warehouses support for IBM's DashDB?

~~~
n2parko
Hey slachterman DashDB isn't yet on the roadmap, mind filling this out so we
can follow up?

[https://segment.com/contact/requests/warehouse](https://segment.com/contact/requests/warehouse)

------
vyrotek
Please add options to use Windows Azure as a warehouse!

~~~
n2parko
hey vyrotek thanks for the request! mind filling this out a request and we'll
follow up?
[https://segment.com/contact/requests/warehouse](https://segment.com/contact/requests/warehouse)

