
Ask HN: What do modern enterprise data warehouses look like? - Ooberdan
I work on a business intelligence team where we&#x27;re currently running an on premise Oracle data warehouse (relational; Kimball&#x2F;dimensional model). Its sources include other relational databases, big data clusters and web APIs. We&#x27;re using fairly current Oracle tools for ETL and reporting (ODI&#x2F;OBIEE), which actually feel pretty dated in their approach.<p>What are some examples of modern BI stacks that would facilitate data insight for both general business users and power users? Bonus points for those that will result in big cost reductions to an Oracle stack (can&#x27;t be too difficult).
======
thenaturalist
While this is a very open question both in terms of your analytical resources
(dedicated data engineers, how tech savy are analysts, can they easily model
their own data) as well as constraints from operating in an enterprise setup
with regards to DevOps (easy to open ports, can data viz solutions be run
online or only on prem), I'll try to mention a bunch of tools assuming they
can be used. Notably, one thing you did not mention is which data
visualization tool is used and how much that allows general business users to
get insights.

Database:

In terms of enterprise ready Data Warehousing databases I would take a good
look at Exasol (on prem & cloud) and Snowflake (cloud only). Exasol will
likely be the pricier of the two and is much closer to the Java based Oracle
world than Snowflake. Personally, I'm not a big fan of Redshift, as Snowflake
takes all the pain that one might experience in Redshift away (manual tuning
of sort & dist keys, tuning of coupled storage & compute clusters). Again, if
you have good data engineering resources readily available it might make sense
to look into a comparative cost analysis of the three in more detail. Notably,
Snowflake has Snowpipe, a native API allowing you to stream data into the DB.

Data modeling:

You mentioned you use Kimball. In my experience while this is the definite
choice for analytical querying and modeling, in an enterprise setup it might
make sense to draw a Data Vault model layer between your raw sources and your
dimensional model layer. Data Vault is more modular and easily extensible,
will allow you to model history (compliance or GDPR readiness) and through the
latter might very well serve you to expand/ extend your dimensional model. Not
my own thought originally, very good read can be found here [0].

Data integration:

A standard enterprise integration tool which comes to mind is Talend, although
licenses are also pretty expensive and it feels very antiquated as well.
Talend bought Stitch last year [1], which follows a much more modern approach.
Worthy to take a look at, not sure if it can cover all of your requirements.
Another modern solution provider for integration more geared towards
enterprise would probably be alooma [2]. There is also Fivetran [3].

ETL:

While some of the tools above can effectively be used for ETL, I'd take a good
look at dbt by Fishtown Analytics [4]. It's SQL only but does a lot of things
right in terms of being a module in a fragmented data stack which does its job
really well. The thinking that goes into dbt is universally a good approach
imho. A player geared towards enterprise is Matillion [5].

Visualization:

The most empowering solution for general business users I know of is hands
down Looker [6]. Of course there is Tableau and many other more mature
solutions out there, but being able to write SQL is always a prerequisite to
getting results there. I acknowledge this is a personal, strongly biased
opinion, so don't take my word alone for it. Looker can be deployed on prem as
well, has LDAP & SSO, and a bunch of other features which would make me
consider it enterprise ready from a pure deployment perspective.

In general I try to stay on top of the most common tools in the BI space, and
maintain a repository of tools here:
[https://github.com/thenaturalist/awesome-business-
intelligen...](https://github.com/thenaturalist/awesome-business-
intelligence/)

I am not associated in any form with any one company mentioned here or listed
in the repo.

0: [https://medium.com/rv-data/overhauling-our-data-warehouse-
wi...](https://medium.com/rv-data/overhauling-our-data-warehouse-with-data-
vault-403fe163271d)

1: [https://www.stitchdata.com](https://www.stitchdata.com)

2: [https://www.alooma.com](https://www.alooma.com)

3: [https://fivetran.com](https://fivetran.com)

4: [https://www.getdbt.com](https://www.getdbt.com)

5: [https://www.matillion.com](https://www.matillion.com)

6: [https://looker.com](https://looker.com)

~~~
Ooberdan
That's great, thank you for the comprehensive response!

