As someone who runs teams deploying BI to internal stakeholders (product, sales) I am pretty fed up with them.
Firstly Tableau, QlikView or PowerBI are all pretty much doing the same thing whatever flavour you prefer.
We find that maybe 10% of users will actually use them to garner new actionable insights and 90% will moan there's too much data for them to process and no one appreciates the immense amount of data munging that goes on behind the scenes to make those pretty graphs.
Where do we go? Personally I see a lot of the insights being automated and turned into actions on the server side without anyone touching a BI tool. This can be achieved with rules based approach and perhaps some correlation, trend analysis over time series data. Where you do need to go beyond that then perhaps AR/VR will provide a novel and more valuable approach.
I see things going the same direction. I've always been pretty anti-dashboard because of the lack of long-term utility. If you want to make data-driven decisions, you need to spend the time codifying the decision making process and automating that. Automated actioning off data is far more impactful than automated visualization of data.
When it comes to business stakeholders, the biggest obstacle is trust. If you're not a data person, or a developer, decision logic being gated behind code is a scary black box. "If I can't control it, I can't trust it". That feeling gets even worse when we build systems that are controlled by AI or ML.
I think we need more solutions for data teams that allow business stakeholders to take part in the automated workflow deployment. Let them see how things are connected together. Let them verify that the decisions are being made correctly every day. Let them tweak levers so they have a say in how things are working. That's the only way to move beyond the current environment where everyone wants a dashboard, but no one looks at it.
That's a big problem I'm aiming to solve right now.
Wonderful comment. But trying to automate actions (write/modify underlying data or system) is often a hugely complex task compared to a visualization that allows you to form an insight. Have you found enough common/repeatable examples that you can automate or develop some type of actions template? What is your business domain?
Previous domain I worked in was digital advertising. We built solutions to automate ad and keyword creation from product feeds and promo calendars, increase/decrease bids and budgets based on hourly data, verify all settings were set to best practices, etc. on Google/Bing/Facebook. We built internal systems that allowed us to map any data view in our warehouse to any API endpoint in a templated fashion.
The technology is getting good enough with cloud warehouses that most of the logic can be defined in SQL, with a script only needing to map the results back to an API.
Current domain is SaaS. I've found many of the same types of opportunities on the sales/marketing side of things, but not at quite the same scale.
Agreed that automation it will always take more time than a visualization. If we can shift the conversation towards the expected action that will be taken off the dashboard, we can hopefully save thousands of wasted hours on visualizations that don't get used, redirecting those hours towards more impactful efforts.
I would love to expand my mindset and hear of domains where the complexity is far higher. Feel free to shoot me an email.
The way I'm (currently!) approaching this is by implementing the decision engine in 2 steps separated by an evaluation period. I refer to these to the stakeholders as "user approves" and "user reviews".
Essentially, I first implement the engine with the output being an automated list of 1-click actions (typically a set of links on an email, but could easily be a dashboard button or anything else), so the system default is "detect, but don't do". After an evaluation period of the system actions, we move onto the second part, in which the system performs the actions and produces a log of activity, plus a list of 1-click UNDO actions to the user.
The idea is that a) the system earns trust from the stakeholders due to their direct involvement, b) the system can enjoy some supervised learning from someone other than the dev team and c) worst case, if it never earns enough trust, I've still saved a stakeholder dozens of hours of work sifting through dashboards as opposed to taking a look at a pre-filtered list. A few systems have turned out well enough that they never move from Phase1 while still being considered a massive win.
I think that's a fantastic approach. You're effectively avoiding the pitfall of automating too soon while still shifting the focus towards "driving action with this data". If people click the buttons frequently, it's ripe for full automation - and they already trust it. If not, there's still an easy way to for users to get the job done quicker.
The data driven process app sounds similar to the ideas behind Knime (https://www.knime.com/) to me.
I think the visualisation matters though. It's much easier to convince people that the automated actions are worth doing once the related data is visualised. And at that point, whatever provides you both may be the best solution.
While I'm not super familiar with Knime, it looks like it falls into the legacy BI category that Benn talks about. An "all-in-one" for your data.
Is the platform still easily used by business users? Or does it primarily become a gated way for Data Engineers to build/action on data?
A lot of decisions can't be easily visualized in your standard dashboard. You can forecast the potential of those automated decisions, but that's still black box logic. You really need more of a middle ground that lets you preview "given these inputs, show me what the output would be".
> Is the platform still easily used by business users?
I don't know and honestly I don't believe there's a system that actually achieves it. But this it does go beyond BI in actually being aimed towards running code based on results. Which makes it closer to Labview-with-graphs than to BI.
This. Knime is similar to BI tools, but it is more action oriented. The reports may not be as pretty, but it can handle the entire ETL pipeline and actually do things besides make interactive pivot tables.
I've found that the vast majority of stakeholders want a dashboard either to (1) spend other people's money and feel important or (2) create opportunities for multiple "version of the truth" and thus prepare multiple narratives to pick and personally gain from when needed. Middle managers would rather reserve the right to say X dashboard has a metric that doesn't reconcile with Y other reliable source, than to actually collaborate with competent analysts and build a holistic info platform. We need to stop casting such a wide net and selling the dream of a glamorous all-knowing BI tool and instead, service analytics on a case by case basis. Over time this can inform practical dashboard design, kept up by a small team of experts and largely ignoring the "laundry lists" of every prospective user.
I've been doing a fair amount of BI work myself recently.
I reckon of the 90% that don't make any data driven decisions with the dashboards they're provided, there are three causes I run into frequently.
1. They can't describe what info they need
2. That info isn't available.
3. They don't make decisions based on data at all.
I feel I'm getting better at addressing 1 and 2 the more I do it, but I figure there's a fair amount from group 3 who demand the reports nonetheless...
As an aside, you mention Tableau, QlikView or PowerBI, have you tried Apache Superset at all?
Let's be fair and provide the other side's view: gathering focused data can prove the need for a decision that faces internal (and often irrational) opposition to necessary change.
One way to solve is to automate the action from insight part and inject into users workflow. I am looking at this now with support of senior sales stakeholders.
Simple example:
Client X activity has been trending down vs it's peer group, Call Client X to understand what's going on.
This is the direction that I'd like to see more people moving in: pushing insights to users rather than expecting users to come to dashboards and get those insights.
Nobody is saying "I'd love another tool that I am expected to check every day", both embedded analytics and insights are how to drive adoption and unlock value from data.
This is part of the expectation of business analysts, that they will surface and communicate data not immediately being asked for by executives or other stakeholders. This usually means running a meeting and presenting findings, and then pairing with another stakeholder to flesh out further opportunity.
If there's no one to pick up the baton, there's very little that can be done about that.
Are there any good examples of this already out there?
It feels like a good idea, but I imagine it would be very fragile in black swan events.
Maybe a mix?
Provide the dashboards, but allow users to set up alerts/workflows, they understand the problem well, and if they know what they're going to be going to the dash to find, let them skip that step. In case something new happens, they can set up a new alert/workflow/whatever for the next time it happens.
We provide power bi access to the data our app runs on and it has been a mixed bag. The customers absolutely love it. But they build dashboards where hitting the refresh button can bring a super expensive db to its knees for half an hour.
So then we have to get our db people in to rewrite all of their reports so they don't do this which eliminated the benefit of having the customers do it.
In the end there is just no avoiding having experts build the dashboards.
Interesting as we're looking at going long with PowerBI embedded for a client facing app and ditching our home grown effort which is always playing catch-up and costs a fortune.
I did the same thing last year for a client. So far it has worked pretty well. Client analysts create the reports and then deploy them for end users to view through a browser. End users can't do toooo much regarding playing with the data. It is slightly better than static data but the real value has been the client analysts being able to modify it without needing dev help (no JS chart libraries to mess with!)
I couldn't agree more! We built Lexio to address exactly that issue. (Our tagline is literally "Stop building dashboards that no one uses".) It does data storytelling (discovering the insights and conveying them in natural language) on the server and gives users a newsfeed experience for consuming them. I'm happy to talk shop if folks are interested in how it works. https://narrativescience.com/lexio
* Provide a clean minimal view of the data that other teams can pivot table on top of or download in the BI tool. We do onboarding, support and all that for this data.
* Provide dashboards and reports for more complicated ongoing questions that require data that's not cleaned yet.
We don't use Looker but based on the sales demo it seems optimized for this type of workflow. The core analytics team maintains the data and complex workflows while other teams have business analysts for day to day questions.
This is the direction where most BI work is heading for: you need a team of data engineers that can make sure that the data for your BI dashboards is delivered and in a good quality. Everything else that doesn’t need it is probably simple enough to be run as some PowerBI dashboard on top of production DB. I think today most effort in those heavier BI cases is spent on data transformations and making sure that data flows/batch jobs run uninterrupted.
If you're not careful, the data engineers team is often overloaded with requests from the analysis team.
With big data warehouse, more often than not, the fancy dashboards would not run fast enough without a dedicate pipeline that materialize data before hand. Too many useless dashboards and your data team would not have enough manpower to do anything else other than operating the existing pipelines. As a result, new dashboards become slower than old dashboards due to lack of maintenance/optimization, and everyone complain about the poor performance.
A lot of what users need can be done this way in my experience. Also giving their people access to customise reports/exports can go even further. I'd advise against direct access to “real” production though: have the dashboard and custom reports run against a (probably readonly) replica of some sort¹ so runaway reports don't impact live app performance.
[1] via replication, log shipping, etc... — all major databases should support at least one method that can achieve this with minimal latency in updating the replica under normal conditions
>A lot of what users need can be done this way in my experience.
In my experience, no, as the company gets to any complexity and size. The production DB is often optimized for use by the ORM/language of choice or there's twenty of them for microservices. This makes it really awkward to do some things in SQL. So now your engineering team is getting requests to keep changing the prod DB schema so analytics can be done more easily. More over the prod DB is likely 10% of the data your business needs. There's a reason companies like Fivetran exist to slurp fifty data sources into a warehouse. Marketing wants google and facebook and mailchimp data. Operations wants saleforce data. Finance wants stripe data. And they want all of this tied to the inhouse data.
Sure you can make reports in each of those services but then you need to tie them together and then people complain of differences in metrics and then you have a bad time.
edit: Not to mention analytics on what users do. Always fun when there's a billion row table in your production db for that.
As I understood it the goal of Looker isn't to make the lives of people who can use something like Shiny better but rather the lives of people who use pivot tables 99% of the time. So rather than having technical analysts making dashboards you have business analysts making dashboards (or at least digging into generic dashboards).
>perhaps AR/VR will provide a novel and more valuable approach
Yes. A lot of darts are being thrown as to where XR fits on which Hypecycles, but one of the most promising uses to me is as a new enduser engagement layer for BI(g) data. The new UI/UX possibilities of immersive 3D have a low level effect on what/how information can be rendered in meaningful ways. It's still early days in this space, but here is a quick list of some work in progress for anyone interested:
[1] https://flowimmersive.com/
Focused on Data driven story-telling through an inhouse 3D visualization tool. Strong on AR collaboration and responsible for "The Data Guy" popular on TikTok.
[2] https://www.badvr.com/
Industry focused on using XR for specific applications like 5G Radio Coverage heatmap modeling and Smart City interfacing tools.
[3] https://3data.io/
Use of XR for IT Operations Center applications (NOC/SOC)
I work as a data engineer in a BI team, with our product being part of a large software suite in healthcare. We got a small number power users who really use the dashboards, who get real result from them and who actively help our development. We also got a larger group of people who use it once in a while.
But most people seem to log in once or twice and then never again. And if you ask why it’s often because they couldn’t find the answer to their incredibly specific one-time only question, so they decided it’s all useless and they continue to work on their own personal homegrown BI-suite in excel.
Also, we use Qlik and I absolutely hate it. Luckily I don’t have to work with it too often but whenever I do I feel like I’m always fighting it. Using Power BI and Tableau felt much more like working together to solve issues.
> And if you ask why it’s often because they couldn’t find the answer to their incredibly specific one-time only question, so they decided it’s all useless and they continue to work on their own personal homegrown BI-suite in excel.
Hah this is probably me. The university I work for uses a BI tool (MicroStrategy) to track students/majors/etc. But I usually find it easier to use the BI tool primarily as a way to get a CSV or XLS export of enough source data to answer my question, and then do the actual analysis in Python+pandas. I can probably formulate my query in their weird proprietary query builder, but it seems unnecessarily complicated and I already know how to do data analysis in Python.
If you're talking about Power Query, the best way to use it is to write code directly once you observe what the interface generates. It's a rather nice kind-of-functional language that really comes into its own when you start using/creating higher order functions.
Power Query gave me an intense craving for a form of SQL with a similar facility for functions.
For example, I have a lot of experience writing Oracle SQL, and Power Query didn't offer "grouping sets". But I realized it could be implemented using functions. It would be so great if SQL supported that.
>they continue to work on their own personal homegrown BI-suite in excel.
Why aren't you providing your dashboards via Excel, then? What exactly does the fancy BI software do that Excel won't? I have used Qlikview but not Tableau or Informatica, and I'm a bit fuzzy on what people mean when they say "Power BI". Is using Power Query what you would call Power BI? Or does that mean using the data model stuff? Power Query is the part of Excel that I particularly like because data kind of stays where you put it.
Power BI is a tool from Microsoft, mostly unrelated to Power Query (although you can pull data from Power BI into Excel via Power Query, but you can also do that with just about anything tabular).
PowerBI is kind of a MS Access for BI. It loads data from your warehouse and gives both visual builders and query interfaces to data. IE, it's a specific product name - the data viz tool from MS to accompany their other services.
It seems to be part of Excel since 2016, no? It appears to me that it's morphed over time and I'm not sure to what extent it remains a separate product.
I've used both DAX and M (Power Query) via recent versions of Excel. Obviously there are plenty of ways to interface with data.
What I'm seeing there looks as though it is targeted at people who make the decision to buy products like this, but not to use them. I'm not seeing what couldn't be done with Excel 365 alone. There are screenshots with verbiage about how well it works with Excel. And they look like...Excel.
I guess it supports R, and maybe makes it easier to seamlessly refresh a report. That's aggravating though, that it makes me suspect Excel might be purposely rough around the edges in order to not displace Power BI.
As I've said a few times, I like M and wish they'd commit to it.
I honestly suspect Microsoft is repackaging functionality that already existed for many years in their "ball of mud" software, with a shiny new wrapper and better defaults, but practically unusable, because people who license software are easily manipulated.
Power Automate is the most awful example of this I've used so far.
You say it's a competitor to Tableau, well, Tableau is a competitor to Qlikview, and I've used Qlikview and Excel enough to consider Excel better in every way that I care about. (My organization uses SharePoint, so that's how reports are usually distributed)
Would love to hear your feedback for https://hal9.ai -- We are building an open source platform for data analysis based on reusable code blocks and a community to build and monetize their contributions. We are pretty early in our journey, launched our alpha and getting ready for our beta release, but would love to hear your thoughts. You can find me at javier at hal9.ai. Cheers!
On the flipside though, I've had some success with self service bi platforms like metabase.
Allowing stakeholders some leeway to conduct their own independent analysis (after a short training session) has allowed our strained data team to hand over simpler analysis to focus on the harder problems.
I am working as a consultant in the field for some time. The approach what I found that works very well is to create a BI org that is driven by the business needs and only have data for what you actually need.
Amplitude has been incredible, at my new workplace. Fast and lots of pretty graphs are easily generated. Also used by tonnes of different people throughout the company!
Firstly Tableau, QlikView or PowerBI are all pretty much doing the same thing whatever flavour you prefer.
We find that maybe 10% of users will actually use them to garner new actionable insights and 90% will moan there's too much data for them to process and no one appreciates the immense amount of data munging that goes on behind the scenes to make those pretty graphs.
Where do we go? Personally I see a lot of the insights being automated and turned into actions on the server side without anyone touching a BI tool. This can be achieved with rules based approach and perhaps some correlation, trend analysis over time series data. Where you do need to go beyond that then perhaps AR/VR will provide a novel and more valuable approach.