
No, you don’t need a real-time data dashboard - sbashyal
http://numeratechoir.com/no-you-dont-need-a-real-time-data-dashboard/
======
matwood
It's important to remember there are different types of analytics, both of
which are important for any organization. Real-time analytics are what I would
consider to be tactical. If your site is getting slammed you need to know
about it - right now! And then you need to act. If you only focus on the RT
analytics though it's easy to get stuck in local minimums, miss the forest
through the trees, or simply not realize you're measuring the wrong things.

There are also strategic analytics. When looking at strategy, sufficient time
periods are generally required. This type of analysis guides decisions in
terms of months, quarters, and years. Strategic analytics are what drive the
business long term. They provide the decions makers the data and visibility
they need to solidify their vision.

Finally, I like to think of analytic data in 3 distinct iterative stages:

First is the _data_. This is the raw data spinning off the multitude of
systems in any company. In any slightly complex system there are many
datapoints that can be captured and counted. Companies often have entire
systems just to _capture_ the data, and this is before any analysis can occur.

Next is the _information_ stage. In this stage all of that raw data is turned
into some sort of reporting. It's still a lot of data, but is in a format that
can start to be analyzed and acted upon (tactical reporting). In the past I've
used various data warehousing solutions to execute on this step.

Finally, the organization turns the information into long term _knowledge_.
This step occurs when the data has been analyzed and used to drive
champion/challenger, A/B testing, etc... and those results are rolled back
into the organization (vision and long term organizational changes). This
knowledge also helps the organization discover what other datapoints need to
be found and/or captured and the process starts again.

~~~
aausch
Different analytics are important at different stages of a company's
development. A place like Microsoft might place little to no value in RT
analytics most of the time. They tend to move too slowly to be able to do
anything in real time. OTOH, a struggling startup trying to grow from 500 to
1000 users might look almost exclusively at the RT analytics.

------
sardonicbryan
I totally agree with the sentiment in the blog post. As someone who has built
and now compulsively checks a real time dashboard however, I'll enumerate one
case where I've found real time really valuable.

I'm the product owner of a social game, and we are blessed with:

1) high traffic

2) tools that make it easy to test sales, promotions, contests in real time

3) a team that is capable of doing multiple meaningful releases a week
reliably

The result of this is that we end up making meaningful changes to our product
on a daily basis. With our high traffic, I can reliably tell you within one
hour (on the outside) whether a promotion, contest or feature is having its
intended impact, especially once I:

1) Benchmark against recent averages (and take into account the volatility in
the metric I'm measuring... everything is framed as how many standard
deviations above or below recent averages)

2) Compare against recent trends.

3) Compare against my expectations/hypothesis. After a couple years of looking
at realtime KPIs move in response to changes, I've gotten pretty good at
forecasting results.

Obviously, I also look at metrics at a daily level and track weekly cohorted
metrics out for months to make sure strategically we're maximizing for a
global maxima and not a local one. But there are a lot of pretty realtime
businesses where tactics are extremely important that could really benefit
from realtime dashboards.

------
edhallen
Great post - if anything, I think this problem goes way beyond just real-time
dashboards to extend to the vast majority of analysis done today (both by web
and more traditional companies).

The statement that particularly resonated for me was [figure out] "if you’re
looking at stats now because you’re curious and impatient, or because those
stats will actually drive business decisions", but I'd take it one step
further.

Analysis and stats are incredibly valuable when:

1) they are applied to a real business decision that is tied to actual value

2) you are willing to change what you are doing based on the result

3) you don't already know the answer

4) you have sufficient confidence in the result to act on it.

We need to get more used to stopping and thinking before we start analysis by
laying out the decision we need help making, the different paths we're willing
to take, and the amount of confidence we'll need to change our decisions. I
could definitely be a lot more disciplined about it.

(I actually wrote a blog post last week on this same topic that lays out the
above criteria in more detail and might be useful - though that was a reaction
to the profusion of ads out there calling Big Data and Analytics "hot", that
ignore how they actually drive real value. Blog post is here:
[http://www.klaviyo.com/blog/2012/07/16/the-curse-
analytics-b...](http://www.klaviyo.com/blog/2012/07/16/the-curse-analytics-
big-data-hype/))

------
sbashyal
There has been a lot of interesting discussions on growth hacking here on HN
lately. This article raises the important point that long term growth has to
be the goal as opposed to increase in sign-ups or page views. Feeding the
Internet troll, for example, may not always lead to long term growth.

------
StavrosK
This is why I chose not to provide real-time analytics for Instahero[1]. Apart
from being very constraining, implementation-wise, it simply does not provide
that much business value.

Much in the same way that looking at an A/B test invalidates it, it's better
if you let your metrics converge before you look.

[1] <http://www.instahero.com>

------
dennisgorelik
Did he mean "Circle of Friends" or "Circle of Moms"?

<http://siteanalytics.compete.com/circleofriends.org/>

<http://siteanalytics.compete.com/circleofmoms.com/>

------
jurre
Yeah, but, they're _cool_!

