
All Dashboards Should be Feeds - DanielRibeiro
http://dashes.com/anil/2013/01/all-dashboards-should-be-feeds.html
======
buro9
Let me propose a maturity scale and suggest where things are...

 _Level 1) Are you recording metrics?_

That's the first basic level of maturity, without recording data you cannot
have data from which to support any decisions. Without data you are blind. The
assumption here is that you are able to record everything that you need to...
recording 3 metrics isn't enough, you need to be able to get to all your data
and record it over time.

 _Level 2) Are you able to visualise your raw data?_

This is the basic dashboard. Whether you build something pretty or use an
existing software package it's just the ability to see the data that you are
collecting. This relies on people to figure out the inter-connectedness of
data and to make inferences from it. Basic graphs and trend analysis fit in
here.

 _Level 3) Are you able to visualise insights in your data?_

This is about changing your dashboard or reports such that you can start to
consider metrics that are abstractions from the raw data but important to you.
Think of cohort charts, revenue relationships to user behaviour, UX funnels,
engagement charts, etc.

 _Level 4) Are you able to record exceptions in your reporting?_

As in, can you create thresholds against a metric such that if the metric goes
above or below the threshold an event is triggered to record it. Exception
tracking coupled with the insights in level 3 should give you the ability to
do things like release a new feature and see the resulting effect on certain
metrics and be alerted when those effects are outside of some normal range.

 _Level 5) Are you able to visualise your exceptional events?_

And here we are at the newsfeed of data, the events which stand-out are
highlighted and all of the other noise drops away. This is great for producing
a list of things you might want to know or react to... but you can't get here
without going through (and still having) levels 1-4 and it's important to
understand that different people in your organisation still need all those
prior levels (try telling your sysadmin that he can't have log files or munin
and that instead he's got a news feed).

~~~
riffraff
I think somewhere there is an hidden level to annotate your data.

Otherwise at day X+60 you'll end up wondering what went wrong since day X
(when your traffic peaked) and blame a new release on X-1 whereas on day X you
had just hit the HN/RWW/BBC homepage.

~~~
buro9
I haven't precluded the ability to treat textual notes as data points in level
1... so they'd be available down at level 4 and 5 as annotations.

I didn't really want to get into any detail on how to make a specific chart,
or capture a specific event... just a general scale of maturity whereby you
can't generally do the higher levels without having the earlier levels in
place and done reasonably well.

------
russell_h
The article is comparing fundamentally different things, which serve different
purposes.

Charts and gauges display "raw data". The downside of this approach is that a
human has to process that data to turn it into useful information. The upside
is that humans are _very_ good at this.

The feed displays higher level information extracted from the raw data. On the
upside, you don't have to have someone sitting around watching it. The
downsides are that it requires a higher up-front investment to program or
train the system to extract this information, and the system will never (given
current technology) be as good as a human for any substantial correlation.

The correct balance depends on your situation (resources, uptime requirements,
implementation details of your system), but dashboards displaying raw or
summary data that enable you to see unusual patterns, monitoring (which is
very similar to a "feed"), and comprehensive stats gathering (which you won't
look at on a day to day basis, but is tremendously powerful when diagnosing
issues) are all tools that any team should be looking at using.

~~~
dimva
I came here to post exactly this. Many professional Chartbeat users spend a
lot of time looking at their dashboards and get an intuitive feel for how
their site typically performs. From that, they can glean many insights, even
some which they can't exactly describe.

Chartbeat's UI tries to provide context. The dial shows the concurrents number
in the context of your 30-day performance, the chart overlays your current
performance on top of the same day last week, and the traffic sources show
typical values, for example.

That being said, we are adding more feed-type features. For example, we
already offer traffic spike detection on a page level in the higher-priced
Publishing product. The reason many of these features aren't present in
regular Chartbeat is that they require data scientists to create a model,
developers to build a system around it, and servers to actually run the
calculations. None of this is free. :)

------
2468ben
Please Both. You need to be able to see real-time and long-term.

All Dashboards should show goals you set or something that's unusual compared
to your past data. THAT is the reason why everyone's confused by most
dashboards - they're visualization for the sake of looking good first. When
they're done right they let you see the state of things, whereas feeds are
like alarms - too many or too few can make you stop giving a damn.

Be like a scientist - ask every question you can think of. Then try to answer
it with data and if that answer changes over time, keep it on a dashboard.
Then set alerts and snapshots to the feed, so only what's really important
will bother you.

What we need to work on is how deep you can ask questions of an external
service. Giving me my choice of 12 pre-baked table columns or API endpoints
isn't going to answer my real goals. 12 pre-designed draggable widgets aren't
either. I want to ask big questions first instead of combining little answers.

------
joshfraser
This is my dream for Torbit (<http://torbit.com>). We currently show
performance data in a typical dashboard, but I wish we had a newsfeed as well.
We're playing with some ideas, but it turns out to be a really hard problem.
I'd love to show a newsfeed with stories that say "your site is 5% faster
because you changed this one JavaScript file" or "your site is 20% slower than
usual right now due to a spike in IE6 usage in China". Those stories are
simple for anyone to understand, but really tough to generate without a ton of
false-positives. You have to account for daily, weekly or other seasonal
changes in traffic patterns, even ones you wouldn't have an easy way to
predict (like a commercial break during the Eastenders). You have to know
what's typical for each site so you can detect anomalies and also be smart
enough not to set off alarms when you get 5 visitors from Liechtenstein
instead of 1. You have to wrestle with correlation vs. causation. It's a fun
challenge & anyone who's interested in tackling it should check out
<http://torbit.com/jobs>. :)

~~~
anildash
"We're playing with some ideas, but it turns out to be a really hard problem."

Yeah, this is definitely true. But it correctly puts the burden on the
developer instead of the user. What's more, generally it makes for a _more_
forgiving environment by which the user can interpret the data. If you say
"you had a huge ecommerce day!" and it's Black Friday, the user interpreting
it will be able to put that together, even if the app (initially) can't.

Part of my bias here is that this is based on our experiences building
ThinkUp, which is open source, and thus we're hoping to iterate on exactly
these kinds of improvements when community members find the insights to be
insufficiently, well, insightful.

------
dhotson
I really liked the idea behind a product called Trendly (unfortunately it got
shut down because Twitter bought the team).

They had some clever algorithms that simplified time series data from google
analytics so that you could see at what points real changes occurred.

Some screenshots here: [http://analytics.blogspot.com.au/2009/08/api-
integration-to-...](http://analytics.blogspot.com.au/2009/08/api-integration-
to-measure-significant.html)

You can see how it's able to take something noisy like time series data and
turn it into a stream of discrete events. It's really quite clever.

------
corresation
There is a tried and true method of getting to the front-page of HN, which is
to take whatever implementation choice you've made today and make broad,
completely unsupported claims that it is universally applicable as a law of
the universe. This applies to a shockingly high number of non-news
submissions, regardless of a lack of rigor to the claims.

~~~
yakiv
HN headline: "The only way to make the HN front page is to make incorrect
generalizations."

~~~
zachrose
The week after: "Headlines that use the words 'every' or 'always' are always
wrong."

------
prostoalex
Except this one
[http://ts4.mm.bing.net/th?id=H.4680604679800715&pid=1.7](http://ts4.mm.bing.net/th?id=H.4680604679800715&pid=1.7)

~~~
2468ben
Exactly. "We're now beginning our descent, which..." (scroll scroll scroll)
"...was 20,000 feet before now"

~~~
anildash
Hah, even though I wrote the original article here, got a genuine LOL at this
one.

------
chandru89new
Now, now.. comparing two very different-intent dashboards isn't cool. I know
I'm late by 10 hours because people have already vented their feelings about
the writeup.

You can partly agree that dashboards should be smart but not every dashboard
_needs_ to be so. Creating smart dashboards requires you to actually identify
the metrics and the requirements of the end-user.

When you generalize this, you end up having a dashboard that's Google
Analytics. It's gigantic and by letting the user decide what labels he wishes
to give to the numbers, it's actually way more smarter than the 'feeds.'

That's just my humble opinion but I sure don't feel comfortable about how
other dashboards are treated in this piece.

And if someone's interested:
<http://thisiscsr.com/post/40088895750/dashboards-smartness>

------
mileszs
What I took away from this, which, reading the other comments, is not at all
what everyone got out of it, is that a bunch of graphs can be a great, but a
system that automatically makes recommendations based on those graphs (taking
some of the decision-making out of the equation) is closer to ideal.

I might be projecting, though. I talk regularly with the people behind Pirate
Metrics (<http://piratemetrics>), which does that sort of thing. (Records,
displays, but makes suggestions as well.) S, that sort of idea has been on my
mind.

------
robotmay
I've been working on my own stats tracker in my spare time (it's gradually
getting there, and aside from accidentally DOSing Pusher, it's going quite
well), and I've taken a mixed approach to the site dashboard. It has the usual
counters and maps around the outside, with a stream of activity in the centre
which shows how users are progressing through your site.

This seemed intuitive to me, and it's kind of a combination of ideas I've
taken from other stat tracking applications like gaug.es, Mixpanel, and
Chartbeat. Each application seemed to be missing one or the other.

------
mipapage
Charts, graphs etc are interfaces to data. Boil those down to a language we
are more used to using, for example, English, and it becomes easier to
interpret the data.

If the best interface is no interface, then here "language", a very common and
familiar interface, approximates "the best interface".

------
dutchbrit
All dashboards should be widgets. A widget could contain a feed. But the most
important part about a dashboard, is giving the end user the flexibility to
add/filter whatever information he desires to see in my opinion.

Show live updates, but allow users to also see older data.

------
dgudkov
Right. Charts should have meaning and this meaning should be easily readable.
We tried to make twitter-like comments in charts (<http://explainum.com>) but
it didn't gain any popularity. Perhaps there is a better approach.

------
vietor
One of the key benefits I get form a dashboard is being able to recognize that
something's wrong, even when it's entirely different than anything I've seen
on that dashboard before.

I'm not convinced a feed would allow me to do that.

~~~
dhotson
I wonder if some clever anomaly detection algorithms could help with this?

<http://en.wikipedia.org/wiki/Anomaly_detection>

