

Ask HN: Making sense of your metrics data - what are your tools of choice? - acgourley

There are many methods for looking at data in an event based fashion. Flurry, Mix Panel, Google Analytics, Omniture, and in-house server logging come to mind.<p>There tools for stream based metrics/querying, which doesn't seem applicable to most startups, but if you one of these that would be cool to hear about.<p>Finally there are tools for enterprise grade business intelligence. I've never had to work with these, so I can't comment.<p>What do you use, what are you forced to use in-house solutions for?
======
DEinspanjer
Mozilla uses WebTrends for most of its web metrics like visitor and session
analysis. We use SiteSpec for multivariate testing. For everything else we use
Kimball's dimensional modeling to build data warehouses and then build
reports, dashboards, and analysis off of them. We make extensive Pentaho's
suite of tools for ETL, multi-dimensional querying of OLAP cubes, report
creation/publishing, and a website to deliver interactive dashboards.

I would strongly recommend that anyone asking these questions do some extended
web reading or pick up a few books on the topics of ETL, Data Warehousing, and
Business Intelligence. I feel it is critical to have experience these fields
if you need to derive insight from data. Using open source ETL/BI tools can
help a startup without blowing the budget. When they discover the value for
their company, they can look for specialists to join the team or they can
investigate support and contracting options available.

In particular, a good ETL tool like Pentaho Data Integration is invaluable to
any company that has to do anything with moving data from one place to
another. Even if you don't use it for traditional BI.

For our Socorro project, we are making use of a Hadoop/HBase cluster to store
large amounts of crash report data. The next two versions of Socorro will be
pushing significantly more analysis into MapReduce and Hive type queries.

We are also working on a new data storage and analysis backend for Test Pilot
which is based on Riak. The interesting part on the analysis side is that it
is performed by submitting MapReduce jobs vis a simple HTTP POST to the REST
API.

------
acgourley
We use flurry for mobile, google analytics for web and in house server logging
for both. I use a custom stack of scripts, SQL and R to make sense of it.

What does everyone do for higher level analysis of their event logs? Lets say
you have a concept of users, and a concept of most actions that they take, how
do you build a picture of who you users are, what they are doing, what the
main user segments are, etc?

Am I missing out on any specialized tools? Note, tools must be able to handle
fairly large datasets, the tools I have seen that slice up SQL databases
usually end up being too slow.

~~~
bsuthoff
You could use a higher-end analytics service. Localytics is real-time,
provides dynamic segmentation analysis and provides an API for the export of
the full session-level data (not just aggregated charts), including events:
www.localytics.com

------
biggitybones
I'm currently developing a custom system to track all requests and events on
my app that will allow me to map visits and see how users move through the
site in addition to specific metrics.

We use google analytics but that doesn't provide the type of segmentation and
exact event tracking I'm looking for. It may be possible, but I find the
integration of specific event types will provide a lot of insight that google
can't. Also I really want some specific data on demo users with the new
registration less sign up I integrated.

------
martian
Other than things mentioned here, we use an in-house graphing solution built
on top of Postgres and custom views. This is great because we log almost
everything and it's easy to join across users, etc.

Tableau also seems fairly promising, though it is really slow on live db
queries. Does anyone here use Tableau?

~~~
acgourley
I haven't tried it, but now I'm curious too...

------
gosuri
We just starting using KISSMetrics <http://kissmetrics.com/> started loving
it, you actually describe your app funnel.. Also has a beautiful A/B Split
Test API

