

Ask HN: Best way to visualize / analyze website stats in realtime? - joevandyk

Say I'm running an e-commerce website.<p>I want to be able to (data updated once a minute):<p>- See how many orders we're getting.  I want to be able to answer questions like "are we taking more orders this evening than normal?" and "what hour do we make the most profit each day?"<p>- How many people people did X? (where X could be commenting, playing games, registering, tweeted about us, etc)<p>There's a lot of analytics software up there, but not many let you break data down into minute or hour chunks of time.  Also, we'd want to be able to customize what data is being monitored.<p>We already use Chartbeat and Google Analytics.  Chartbeat doesn't let you customize data (so we couldn't see how much profit we make per hour, for example), and Analytics isn't realtime at all.
======
pmjoyce
Excuse the blatant plug, but my startup <http://geckoboard.com> might be a
reasonable fit for the visualise side of your requirement. We pull data from a
bunch of different sources (e.g. Google Analytics, Chartbeat, Zendesk,
Freshbooks etc out of the box, but we also allow custom data uploads) and
display them on a status board.

You can use a script on your own server or a YQL query to create the file for
custom data and we pull it at regular intervals into your status board.
Earlier this week we added support for custom chart widgets.

We're in an invite freeze period for our private beta right now, but if you
mail me (details in my profile) I'll send yours out.

Just to re-iterate; we don't help you gather stats; we display stats you've
already got and want to visualise in near real time.

~~~
joevandyk
Is it possible to view stats in the past?

~~~
pmjoyce
That depends on what you mean, for the custom data uploads you can feed it
anything. For Google Analytics you can view the stats (e.g. pageviews) for
today (partial), yesterday, past 7 days & past 30 days and compare them to the
previous period e.g. page views in the past 30 days are up 12.5% on the
previous 30 days.

However, we don't store data (besides temporarily caching it) so if you want
to view history it has to be present in the existing feed.

~~~
joevandyk
When you query the feed, can you pass in date/time ranges? So, we could query
last month?

~~~
pmjoyce
There are 2 types of feed we accept. The first is a standard connection to one
of a variety of web services such as Google Analytics where you authenticate
(in this case via OAuth) then choose what you want to display. The second type
is custom data uploads where you simply point widgets on your dashboard to an
XML/JSON source with an explicitly defined structure and we pull it on a
regular basis.

The first type use logic which we have defined up front to extract the
appropriate information for display. We apply little or no logic on the custom
data feeds as that is assumed to have been carried out by the provider,
usually when the XML/JSON file is generated.

If you have an obscure source of data, or it's sourced from a service we don't
yet extract from, then I'd suggest writing a script to extract and massage the
data into a format we accept and point your dashboard at it. We have
definitions and example scripts in our support forum and I'm more than happy
to give you a hand setting it up if that would be useful for you.

------
revorad
I'm building such a product at my startup Pretty Graph. Please email me at
hrishi@prettygraph.com if you'd like to discuss your needs.

Also, check out <http://www.metricly.com> and <http://chart.io>.

------
elliottcarlson
You could look in to customizing Hummingbird Stats
(<http://hummingbirdstats.com/>) - the real time factor is already there; you
would just have to add some additional things to track against.

~~~
joevandyk
Hummingbird doesn't look it supports going back in time well?

i.e. we'd want to ask "how many t-shirts did we sell yesterday".

~~~
joevandyk
Plus I think we want something that doesn't require javascript.

~~~
elliottcarlson
I guess I misunderstood your requirement of real time - I am guessing you
meant that you want to see up to date information when visiting a page?

~~~
joevandyk
Yes, pretty much. It's fine to hit refresh and see the data as of a couple
minutes ago.

Needing to wait a day (as in Google Analytics) would be too much.

------
fudge
A good way to visualize what happens on your site right now is glTail at
<http://www.fudgie.org/> \- you can do a lot of the real-time queries you want
with a custom parser.

------
gilbax
Is Chartbeat the only real time analytics engine on the market ? ( I'm asking
this because I assume you invested more than me, and I'm interested in knowing
the current state of this market )

------
dannytatom
I've never used it myself, but <http://mixpanel.com> sounds similar to what
you're looking for.

