

Ask HN: What analytics do you use when Google Analytics reaches its limits? - dumbfounder

From Google's page on data limits: "Data collection limit: If you exceed 10 million hits per month, there is no assurance that the excess hits will be processed." - We are doing more than a million hits per day, so what now?<p>Right now Twicsy continues to use GA, and also Quantcast, but their numbers (on page views in particular) have started to diverge. On some days the page view counts are 60-70% different. We are not sure who to trust.<p>We have tried hosting Piwik ourselves and also a hosting provider, it just stopped working after a few days for some reason in both cases. Clicky doesn't support websites that get more than 1 million page views per day on one website. We can't find anything else appealing.<p>We don't have a lot of money to spend, at most we would like to spend a few hundred dollars per month. Is there something out there that is worthwhile at that price? If not, what are our alternatives?
======
omarrr
My suggestion would be to look into a workaround _inside_ of GA.

Eg: Create a new suite within GA (UA-XXXXX-2), and use the new ID for a subset
of your page views. Look into how your page views are happening across your
site and plan on distributing the analytics to fall under 10M. For instance,
use your main ID for landing, top and trending pages and a second one for the
rest of the site. If you are near or over 20M views you might want to consider
a third or forth ID.

Good luck!

===

Edit: On a second thought, one of our sites reports over 10M a month and we
never run into issues. GA doesn't process more than a small percentage of
visits for the reporting, so your issue might not be related to the 10M cap.

~~~
dumbfounder
Thanks for the suggestion, but I see a few issues with this approach: 1\. You
can't track the number of unique visitors anymore 2\. I will lose the data
history I have now with GA. I am actually not sure that it is incorrect, I
just am suspect at this point and need validation. If it is indeed correct
then I don't want a break in the data.

But, something to think about, so thanks.

Re: Edit: Are you sure? I mean, really sure?

------
centdev
Are you sure it's GA? Our numbers differ between GA and quantcast because
google isn't tracking opera mini on feature phones but quantcast is. We are
running 450m pageviews a month through ga and its typically a day behind
processing. Outside of that its missing feature phones, but I don't believe
it's because of our numbers.

~~~
dumbfounder
Nope, I am not sure, so thanks for the input. The numbers are 60-70% different
for us between Quantcast and GA, so I don't think opera mini would make up
anywhere near that difference.

And, are you sure it isn't dropping data? I mean, really sure?

------
t0
Reddit uses GA. There may be some special arrangements you need to make.
<http://blog.reddit.com/2011/02/reddit-billions-served.html>

~~~
itsprofitbaron
They'll use the GA Enterprise version which serves up to 1 Billion "hits
processed per month" its £90,000[1]/$150,000[2] as a flat fee per year.

[1]
[http://www.google.co.uk/intl/en_uk/analytics/premium/feature...](http://www.google.co.uk/intl/en_uk/analytics/premium/features.html)

[2]
[http://www.google.co.uk/intl/en_us/analytics/premium/feature...](http://www.google.co.uk/intl/en_us/analytics/premium/features.html)

------
dangrossman
Server access log analyzers (awstats, webalizer, etc)? Those are free, and get
you most of the same basic reports. I don't know of any SaaS analytics that
aren't out of your price range at 30M+ data points per month.

------
staunch
<http://clicky.com/>

<http://clicky.com/help/pricing#/compare/>

~~~
dumbfounder
They only allow up to 1 million page views per day per site. We are over that
number.

~~~
staunch
On the register page it says

"If your site has more than 1,000,000 daily page views, please contact us
before registering!"

I take that to mean they'll work something out with you, and based on their
pricing it should be well within your price range.

------
lifeguard
Consolidated logs + Splunk.

<http://www.splunk.com/>

~~~
dumbfounder
Maybe I am wrong, but I don't think that analyzing logs is anywhere near as
effective as the javascript based solutions that use cookies to track users.
We receive a lot of bot traffic and I don't think log analyzers do a good job
of telling the difference.

~~~
lifeguard
It is trivial to filter out the bots IMO. You can also correlate system logs
with application logs.

