I think the post is spot on and it shows the Satchel team put did a ton of research into the post. There are two comments I would make on top of the post. First, know if the advice applies to you. If you are pre-product market fit it's probably too early to think about event based analytics. If you have a small number of users and are able to talk with all of them, you will get much more meaningful data getting to know them than if you were to set up product analytics. You probably don't have enough users to get meaningful data from product analytics anyways.
Second, while the autotrack functionality at Heap is fantastic, what I saw was that a significant portion of Heap's customers were not able to use it. This primarily happened because in addition to using Heap's autotrack to collect data, a lot of Heap's customers were also using Segment to collect and route the data between different tools. This created two different sources of truth for the data and Segment usually wound up winning. For that reason, I left Heap ~18 months ago to start Freshpaint (freshpaint.io). Freshpaint is an autotrack based alternative to Segment, allowing you to autotrack data and feed that same dataset into all your different tools. That way you get all the advantages of autotrack without needing to maintain two sources of truth.
Another autotracking tool is https://useitbetter.com which combines quant analytics with heatmapping and a/b testing.
I believe Mixpanel also offered autotracking for a while, but it didn't work out for some reason? It's probably difficult to mix pricing for manual tagging nad autotracking since the latter produce massive amounts of data.
I use ublock, but the site works fine for me. Are you blocking JS? Looking at the source code it's some kind of a javascript single-pager, built entirely in the app itself.
I’m always suspicious of articles like these that keep bringing up one company that fits their criteria. That tells me the criteria was constructed around a specific offering, not the other way around.
We updated our packaging/pricing a few weeks ago. Our Free plan allows for 100k MTUs now (100x more volume), and our Growth plan is cheaper. https://mixpanel.com/pricing/
We wanted to make a big improvement in both these categories for customers:
- "How far you can get without paying"
- "Paid Plan Affordability"
Just an FYI since that change happened since this Satchel review was published in May.
We're kicking the tires on Mixpanel - currently we're prepping for an alpha release, and there are a few things about the dashboards I wish either don't exist, or are not easy to find:
1. I wish I had more options for visualizations. Fot instance, at this scale, it would be really nice to have a scatter plot of events at exactly the time they occur, rather than by the hour
2. I really wish I could customize the graphs more - like if I have a 3 day view, I would rather group the events by the hour rather than the day, and it seems like I can't do that.
3. Is there a place to just query all the events, and get them in a table form? Like I would love to just be able to get all of a particular event in a certain time period, and then look through the properties and exact timings one by one.
Anyway, just some casual observations from a potential user.
I know PostHog is only lightly mentioned in the article, but I would highly recommend checking them out! [0] (They're also open-source and provide a self-hosted solution. [1])
Yeah they do Fullstory-type stuff, Heap-type stuff, plus they have an SDK in most popular languages (which no other company has.) If I didn't already set my stuff up before I had heard of them, I would have gone with them. They have the best parts of other tools.
Unless their stuff doesn't work well. I only spent a couple of hours so I might not have noticed UX issues that would have been painful to deal with.
I was also befuddled by this. I was using event tracking with GA at least 3 years ago I think, so it was difficult for me to take this article seriously as it hammers on this point repeatedly
I’ve implemented both amplitude and mixpanel for event tracking on Django apps.
I currently have a project that uses both auto JS and python-based track() calls using mixpanel.
I’ve found people commonly block js analytics and it isn’t safe to rely on for user behavior.
For example, you can instrument react hooks, but you really aught to instrument the api calls they rely on or they will not get captured in many circumstances.
I also have found mixpanel’s anon / known user ID merge to be poorly documented and the community support to be kind of slow and did not get the sense the folks answering had not personally done the implementations being discussed.
One of the biggest things I wanted out of event tracking was to build event bases web hooks to trigger drip-contact events.
Mixpanel quietly deprecates web hooks a few months ago and now wants you to use partners that process your data and do this for you.
Out of curiosity, what category is your product in? I don't think i've found a huge amount of people block js analytics, but thats probably due to the categories of products I've worked in.
We use event-based analytics. A super easy way to do it is to create an AWS Lambda function that you can hit using a Rest API. A simple `console-log` statement with JSON.stringify will log it to Cloudwatch logs. And bam, you can create basic charts and stuff with [Log Insights](https://docs.aws.amazon.com/AmazonCloudWatch/latest/logs/CWL...). Up to 5Gb in the free tier. Easy dashboards. All data is in AWS (so less tools to keep up with)
Tradeoff - their query language is poor, and I really want to do cohort analysis. But you can export as CSV, or easily push to Datadog.
"Paid Plan Affordability" for Heap is listed as "noticeably better" (green), but on Heap's website, they either have a limited free tier, or a business plan "Starting at $12,000/year". Is that what's considered affordable for a startup? This whole website seems like a shill for YC companies.
Great writeup here but I have a question on whether Product Analytics is at all appropriate for Startups. I'm incredibly biased (this problem is why I built Chartio), but I think these tools end up being quite bulky, expensive and a non-agile way to do analytics on your product. I feel these products are best suited for later stage companies trying with great tracking detail to improve a funnel or engagement metric, not simply monitor usage.
The majority of the important things for a startup are usually already being tracked with timestamps in the database (new signups, new users, churned users, new todo items, etc). If a key metric is not directly tracked, there's usually a good enough proxy available somewhere in the database. A Busineess Intelligence product, or a SQL to chart tool is much more applicable and affordable in the startup stage.
There are likely going to be responses here "But what about..." and I'll premptively respond to those with the question: Could that be answered/solved almost just as well with a query to the database, without sacrificing the agility, extra setup and data collection time, and expense overhead? Those are all very costly things for a startup.
I think the post is spot on and it shows the Satchel team put did a ton of research into the post. There are two comments I would make on top of the post. First, know if the advice applies to you. If you are pre-product market fit it's probably too early to think about event based analytics. If you have a small number of users and are able to talk with all of them, you will get much more meaningful data getting to know them than if you were to set up product analytics. You probably don't have enough users to get meaningful data from product analytics anyways.
Second, while the autotrack functionality at Heap is fantastic, what I saw was that a significant portion of Heap's customers were not able to use it. This primarily happened because in addition to using Heap's autotrack to collect data, a lot of Heap's customers were also using Segment to collect and route the data between different tools. This created two different sources of truth for the data and Segment usually wound up winning. For that reason, I left Heap ~18 months ago to start Freshpaint (freshpaint.io). Freshpaint is an autotrack based alternative to Segment, allowing you to autotrack data and feed that same dataset into all your different tools. That way you get all the advantages of autotrack without needing to maintain two sources of truth.