I just wanted to share a piece I recently wrote on Dev.to about some of the work we've been doing at Dittofeed. Dittofeed is an open-source alternative to platforms like customer.io, Klaviyo, and Braze.
If you're interested in how to segment users by processing semi-structured event logs at scale, keep reading!
This article includes code samples, as well as a link to a git repo you can use to run tests and follow along. These code samples demonstrate a simplified user segmentation setup in ClickHouse.
User segmentation is key to understanding your business or application’s users’ behaviors. For example, you might want to keep track of which users have gotten stuck in an onboarding flow or which users haven’t logged in within the last 30 days.
Here's a quick rundown of what I covered:
Diving into ClickHouse: Why we (and you) should use ClickHouse for your analytical workloads, and user segmentation in particular.
On User Segmentation: What are user segments, and how do we calculate them. We cover some basic pseudocode.
Tackling Idempotency: To avoid duplicate data messing up our calculations, we've made sure our setup is idempotent.
Scaling with Micro-Batching: Simulate stream processing by incrementally aggregating user events in small batches. This is really the core of the article!
"Event Time" vs. "Processing Time": When did an event happen and what are the different ways we can measure time? How do we accommodate different measures of time in our application?
I'd really love it if you could take a moment to check out the article and maybe even drop a star on the Dittofeed GitHub repo. Your support and feedback would mean a lot!
If you're interested in how to segment users by processing semi-structured event logs at scale, keep reading!
Article: https://dev.to/dittofeed-max/how-we-stopped-our-clickhouse-d... Dittofeed's primary repo: https://github.com/dittofeed/dittofeed
This article includes code samples, as well as a link to a git repo you can use to run tests and follow along. These code samples demonstrate a simplified user segmentation setup in ClickHouse.
Accompanying repo for the article: https://github.com/dittofeed/clickhouse-segments-tutorial
User segmentation is key to understanding your business or application’s users’ behaviors. For example, you might want to keep track of which users have gotten stuck in an onboarding flow or which users haven’t logged in within the last 30 days.
Here's a quick rundown of what I covered:
Diving into ClickHouse: Why we (and you) should use ClickHouse for your analytical workloads, and user segmentation in particular.
On User Segmentation: What are user segments, and how do we calculate them. We cover some basic pseudocode.
Tackling Idempotency: To avoid duplicate data messing up our calculations, we've made sure our setup is idempotent.
Scaling with Micro-Batching: Simulate stream processing by incrementally aggregating user events in small batches. This is really the core of the article!
"Event Time" vs. "Processing Time": When did an event happen and what are the different ways we can measure time? How do we accommodate different measures of time in our application?
I'd really love it if you could take a moment to check out the article and maybe even drop a star on the Dittofeed GitHub repo. Your support and feedback would mean a lot!
Catch you all soon!