
How to use Apache Kafka to transform a batch pipeline into a real-time one - ishikawa
https://medium.com/@stephane.maarek/how-to-use-apache-kafka-to-transform-a-batch-pipeline-into-a-real-time-one-831b48a6ad85
======
sidlls
The examples in these articles are constrained ("toys") for demonstration
purposes, but too often are taken as a recipe by people who should know better
but want to build their resume.

LinkedIn didn't develop Kafka _just_ for real-time processing: dozens of
solutions for that existed for years before LinkedIn existed. They developed
Kafka for a problem _at huge scale._

A little 100MB/day of traffic doesn't need Kafka for real time processing, and
neither does a 100GB monthly data dump/ETL batch job.

