
Show HN: Live visualization of Kafka streams - yairwein
https://www.alooma.com/live
======
rekwah
I like the color bomb when you switch away from the tab for a bit and come
back to an onslaught of events get processed by the UI.

~~~
olalonde
Is this a side effect of how the browser handles canvas rendering or did it
have to be explicitly coded?

~~~
peburrows
Many browsers throttle javascript performance (particularly looping timers)
when the tab doesn't have focus. My guess is the "color bomb" is just a side
effect of the JS performance returning to a normal level when focus returns to
the tab.

------
jodrellblank
60 events per second and it's pretty but basically unfollowable.

It's something I always wonder about visualizations - no human can follow the
amount of data in even a small system. How much are they just hype and
feelgood, and how much are they genuinely useful?

~~~
vegabook
In finance this is a classic problem and is why you see traders with
proliferations of monitors, trying to track everything, and they are anyway
outmatched by the machines. The irony is that the vast majority of tick data
points do not contain much incremental information. They're very correlated.
Think how most stocks tend to move in lockstep to the index.

We use principal components analysis a lot to cut thousands of feeds and get
the "big picture" in usually 4-6 "global" variables, and then we use PCA
regression to find the "outliers" in the rest of the data and show those. Thus
we get at least 2-3 orders of magnitude less data that allows mere humans to
actually interpret - big picture + outliers - and it's very rare that using
this simple technique we ever miss much. And it can literally cut thousands of
feeds into a couple of dozen. We've found this to be much more effective than
creating animated "dot swarms" which look beautiful but are very poor at
conveying rich information.

------
user5994461
Coincidence: I was checking out your product the other day and it seemed quite
interesting.

There was just one huge issue: Where is the price tag?

~~~
yairwein
Alooma pricing varies greatly. Our customers are paying anywhere between $1000
and $15000 per month. Because the variance is so big, we prefer to have a
conversation before providing a quote. There is a two weeks free trial though,
to test things out.

~~~
arcticfox
It would be great if you have this on the pricing page. It's really
frustrating for there to be nothing there.

My startup has a ~$50 / mo Kafka ETL system hooked up to a few things and it
seemed like a good candidate to move to hosted. But if your small clients are
$1000 / mo... Noooope, not our scale! Good thing I saw this post or I might
have spent a lot of time trying it out.

~~~
yairwein
We could tell you that over a quick email / phone conversation as well before
spending any time.

At small scale Alooma probably is an overkill.

Here is a short write-up from one of our co-founders for when it starts to
break: [https://www.alooma.com/blog/building-a-professional-grade-
da...](https://www.alooma.com/blog/building-a-professional-grade-data-
pipeline)

------
saint-loup
I thought it would be some kind of visualization of Franz Kafka's streams of
consciousness. Oh well, another context collision.

------
bjoernw
Is the source available somewhere?

~~~
yairwein
Not yet, we are still debating internally if to open source it or release it
as a docker image.

Feedback will be welcome!

~~~
kkirsche
Um..why is it one or the other rather than both?

------
apazzolini
This is _awesome_. Would love to be able to hook this up to our Kafka queues.
Please open source it!

~~~
Godel_unicode
s/queues/topics

------
ecesena
Curious, what do you use for mysql -> kafka?

~~~
yairwein
pymysqlreplication ([https://github.com/noplay/python-mysql-
replication](https://github.com/noplay/python-mysql-replication)) + home brew
Kafka producer.

------
znt
Very beautiful interface.

~~~
itamarwe
Kudos to Itay Niv for the amazing design, Stav Mizrahi (@stavmaz) for the
implementation! And @ronihcohen for the inspiration!

------
iblaine
What does alooma generally charge? Just curious.

~~~
yairwein
Alooma pricing varies greatly. Our customers are paying anywhere between $1000
and $15000 per month. Because the variance is so big, we prefer to have a
conversation before providing a quote. There is a two weeks free trial though,
to test things out.

------
giaosudau
I want this deeply!

------
alvaromuir
That is effn' cool.

~~~
yairwein
Thanks

------
andrewvijay
Mozilla in android crashed within 10s. But it was totally worth it.

------
bubuli
i need this.

~~~
yairwein
Coming soon!

------
shyrka
Want

