Hacker News new | past | comments | ask | show | jobs | submit login

So tapping into the change-logs of the underlying data-sources and providing a stream processing layer that's expressible in a stream SQL dialect?

btw, not being critical of your project, just trying to understand it.

Mainly thinking of explicitly stream oriented data sources like kafka, but yeah, change logs are really solved with an analogous abstraction.

There's a great paper on that: "One SQL to Rule Them All", check it out.

We also want to scale well from single computer one-of data exploration queries, to full blown clustered long-term stateful stream processing.

The point is to provide a well thought out SQL interface to as many data sources as possible, and like drill does, push down as much computation as possible.

We actually learned about drill only after creating OctoSQL, but that's another story. (We're definitely less mature currently and support fewer datasources)

Ha! I was going to link that to you once I understood what you meant.


I believe there is a similar effort going on in the Flink project. They're slowly merging in work from the Blink project to provide a unified SQL paradigm (batch + stream).

Yup, Akidaus book is a great resource too.

We're basically aiming for a middle ground between Flink, Presto and Drill.

Guidelines | FAQ | Support | API | Security | Lists | Bookmarklet | Legal | Apply to YC | Contact