

Norikra: Schema-less Stream Processing with SQL - matsuu
http://norikra.github.io/

======
loevborg
This looks like an interesting, simpler alternative to stream-processing
solutions like Spark or Storm, which require quite a lot of upfront investment
in terms of infrastructure and learning.

I'd be interested to know how this works behind the scenes. Does it
recalculate answers every time you run a query, or does it update the query
answers as the events flow in?

Also, the examples all seem to be using a time window, like this one from the
documentation:

    
    
         SELECT COUNT(DISTINCT cookie.id) AS uu,
                COUNT(*) AS imps
         FROM impressions.win:time_batch(1 hours)
         WHERE cookie.valid
    

So does this only answer questions of the type "how many unique users did we
have in the last hour"? Or can you also calculate a total count: "how many
users have visited this page in total?".

~~~
tagomoris
Norikra's queries are once registered, and then, outputs answers automatically
(until it is removed).

The query above puts "how many unique users in an hour" per an hour. Of
course, we can write queries like: * how many unique users in the last hour
per every input event * how many unique users in the last 1000 inputs * and
many more way to calculate metrics

------
repeatedly
Norikra is often combined with Fluentd log collector.

[http://fluentd.org/](http://fluentd.org/)

Fluentd document has the article for Fluentd and Norikra use-case :)

[http://docs.fluentd.org/articles/cep-
norikra](http://docs.fluentd.org/articles/cep-norikra)

------
profquail
How does this compare to the IQbservable interface in Rx.NET? (IQbservable =
Queryable IObservable)

IQbservable doesn't seem to get much press -- but here's an interesting
implementation that works over TCP:
[https://rxx.codeplex.com/wikipage?title=TCP%20Qbservable%20P...](https://rxx.codeplex.com/wikipage?title=TCP%20Qbservable%20Provider&referringTitle=Documentation)

------
gopalv
This uses Esper, right?

Esper is awesome all by itself.

~~~
tagomoris
Yes, Norikra uses Esper. That is great software.

Norikra provides:

1\. schema-less processing: Esper requires schema for all events and queries,
but Norikra query does not requires any schema

2\. nested hash/array access by extended SQL(EPL)

3\. standalone server software: applications in any languages can use stream
processing over RPC

4\. lightening deployment: only 3 minutes by jruby+gem install

And UDF over rubygems, connecting query output into other query inputs, and
much more.

