
Show HN: Upstream – Stream processing, for humans - lucasmg
https://www.upstreamapp.io/
======
dozzie
Being a sysadmin and being interested in monitoring systems, I know quite well
where and how to apply stream processing as a general paradigm. Having this
"stream-processing-as-a-service", I don't have even the slightiest idea where
it could be useful. The fact that the service only provides webUI for defining
things, and thus the processing capabilities are very limited, doesn't help.

~~~
lucasmg
I'm curious, when you say a limited set of capabilities, what libraries or
systems would you juxtapose us with?

Our large goal was to make the sharing and collaboration piece much easier
than other systems allow for. So let's say that one person might be
responsible for instrumenting a system, but others want to be able to create
their own notifications or visualizations. One user can publish a stream of
events, and then allow others to subscribe and create their own
transformations/visualizations. There are other pub/sub systems that achieve a
similar result, but generally access control doesn't exist and usually lacks
any real system of discover-ability. In the same way that github/bitbucket/etc
make repositories easier, we wanted to enable a similar effect with streams of
data. And in the same way, we built in a number of tools that make
manipulating or visualizing a stream of data easy without leaving the system
(sure, you shouldn't probably do the bulk of your code editing out of github's
text editor, but it's very handy for quick changes).

I'm not sure if that helps frame our system better. We're still working a bit
on how to position the system, so we might even try and revise our landing
page to emphasize this more.

~~~
dozzie
So your service basically provides (a) place where one can send data for
further processing, (b) place where one can process data that was collected,
(c) tools for plotting and charting available data, and (d) tools for sending
alerts on available data. And registry with available data streams.

Mind you, access control is most probably not a feature for _your customers_ ,
but for _your company_. Limiting data visibility for employees is usually
unnecessary for smallish businesses, and big businesses rarely take part in
developing somebody else's product like this.

With all this, I would say that you are like AMQP broker, except you have a
stream registry, can do a little of trivial data processing (handful of set
functions on a window), have a charting UI, and no way of extracting data
back, from what I've seen in the documentation.

And you asked what I compare your service with. These would be Splunk and
Riemann from finished software, and in terms of processing capabilities,
plenty of stream processing systems invented in academia, like Aurora.

------
lucasmg
Hi folks, I'm one of the creators of Upstream. We'll try to be around all day
to answer any questions you might have. We're feeling pretty nervous, but
excited to get your feedback!

~~~
craiglgm
I'm one of the creators too. We're excited to see what you think.

