
Show HN: Grafana Live Streaming Datasource Implementation - seanlaff
https://github.com/seanlaff/simple-streaming-datasource
======
dkhenry
I saw this presentation at grafanacon and it was awesome. I really hope the
team takes this and turns it into a first class feature.

------
kthejoker2
Very cool, but real time visualization use cases are surprisingly hard to
concoct. Almost any signal worth capturing visually at one second or sooner
intervals would be better served through automation.

For example, all SCADA systems already have command and control alerting for
instant degradation response.

~~~
iluminae
Also think about having a dashboard refresh at 5s with 50k datapoints,
literally transferring gigabytes of data to every browser looking at that
dashboard. If you streamed the data (even at slow rates) it would be a massive
traffic savings.

~~~
minimaxir
Recently I was looking into Plot.ly Dash
([https://plot.ly/products/dash/](https://plot.ly/products/dash/)) for a
Python-based real-time updating dashboard.

...until I noticed that it transfers the entire chart and all the datapoints
each update. (example: [https://dash-gallery.plotly.host/dash-live-model-
training/](https://dash-gallery.plotly.host/dash-live-model-training/))

~~~
nicolaskruchten
I'll be honest, before I started working at Plotly I had the same reaction to
Dash at first glance, but as I worked with it/on it, I realized that most of
the time, this client-server chattiness is not much of a problem from a user
perspective, and is a great developer-productivity tradeoff.

More to your point, though, this is something we're trying to address in
collaboration with the Dash community, in this PR for example if you want to
follow along: [https://github.com/plotly/dash-core-
components/pull/461](https://github.com/plotly/dash-core-components/pull/461)

------
rixed
After having looked at what this actually is, I'm a bit disappointed. I was
hopping that they wound change the various panels data API so that they would
request only the new time slices; ie. instead of querying the last hour of
data every minute, query the last minute every minute and reuse the previous
49 data points. But that's not what this is. Instead, you need a specific
panel and a specific data source for streaming to work. :-(

~~~
seanlaff
This datasource doesn't require any custom panels to function. To your point
though, it would be nice if grafana supported incremental time range querying
out of the box, since many backends don't support streaming data.

~~~
nopzor
Something we are looking at.

Problem is complicated when the query applies time series transformations such
as moving averages, where requesting only incremental data can be problematic.

