
Show HN: Streaming thousands of data points in real time with redis and webgl - sillysaurusx
http://test.tensorfork.com
======
sillysaurusx
Being able to stream thousands of data points per second was challenging but
satisfying. All of this data is being generated in real time, on a server. You
can watch your training runs, to keep a pulse on them.

I had to fix webdis websockets; no one streamed so much data before.

All the data is also _stored_. Redis can handle as much data as you can throw
at it. Theoretically I can view the entire history of every one of these
streams. Every data point, at every step, with zero dropouts.

You lose so much information when you average losses over a batch.

Since this link will die eventually, you can find a video here:
[https://m.youtube.com/watch?v=vxwjh9gcc2U](https://m.youtube.com/watch?v=vxwjh9gcc2U)

