Ultimately, objects are always drawn in the screen by setting pixels into it. Plotting a point by setting a pixel is entirely reasonable, and can be indeed done directly, in realtime, for several millions of points. I just tested the C program below, compiled with gcc without optimizations, and it gives about 80 fps for three million points (on my 6 year-old thinkpad). My point: CPUs are ridiculously fast, and you can indeed do a lot of large-scale data visualization without need to meddle with the GPU.
#define FPS 80
void plot_points(
float *o, // output raster array (w*h)
int w, // width of raster
int h, // height of raster
float *x, // input point coordinates (2*n)
int n // number of input points
)
{
// initialize the output raster
for (int i = 0; i < 2*w*h; i++)
o[i] = 0;
// accumulate the points that fall inside the raster
for (int i = 0; i < n; i++)
{
int p = x[2*i+0];
int q = x[2*i+1];
if (p >= 0 && p < w && q >= 0 && q < h)
o[w*q+p] += 1;
}
}
#include <stdlib.h>
int main(void)
{
int w = 1000;
int h = 1000;
int n = 3000000;
float *x = malloc(2*n*sizeof*x);
float *o = malloc(w*h*sizeof*o);
for (int i = 0; i < 2*n; i++)
x[i] = 1000*(rand()/(1.0+RAND_MAX));
for (int i = 0; i < FPS ; i++)
plot_points(o, w, h, x, n);
return 0;
}
// NOTE: if this program runs in less than 1 second, it means that it
// is faster than "FPS"
Fastplotlib is very different from bokeh and holoviz, and has different use cases.
Bokeh and holoviz send data to a JS front end that draws (to the best of my knowledge), whereas fastplotlib does everything on the python side and uses jupyter_rfb to send a compressed frame buffer when used in jupyter. Fastplotlib also works as a native desktop application in Qt and glfw, which is very different from bokeh/holoviz.
Fastplotlib also has higher raw render speed, you can scroll though a 4k video at 60Hz with thousands of extra objects on your desktop which I haven't ever been able to accomplish with bokeh (I haven't tried it in years, not sure if things have changed)
The events system is also very different, we try to keep the API to simple function callbacks in fastplotlib.
At the end of the day use the best tool for your use case :)
What is GSP in this context? Searching Python GSP brings up Generalized Sequence Pattern (GSP) algorithm [1] and Graph Signal Processing [2], neither of which seem to be a protocol. I also found "Generic Signaling Protocol" and "Global Sequence Protocol" which also don't seem relevant. Forgive me if GSP is some well know thing which I am just not familiar with.
Forgive me for doing this, but I used an LLM to find that. They’re exceptionally useful for disambiguation tasks like this. Knowing what an acronym refers to is very useful for next token prediction, so they’re quite good at it. It’s usually trivial to figure out if they’re hallucinating with a search engine.
Thanks, and the purpose was to show what's possible on modest hardware that most people have. We have created gigabytes of graphics that live on the gpu for more complex use cases and they remain performant, but you need a gaming gpu.
But why do you want to fit the whole dataset in memory? If the dataset is stored in a tiled and multi-scaled representation you need to only grab the part of it that is needed to fit your screen (which is a constant, small amount of data, even if the dataset is arbitrarily large).
If you insist to fit the entire thing in memory, it may seem better to do so in the plain RAM, which nowadays is of humongous size even in "modest" systems.
Maybe it's an instance of Parkinson's law [1]: if it all fits in GPU memory, just put it all in and plot it. This is much simpler to implement than any out-of-memory technique. It's also easier for the user—`scatter(x, y)` would work effortlessly with, say, 10 million points.
But with 10 billion points, you need to consider more sophisticated approaches.
I'm in the same boat as the person you replied to, but have zero experience with remote plotting other that doing static plots in in a remote session in the interactive window provided by VS Code's python extension. Would this also work there, or would I have to start using jupyter notebooks?
non-jupyter notebook implementations have their quirks, eventually we hope to make a more universal jupyter-rfb kind of library, perhaps using anywidget. Anywidget is awesome: https://github.com/manzt/anywidget
People have used fastplotlib and jupyter-rfb in vscode, but it can be troublesome and we don't currently have the resources to figure out exactly why.
Hi! I've seen some of your work on wgpu-py! Definitely let us know if you need help or have ideas, if you're on the main branch we recently merged a PR that allows events to be bidirectional.
reply