
Superconductor – real-time data visualization using WebCL, WebGL, Web Workers - iso8859-1
http://superconductor.github.io/superconductor/
======
tristanz
Definitely a big need for this stuff. Nice job!

Another approach that uses WebGL only and scales to billions of linked points
at 50fps is the inMems project by the Stanford Visualization Group:
[http://vis.stanford.edu/papers/immens](http://vis.stanford.edu/papers/immens).
The key idea there is binning and data tiles.

Is there a hard dependency on WebCL? I believe the Chrome team recently said
they have no existing plans to adopt it.

~~~
lmeyerov
I talked to Jeff recently -- they're doing cool stuff with immens. AFAICT,
Superconductor does a couple of things differently. First, it is DSLs for
making new visualizations, not running analytics within ones they've prebuilt.
Second, Immens uses server preprocessing -- we do not require it, and only use
it to drop data loading from 5-10s to < 1s. The idea of a render cloud is
great (others do this as well), so I've been chatting to Spark folks
([http://spark-project.org/](http://spark-project.org/)) about a backend to
compile our DSL into real-time graph processing on the cloud. A shorter-term
option may be to extend our CUDA/OpenCL backend to use new multi-GPU
extensions.

RE:WebCL, WebCL is simply one of our compilation targets. We can also compile
to multithreaded C++, JavaScript (naive or typed arrays), OpenCL, and CUDA as
well. For the web space, I'm working on an accelerated alternative of
(workers+asm.js)+WebGL, and going from WebCL to RiverTrail isn't too big of a
leap.

------
pyalot2
Found no live-examples to run. WebCL's not supported by my browser. WebCL's
not integrating with the WebGL rendering pipeline (no buffer sharing, no
atomic synchroniczation primitives etc.)

I'd suggest solving a few fundamental problems with the technology first,
sorry to rain on the parade.

~~~
lmeyerov
Yeah, WebCL->WebGL buffer sharing is a big deal: having to copy the data from
GPU->CPU->GPU for the layout->rendering hand-off is a ~5-10ms hit on the
animation loop.

Luckily, the Safari version of WebCL we use has integrated buffer sharing
([https://github.com/SRA-SiliconValley/webkit-webcl](https://github.com/SRA-
SiliconValley/webkit-webcl)). I'd expect that to be in the Chrome port being
developed. The Nokia port for Firefox a couple years back didn't have it,
though I don't know what's going on with the new one @
[https://www.khronos.org/bugzilla/show_bug.cgi?id=792](https://www.khronos.org/bugzilla/show_bug.cgi?id=792)
.

My benchmarks show that asm.js + workers + webgl will have that 5-10ms hit,
but otherwise performs at similar scales. If we figure out streaming data
transfers to overlap communication with computation, it would be even less.

~~~
pyalot2
Compute shaders solve essentially all these issues, and more.

The graphics pipeline consists of programmable and semi-fixed components and
it is quite often useful to interact with it. For that purpose compute shaders
introduce compute kernels, atomic counters, atomic locks, sharable buffers and
some hookups to the pipeline stages.

This is functionality gleaned from Cuda, but repackaged in a more or less
portable way. Of course compute shader kernels aren't as flexible as either
Cuda or OpenCL programs. However OpenCL is by now quite far behind "the state
of the art".

~~~
lmeyerov
Ah, interesting. OpenCL/WebCL solve this problem, so I think the question is
portability. Will compute shaders be landing in most WebGL implementations?
WebGL seems generally limited to OpenGL ES, so I had only explored BrookGPU-
style alternatives.

Targeting CUDA/OpenCL/WebCL was so that we can rely upon intermediate GPGPU
compilers. Our DSL compiler outputs a bunch of for-loops, which traditional
compilers love to optimize (the JIT for JS, LLVM for OpenCL, etc.). If we
targeted a low-level language such as nvidia's PTX, I suspect we'd still want
something like LLVM inbetween. Is there a good preprocessor for compute
shaders?

~~~
pyalot2
At least in the OpenGL world, compute shaders use more or less the same syntax
as other kinds of shaders (GLSL).

It's true that WebGL does not have compute shaders (and neither does OpenGL
ES), yet.

However OpenGL ES roughly tracks OpenGL's development with OpenGL ES 2.0 ==
OGL 2.0, OpenGL ES 3.0 == OGL 3.0, in each case with a bunch of features
stripped but with the essential advances in place. OpenGL ES 4.0 which should
come about in the next 2-3 years is very likely to include compute shaders.

------
chenster
The author should add "large dataset" to the title.

~~~
iso8859-1
It's 80 chars right now, so that can't be done without sacrificing some golden
buzz tech. And without that buzz tech, you never would have seen this...

------
koolkao
Mac only at the moment

~~~
lmeyerov
Language creator here (didn't expect to see this!). Spot on about WebCL not
being widely deployed yet, which means code generating it is mostly for kiosk
etc. purposes.

For more immediate deployment, I have most of an alternative codegen working
that emits multicore + GPU code. Basically, CSS selectors + layout are
multicore via asm.js + web workers + typed arrays, and GPU rendering via
WebGL.

If anyone is interested in an alternative GPU backend for layout + CSS
selectors, say via Rivertrail or old school GPGPU, drop a line :)

~~~
rektide
I'd love some kind of posting on FTL: on why it was introduced, what
constraints it has, what it enables Superconductor to do that couldn't be done
with a general purpose language.

Did you give any thought to trying to intake JavaScript itself as the target
language?

~~~
lmeyerov
We originally built FTL to help formalize and automatically parallelize CSS.
(Results of that work are going into Mozilla's Servo browser.) That includes
constructs for both declarative layout and declarative parallelization. Our
PPOPP 2013 paper is the best place to look right now, and a video of my talk
at SFJS last week should be posted soon.

We already have JS as a target language ;-) You can try an old version here:
[http://jsfiddle.net/lmeyerov/Awubf/](http://jsfiddle.net/lmeyerov/Awubf/) .
It's similar to the current lang version except without macros. I'm hoping to
have a full release with tutorials and a fast non-WebCL version by September,
but we'll see :)

