I thought there are only vertex/fragment shaders and compute shaders aren't supported yet? Do you just pretend everything is pixel data?
We store NDArrays as floating point WebGLTextures (in rgba channels). Mathematical operations are defined as fragment shaders that operate on WebGLTextures and produce new WebGLTextures.
The fragment shaders we write operate in the context of a single output value of our result NDArray, which gets parallelized by the WebGL stack. This is how we get the performance that we do.
It will be interesting to see if the industry will produce a standard for GPGPU in the browser. Giving that the desktop standard is less common than a proprietary standard.
They did: webcl Sadly, it had multiple security issues so the browsers that had implemented it in their beta channels (just Chrome and Firefox, I believe) ended up removing it. And now, I think it's totally stalled and no one is planning on implementing it.
Also sadly, SIMD.js support is coming along extremely slowly.
WebAssembly is coming along quite nicely.
And SwiftShader is a quite nice fallback for blacklisted GPUs. They simulate WebGL on the CPU and take advantage of SIMD:
We wanted to do hardware accelerated deep learning on the web, but we realized there was no NumPy equivalence. Our linear algebra layer has now matured to a place where we can start building a more functional automatic differentiation layer. We're going to completely remove the Graph in favor of a much simpler API by end of January.
Once that happens, we'll continue to build higher level abstractions that folks are familiar with: layers, networks, etc.
We really started from nothing, but we're getting there :)
I ended up switching to OpenCL since I am running this on my desktop. Just curious to see what you did. Thanks!
It's inconvenient because shaders are meant for vertices/fragments but it still works.
You want fast matrix multiplication? Pretend you're doing texture shading in a language primarily used for dynamic web frontends!
A texture is just data and a shader mutates that in a highly parallel way with specific coherence semantics. Before knocking it, I suggest educating yourself a bit more.
Don't expect anything better soon as companies focus on performance per watt and will only develop things in that path.
Node adds file support, but libraries for stuff like dataframes are still in their early days. That might not be a problem for example problems, but real world data needs cleaning up before you can start doing ML.
JS has improved over the year but you can also go with a typed language if you wanted like purescript or typescript.
You have created an abstraction that is pretty portable. You'll probably be able to capture new performance enhancements as they occur on web runtimes. Maybe I'll try it out.
* Error in line 392
After an hour of searching, the solution is:
"Just install these 100 packages. Package 1-20 need to be compiled from source, because the project requires an old version. And they all have dependencies of their own."
Would love to see some exciting discussions around machine learning in JS. I am exploring the topic heavily at the moment and I am keen to apply/write/educate/learn more about it.
I personally like this one the best (also uses deeplearn.js): https://teachablemachine.withgoogle.com/
I dig it.
I also hope internet bandwidth grows to the point where hundreds of megabytes takes milliseconds to download, and that computer memory increases until hundreds of megabytes is trivial, and that the economics of ISPs mean I get truly unlimited data without incurring charges, but when all that happens I definitely do hope we're utilising technology in ways that includes embedding complex neural networks in webpages as a normal, typical thing we do.
That particular future will be awesome.
This is what's actually happening right now. This is the near future.
1) The theoretical speed is one thing, the real one is another one.
2) There is more than one node connected to the network
3) TCP needs some package interchange to increase the window size
4) Latency with the server where the page is hosted is not going to be 1ms, you are lucky if it is around 20ms.
5) Serves have more RAM because they use better CPUs, it means they are more expensive.
6) Who connects to websites nowadays? Mobile is the way to go now (It means slower CPUs, less RAM, less power consumption, etc)
7) Why are you going to transfer a complete netural network instead of sending the data to the server. The data is going to be smaller than the network and you avoid the risk of someone cloning the neural network for their web site.
8) There are no useful applications in web sites for neural networks. It is a non sense for me (please probe me wrong and show me an use case where it makes sense)
Privacy is one reason I can think of. By running the trained network locally I can benefit from it without sharing all my data with the company serving it.
Data size is another reason. If I want to categorize 50GB of images using a 300MB model it makes a lot more sense to download the model than upload the data.
8) There are no useful applications in web sites for neural networks.
I admit that I haven't got a clue how these things will be useful. If I did then I'd have started a business to implement it already. My lack of insight isn't a good reason to suggest they'll never be useful though.
The faster the network is the more reasons you have to upload your data. The time to process an image in the cloud is going to be virtually 0 with small latency as the resources in the cloud are huge compared with what you can have at home.
Don't forget that folks were saying this about the web when images / rich media were becoming prevalent!
Details please. What techniques are used to reduce the model size?
The API mimics the structure of TensorFlow and NumPy, with a delayed execution model for training (like TensorFlow), and an immediate execution model for inference (like NumPy). We have also implemented versions of some of the most commonly-used TensorFlow operations. With the release of deeplearn.js, we will be providing tools to export weights from TensorFlow checkpoints, which will allow authors to import them into web pages for deeplearn.js inference.
Is is 10x 100x 3x slower than standalone GPU lib ?!
It goes without saying that a framework that does define-by-run also knows the whole graph.
By the way, if you'd make your interface more general than deep learning, your library could be the start of an alternative for numpy/scipy on JS, and it would be even faster than the original Python version because it uses the GPU. Just a thought ...
(One small downside is that JS doesn't have the nice operator overloading that Python has, afaik)