Hacker News new | past | comments | ask | show | jobs | submit login
CanvasKit – Skia and WebAssembly (skia.org)
127 points by espeed 36 days ago | hide | past | web | favorite | 70 comments



This page is infuriating to use on my phone. I can barely scroll, and when I do get lucky, it bounces back to the top and more things load.


Web development-centric websites often seem to totally ignore the need to provide a usable mobile experience. It’s maddeningly weird.


It's not unsurprising. Web development happens practically exclusively on desktop/laptops.


Maybe I'm a small ignorable minority but almost all my library skimming is done on a phone. I explore new technologies when rocking babies to bed or dozing off myself.


And yet, desktop browsers have no more need of hijacking the browser's scrolling behaviour than any other browser.


And leave scroll bar rendering to the browser or OS vendors!?

Think of the pixels!

/s

Seriously though I believe it's often done to make infinite scrolling and scroll-transitions more visually pleasant. (Not that I agree with the sentiment, but if visuals are more important than anything else...)


And yet many of the sites I build have majority mobile traffic. It's borderline incompetent to ignore the mobile ux these days.


And the clicking/drawing/dragging interactive demos ignore touch input, but also block touch scrolling.


I would like a Skia + v8 runtime. Some of my web apps are just a 2d canvas and JavaScript. Would be nice to be able to use them without a browser and bundle with something more lightweight then Electron.


I would assume the skia wasm blob can be integrated with cairo canvas, as it has the same exposed api.


What is this for and why would someone want to use it?

There's basically nothing on the website but the API. No decent overview or tutorial or even why someone would want to use this project.


Skia is a graphics engine developed in C++. Think of it as the building block for graphical interfaces, animations and everything else that involves displaying some 2D data on a screen, specially vectorial data.

For example, Flutter uses it to draw its UIs and Chrome uses it for almost everything, including rendering text parsed from HTML. Sublime Text, Firefox, Xamarin and many other projects also rely on Skia for the same sort of thing.

However, unlike, let's say, Qt or GTK+, Skia does not provide already done widgets (i.e., drop-in buttons, windows, text inputs etc.). It's like the fundamental building block on top of which you can create your buttons, animations and everything else and then display it on screen.

Now, CanvasKit is Skia ported to the browser via WebAssembly. Since Skia is a mature project and very performant (as you can guess from who is using it), it's now an alternative to Canvas API and DOM to render things on websites and will also let existing native applications to be ported to the web more easily.


I had a really hard time understanding it as well, and as someone who is supposed to at least have some understanding in this space (HTML5 gamedev) it made me feel a bit dumb.

However, from what I understand, there seems to be two things needed to be noted here:

Skia - a standardised library that can 'render' to different backends - i.e. it can render to OpenGL or SVG or PDFs.

CanvasKit (the actual one linked) - basically use Skia on the web (with webgl).

A huge boon here would be performance. For example, in WebGL, if you want to draw text, the current method is actually to render it in the canvas first, and reupload the texture to the GPU. From what I understand Skia seems to handle the rendering 'natively' so this step is not necessary. So basically it would just make it easier to draw 'primitives' since they are provided by the Skia API without having to do workarounds or writing really low level webgl fragments/shaders.

This is just from what I gathered, but yeah it is confusing. If anyone knowledgeable reads this, let me know if I got anything wrong!


Skia is just a 2D drawing API, like Cairo. It's what Chrome uses under the hood to render their whole UI I think, including (non-WebGL) HTML5 canvas stuff.

Why you would want to use this over HTML5 canvas? Good question. Maybe you want some features in Skia that aren't in the canvas API? Maybe you have a Skia app that you want to port to the web?


I can think of a use case. Writing a GPU accelerated text editor with syntax highlighting that can interface with a back end like neovim. Maybe not super useful but I could see myself doing this since my only expertise is with web technologies.


C/C++ for WebAssembly mainly used for porting existing C/C++ application to the web. I think it is one of them. You know what? Flutter uses Skia as a GUI backend. So maybe we could run a Flutter app using this someday.


I'm not sure if it's been publicly released but "project hemmingway" (I think) is Flutter for web. I've been following it for awhile.


This is for Skia devs to easily exchange code examples in the manner of JSFiddle.


Can someone explain to me what this drink example is doing?

https://jsfiddle.skia.org/canvaskit/e7ac983d9859f89aff1b6d38...

Just from a brief parse of the code I don't see how the drink could possibly be transforming the way that it is doing. The renderer - drawFrame in this case - isn't doing any heavy lifting to really make all the side animations (drink splash etc) occur.

It appears that at least with this example, possibly the others (the lego one?) if they work they've hidden a lot of data in the fetched json data. Which seems not great. I mean the drink json (https://storage.googleapis.com/skia-cdn/misc/drinks.json) is not human parse-able so I don't see how they made these examples at all. Maybe they made them in Adobe After Effects and then exported the keyframes as json or something, but there's no tutorial or directions that I can see.

This is incredibly frustrating because a decent canvas tool is sorely needed (generally). Does anyone see what is going on?

EDIT: So they do use lottie files (https://lottiefiles.com/410-lego-loader). But lottie files can already be exported to javascript incredibly easily, so unless you really need the extra performance of WASM this doesn't appear to add anything. Am I still missing something?


They should use the desynchronized canvas flag for lower latency on the ink example.

canvas.getContext('webgl', {desynchronized: true})

https://www.chromestatus.com/feature/6360971442388992


That seems like it would have to happen within CanvasKit itself, as the code using it only uses abstractions over the canvas itself. I can't find anywhere in CanvasKit where it does getContext('webgl'), only '2d', so I'm not sure how it's doing it.

The latency of that example seems great with a mouse, and a little laggy with a pen (a bit better than a pure canvas drawing app I've been working on). The frame rate is much better than my app.


I just created a proposal to support lottie based animated stickers in Discourse two days ago, and this looks very useful to get more performance when playing those using the new Skottie module. Very cool!

https://meta.discourse.org/t/plugin-for-animated-stickers/12...


I wonder if this is in part driven by Flutter for Web?


I can't think of another reason for Google to pour in all those resources (Google apparently being the biggest sponsor and owner of Skia). This could also more generally help give native ChromeOS apps a much needed kick (instead of needing Android apps for so many things).


See the Skia/Flutter note below. Skia has been open-source since 2008 -- it's the wasm bit that's new.

https://skia.org/dev/flutter


Right, I just mean specifically the new web stuff in Flutter:

https://flutter.dev/web

Presumably having Skia available in the browser makes supporting the web platform easier.


Here's one of the Skia Path Ops videos referenced in the Presentation section -- for some reason it links to slides but not the videos...

Skia Path Ops : High Performance Set Operations for Geometry

https://www.youtube.com/watch?v=OmfliNQsk88


Anyone knows why text looks so blurry?

Where are the docs for CanvasKit?


> Anyone knows why text looks so blurry?

If you're on desktop it's probably because it's not doing any form of subpixel hinting or AA.

It's both harder to do that in a GL context, and I don't know if there's even any way to get the display's subpixel layout in JS in the first place.


It looks like there is antialiasing, although it's huge. It only happens for text resizing demo though, the text on the star demo looks better.


It should be doing AA but not subpixel AA (aka, it's not going to match cleartype). Those are 2 different things.


I'm guessing @pier25 is viewing on a retina (2X pixel density) screen. All the demos are rendered using 1X canvases, so they appear blurry on retina (the default is bicubic interp).


Yep, I'm on retina. But the vector shapes and the text on the star demo look better than the text shape demo.


Can anyone show the url of the source code (C API?) which is outputted to the wasm file? I mean the subfolder in the skia project.



Almost. Thanks.


It sounds like this allows you to draw to a WebGL surface as if it were Canvas?


It means you can now include Chrome's Skia vector graphics rendering engine as part of your custom code in WebAssembly (skia/wasm in Chrome or elsewhere).

See the Slug thread from earlier today for context...

Slug: Dynamic GPU Font Rendering and Advanced Text Layout https://news.ycombinator.com/item?id=20475111

https://en.wikipedia.org/wiki/Skia_Graphics_Engine


Would be terrible if a browser decides to "optimize" the Wasm and upon detection of code that it has natively, directly runs the native version and completely bypasses any security benefits. (If you work on Skia and are reading this, please DON'T compromise on security. We already have too many vulnerabilities on the hardware level thanks to relentless optimizations that was not or unable to be proved correct.)


No browser would ever do this. It's a pain in the ass to do to begin with but it's also an absurdly complex optimization for a specific use case that would barely pay off in practice. It wouldn't work unless you were shipping the same version of skia as the embedded application and you'd have to do all sorts of weird stuff to make the ABIs match up (the native C/C++ ABI is not the same as the one wasm uses at runtime)


Knowing Google, they might be willing to go through the effort so that Skia sites only work in Chrome.


Right, but I'm wondering if it runs entirely on the CPU or uses GPU-based acceleration via WebGL.


It is certainly a WebGL Context.

Running this on one of the examples.

let c=document.querySelector("#canvas");

c.getContext("2d"); >>>> null

c.getContext("webgl"); >>>> WebGLRenderingContext { vertexAttribDivisor: vertexAttribDivisor(), drawArraysInstanced: drawArraysInstanced(), drawElementsInstanced: drawElementsInstanced(), createVertexArray: createVertexArray(), deleteVertexArray: deleteVertexArray(), bindVertexArray: bindVertexArray(), isVertexArray: isVertexArray(), drawBuffers: drawBuffers(), Yu: null, canvas: canvas#canvas }

c.getContext("webgl2"); >>>> null

Of course that doesn't stop it from using any amount of software rendering layers before it hits the canvas, but it's a good sign.


The docs say it uses WebGL [1]. And I found a "fixed" pull-request saying it too [2].

   WebGL context encapsulated as an SkSurface, 
   allowing for direct drawing to an HTML canvas.
However, I just discovered this project a few hours ago so I don't know if there's any "gotchas" -- hopefully someone more knowledgeable or on the chrome/skia team will chime in.

[1] https://skia.org/user/modules/canvaskit

[2] https://bugs.chromium.org/p/skia/issues/detail?id=8378


As far as I'm aware, browsers have been making use of any available GPU acceleration for rendering 2d contexts since 2012[1]? The example supplied by Google[2] (on the first link) definitely requests a 2d context for the canvas.

[1] - https://developers.google.com/web/updates/2012/07/Taking-adv... [2] - http://fhtr.org/gravityring/sprites.html


It takes 1 minute to draw star (first demo). Why is it so slow?


There are a few MB of .wasm included in the page. Takes about half a second to load it and another half second to initialize that for me on Chrome/FF. If it's not connection speed related then there is probably something buggy going on with running WASM in your browser.


Not for me and I’m on mobile.


Yesterday I saw a post about tinygo (tinygo.org) which can produce WebAssembly code and now Skia + WebAssembly. Fun times are ahead with Go :)


Would be great if anyone here has some experience with this and examples of use cases. I have been looking at this for a few weeks now for a tile based view with multi threaded rendering and server side prerendering. But I just can't figure out what the benefits would be over canvas natively, if there even are any. As I wrote in https://news.ycombinator.com/item?id=20339574

" context: I want to build a tile based view (tiles in x/y dimensions + zoom levels). The content of the tiles is loaded from server (shapes mostly) and rendered into tile images client side. I also want the same tiles prerendered as identical images on the server.

For this I have a feeling that something like skia is the way to go. Skia can be used via wasm bindings. How I would fetch the shapes and render the tiles (or fetch the prerendered ones) transparently and then where to render the tiles into, that is what I am trying to figure out. It feels like multithreading could be very useful here. Right now only chrome appears to support OffscreenCanvas (which can be accessed from webworkers), hence the idea of using skia directly and possibly going a level higher to write whatever kind of multithreaded render logic in rust and run it with wasm and a single "canvas output". Whether skia is the right choice here or not is also something I have yet to figure out

The ultimate goal is quick startup (prerendered tiles) while simultaneously high performance when updating the entire view (=multiple tiles in parallel). This is mostly a learning project for me context: I want to build a tile based view (tiles in x/y dimensions + zoom levels). The content of the tiles is loaded from server (shapes mostly) and rendered into tile images client side. I also want the same tiles prerendered as identical images on the server.

For this I have a feeling that something like skia is the way to go. Skia can be used via wasm bindings. How I would fetch the shapes and render the tiles (or fetch the prerendered ones) transparently and then where to render the tiles into, that is what I am trying to figure out. It feels like multithreading could be very useful here. Right now only chrome appears to support OffscreenCanvas (which can be accessed from webworkers), hence the idea of using skia directly and possibly going a level higher to write whatever kind of multithreaded render logic in rust and run it with wasm and a single "canvas output". Whether skia is the right choice here or not is also something I have yet to figure out

The ultimate goal is quick startup (prerendered tiles) while simultaneously high performance when updating the entire view (=multiple tiles in parallel). This is mostly a learning project for me "

Essentially what I am wondering is if using skia via wasm would allow me to bypass limits on canvas natively (and canvas in webworkers) so that I can render multiple tile images in parallel and then draw them into a single view for the user to interact with. Think google maps or similar but tiles rendered client side

Overall this feels like there might be a better way and I am just not seeing it


It's less than 50KB Gzipped, not bad at all.

https://bundlephobia.com/result?p=canvaskit-wasm@0.6.0


I don't think that's accurate at all. Watching Chrome's network profiler it's pulling down 2.5MB .wasm for the star fiddle: https://jsfiddle.skia.org/canvaskit/ea89749ae8c90bce807ea2e7...

50KB Gzipped seems like it's just the wrapped JS code, not the .wasm code.


The NPM package doesn't list any dependencies:

https://github.com/google/skia/blob/master/modules/canvaskit...

I'm not saying you are wrong, but if the NPM package needed more stuff it would be in the package.json, no?


I'm more saying bundlephobia.com is wrong in that it doesn't appear to be including .wasm files at all.

If you just do an npm install of canvaskit-wasm you'll find a 6.4M canvaskit.wasm file. That'd have to be some insane gzip magic to bring that down to 50Kb.

If I gzip the entire canvaskit module (including the font files) I'm getting a size of 3.4M. If I just gzip the .wasm file I get a size of 2.4M.


Thanks for checking that out. You are right of course.


Cool, but why?

Any browser already includes the very same set of graphic primitives that Skia provides. Just to render what you see on the screen right now.

Why not just to expose all that as API for WASM or whatever?


HTML canvas is incredibly slow and just a generally bad API. Using it from WASM is a mistake. It uses strings for everything and has a particularly slow imperative API, and the spec/de-facto-spec behaviors perform very slowly even on high-end hardware. I've seen relatively simple 2D canvas-based games drop frames in chrome on a GTX 2080ti because canvas is that bad.


The slow downs usually comes from abstraction layers. If you just use the api it will be fast. But if you create 1 million new objects and have 2 million function calls in excess on every frame it will be slow. Eg. If you dont like imperative style it will be slow. A compilation step and a bunch of optimization runs could make anything fast thought.


It really is not. If you want to set the current color you have to assemble a css color string from your color values. There is no way to make this fast. When you ship production 2D games using canvas, stuff like this shows up in your profiles and there's no way to optimize it out other than not using canvas. (Many people switch to using WebGL to do 2D themselves for this reason - it's faster.)


If setting the color is the bottleneck you can for example paint all red shapes, then paint the blue shapes, etc, instead of setting the color for every shape. You can also use different canvases . Offscreen and worker canvases is in experimental which enables further optimizations. 2d canvas has some very expensive calls, the trick is to only call them once, not for each object, not even for each frame. If you have 3d objects eg. Z-index its better to use 3d canvas. Yes 2d canvas is slow, but 2d scenes should be able to be optimized by only painting stuff once, then translate, rotate, reset and reuse. Set some upper limit of moving parts on the screen, then use pooling and caching. Put all canvas calls in the same file/ function then it will be easy to see what code can be removed/re-ordered eg. set the color once, not 1000 times after each other.


"HTML canvas ... generally bad API"

Agree on that. But this is not about Canvas::Graphics.


The original post appeared to be suggesting just using HTML Canvas (since it's backed by Skia). Was it suggesting something else?


There is no guarantee that canvas is hardware accelerated, or how whatever they are optimized, so a low level software rendering library might turn out to execute faster, regardless of the browser.


This is not about Canvas but rather set of primitives that browser uses for rendering web pages.

Like in my Sciter (https://sciter.com) where you can do

    var anyEl = …;
    
    anyEl.paintBackground = function(gfx) {
      gfx.fillColor(rgb(0,0,0))
         .rectangle(0,0,100cm,100cm);
      return true; // to suppress CSS background drawing 
    }
paintBackground/Content/Foreground methods are called while rendering HTML tree and the `gfx` passed to that function is the very same graphics that is used for rendering HTML/CSS tree.

That thing allows to combine immediate mode graphics a la ImGui with retained one to achieve more performant solution at the end.

The same approach can be used with WASM I think.


What if you want a common C++ code base that can run on web or desktop?


If you want that you will create set of C++ classes wrapping Skia, Direct2D or Cairo with the same API as browser provides.

That is actually what I did in Sciter that is using these libraries for rendering on different platforms - they all have very close feature set - umbrella wrapper is quite simple.


Sure, and that's actually what FireFox does, but not everyone is going to prioritize that approach.


So it is doable. QED.


So because someone can abstract away the underlying graphics library, CanvasKit shouldn't exist?


No.

Because browser already contains Skia (or Direct2D). In native (most performant) form.

Why do you need to download and run Skia inside Skia?




Guidelines | FAQ | Support | API | Security | Lists | Bookmarklet | Legal | Apply to YC | Contact

Search: