
Show HN: Scrawl-canvas – build responsive, accessible HTML5 canvases - rikroots
https://scrawl-v8.rikweb.org.uk/
======
rikroots
Maintainer here. dang suggested I add a comment to the thread to help start
possible conversations about the project. So I'll answer 3 self-asked
questions.

"Why did you start the project?"

Short answer: I'd spent 3 years trying to get my first job in tech (London). A
recruiter took pity on me in 2013 and suggested I build up a portfolio of
projects on GitHub to showcase my abilities. I started a project which
included an animation on canvas, and took the decision to code the animation
using the canvas API rather than an existing library. I got a bit obsessive
about the canvas element over the next few years. 7 years later, I'm finally
beginning to feel happy and confident in the work.

"What problems does the project solve?"

My view is that there's three big issues with the <canvas> element:
accessibility; responsiveness; fun.

Accessibility - because the current canvas element completely fails users who
access websites with non-visual browsers and devices. All information
contained in the graphic output is a black box to them. This has to change.

Responsiveness - because the current canvas element doesn't know how to play
nicely with CSS responsive design patterns.

Fun - because trying to do anything reasonably complex with the Canvas API is
not fun.

"What are your hopes for the project?"

I'm not looking for the library to take the frontend world by storm (though it
is a nice fantasy).

What would really make me happy would be if maintainers of other canvas
libraries look at what I've tried to do with Scrawl-canvas around the areas of
accessibility and responsiveness, and add that sort of functionality to their
projects - hopefully in even better, more efficient ways.

Oh - and finding some collaborators to work on the project would also be good,
because I've spent too long working on it by myself and it could really do
with some fresh eyes and opinions.

~~~
BrowserMeeting
I can’t get it to load on the latest Safari on iOS.

It looks fine in Chrome though! Super cool.

~~~
LunaSea
[https://caniuse.com/#feat=intersectionobserver](https://caniuse.com/#feat=intersectionobserver)

Lots of Safari versions don't support IntersectionObserver it would seem.

Then again, Safari is the new IE.

------
chrismorgan
This fails my very first test of anything canvasy: it’s not scaling by
devicePixelRatio, so it looks bad on a high-DPI display, especially if text is
involved. I usually find this to be a bad sign.

This puzzled me, because it mostly looks very good (even if it rather
encourages CPU-guzzling). Have you just never had access to a high-DPI
environment and thus never thought about it? Anyway, this one should be very
quick to fix.

Accessibility: yeah, you can tab between the links, which is good, but when
you’re just hovering you’re not hovering over a real link, so you can’t see
the link target, the cursor is wrong (though one that could be fixed), and
modifiers don’t work. Right-click triggers the “links” too, which it
shouldn’t. Things like this _must_ be handled by drawing real DOM elements
persistently on top of the canvas, and letting the user _genuinely_ interact
with them. No other approach is capable of achieving the expected and desired
result. Not `location.href = …`, not `open(…)`, not `link.click()`. Nothing.

One straightforward (but inferior) approach to make this work could be to put
an <a tabindex=-1> above the entire canvas (or even wrapping the canvas!) and
set its href when hovering a “link”, and remove it again when not.

I haven’t delved into any more of the accessibility stuff, but it looks much
better than most, and just needs to work on any real DOM counterparts to
canvas pieces actually being in the right place, so that interactions, text
selection, &c. can work properly.

~~~
rikroots
Hi chrismorgan. Many thanks for taking the time to check over the site, and
giving me feedback!

Can I ask which device/screen you used to view the site? I'm currently limited
to a Macbook Pro, so can't get the insight of seeing the page on other
devices.

>> This fails my very first test of anything canvasy: it’s not scaling by
devicePixelRatio, so it looks bad on a high-DPI display, especially if text is
involved. I usually find this to be a bad sign. [...] Anyway, this one should
be very quick to fix

Noted. Fixing will take a bit of thought as it won't be a simple case of
scaling (as suggested here[1]). The canvas Cell where most of the work happens
can already have different dimensions to the display canvas, which goes
partway to addressing the problem. The library should also play nicely with
<img srcset="..." sizes="..."> \- I'm only supplying a single src value for
the images used on the site.

>> (even if it rather encourages CPU-guzzling)

Yes. Minimising the impact on the CPU will remain a key priority for the
library going forward.

>> Accessibility: yeah, you can tab between the links, which is good, but when
you’re just hovering you’re not hovering over a real link, so you can’t see
the link target,

That's an issue with the page code, not the library, I think. Hovering over a
link with the mouse cursor should bring up a tooltip (working on my machine
for Chrome, Firefox, Edge; fails on Safari) - I didn't think to include the
link url when designing the page, but that can be easily added. Tabbing (which
doesn't work on Firefox because[2], and is buggy on Safari) shows the link at
the bottom-left of the browser window on Chrome, Edge.

>> the cursor is wrong (though one that could be fixed),

Agreed, fixable.

>> and modifiers don’t work.

On my Macbook Pro hovering over a link (and tabbing to it) changes the size of
the pin and displays a graphic text label. Is that not what you see? The
library has functionality to handle focus and blur states, but I've not
thought to include anything to handle visited and active states.

>> Right-click triggers the “links” too, which it shouldn’t.

I can replicate this on Firefox, when I previously agree to "allow popups".
Without that user agreement, the right click shows the small menu with links
to "save image" etc. Does not seem to be occurring on Chrome, Edge, Safari.

... I'm beginning to fall out-of-love with Firefox on Mac

>> Things like this must be handled by drawing real DOM elements persistently
on top of the canvas, and letting the user genuinely interact with them. No
other approach is capable of achieving the expected and desired result. Not
`location.href = …`, not `open(…)`, not `link.click()`. Nothing. One
straightforward (but inferior) approach to make this work could be to put an
<a tabindex=-1> above the entire canvas (or even wrapping the canvas!) and set
its href when hovering a “link”, and remove it again when not.

This can be achieved by wrapping the canvas element in a Scrawl-canvas stack,
which then allows DOM elements to be positioned over (or beneath) the canvas
in the same way as canvas entitys. By making the DOM element "mimic" its
entity, DOM element positions and dimensions can be handled automatically.

>> I haven’t delved into any more of the accessibility stuff, but it looks
much better than most, and just needs to work on any real DOM counterparts to
canvas pieces actually being in the right place, so that interactions, text
selection, &c. can work properly.

Again, thank you!

[1] -
[https://www.html5rocks.com/en/tutorials/canvas/hidpi/](https://www.html5rocks.com/en/tutorials/canvas/hidpi/)

[2] - [https://stackoverflow.com/questions/11704828/how-to-allow-
ke...](https://stackoverflow.com/questions/11704828/how-to-allow-keyboard-
focus-of-links-in-firefox/11713537#11713537)

~~~
chrismorgan
The main thing I’m talking about is links, that links _need_ to be real, or
the interactions are all wrong, irredeemably. It _must_ be a real <a href>
element that the user hovers over and clicks on, or else the link destination
isn’t shown in status text (and you can’t just set window.status like you used
to be able to!), and modifiers don’t work.

Modifiers: Ctrl+click, Shift+click, Ctrl+Shift+click, that sort of thing,
which change where the link will open—current window, new window, new
background tab, new foreground tab, it’s up to the user agent to decide what
it all means, and target=_blank is the only slight control authors can have.
If you use open(…) or similar, it will do the wrong thing.

open() is a very flimsy tool that has (or should have) very limited use.
Browsers are likely to block it, though they _may_ allow it to operate as part
of an event handler.

Concerning DOM elements matching the canvas contents, yes, that’s the approach
that needs to be taken; and it _needs_ to be taken. This leads me to suspect
that for many applications minimising the part that happens in the canvas, or
making an SVG/canvas hybrid, may be more sensible.

The few examples I looked at seemed to _start_ in the direction of doing this,
but not take it as far as it needs to be taken to satisfy accessibility.

~~~
rikroots
>> It must be a real <a href> element that the user hovers over and clicks on

For each <canvas> element that Scrawl-canvas wraps, it adds a hidden <nav>
element to the DOM, immediately after that <canvas> element. Creating a
graphical entity with an anchor attribute will trigger the library to add an
<a> element to the hidden <nav> element. When the user clicks on the graphical
entity in the canvas, the library creates a new MouseEvent 'click' event and
then dispatches it from the <a> element.

The anchors are not directly placed over the graphical entitys in the canvas,
but they are there in the DOM.

>> Modifiers: Ctrl+click, Shift+click, Ctrl+Shift+click, that sort of thing,
which change where the link will open—current window, new window, new
background tab, new foreground tab, it’s up to the user agent to decide what
it all means, and target=_blank is the only slight control authors can have.

For Chrome running on a Macbook Pro: Shift+click opens the link in a new
window; Command+click opens the link in a new tab but remains on the original
page. I've not tested this functionality yet for other browsers, or for any
browser running in a different OS/device.

>> If you use open(…) or similar, it will do the wrong thing. open() is a very
flimsy tool that has (or should have) very limited use. Browsers are likely to
block it, though they may allow it to operate as part of an event handler.

The library makes no use of window.open(), or similar. Links are driven by
MouseEvents:

    
    
        let e = new MouseEvent('click', {
            view: window,
            bubbles: true,
            cancelable: true
        });
    

>> Concerning DOM elements matching the canvas contents, yes, that’s the
approach that needs to be taken; and it needs to be taken. This leads me to
suspect that for many applications minimising the part that happens in the
canvas, or making an SVG/canvas hybrid, may be more sensible. The few examples
I looked at seemed to start in the direction of doing this, but not take it as
far as it needs to be taken to satisfy accessibility.

I am grateful for the feedback! If the responses I've given above indicate
that my thinking is flawed, then it is much better for this flawed thinking to
be exposed and examined - and, if possible, fixed in the near future - rather
than letting them lie hidden and festering in the library, making it unusable
for serious production sites.

~~~
chrismorgan
Interesting. I had forgotten that synthesising events and dispatching them
triggered the appropriate functionality. I knew it once, but for some reason
I’d ended up thinking that .click() was special.

Anyway, it’s not enough to simulate clicking on the anchor element, for two
reasons:

• As hitherto stated, the anchor element is not there under the mouse; so any
_other_ interactions definitely won’t work properly. e.g. hover should show
the link destination in the status bar, and right click should open a context
menu with items like “open link”, “open link in new tab”, “open link in
private window”, “send link”, &c. I don’t believe that either of these can be
triggered programmatically.

• It doesn’t have the desired effect across all browsers. In fact, I find it
surprising that a synthesised click event would have modifiers work in _any_
browser, and I’m inlined to call it a bug. The synthesised event doesn’t have
the modifiers like ctrlKey set on it, so if Chrome is trying to be helpful by
guessing that you probably want the modifiers to apply even to the faked event
(which is a dubious assumption) then it puts it at odds with event handlers,
which will see events without ctrlKey _et al._ And if you try to set ctrlKey
in Firefox (haven’t tried in any other browser) then it ignores the event
(which I find a tad surprising, but I’m guessing it’s a form of browser hijack
protection), so there doesn’t seem to be any way to make this simulation
approach work in Firefox.

There’s a lot you can do in browsers, but there’s also rather a lot of user
agent functionality that can’t be faked, like <a href> and scrolling. And so
for those sorts of things, I say: use the native functionality, and accept no
substitutes.

I am heartened to see how carefully you’ve been thinking all these things
through. Nice job!

------
visarga
Cool project. I used a somewhat similar project fabric.js to build an
interface to display OCR boxes on scanned documents and label them them for
document understanding. The essential feature was to display polygons on top
of the document image, which were editable by hand (resize, translate, zoom,
rotate).

~~~
rikroots
I agree that Fabric.js[1] is an excellent Canvas library. I still follow
Kangax, who started that project, on Twitter.

[1][http://fabricjs.com/](http://fabricjs.com/)

------
jmiskovic
On Linux Firefox 75 the page shows only teal background.

~~~
rikroots
Thanks for the bug report!

Firefox is choking on this regular expression: `let matches =
val.toString().match(/\\((. _?)\\)._?\\{(.*)\\}/s);`

Firefox doesn't support the dotAll /s flag yet -
[https://developer.mozilla.org/en-
US/docs/Web/JavaScript/Refe...](https://developer.mozilla.org/en-
US/docs/Web/JavaScript/Reference/Global_Objects/RegExp/dotAll)

I'll investigate - see if there's a way around this issue. Do you/anyone have
any pointers towards a possible fix?

~~~
rikroots
Ah - no worries. Found a potential solution. Will try it and, if it works, do
a new release later today

[https://stackoverflow.com/questions/1068280/javascript-
regex...](https://stackoverflow.com/questions/1068280/javascript-regex-
multiline-flag-doesnt-work)

------
dhimes
This is beautiful! Congratulations!

