
Building the most inaccessible site with a perfect Lighthouse score (2019) - im_dario
https://www.matuzo.at/blog/building-the-most-inaccessible-site-possible-with-a-perfect-lighthouse-score/
======
luckylion
Imho, the problem with Lighthouse (and Pagespeed before it) isn't that they're
not perfect, it's that they assign scores/grades.

When Google assigns a score to something, people understand it to mean highest
score = best and start optimizing for the grade Google gives them, not for
performance and user experience, which the grade is supposed to represent.

It would be more fruitful to list the issues and their severity but not add
overall scores, because scores change the objective from fixing the problems
to getting high scores. They occasionally also have the have bugs where they
will punish something with a worse score that is actually an improvement in
the real world, discouraging people from doing the right thing ("I want my
site to load faster, but Google is buggy and will rank me down if I do").

~~~
ljm
The problem, as I see it, is it more or less makes Google the arbiter of the
internet, and when their purpose is to provide internet advertising then I see
a conflict of interest. Especially when things like "supporting AMP" will
likely impact your score.

While unfortunately many businesses have to care about Google SEO due to their
complete monopoly status in the search field (one that is of course growing
increasingly unhealthy), I'd prefer to not let Google be an ML-driven judge,
jury and executioner when it comes to being visible on the net.

~~~
anoncareer0212
I'm not sure the argument in your post or the parents hold water - lighthouse
scores aren't a "judge, jury, and executioner"

sometimes dev tools are just dev tools, and nothing is perfect.

~~~
the_pwner224
Wouldn't be surprised if they use it in search rankings, chances are they
already do. And not being on Google effectively == dead. They already take
into account page load speed (which discriminates against complex but useful &
good web apps) and iirc also whether you have AMP.

~~~
aww_dang
Do a test and compare your own optimized site to an amp site. See if you can't
beat it. Not that hard when you consider the 3rd party requests amp requires.

~~~
the_pwner224
I never said amp is good or fast, just that (iirc, but 90% sure) Google
penalizes you for not having amp.

~~~
aww_dang
My experience is that AMP is necessary for google news etc. Haven't seen the
outcome you speak of as compared to regular pages which are optimized.

------
joan_kode
Wow, CSS system color keywords seem like a massive privacy leak. I just tested
setting the property:

    
    
      background: Background;
    

on an element, and then changing my Windows desktop background. The element
immediately changes color to match my desktop. Then if I call getComputedStyle
on the element, I get my desktop background color in javascript. This is in
Firefox private mode, and apparently every website can read all my system
colors. Why in the world is this enabled by default?

[https://www.w3.org/wiki/CSS/Properties/color/keywords#System...](https://www.w3.org/wiki/CSS/Properties/color/keywords#System_Colors)

~~~
dharmab
It's already trivially easy to fingerprint a user in about a dozen other ways
via:

User Agent String (being fixed soon by the Chrome team)

HTTP_ACCEPT headers

Browser plugin metadata

Time zone

System fonts

Supercookies

Canvas and WebGL fingerprinting

AudioContext

Device CPU and memory

What's one more bit of information?

~~~
account42
Many of these should also not be exposed to websites without explicit
authorization.

------
ChrisMarshallNY
One of the "philosophers' stone" goals of the software industry is to
completely replace human testing with automated testing.

Basically, I think automated testing is a very good thing, and we should
_definitely_ try to do as much of it as possible.

So we can clear the way for more useful and meaningful human testing.

I've always thought that the engineers in QC should be just as skilled and
qualified as the ones building the product.

Part of what they should do, is design and build really cool automated tests,
but I think that they should also be figuring out how to "monkey-test" the
products, and get true users (this is 1000% required for usability and
accessibility testing) banging on the product.

"True users" != anyone even _remotely_ connected with the product development
or testing, beyond a contract agreement.

But I'm kind of a curmudgeonly guy, and my opinions are not always welcomed.

I do write about why I prefer using test harnesses, as opposed to [automated]
unit tests, here: [https://medium.com/chrismarshallny/testing-harness-vs-
unit-4...](https://medium.com/chrismarshallny/testing-harness-vs-
unit-498766c499aa)

~~~
naringas
>I've always thought that the engineers in QC should be just as skilled and
qualified as the ones building the product.

I agree, but then they need to be equally well payed... and I don't think it
is possible to find a manager willing to pay them as much

~~~
ChrisMarshallNY
I worked for a Japanese company that was _renowned_ for Quality (with a
capital "Q").

In the US, having "Quality" in your title often means that you are in a dead-
end job.

In that company, it meant that you were an elite, and that you had
considerable power over Engineering.

It also meant that you were a completely anal-retentive S.O.B.

They had spreadsheets with 3,000 rows (each row was a test -usually "monkey"
test).

If even _one_ of those rows got a red "X," the whole shooting match would come
to a screeching halt, with heads rolling around like foozballs.

It also meant that they double-checked every bug report six ways to Sunday. If
they reported a bug, It. Was. A. Bug. No ifs, ands, or buts. If you questioned
it, they would get quite huffy; which was not a good thing (see "power,"
above).

Management kept the QC organization quite separate from Engineering, and they
often had an adversarial relationship; which was sometimes encouraged.

This led to engineering departments having some very large testing teams;
often outnumbering the engineers. The engineering departments would be
penalized for bugs found by the official QC organization, so having large in-
house testing teams was worth it.

Their QC doesn't work especially well for software. They would get frowny
faces, when I'd suggest automated testing, or process quality best practices.

Quality was always treated separately from construction. I could never quite
agree with that, but it also meant that I was "on my own," if I wanted to try
using modern quality engineering techniques.

Their [hardware] products are damn good, though. What many companies would
consider minor quality issues are treated like Extinction-Level Events, at
that company. They've been doing it for 100 years, so it's difficult to argue
with them.

~~~
redis_mlc
> The engineering departments would be penalized for bugs found by the
> official QC organization, so having large in-house testing teams was worth
> it.

Some more details about that behavior are:

In Japan, employees start with a zero score, then for each mistake lose a
point. So they will analyze/block anything that could potentially cause a
demerit. Obviously that's the polar opposite of "move fast and break things."

Regarding the "3,000 line spreadsheet", yup, that's normal there and part of
why their meetings often run to midnight. Gotta check and double-check every
row before moving on to the next one. :)

If you want to learn more about these cultural differences, read patio11's
excellent posts about working in Japan.

~~~
ChrisMarshallNY
> their meetings often run to midnight.

I still have PTSD from that.

Nothing quite like a 14-hour flight, massive jet lag, and an all-day (and all-
night) meeting in a packed, cramped, overheated conference room.

------
userbinator
Having seen many other instances where chasing numbers has lead to a worse
outcome, I think "metrics driven development" is an abomination that must be
abolished. Unfortunately, management seems to really like the idea of turning
everything into a number and increasing it at all costs --- I have fought
against such things, and when I pointed out all the negatives associated with
it, they would often agree; but then dismiss the thought completely with a
response that essentially means "but it makes the numbers look better."

As the saying goes: "Not everything that counts can be counted, and not
everything that can be counted, counts."

~~~
theomh
I think what you are describing is a case of Goodhart's Law: "When a measure
becomes a target, it ceases to be a good measure."

[https://en.wikipedia.org/wiki/Goodhart%27s_law](https://en.wikipedia.org/wiki/Goodhart%27s_law)

On the other hand, a true accessibility score (for example, one awarded by a
human or a robust grading process) could reliably quantify accessiblity.
Goodhart wouldn't apply in that case.

------
choxi
Cool, but this article would have been more useful with some practical
examples of things Lighthouse doesn't catch. If the point is "this automated
metric isn't perfect", no automated metric is but how bad is it exactly?

I still don't have a sense for how bad Lighthouse is because I've never
disabled all keyboard events, disabled all mouse events, or changed the high
contrast stylings. The article almost makes the opposite point to me -- how
bad can Lighthouse be if the only loopholes are things that would pretty
obviously have accessibility issues?

The only useful examples I could see were the ones at the bottom of the
article, which show up in Lighthouse next to the score.

~~~
imtringued
You didn't understand the article. The article doesn't contain a single
loophole. Each of those features has legitimate use cases and it is impossible
to detect whether they are illegitimate.

Just look at the first example. It straight up denies access to the content
and the reason why lighthouse accepts it is because hiding content just hides
content. Usually when content is marked hidden the user is not supposed to see
it. It is therefore not a concern if a user can't see hidden content. Imagine
if you had a hidden dialog that is only shown on user interaction. Any
automated tool cannot tell if this is an accessibility issue or not. It's like
deciding if your car should be blue or black. There is no objective answer and
choosing the wrong color won't reduce your score.

All the other tricks in the article follow the same pattern.

~~~
aidenn0
Mostly; I can't think of a good reason for 1px sized text.

~~~
darekkay
The only thing I can imagine are screen reader-only labels (although you're
usually achieving this with a special "sr-only" or "visually-hidden" utility
class that doesn't touch the font size)

------
runawaybottle
This reminds me of a time when a team member was adamant about code coverage
metrics. It felt like an intense amount of busy work that really didn’t
improve our codebase or ensure thoughtful tests that actually, you know,
caught stuff.

It was just some weird metric we were chasing that involved making sure we go
through each function and superficially test calls, regardless of the fact
that some of the stuff we were testing gave us no confidence on the actual
internals. I will not even mention that code coverage became this _number_
that he/she believed was this standard, even though no attempts were made to
build the codebase via TDD from the get to (making chasing the code coverage
metric after the fact laughable). What could I say? The person appealed to the
authority of the code coverage metric.

But hey, we got that code coverage percentage up :)

~~~
Izkata
As one of those people who pushes for code coverage, I'd just like to jump in
and dump my thoughts, because no tool does what I want and because of that
most people don't understand what I'm going for.

The single most important thing is getting code where no consideration was
given down to 0%.

This doesn't necessarily mean "all code is covered by tests", though
definitely I'd prefer that be a high percent. What it means is all code was
either covered by tests, or deemed not worth the time to add a test for _and
marked as such so the coverage tool ignores it_.

Unfortunately, all coverage tools I've ever seen only allow "skip this for
coverage", with no nuance. I want at least two reasons for skipping to be
built in to the tool, that would be separated in the report:

* Things intentionally not covered because they're simple helpers in the realm of "obviously no bugs", or a small manually-tested API wrapper that works and shouldn't ever change again but that would be a pain or waste of time to write a worthwhile test (significantly reducing the value-to-effort ratio). There are probably other reasons, but those are the two that jump into my mind as the most common in the codebases I've worked with.

* Things that you're not writing a test for because of some other reason, but a test probably could be written for - such as a time crunch while fixing a bug, something complicated you know should have a test but are uncertain about, and so on. This one is measurable technical debt, and ideally should trend towards 0%, but isn't terribly important to do right away - it's a place where a decision was actually made, where losing test coverage is acceptable.

And of course, that leaves code that's covered by no tests and was not
intentionally left uncovered. This is what I mean by "no consideration was
given" \- it's not covered by tests by accident. It's a likely place to find
bugs. This is what, IMO, should always be at 0%.

(Quick aside, adding coverage to a legacy codebase would involve marking
everything as the second exception. The project functions, but adding all
tests immediately is not feasible, so it becomes explicitly marked as
technical debt to be reduced over time. All new code then becomes unmarked-by-
default where a decision should be made or a test added as the new code is
written.)

Having the separate ways to mark uncovered code, and actually using them, is a
way to signal to future developers "here be dragons!" when they look at the
uncovered and unmarked code.

------
bhrgunatha
An Interesting and entertaining article.

The most interesting part to me (as someone with vision problems) was the
WebAIM link [1]. The biggest problem I have is with the almost total blind
adoption of low contrast (so often too low for me to even read) and sure
enough the section about low contrast [2] says:

"found on 86.3% of home pages. _This was the most commonly-detected
accessibility issue_.

My basic question then is why do so many designers and websites choose to
break the WCAG guidelines?

[1] [https://webaim.org/projects/million](https://webaim.org/projects/million)

[2]
[https://webaim.org/projects/million/#contrast](https://webaim.org/projects/million/#contrast)

~~~
darekkay
From my experience, it is often not an active choice to break the guidelines.
It's rather the lack of knowledge/experience or ignorance - or a mix of both
("What is this accessibility thing? Do we need it?").

For contrast ratio specifically, I wished more people would adapt the approach
that USWDS uses [1]. It enforces accessible color combinations by using
standardized naming with a special property. E.g., I know that `blue-60` on
`purple-10` is accessible (WCAG AA), because the absolute difference is 50+
(60 - 10). I'm currently writing a blog post regarding this approach to spread
the awareness.

[1] [https://designsystem.digital.gov/design-
tokens/color/overvie...](https://designsystem.digital.gov/design-
tokens/color/overview/)

------
kopite
First, let me start by saying that this is a good article and sheds light on
one of the challenges of accessibility adoption.

All of tools like lighthouse, axe-core etc. run a subset of tests that gives a
false sense of security about accessibility. Similarly, tools like
Accessibility Insights for Web has a fast-pass option, which does the same
thing where it runs a subset of tests to catch the most common issues on a
website.

But it does not and cannot(at this moment) catch all the issues that require
semantic analysis of a website, like checking that the alt text on an image
has meaningful text. For tests like those, a human is needed to perform a
comprehensive assessment, something like Accessibility insights for web offers
as an Assessment option.

In my opinion, all of the tools are doing one thing good and that is raise
awareness about the problems that users with a disability face daily, when
trying to use a website. They are making accessibility a must. Tools still
needs more work and I feel confident that it will continue to improve. It all
kind of comes down to how much time a development team puts in to make their
website completely accessible, which ideally every team should budget and plan
for.

~~~
mattlondon
Where I am, we use axe-core as part of our automated integration testing. Any
change that regresses on accessibility (i.e. new severe a11y issue etc) is
blocked from submission in the same way that it would be blocked from
submission by causing tests to fail.

I find this useful as it prevents "accessibility rot" where people think "Oh
we'll sort out the aria stuff later".

But yeah I agree that a human test is best. I think of axe et al as an
equivalent of a "static analysis" that can pick out the obvious mistakes, but
wont understand the dynamic nature of the application - it is after all great
for flagging the "easy" problems, allowing the human tester to focus on the
major a11y issues without ending up raising hundreds of bugs for every button
and link and colour combination that should have been sorted out way before
any human testing is involved.

~~~
darekkay
I usually rephrase the same point as: "Automatic testing is able to tell you
if a page is _inaccessible_. However, it cannot tell you if a page is
_accessible_."

------
sixhobbits
> Oh, man. This is so evil. My LinkedIn inbox will be filled with job
> offerings by companies like Facebook and Uber.

I really enjoyed this part

------
ancarda
So this is what people mean by "following the letter of the law instead of the
spirit of the law"?

------
chrismorgan
Another technique to mess up keyboard users: don’t use the document scroll
area, but make your own (and don’t focus it or anything in JavaScript). Thus
the user will have to press Tab or click in the area before keyboard
navigation keys will work. So for best results put a large number of focusable
elements before the scrollable pane, so that the keyboard user must press Tab
a large and unpredictable number of times before it works.

It’ll be something like this:

    
    
      <body>
        <div tabindex=0></div>
        <div tabindex=0></div>
        <div tabindex=0></div>
        <div style="position:absolute;top:0;left:0;right:0;bottom:0;overflow:auto">
          …
        </div>
      </body>
    

You could probably mess with other tabindexes (randomly jump through the
document with Tab!) without Lighthouse baulking.

I was going to suggest adding `pointer-events: none` so that the user can’t
just click to focus it, but that was already done!

(I mentioned focusing your scroll area element as something you need to do if
you roll your own rather than using the document scroll area; but that’s not
all you need to do. You also need to monitor blur events and change any
.blur() calls, so as to avoid the document element ever retaining focus. It
inherently depends on JavaScript, and is very fiddly to get fully right—I’m
actually not sure if _anyone_ gets it _fully_ right; the interactions of focus
and selection are nuanced and inconsistent, and it’s extremely easy to mess up
accessibility software; I haven’t finished my research on the topic. I
strongly recommend against the technique on web pages; web apps can
_occasionally_ warrant it.)

------
dccoolgai
I just read an article a couple days ago about how even YouTube widgets and
stuff have huge a11y problems. I think it's time to admit Google is terrible
at accessibility. All their devrels talk like it's important and have all
these beautiful demos but whenever you look behind the curtain at their
products it's terrible.

~~~
darekkay
The classic "do as I say and not as I do". That said, the YouTube widget is
actually quite accessible compared to most other video players [1].

[1] [https://accessibility.blog.gov.uk/2020/03/16/why-videos-
on-g...](https://accessibility.blog.gov.uk/2020/03/16/why-videos-on-gov-uk-
use-the-youtube-video-player/)

------
irrational
Stop giving people ideas. I can almost guarantee that there is someone out
there getting ideas of things to add to their site from this post ;-)

------
calibas
Google invents new metric, web shifts to accommodate new Google metric, people
learn how to abuse metric, Google invents new metric...

------
ornxka
I think this actually outlines the standard design process for most major web
sites

------
staycoolboy
This is hilarious. It's like if McSweeny's staff were programmers.

------
SiempreViernes
So disapointed this isn't about building a tall building at a super remote
site on antarctica or something like that :(

~~~
gostsamo
I'm sure that you find it funny. Consider though those for whom accessibility
is matter of great importance, because for those of us taxes, health and
employment might depend on the diligence of some motherfucker who was not in
the mood or just decided to copy/paste from Stack Overflow and just ruined
someones experience on a rather important webpage. Not to talk about Atlasian
who had the habit of adding aria-hidden to Jira so that enterprises felt
insentivized to purchase the accessibility plugin for their employees.

