
Blind Apple engineer is transforming the tech world at only 22 - davidbarker
http://mashable.com/2016/07/10/apple-innovation-blind-engineer/
======
escapologybb
There's a lot to gripe about with Apple, but not when it comes to
accessibility. I am quadriplegic and usually that means putting up with some
terrible subset of functions for any given application, tablet, phone or
whatever but not with Apple. They've made it so that anything able-bodied
person can do with an iPhone, I can do with their Switch Control interface
which I use connected up to the chin controller on my wheelchair. It's all
done via a little Bluetooth bridge called a Tecla shield and it's absolutely
marvellous.

There really isn't another company in the world, certainly not another phone
Manufacturer that has thought of people with motor skill problems in the way
that Apple has. When people think of accessibility they think of the blind and
that deaf, people with motor function problems are definitely the red-headed
stepchildren of the accessibility software world.

Apple, pay your f __ __ng taxes but thank you very very very much for the
accessibility software, my life would be much much poorer intellectually
without it.

~~~
tezza
Microsoft also deserves applause in the accessibility stakes.

My cousin's child is quadriplegic with cerebral palsy and a lot of his eye
controlled tech runs on Windows

~~~
Pxtl
We just hired a 100% visually impaired student and he said that most MS apps
provide enough metadata cleanly to play with screen reader apps like JAWS, and
that the real bane of his existence was GTK-based applications on Windows.

~~~
blakeyrat
I think GTK+ apps on Windows are the bane of _everybody 's_ existence. What a
crappy windowing framework.

------
andrewstuart2
One thing that frustrates me significantly about accessibility and assistive
technology (specifically for the web at least) is the lack of consistency
implementing standards. It's much more insidious than most other standards
inconsistencies because rather than your page looking slightly off in one
browser, you can have your site reading perfectly on two of the major screen
readers and the third refuses to read it.

It's a massive waste of resources to find the right combination of tech that
ends up magically working for all three. The running joke at my workplace is
"fix JAWS, break VoiceOver; fix VoiceOver, break NVDA" etc. And yet it's not
the kind of thing you want to let go because not many people will have the
chance to switch to another device/reader to see if the site works there. Nor
should they have to.

~~~
microcolonel
In my experience, TalkBack works great. It is especially nice that in concert
with Chromium's aria implementation, it also works quite well. The menace has
been VoiceOver, which is inconsistent. In web views, VoiceOver segments
incorrectly on elements and will regularly ignore aria labels. Not sure how
well JAWS works, NVDA seems to work well enough, and I've admittedly not tried
Orca with a web browser.

Though honestly, I think that we should give up on trying to present exactly
the same interface structure to people who are completely blind. An
application design which works well for sighted people can easily seem
incredibly convoluted when described by your screen reader. I've not had time
to build out a framework for this, but I think it would not be hard to start
delivering something better. The idea being that you would provide a
structural description of interactions and content, and the framework would
present this through whatever means is available on the platform. This way, in
combination with platform detection, it will always present something
functioning.

~~~
DonHopkins
I wrote up some ideas about a scriptable cross platform accessibility
integration system called aQuery, like jQuery for Accessibility.

[http://donhopkins.com/mediawiki/index.php/AQuery](http://donhopkins.com/mediawiki/index.php/AQuery)

Morgan Dixon did some wonderful work at the University of Washington called
Prefab. Some of the links from my page to his papers are broken, but here's
his web site and a demo:

[http://morgandixon.net/](http://morgandixon.net/)

[https://prefab.github.io/](https://prefab.github.io/)

Prefab: The Pixel-Based Reverse Engineering Toolkit Prefab is a system for
reverse engineering the interface structure of graphical interfaces from their
pixels. In other words, Prefab looks at the pixels of an existing interface
and returns a tree structure, like a web-page's Document Object Model, that
you can then use to modify the original interface in some way. Prefab works
from example images of widgets; it decomposes those widgets into small parts,
and exactly matches those parts in screenshots of an interface. Prefab does
this many times per second to help you modify interfaces in real time. Imagine
if you could modify any graphical interface? With Prefab, you can explore this
question!

[https://www.youtube.com/watch?v=w4S5ZtnaUKE](https://www.youtube.com/watch?v=w4S5ZtnaUKE)

Imagine if every interface was open source. Any of us could modify the
software we use every day. Unfortunately, we don't have the source.

Prefab realizes this vision using only the pixels of everyday interfaces. This
video shows how we advanced the capabilities of Prefab to understand interface
content and hierarchy. We use Prefab to add new functionality to Microsoft
Word, Skype, and Google Chrome. These demonstrations show how Prefab can be
used to translate the language of interfaces, add tutorials to interfaces, and
add or remove content from interfaces solely from their pixels. Prefab
represents a new approach to deploying HCI research in everyday software, and
is also the first step toward a future where anybody can modify any interface.

~~~
microcolonel
That is very cool, thank you for sharing. I'll see if I can put some weekends
into my part of the solution. Prefab seems like it may be the only hope for
making existing proprietary software accessible; I want to make it easy for
businesses with accessibility requirements to do the right thing well.

Maybe in the future we won't need computer vision to reverse-render display
elements, one can dream.

~~~
DonHopkins
I think it's great to use JavaScript to control and combine the best of both
worlds: accessibility APIs plus screen scraping / selective screencasting /
pattern recognition / computer vision.

For example, you could use the accessibility APIs to find the screen position
of the video window in the Skype application, perform facial recognition and
tracking, and screencast the video onto a texture of a VR chat application.

------
danso
> _Castor told Apple reps how amazed she was by the iPad she received as a
> gift for her 17th birthday just a few years earlier. It raised her passion
> for tech to another level — mainly due to the iPad 's immediate
> accessibility._

Just goes to show how different things can be between perception and reality.
I've worried that the move away from keyboard-based inputs would increase the
marginalization of disabled computer users. Now that I think (more) about it,
the physical form of a tablet can be similarly as interpretable as a keyboard,
with the added bonus that app designers don't have the choice to build (and
prioritize) a mouse-based interface -- I'm assuming that mouse-driven
interfaces are especially difficult for the visually-impaired [0].

The uniformity of interface that iOS imposes is probably especially useful for
the visually-impaired, provided that they have employees (like the one
featured in the OP) who are on the engineering and design teams.

[0]
[http://webaim.org/articles/visual/blind](http://webaim.org/articles/visual/blind)

~~~
Klathmon
A blind guy I know loves android significantly more than any other platform.
To him he can just run his finger over the screen to "feel" what's there by
listening to a voice that is WAY too fast for me to even really understand.

I've tried it myself (mostly to test out how some of our web apps work through
that system) and it's actually pretty damn intuitive after you spend like 15
minutes getting the basics down.

~~~
digi_owl
Makes me wonder what could be done with a pressure sensitive screen and
something like electrostatics to produce "texture".

~~~
323454
What if you could produce braille on a touch screen?

~~~
bpicolo
There are a few such things out there, [http://www.popsci.com/new-touch-
screen-design-could-display-...](http://www.popsci.com/new-touch-screen-
design-could-display-in-braille)

The cost to produce them is probably very high, I imagine, especially when
alternatives (like good audio/voice stuff) can do the job.

------
mattbgates
I have quite a few blind readers to my website at
[http://www.confessionsoftheprofessions.com](http://www.confessionsoftheprofessions.com).
I'm not sure if they came before or after I made the decision when starting
the website to write out every single infographic in detail with a text-
friendly version. It takes me an extra 10-20 minutes, depending on the length
of the infographic, when I'm processing my articles, but it has its advantages
for both readers and SEO.

It was actually unintentional when I started doing it and I had only done it
with search engines in mind, because they can't read infographics, and I
wanted them to be able to categorize the article better. Turns out, there are
hundreds to thousands of people who also can't read infographics.

The side effect was that it began attracting a good amount of people who just
need everything written out for good reason. Not just the blind, but I have
gotten emails from non-blind people who are autistic, dyslexic, or just have
trouble reading who use e-readers both on computer and mobile devices thanking
me for doing it.

Highly recommended for everyone to start taking the handicapped into mind when
it comes to the Internet. EVERYONE is using the Internet. Handicapped, blind..
they aren't just sitting there twittling their thumbs. They are searching the
Internet!

------
gok
Everyone (especially sighted developers!) should try enabling a screen reader
on their phone once in while. Of course it's great for finding accessibility
bugs, but it's also a cool trick to be able to read emails and stuff without
someone being able to look over your shoulder.

~~~
kens
If you want to try the Android screen reader, turn on accessibility with
Settings -> Accessibility, TalkBack On. Then, tap a control once to hear what
it is, then double tap to activate. (This is very important to know, or else
you won't be able to turn TalkBack off again.)

Tip: don't try TalkBack in your office, or you will rapidly drive your
officemates crazy.

~~~
teamonkey
I often have to help a family member who is blind. Trying to use a phone with
Voiceover or Talkback is the most horrific experience. Each button or key
press takes 3 time-sensitive taps, while it speaks the contents of that
button/key in a robotic voice.

I say this as a sighted person, but is this really the best we can do? If
you're blind I can imagine how important it is that it works at all, but it
still sucks.

Annoyingly, what my family member really needs for their use case is an old-
style phone with tactile keyboard - if it weren't for the fact that they also
need to voice dial, hear who's calling, and hear incoming texts.

------
michael-the1
This thread comes at a funny time. I just saw my cousin and roommate come back
from surgery. He had the same surgery before in one eye, now he's impaired the
other as well. Now it's just waiting and hoping for recovery.

The thing he wants to do the most, in his current blinds state, is not rely on
other people. Accessibility allows him to be that independent, empowered
person. Thank you to everyone who furthers that cause.

------
WalterBright
Apple is doing great things here. I welcome feedback and suggestions from any
visually impaired programmer that can improve the usability of the D
programming language website for them. dlang.org

------
samfisher83
In the article it mentioned 70% of blind people are unemployed. That seems
like a big problem.

------
samnwa
Is "transforming the tech world" a bit strong? I get turned off by all this
headline hyperbole.

~~~
Mao_Zedang
"Apple hires blind woman" would be a much fairer title.

------
rmason
Though I never met her when she was at Michigan State, I've got a number of
friends that did and she was a rockstar developer at the school even as an
undergrad.

She gets mentioned on my Twitter on at least a monthly basis and it's kind of
neat to see someone who was once a local on the front page of HN.

~~~
jscholes
> rockstar developer

Could you expand a little on what you mean by this?

~~~
tamana
It means "exceptionally good, and known for it"

------
nxzero
Apple's accessibility features even useful as workarounds for hardware
failure. If one of the buttons fails, turning on the accessibility features
to: on/off, screen-lock, volume, etc.

~~~
stephenr
The assistive touch widget on my iPhone has meant I can wait a little before
replacing it (failed home button).

It gets more extreme here in Thailand though - my sister in law and mother in
law, (and I am led to believe, a lot of other people) use the assistive touch
widget exclusively, instead of using the _working_ home button, as they don't
want the hardware button to break/wear out.

I don't quite understand the logic myself, but its definitely a thing.

~~~
kalleboo
It's the iOS equivalent of never removing the protective film from the
factory. Keeps your device looking pristine, too bad you never get to actually
see it.

~~~
stephenr
I've seen people here with 5+ year old TV's with the promotional stickers
still on them, obscuring part of the screen.

------
newscracker
This reminds me of a few things with respect to Apple and accessibility
(listed in chronological order here).

* In a speech Tim Cook made in December 2013 on receiving the "International Quality of Life Awards" at Auburn University [1] (link goes to the exact time in the video when this was said):

> "These values guide us to make our products accessible for everyone. People
> with disabilities often find themselves in a struggle to have their human
> dignity acknowledged, they frequently are left in the shadows of
> technological advancements that are a source of empowerment and attainment
> for others, but Apple’s engineers push back against this unacceptable
> reality, they go to extraordinary lengths to make our products accessible to
> people with various disabilities from blindness and deafness to various
> muscular disorders."

* In early 2014, in the context of an NCPPR representative asking Apple CEO Tim Cook to "commit right then and there to doing only those things that were profitable", this is how Tim Cook responded [2]:

> When we work on making our devices accessible by the blind," he said, "I
> don't consider the bloody ROI."

* In July 2015, marking 25 years of the "Americans with Disability Act", Tim Cook tweeted: [3]

> "Accessibility rights are human rights. Celebrating 25yrs of the ADA, we’re
> humbled to improve lives with our products. #ADA25"

Regardless of profit motives and the higher prices of iDevices (in comparison
to others), Apple is leading on a few different fronts, one of them being
accessibility and its commitment to improving it continuously. This kind of
inclusivity, which should be a default, is what progress of humankind is
about.

[1]: [https://youtu.be/dNEafGCf-kw?t=272](https://youtu.be/dNEafGCf-kw?t=272)

[2]: [https://www.macobserver.com/tmo/article/tim-cook-soundly-
rej...](https://www.macobserver.com/tmo/article/tim-cook-soundly-rejects-
politics-of-the-ncppr-suggests-group-sell-apples-s)

[3]:
[https://twitter.com/tim_cook/status/624584736862679040](https://twitter.com/tim_cook/status/624584736862679040)

------
dvillega
This reminds me of the "SignAloud" gloves that vocalize American Sign
Language.

[http://lemelson.mit.edu/winners/thomas-pryor-and-navid-
azodi](http://lemelson.mit.edu/winners/thomas-pryor-and-navid-azodi)

It's exciting to think about how tech is unlocking accessibility barriers.
Huge :+1: to Apple for innovating in the accessibility space.

------
mark_l_watson
Great article. I became interested in how blind programmers work when a blind
programmer emailed me about 20 years ago with some questions about the code in
my book Portable GUI Applications with C++. Apple, and other companies who
support people with disabilities, deserve a lot of credit, but people who work
through their disabilities greatly impress me.

------
tommynicholas
Was just talking about this yesterday with my newly blind friend.

The unemployment rate among the blind is at least 63% right now. It's
remarkable.

------
alannallama
If any developers are thinking about getting started with accessibility, I
would definitely encourage you to!

Here's a short video of my co-founder talking about our journey with this in
our startup, and how you can take some first steps:
[https://youtu.be/g0bjrTCKxZw](https://youtu.be/g0bjrTCKxZw)

------
javajosh
A bit of a technical question about blind people, especially birth-blind
people. Does the spatiality of the screen cease to be a factor? How do they
"visualize" internally a computer interface, especially a GUI one? Thanks.

------
necessity
Could we get a less descriptive title? Not clicking on clickbaits.

------
quakerquickoats
this is marketing tactics. how would she know if the ipad she received is
working great out of the box? seriously, our screens are not yet tactile or
programmably textured or termperatured yet. she should work on that stuff
perhaps.

~~~
escapologybb
Just curious, but have you tried a brand-new iDevice out-of-the-box as a
disabled person? I have. I am quadriplegic and within five minutes of help
from a able-bodied person, I had as much access to my iPhone as any able-
bodied person does.

Sometimes good news is just good news.

------
mankash666
Just noting that if accessibility was front and center, the tablet wouldn't be
priced exorbitantly. Let's take this for what it is, good PR

~~~
freshyill
That's an extremely cynical take.

~~~
mankash666
Sad, but true. There couldn't be a better PR plug than this for Apple, as they
continue to price their products outside the affordability of the disabled. If
70% of the blind are unemployed, how can they afford an iPad?

~~~
rimantas
Go check the price of JAWS. By your logic anyone making great is just doing
that for PR.

~~~
jscholes
> Go check the price of JAWS. By your logic anyone making great is just doing
> that for PR.

Worth noting, perhaps, that very few blind people actually own a JAWS license
that they paid for themselves. The majority[citation needed] either get grants
from charities, financial help if they're in work or studying, or use a
pirated copy.

