How did we not have this 10-15 years ago? It just feels like not enough people work on, or care enough about assistive tech till it impacts them personally.
I work in a lab at a university where I'm currently working on building assistive music controls for disabled musicians. From what I've seen in my short time here is it seems like a lot of these kinds of projects don't often translate from the research space into viable businesses. Also, due to the highly variable physical and neurological characteristics, often assistive devices will need specific customization, tuning, and support. So these things also tend to be expensive.
This is exactly correct (I worked in AAC and BCI for years, and still collaborate with the group that I used to work closely with). To elaborate a little bit on your excelelnt answer: In addition to being enormously expensive and requiring heavy customization and personalization, assistive technology has other issues that go well beyond the core technical functionality. It also often requires significant up-front investment in configuration and training for the user, which needs to be done by a professional in the field, and then --- crucially --- also requires ongoing maintenance and re-training by skilled practitioners, which means there are ongoing operational expenses.
One of the reasons for this last bit is that many users of assistive technologies have very dynamic situations. For example, if they have a progressive condition like ALS, their mobility and other capabilities and needs will change over time; even if the user doesn't have a progressive condition, their living situation very likely will change over time, which may involve needing to re-work their setup. This might involve something as major as moving from one care facility to another, or having to adjust to the presence of a new pump or something that causes EM interference with your device.
Even for somebody with a very static medical condition and very stable living situation, their computing setup won't be static, so will need ongoing maintenance; or, alternatively, their computing setup will become static if they end up dependent on a particular piece of assistive technology, which can lead to a whole other set of issues (which is how you get people running ancient versions of Windows for years and years and years, because it's what their eye-tracker works on, and the vendor is long out of business and so never got it working on Windows 10).
The other piece to the puzzle is that, speaking as a gross generalization, users of assistive technologies tend not to be wealthy people (many speech and motor disabilities make it challenging to work). So, you've got an expensive product, often involving ongoing operational costs, with zero economy of scale, for a small and relatively poor customer base; this all adds up to what may fairly be described as "a very challenging business". Adding another layer of complexity, the economics of commercial assistive tech in the US have a lot to do with insurance Medicaid rules; the only reason there is any commercial activity at all in assistive technology and AAC is because under some circumstances Medicaid will pay for a device, within certain very rigid and complicated boundaries. So now your business also is going to have to juggle that problem in order to keep the lights on, which will add another layer of constraint to what is already a complicated situation.
On the other hand, when the right technology is paired with the right user and the right support, the impact on the quality of life for the user and their family can be truly enormous. Which is what keeps people working in the space...
At least one company I know of (though this was 15 years ago) had ridiculously high prices for their products. But they had extensive after-sales support, and the warranty was such that you could basically run over the unit with a car, and they'd still repair / replace it under warranty for free.
We're currently building a new AAC device which allows users to write/speak quite a bit faster, and we're encountering exactly the hurdles you mention. Would you be willing to exchange some helpful pointers about how to bring an AAC device to market? Contact info in my profile.
I suspect it’s a couple things. First, the skill gap - maybe someone begins needing assistive tech and has no idea how to start making. Second, perspective - as an able-bodied techy, I lack the understanding of the problem to build the right thing. Third, communication - can the one in need describe their need? This really requires reflection, and maybe skill. And lastly: money - how many of is can afford to work for months or years on a project that doesn’t (and likely won’t) pay?
We tried about 9 years ago :) we even open sourced the solution. At the time we felt there was not a big enough market so we decided to make it open for anyone to build upon our research.
We have working systems like the Tongue Drive System (TDS) [0]. Tech companies seem out there [1] or [2] that offer solutions, but the a11y market is largely dysfunctional across the globe.
It is so difficult to get everyone the tech that works best for them. I was part of large EU funded project that wanted to fix this with an open source ecosystem [2]. However, I had to learn that although there are many innovative and engaged people it is really hard to fix seemingly simple problems.
Exactly. Its unfortunate that this is the way the world works...but we hope that the MouthPad^ can change that tired script! Universal design benefits us all, not just those at the extremes -- we should all care. See the history of curb cuts on our sidewalks as a small example.
For a short time in DC (District of Columbia, USA) government, I worked with Age-Friendly DC. They were all about universal design, obviously. I learned a ton - like how curb cuts are good for folks using canes, walkers, and wheelchairs, and those who push things like strollers. However, the Age-Friendly DC staff could have done much more to work with non-aged policy advocates and organizations, to show them the benefits of things like curb cuts.
Personally, I'm using a lot of what I learned from the Age-Friendly folks; my 83yo mom has moved in and the learning curve, while steep, got a head start with my government experience.
I went to grad school for architecture at the end of the last century.
Tim was a classmate. He was and remains paralyzed from the neck down. At the computer Tim manipulated a trackball with a tool — rod connected to a flat mouth piece.
This is different, but I am not sure it is better. The trackball and stick were mostly mechanical, reliable, and straightforward to repair.
No special software. No need to charge. Tim’s tools were as reliable as a mouse. Not much of a choking hazard because of the stick either.
I’m not saying the mouth-pad is a toaster fridge. Only that it has technical complexity rather than mechanical simplicity.
It’s a radio and computer that goes inside a person’s mouth. That person can’t use their hands. How do they take it out and put it in?
Tim had a red solo cup size container next to his keyboard — he also typed with the tool in his mouth. [1] When he wanted to talk to someone, he stuck the stick in the cup and stood the tool up where he could pick it up again.
[1] I hadn’t remembered that because it’s been 25 years. Tim could also out sketch most people with a pencil in his mouth as well. He had a bachelor in fine art before the motorcycle wreck.
MITs MO is to try to create a breakthrough tech and rapidly spin off startups from the tech. But I think the main barrier is that to operationalize hardware requires massive capital investment upfront for testing and then regulatory approval, this is mostly not possible for VC. So the students/profs who make the breakthrough do the next best thing: Patent and then sell it to a company that has the capital to do it.
But someone can correct me if I am wrong about this.
I can imagine this helping anyone that has struggled with computer-related overuse injuries. I had a bad case of tendinitis that made using a mouse or trackpad very painful. I would have loved to have this kind of device as another option.
Best I personally managed was push-to-talk via a foot pedal, to help with mic discipline in cooperative games without reducing my reaction time.
I'm not sure my feet have the accuracy to work as a mouse.
I think assistive devices do better as a class when some fraction of them are also usable or even attractive to able people as well. It gets us thinking about other modes of interaction and of course it brings the unit price for those items down out of the stratosphere.
Foot mouse will unfortunately stay niche, but when I saw the device in this article, I thought, "Why not for everybody?"
Regarding fatigue: We constantly use our tongues for talking and eating without feeling tired and so far, current users haven’t reported tongue fatigue issues.
I don't get the tongue idea either, any kind of device you'd put in would make it hard to talk and who wants to eat a battery? Plus the movement area is pretty tiny.
Accurate foot movements are just training I bet, just like with anything else. Arm amputees have shown again and again that it's just a matter of practice. Plus, were any of us really that proficient with a mouse when we first started? We take it for granted but we've been honing that for decades.
> trackpoint
Looking at how that one works, it's just four load cells. That would be exceedingly simple to reproduce with a microcontroller that supports HID. I might actually try to make one.
I did experiment with using a trackball taped to the floor. It's hard to get precise positioning moving your foot. I think using the tongue along the roof of your mouth would be more precise and less fatiguing. What was more useful was a set of 9 foot switches that could be programmed to send arbitrary keystrokes. I could off load from my hands the most common keypresses I do all day, for example: pageup, pagedown, tab, enter, backspace, mouse click, passwords... along with speech recognition software you can get at least non-coding tasks done without using your hands too much.
> It's hard to get precise positioning moving your foot.
How long did you do your experiment? Obviously most of us are not typically used to point and move precisely with our feet. So to be efficient we would need to train for months but that doesn't mean we can't be precise with our feet if we were to dedicate enough time for it.
I once saw a documentary about a german woman who was born without arms. She would do everything with her feet, writing, drawing, smoking a cigarette or mounting a horse and holding the reins with one foot. We are much more adaptive than we think we are but we don't realize it when we have the luxury to not needing it.
I went to grad school with a guy w/o arms. Sure, he could write with his feet. It was extraordinarily hard and difficult. Nothing changes the physical facts - we neither have the nerves or fine control in our legs/feet that are present in our hands.
We also build fine motor control in very young childhood; children can't color within the lines (for example) because the nerves just aren't there yet. I would expect someone born without arms to have better fine motor control of their legs than an adult that tries to develop the ability. And it takes them years - 7 or 8 years to get pretty decent control. Even if I had the neuroplasticity of a young child (I surely don't) I wouldn't want to spend years trying to develop that control.
I face this every day on the piano. You distribute motion between fingers, wrist, forearm, elbow, and shoulder based on the amount of fine control needed. Sure, you can use your shoulder to trill, but it is always going to be very slow and clumsy. finges are best, but you face fatigue, so you usually use wrist rotation to get muscles with more endurance suppling the gross motions, and then fingers for the fine control of dynamics. You can't change this physical reality, at the most you can compensate (e.g. someone with a fused wrist would have to find alternatives).
Can you link the hardware you're using? I have one foot pedal for push-to-talk and I think more could be useful, but I don't want to tie up 9 USB ports.
Yes, push to toggle microphone is another great use for these!
I was using the Kinesis Programmable Foot Switch 26 years ago with ps/2 connectors (later they switched to USB). You would get 3 foot switches in each unit and chain them together along with your keyboard so they only used one host port. The most similar thing I see now would be https://www.amazon.com/X-keys-Foot-Pedal-Playback-Control/dp...
It's frustrating that I can't reprogram my old foot switches since the manufacturer stopped supporting them. If buying today, I'd look for something based on open source QMK firmware like my keyboard uses. This project looks cool: https://github.com/christrotter/qmk_firmware/tree/arcboard-s...
Lift your foot up and try to balance in your seat and see how it feels. Try using a standing desk with it. Put a pen between your toes and try to draw a square and feel major muscles in your upper thighs struggling to make these fine control motions. Put on high heels and try to do anything that isn't gross movement. Sit in a train/plane/car and think of trying to use a laptop with a foot controller. Heck, put your mouse on the ground right now, turn pointer speed down as slow as possible to accommodate the gross movements of your legs, and try to use it.
FWIW, I've considered mapping a midi pedal board to meta keys and macros before. That alone could be useful - you'd go from a typist to something like an organist.
The recently launched Vision Pro and its eye tracking also offers a relatively niche and very accessible input method. It is already being used in this context:
> “I was having trouble getting around that day,” he said, so he took the day off and did work for his consulting agency, Equal Accessibility, from bed, wearing the Vision Pro, surrounded by screens he controlled with his eyes and a series of custom mouth-sounds that triggered selections. “As I get older, and this happens more often for me,” he said, “I envision myself working virtually with the AVP even more.”
Eye tracking is coming to iPhone and iPad too. I’m sure it won’t work quite as well as the Vision Pro’s hardware, but for someone who can’t take a headset on and off by themselves it’s nice to have options.
I just want this for extra control options in gaming. I bet I could get good at aiming with my tongue. Shorter distance brain to tongue than brain to hands too? Faster response time? Tongue control is going to dominate in esports ;) RGB mouth guards incoming.
There is a streamer named RockyNoHands[0]
He plays Warzone entirely with his mouth. His controller has tubes that he blows and sucks to control the character.
It is really impressive watching him live on such a competitive game.
Some people have much stronger and more nimble tongues than others due to physical differences (e.g., tongue size, the length of the tongue frenulum, etc.). But this is the case for the hands as well, which is why people have different input preferences and different gameplay preferences.
Personally, I think I would prefer a mouth controller over a traditional controller. I am very prone to tendinitis in my hands, plus I have small hands and short fingers, and can't play twitchy games well because of it.
The picture of a tongue mouse from article (maybe it's a render I don't know) seems to have maybe 16 dpi so it is probably fits for secondary mouse just for rotating the world, but for tiny pixel-hunting I would like to have something with a multiplicator.
Assistive technology for the disabled is the higher use for mouth-input, but it'd be great to have a simple bluetooth mouthguard with media playback controls (play, pause, next previous) for action sports.
- Single tap on the roof of your mouth: play/pause.
- Tap on the inside of your upper left/right teeth: previous/next.
- Drag your tongue along the inside of your teeth: playback scroll.
- Drag your tongue on the roof of your mouth from front to back and back to front: volume up/down.
It seems very intuitive, and like it would be much more precise than any of the finger gestures we use today. My only concerns would be comfort, mistaken inputs, and impact on speech. But then again, some people wear dental jewelry and it doesn't seem so bad. :)
While snowboarding or skiing, it is difficult to control music playback because bulky gloves are being worn. Cycling might be another sport where you want to control music without taking your hands off the handlebars. A single button (like tapping an earbud) that could be gently bit down on could control song playback. Prior art for this type of device would be something like this: https://www.airturn.com/products/airturn-bite-switch-remote-...
It's very strange to me to listen to music while snowboarding or cycling. Especially while cycling you need a lot of situational awareness of what's going on around you. I think it's very dangerous not to be able to hear cars coming up from behind.
You’re absolutely right, but a truly startling amount of people on the trails give absolutely zero care about other trail users and are just reckless antisocial parasites.
Some headphones (eg airpods) have pass-through audio to allow situational awareness. There are also trails in my area that allow bikes but that are very remote and are unlikely to have much bike and pedestrian traffic. It wouldn’t be unsafe to have headphones in there.
Kind of tangential, but I don't understand why keyboard/mouse layers haven't caught on more. I kickstarted the UHK[1] years ago, and haven't had a mouse ever since. It does take a little bit of getting used to, but controlling the mouse with a keyboard is really amazing.
Can we talk about the absolute perfection of the name? MouthPad sounds like someone trying to say mouse pad with a MouthPad in their mouth. It's beautiful.
Exactly! I also started to pretend scroll with my tongue and it actually felt quite intuitive. It is a very clever solution! But they will probably need a well written driver which rejects accidental touches since at any point in time my tongue touches the roof of my mouth.
People living with such extreme disabilities are so much stronger than me.
I put a lot of pride in my independence. If I needed someone else to feed, clothe, and bathe me, I think I'd rather die. I'd feel like I was no longer living, but merely surviving.
Probably most people would think like that, but it all depends on what you can still do. If you become a good writer or even if you stick to software, but make valuable contributions, that improve the world a bit, or become a great orator, or give people good counsel, or any other great activity, you might find, that that is a goal worth still continuing to live.
Does anyone here know how the sensor technology actually works? Is it capacitive touch, a low voltage electrical connection, or something else? Also, how would something like this differentiate mouse use from speech?
Oh hey, I remember this, signed up to beta test a year or so ago and 6 months later got an email that said I was rejected for being outside of California. I think at the time I found it weird that it took them 6 months.
Hi Logan, thanks for your comment, and apologies for the delay in our response. We're a small company with limited resources, so we've prioritized waitlist applications from those who really need our solution and are close to us. We're working on scaling our operations and hope to serve everyone in the US soon. Stay tuned!
Also consider perforations in the inert parts of the MouthPad. Especially considering its intended users, it's a choking hazard and sooner or later somebody will get one of these stuck in their throat. You'd rather it not kill them when that happens.
Better than eye-tracking? It's all though-experiment land for me, but when I think about the precision and speed with which I can move and "click" something, nothing beats the eyes.
Maybe. I just think of all the other things that my eyes want to be doing. I figure that would interfere too much in the usefulness. But i agree. We should just test out all the wacky ideas and see what works best.
Great point, maybe that is the weakness in eye tracking. What about a small switch in the mouth you press with your talk, similar to a push-to-talk that activates the eye tracking. When you aren't pressing, you can look around and do other things with your eyes without affecting the computer?
Wish I had the bandwidth to take this on right now, because it sounds super fun and something I could be very passionate about.
Yes, Tobii sells cheap cameras and SDKs that have done exactly this for years now, to the point that several random games have Eye Tracking integration for no reason.
People on HN complain how a substantial amount of the comments are negative. I give my honest opinion, as a doctor, knowing full well how hard it is to not only clear the hurdles to get to where they are at, but I also appreciate that it is a worthwhile endeavor. In an era where half of the people in tech are wasting their lives optimizing SEO and to sell more ads for useless crap, these people are out there making tangible improvements in people's lives.
Not exactly related to disabled people, but a though crossed my mind on how cluttered F1 pilot wheels are with buttons, I imagined some F1 Engineer already thinking on how feature dense these pads can get and how to integrate them into races.
reply