

Paralyzed Woman Controls Robotic Arm With Her Mind - bcl
http://www.wired.com/wiredscience/2012/05/thoughts-control-robotic-arm/

======
kevinalexbrown
Disclaimer: I work in a similar lab (my adviser was advised by the senior
author on the paper, who runs the lab) on a brain-machine interface project
funded by DARPA.

What makes this awesome from a scientific point of view:

1) 2-dimensional decoders, where an individual controls a mouse with their
mind, have existed for a while. What makes this really cool is that there are
actually quite a few degrees of freedom in a robot arm, but they're basically
using the same hardware. So the influence of _software/algorithms_ in this
case is pretty fundamental. There are a lot of papers on improved neural-
decoding methods that at first glance appear really dry, boring, not the
`sexy' kind of science with huge breakthroughs, but they end up being crucial
to good performance as the complexity of the robot grows.

2) One participant was implanted with the electrode array 5 years before the
study, and had the injury 10 years before that. Usually the signals don't last
that long in monkey models. And we know that your cortex changes with disuse,
so it's awesome that they were able to get usable signals so many years later.

What needs to get better are a couple things:

First, these decoders aren't perfect yet. What the Wired article didn't tell
you is that the performance for the woman's implant was around 20-50%
successful trials (still awesome from a comparison to no interaction)[1].

Second, incorporating sensory feedback is another challenge that is really
hard to address, but also very important. Imagine building a robotic
controller in which the only information you received about the robot's
position was visual. That's the way this works. If we find a reasonable way to
mimic sensors of muscle extension (a proxy for joint angle) then we can create
more controllable devices.

[1] [http://www.nature.com/news/mind-controlled-robot-arms-
show-p...](http://www.nature.com/news/mind-controlled-robot-arms-show-
promise-1.10652)

~~~
johnpaulhayes
No doubt this is amazing. But a question strikes me... why is a military
organisation funding this research? By their very nature, are they not in the
business of disabling enemies (a.k.a people)?

Just strikes me as odd but I can see the potential of weaponizing this
technology.

~~~
cobrausn
Dismemberment and lost limbs is fairly rare among the civilian population.
This is not the case with the military - it makes perfect sense they would be
the ones funding this, as they are regularly sending home guys with fewer
limbs than they went out with.

I think they should be funding this - they owe it to their disabled veterans.

------
jacquesm
This is extremely impressive, what a beautiful piece of work.

The smile on that lady's face says more than any number of facebook 'likes' or
google +1's could ever do for me, that's a real and measurable quality of life
improvement for a single person, this really makes a difference.

The engineering on that arm is pretty heavy duty, I wonder what it weighs and
how fast it could move and what kind of safeguards are built in to avoid the
operator injuring themselves due to glitches in the system. If you look at the
way the arm moves it is actually quite comparable to the arm of a baby that
moves an object to its mouth the first couple of times.

With one big difference, a baby may get it wrong but it is not strong enough
to do too much damage, even if it pokes itself in the eye every now and then
it is usually with very little force. This arm however looks engineered to be
strong enough and fast enough that it could do real damage to the operator or
its environment. If that's servo driven there has to be a whole slew of safety
systems in case a driver decides to hook a motor to V+ or V- because of a
blown FET.

In any real-world machinery situation where the machinery is under software
control and has the capability of doing real harm (for instance: machine shops
with CNC gear) there are normally countless interlock systems that you'd have
to bypass before you could get yourself in contact with a piece of it moving
under control of the computer. In this case the operator is extremely exposed
and I cringed when the arm sped up towards her face with the bottle. I also
found myself sort of 'willing the bottle' in the right direction, they way
some movies will have you react to something on the screen. Hard to describe.

This is not a soft hand moving either, it looks like it is made of pretty
strong aluminum and fairly heavy.

~~~
saulrh
They're using a Kuka DLR lightweight arm. It's a rather high-end piece of
equipment. It's strong enough to do a pull-up, fast enough to catch a ball,
and every joint incorporates a six-axis force-torque sensor. It's definitely
strong enough to do damage. Of course, that just means that Kuka loaded it up
with safety systems. The FT sensors let it know exactly how much force it's
applying, and if they read too high a value the brakes slam on. I've never
worked with those arms before, but I'd bet that there's a similar system
completely in hardware. You'll also notice that the operator's left hand and
the entire back half of the room are never seen; there are at least two
emergency stop buttons there, probably more, and one of them is wired directly
the arm's power supply.

Really, though, all that engineering is of secondary interest. That look on
her face is exactly why I got into robotics. And we've only been at it for 15
years; imagine what we'll be able to do in another 15.

------
rmATinnovafy
This is an area where hackers can really make an impact.

We tend to forget how necessary movement and control is for our daily
routines. There are people out there who even lack the ability to turn on|off
the TV. A little hacking with an arduino, and some buttons will make it
possible for them.

In hackaday.com, one consistently sees hacks that are for the disabled.
Special game controllers for people missing an arm, or some other limb are
popular and simple to re-create.

I do have a personal anecdote.

I was once buying some resistors on the local radio shack. While I looked in
the mess of tiny plastic bags, another person started to browse for electronic
bits. I started to talk to him, and found out he was a doctor. His work was in
dealing with disabled people. He had learned how to work with embedded
electronics and fiber composites in order to build prosthetic limbs for
impoverished patients.

Just amazing.

~~~
jowiar
> This is an area where hackers can really make an impact.

One of my friends has coauthored several papers with some of the BrainGate
folks.

We spent a road trip a couple years ago discussing his research. One thing
that jumped out is how similar many of the tools used in this field are to
those in a startup. In particular, many big-data analysis tools and techniques
that can be developed, optimized, and funded within the context of the web
sphere are applied to the bioengineering problems. A few months ago an article
floated around HN on how the current tech bubble, if it's such, lacks a
"byproduct" that will benefit society when the scraps are cleaned up. You can
make a pretty good argument that it's this.

~~~
rmATinnovafy
Yes, good point.

Once the photo sharing app market dries up, people will have to look into
other areas. I know that in the next ten years we will have startups (even 1-2
person teams) building much more than simple web apps. Innovation will slowly
seep to other areas.

(And thus, Jeff, that is why people need to learn how to program)

------
etrautmann
I also work in a similar lab as a phd student. Kevinalexbrown's points are
spot on. Currently, the main challenges for implementing this clinically are

1) stability of the electrode array over long timescales.

2) increasing the number of degrees of freedom of robust control that we can
decode from the neural data. Activity in the motor cortex inherently lives in
a (very) roughly 10 dimensional space, which is a bit of a mystery since we
use thy activity to control hundreds of muscles independently. What this
means, is that when you record from 96 electrodes simultaneously, many of the
neurons picked up by the array are correlated, such that the resulting
dimensionality is much lower than the number of neurons

3) An important step forward will be to develope optogenetic (or other)
sensory write in, using pulses of light to activate neutrons in specific
patterns that mimic priprioceptive signals from your limbs. This will increase
the speed and robustness of movent via faster closed loop feedback.

4) Processing power. Currently it's typical to run decoders on real-time PC's.
Creating an embedded processor capable of sufficiently low power operation
with enough processing power to run the decode is non-trivial, especially if
you want to implant this in the brain.

It is very unlikely, as suggested elsewhere, that current web tech will impact
BMI research at all. The primary interesting tech is in the decode algorithm,
which is a modified form of the Kalman filter. There is, however, a lot of
room for hackers in research. I recently made the switch from working in
defense, building robots and designing sensor fusion algorithms, to a phd in
neuroscience. Probably won't pay as well long term, but it's far more
rewarding and interesting! I'm surrounded by scientists that need better
(software and hardware) tools and analysis methods. Also, an infusion of ideas
and values aligned with open access publishing (or changing the scientific
publishing model altogether), open source software, data and code sharing,
etc. would generally benefit all and accelerate scientific research, but
that's another topic worthy of more discussion elsewhere.

------
panic
If you're interested in helping out with the next stage of this research and
you know someone with limited or no ability to use both hands who might also
be interested, please visit <http://www.braingate2.org/clinicalTrials.asp> and
let them know.

------
clicks
I know I know, it's not good etiquette here to make short, emotional comments,
but -

 _By thinking about moving her own paralyzed arm, one woman in the experiment
used an artificial limb to serve herself coffee for the first time in 15
years._

Seeing her smile at the end of that video just made my otherwise terrible day
rather special. And it's such great validation for the team that's working on
this technology, I hope they continue to see much success in the future.

~~~
icarus_drowning
Its just so wonderful to see what could otherwise be a frustrating,
debilitating experience become something incredibly _empowering_.

~~~
alan_cx
Finally, a decent use of the word "empowering". Shame I can only up vote you
once :)

------
jezclaremurugan
The real deal is that this needn't be confined only to people with
disabilities. We all can use an extra limb or too. But, I think it will take a
little more time to become such a sci-fi toy, after the urgent needs of
disabled people are met..., but the size of the market if we look at this as
applicable for everyone is huge.

------
andynosebone
Hehe, this is friggin' awesome. The only thing I could think of that who's the
first one to go all Deus Ex about this!

------
mukaiji
That woman is adorable.

------
grantgrant
what is she thinks dirty?

~~~
jacquesm
The only person in this thread that thinks dirty is you. Really, someone is
capable of some level of direct control over her environment for the first
time in years and that's the only thing you can think of? Look at the video
and watch that lady smile at the end and then ask yourself what your comment
said about you instead of about her.

~~~
pazimzadeh
I think grantgrant has a point. I wonder what it's like to use an arm like
that - she seems to be focusing hard on thinking "up" "down" "left" "twist"
but as this stuff gets more powerful, how do you separate the signal from the
noise? I have many thoughts bouncing around my head, but somehow my brain
knows not to act on them.

~~~
etrautmann
The brain is a "parallel processor" in that distinct regions are responsible
for different tasks. These arrays are recording from the primary motor area
(M1) which directly controls output muscles. While this does require conscious
control right now, it's not the case that irrelevant thoughts are likely to
significantly alter the control signals sent to the arm.

~~~
Symmetry
Probably less, since you don't have to worry about, say, adrenaline dampening
the negative feedback loops between brain and muscles.

