
Musician Who Lost His Arm Plays Piano Again with AI Prosthesis - bcaulfield
https://blogs.nvidia.com/blog/2017/12/28/ai-prosthesis-skywalker/
======
hypersoar
This reminds me of Paul Wittgenstein (brother of philospher Ludwig)[1]. He was
a concert pianist who lost his right arm in WWI and the commissioned a bunch
of piano music playable with only the left hand, including works by still-
famous names like Ravel and Prokofiev. My personal favorite is Korngold's [2].

More recently, there is Leon Flescher, who was a highly-renowned pianist until
he lost the use of his right hand in the 60s. Amazingly, he got it _back_ 40
years later and has enjoyed recent success again as a two-handed pianist.

[1]
[https://en.wikipedia.org/wiki/Paul_Wittgenstein](https://en.wikipedia.org/wiki/Paul_Wittgenstein)
[2]
[https://www.youtube.com/watch?v=bGulyl8bzgQ](https://www.youtube.com/watch?v=bGulyl8bzgQ)

~~~
joshvm
Some famous (ish?) modified versions of Chopin's Etudes were written by
Godowsky. His mission appears to have been to make Chopin's hardest pieces
just a bit harder, including the Revolutionary for the left hand only:
[https://youtu.be/X2nMUwdh1Wk?t=2m41s](https://youtu.be/X2nMUwdh1Wk?t=2m41s)

~~~
asimpletune
Woah

------
dpark
Typically these “futuristic” prosthetics are still relying on myoelectric
sensing and a lot of wishful thinking. It’s nice to see some meaningful
progress in more detailed sensing technologies. This actually seems promising.

~~~
melling
This was a hot thing a few years ago. These guys still seem to be around:

[https://techcrunch.com/2013/06/05/thalmic-labs-
raises-14-5m-...](https://techcrunch.com/2013/06/05/thalmic-labs-
raises-14-5m-to-make-the-myo-armband-the-next-big-thing-in-gesture-control/)

[https://www.myo.com](https://www.myo.com)

------
bcaulfield
Once these are working well will factory workers be able to attach a third and
fourth arm for extra productivity?

~~~
osteele
The brain has a couple of regions (the somatosensory context, the primary and
supplementary motor cortex) with maps of the body. You'd need to add the limbs
to those maps. Brains have enough plasticity to re-allocate the area
responsible for limbs or fingers by a minor amount, but nowhere near enough to
add a new limb, at least in adulthood.

~~~
ars
I'm not so sure that's true.

People who use a tool for a long time report the tool feeling like an
extension of their body.

Also, try this thought experiment - try to flap your wings. I know you don't
have any, just try to activate them anyway. Do you sense yourself trying to
move "something"? Your brain clearly is able to map anything it wants.

~~~
icegreentea2
For the technology specifically covered in this article, all they are doing is
reading the muscle activation of existing muscles, and then mapping to the
prosthetic's actuators. The "twist" in this story is that sensing is done with
ultrasound instead of EMG (better signal to noise if you can throw enough
compute at it), and I believe some cool deep learning tricks to map sets of
muscle actuations to prosthetic motion "better".

In order for this approach to work, a nervous signal has to travel out of the
brain, and at least into the brain stem. In these cases, they actually have to
get all the way to a giant amplifier (your muscle) in order to detect it.

When your talking about your brain imagining to flap your wings, unless your
brain has mapped your swings to your lats, traps and delts, you're not sending
any actual motor signals out of your brain and we can't read it. If it is
mapped to your lats, traps and delts, then you're actually moving those
muscles right now. If we used a system where we tried using your lats, traps
and delts as points to control prothestic wings, then you run into the problem
of what do you with your wings when you're not trying to use those muscles as
a wing controller. In other words, you'll get a modal system... which are
tricky. You'd basically have to switch off parts of 'baseline human'
functionality at a time to get your 'human+' stuff.

~~~
red75prime
We have useless muscles in outer ears though. Maybe it is possible to
repurpose them.

------
partycoder
It is progress, and it is very impressive but the title is a bit misleading.

I think there is still a gap between what is being demonstrated to fully "play
piano again", in the context of playing piano professionally as a musician.

For composition it can be still very valuable.

------
adrya407
What really frustrates me when I watch this kind of demonstrations with
bionics is the delay and inaccuracy between desired action and the real
effect. I know it's not that simple, but from my perspective since the desired
actions are transmitted through our nerves at 100 m/s and we can send an
message thousands of kms away in milliseconds, I think there is always space
for improvement. I am curious which is the real bottleneck in all the process.

~~~
JamieBeckett
As you can see in the video
[https://www.youtube.com/watch?v=HjW1kIt5iQg](https://www.youtube.com/watch?v=HjW1kIt5iQg)
there is some latency. Weinberg and Barnes are working to fix that, but I
don't know details. Weinberg said he wants to gather more data from more users
so the arm is more useful to more people.

~~~
adrya407
But why should one rely on the termination of the nerves instead of the
implating an electrode on the main cortex which controls the motor activity?
Can't they translate directly the actions from the energy flow at that level?

------
rjblackman
how hard would it be to translate the movements to midi? I'm guessing there
would be some way to capture the signals the prosthesis and characterise the
mapping from movement to midi.

