
Recognising gym excerises in real-time using a neural network - argilium
http://dilpreetsingh.me/activity-recognition
======
argonaut
Here's a problem: I'm pretty sure you'd get just as good (probably _superior_
) results with a simpler, faster out-of-the-box algorithm like logistic
regression, random forests or boosted trees (depending on the amount of data).
This just isn't a problem that seems well suited for neural networks.

Gyroscope/accelerometer data might seem like the kind of extremely-noisy,
high-dimensional data well-suited for neural nets, but 1) you don't have a lot
of data, and 2) intuitively the data isn't actually that noisy; my guess is
the data separates out quite nicely (not necessarily linearly).

~~~
blennon
I agree and disagree. I would definitely start with a simpler algorithm like
logistic regression or even trees and use temporal features like power at
different frequencies by taking an FFT. I would even add time lagged features.
After that I'd graduate to a single hidden layer MLP.

If I had tons of data (a lot of different people using it), I'd experiment
with LSTM neural networks because I think the temporal information is crucial
for determining a movement.

I think a killer app would be aimed at weight lifters. If you go to the gym,
the serious weight lifters record their reps and weight for each exercise. The
app would utilize the iWatch or some other wearable and detect the exercise
and count the reps. Then it would prompt the user for how much weight they
used.

~~~
argilium
You're right on the ball. I'm indeed using temporal features to classify,
along with various other statistical ones.

The repetition counting in my next aim. Once apple opens up the gyro on the
Watch, I think there could be a good chance to get something like this out the
door.

------
recuter
"Wouldn’t a better approach be to initially train the neural network on more
than just one person? That is a great point! In fact Microsoft’s research arm
published a paper last year doing exactly that. Although they achieved great
results, their initial training cohort consisted of 94 participants! I, as a
one man team, can’t possibly duplicate that. This is why I created a system
that can adapt, eliminating the need of Microsoft level resources."

\--

Rather than using an accelerometer (OP attached an old iPhone to his person
while doing the movements) - you could plausibly point a camera and try to use
video as the input.

If that pans out the next step would be using Mechanical Turk to cheaply and
quickly build the initial training set (no pun intended) using publicly
available videos of people working out.

~~~
argilium
That's an interesting point. The problem here would be that video input would
not provide accel+gryo motion information, which is what the network needs in
order to learn.

If we instead simply used video information to track exercises the problem
would be scaling that to consumers. They'd require an external camera to watch
them.

But the idea of using Mechanical Turk is quite smart. It'd help me get varied
form - especially if I can get them to wear a band/phone that has the sensors.

------
jasonjei
And this what happens when you bring a Computer Scientist into the gym! One
question: how well would a neural network recognize a more complex and
involved exercise like Kettlebell Turkish Getup compared to an exercise with
simple movement like Push-ups?

~~~
argilium
That's a good question! Turkish Getup's are very complex like you say. Given
my current setup and window length, it's unlikely that it'll recognise them,
because one rep is very long. Maybe if I did exercise based window lengths it
could work better? I'm honestly not sure. But that is something to try out!

~~~
jasonjei
To be honest, a human seeing it for the first time would have a hard time
identifying Turkish Getups too :)

Also, identifying good form--that's an app that could help a lot of amateur
athletes.

------
SebKba
That's awesome! I didn't think this would work but it does. It would be
amazing to combine this with a training tracking app. So far the data entry
has always been really annoying so only machine exercises could really be
captured properly. Really exciting stuff. I think this has a lot of potential
for a startup!

------
seagreen
This might be a good chance to ask something that's been bothering me -- is
there a good training log format for weightlifting?

I can find apps for recording training data, but they all use their own
formats. Anyway, if there is one it would be cool if this program could use it
as output, instead of making up its own ad-hoc format.

~~~
argilium
That's a good question. I'm also quite sick of all the logging apps that have
their own unique formats.

If there is a standard out there, I'll definitely look at integrating that for
the output.

~~~
seagreen
Thanks! If you find one ping me, I'll do the same.

------
mpg33
Actually thought about an app that would do this on a smartwatch. I would
definitely buy a smartwatch if I could have this level of accuracy of body
movement tracking.

~~~
MaxGabriel
WatchOS can detect the type of exercise, based on a recent talk I saw. Here's
the docs on the types of exercises detected:
[https://developer.apple.com/library/watchos/documentation/He...](https://developer.apple.com/library/watchos/documentation/HealthKit/Reference/HealthKit_Constants/index.html#//apple_ref/c/tdef/HKWorkoutActivityType)

------
dshah22
This is fantastic! Do you think it's feasible to develop this into a 1 phone
solution? I can see a lot of applications in the rehab sciences area.

~~~
barake
The article says the second phone is just for accelerometer data, so it could
be replaced by some sort of Bluetooth device. I'm sure one of the many fitness
trackers on the market provide real time data.

------
santa_boy
Cool stuff. Is this open sourced by any chance? Just want to tinker with this
a bit

~~~
cavancanavan
Not truly open, but we have an SDK similar to this that you can play with.

It works with any open accelerometer (Apple Watch, Android Wear, Microsoft
Band, Pebble). We track 50 exercises, auto classify 22, and there's a machine
learning tool that let's you add new movements to the system and improve it
over time.

www.focusmotion.io

~~~
santa_boy
Ok. I'll try it. Does this have a NodeJS SDK?

------
satjot
This is great

