
Introduction to Apple WatchKit with Core Motion – Tracking Jumping Jacks - hsiaoer
https://heartbeat.fritz.ai/introduction-to-apple-watchkit-with-core-motion-tracking-jumping-jacks-259ee80d1210
======
moflome
Thanks for posting @hsiaoer and looking forward to your ML classification
follow-up post. I've been focused on similar research and read with interest
the ARM DynamIQ ML focus [0] as I believe this will open up ML edge processing
on wearables, although hard to beat what Johny Srouji and team are doing.

WatchOS 4 seems to support CoreML [1], do you plan to do inferencing on the
device eventually? If so, cool!

[0] [https://community.arm.com/processors/b/blog/posts/arm-
dynami...](https://community.arm.com/processors/b/blog/posts/arm-dynamiq-
expanding-the-possibilities-for-artificial-intelligence)

[1]
[https://developer.apple.com/library/content/releasenotes/Gen...](https://developer.apple.com/library/content/releasenotes/General/WhatsNewInwatchOS/Articles/watchOS_4_0.html)

~~~
hsiaoer
Thanks for the link!

Yes I would love to get the CoreML model onto the watch itself. I think the
interesting part will be tweaking the model and exploring how we can build it
with space, battery consumption, and resource utilization in mind. The nice
thing is that we can possibly deploy 2 models (one to the phone and one on the
watch): the one on the phone runs when the watch is nearby and is more robust.
The other runs directly on the watch itself with some tradeoffs). We might
discover that we need better hardware on the watch but that would be
interesting to explore too.

------
saagarjha
> Unless you’re extremely flexible, changes are that your palms won’t be
> facing away from your body (if you’ve put the watch on correctly).

Keep in mind that some users wear Apple Watch so that the screen face is on
the inside of their wrist, rather than the outside. To a naïve interpretation
this would appear as "palms facing away".

~~~
defap
So many features are broken in this orientation that I doubt anyone would be
surprised by this.

~~~
AckSyn
iirc, you can set the orientation in the Watch settings. The OS and software
should adjust.

~~~
comex
The orientation settings let you choose left or right wrist and, separately,
left or right facing Digital Crown, which means you can turn the watch "upside
down" and wear it that way. But it doesn't have a setting for wearing it on
the inside of the wrist.

~~~
zakki
What if I use the watch at my right wrist and set is as left wrist? Edit for
clarity: this is to make screen facing my inside and watchOS knows about this.

------
srikris
You can also consider using creating a model using the Activity Classifier
toolkit in Turi Create which automatically exports in CoreML format for use
in. your watchOS app

[https://github.com/apple/turicreate/tree/master/userguide/ac...](https://github.com/apple/turicreate/tree/master/userguide/activity_classifier)

~~~
hsiaoer
Awesome. Been meaning to play around with Turi Create. Seems pretty easy to
get started with. One of my colleagues wrote a post on building the hotdog/not
hotdog app with it in an afternoon. [https://heartbeat.fritz.ai/building-not-
hotdog-with-turi-cre...](https://heartbeat.fritz.ai/building-not-hotdog-with-
turi-create-and-core-ml-in-an-afternoon-a87fd1967d10)

------
mshenfield
I know it's just for educational use (thanks!), but fun to think through what
it would take to make this into a fully functional app.

Exercises that don't involve arm movement would be impossible to classify. An
extra sensor on the leg might be enough to a) handle cases where just leg
movement is involved and b) capture body position for static exercises like
planks and wall sits.

~~~
coolioxlr
check out Gymatic featured in Apple Watch app store. Exercise classification +
rep counting. I am the developer and we already have a lot of active users
using the solution in Gym.

~~~
mshenfield
I'll take the bait. Great job on the app - seems like it really works. Also
answers most questions about how this would be feasible

* How to track workouts where arms don't move? Add a second sensor, or create artificial arm movement.

 _\- Want to track Leg Press? Strap phone to your leg. Or keep hands on your
leg to move along._

* How to distinguish between similar workouts/add custom workouts? Custom classification per user means a smaller pool of workouts to distinguish.

 _\- It takes just 3 seconds to train any new exercise_

------
aphextron
I’ve just bought a new 3 series and it really bothers me how neuterd most of
the apps feel. I have what’s equivalent to a desktop PC from 2005 on my wrist
yet there is no web browser, decent games, or anything else actually pushing
the hardware. The built in email client wont even render html or images.
Google has dropped all app support but News. I’m not sure why people aren’t
doing more with this platform, as the hardware specs are amazing.

~~~
coldtea
> _I’ve just bought a new 3 series and it really bothers me how neuterd most
> of the apps feel. I have what’s equivalent to a desktop PC from 2005 on my
> wrist yet there is no web browser, decent games, or anything else actually
> pushing the hardware._

Because you'd e.g. be web browsing on a watch if it did have a web browser?

Besides, even with the "neutered apps" and only cursory use of them, the
battery barely lasts a full day. Imagine with what you're suggesting...

~~~
djrogers
With a series 3, even the cellular version, the battery will easily last 2
days for most people. Even with heavy use I usually have over 50% battery left
at the end of the day...

~~~
gaelenh
Curious: I just got a series 3 last week and use it as my sole phone (hate
smartphone form factor, flip phone user since forever). All I use it for is
heart rate, phone, messaging, and music (very happy with it).

I've had to charge my series 3 every night. I guess not having it paired with
a phone somehow uses more battery power? Technically, I reset my wife's phone
with my sim+apple id, paired, reset back to wife's backup and sim. So it has
no real phone to pair with, though I guess maybe that causes it to use
cellular a lot more than most.

------
drudru11
Any insight into battery life when collecting data?

~~~
hsiaoer
I didn't get a chance to profile the app for battery consumption since my
recordings were only 5-10s each but I'd assume that while the watch is
recording the battery life would be significantly less if you had it on for an
extended period of time.

~~~
drudru11
Thanks

