I remember when pose detection was announced, showing an app that corrected your workout movements.
i have yet to see an app that actually does that. i'd love to have the equivalent of a personal trainer showing me where i need to adjust my pose in say pushups or other simple excercies.
thus im equally sceptical of seeing these apis used. it seems developers are mostly porting web apps to all platforms ignoring neat but platform specific apis like this.
please prove me wrong and link some awesome apps that use pose detection.
There are multiple apps in the App Store that do this. I spent last year implementing pose detection in an exercise app and we used both Apple’s pose detection and a 3rd party’s. The pose (each point of the human form) itself was sent to a machine learning backend at around 30 fps, analyzed, and data returned at about the same speed using gRPC. Each exercise had a set of specific feedback for both positioning (“Stand facing the camera with you arms at your side/Stand sideways to the camera/etc”) and form correction (“Raise your right arm higher above your head etc”). Feedback was spoken out loud to the user and there was a relatively complex set of rules governing which feedback got priority and how often feedback was spoken. I also implemented an on-screen “skeleton” of the user’s human form points that rendered on top of the camera view. Pretty fun project from a tech point of view.
The signal to noise in fitness apps is high. The mainstream ones don’t do this, or if they do the implementation is so bad it’s not worth using, and discovery of anything else is fraught with shitware that wants a subscription to “unlock” it’s unknown potential.
Did you mean to say signal-to-noise [ratio] is _low_? Meaning that you get way too much noise for the amount of signal. Or did you mean to say it needs to be high (I.e. low noise) to be useful?
For every good fitness app that does what it promises (-> signal) there are at least 50 bad fitness apps that promise too much and let you pre-pay for the (broken) features you wanted, money you'll never get back (-> noise).
The amount of noise in Fitness apps is so high, nobody really dares to try out small apps. Therefore cool implementations from small devs like the workout-correction might stay unnoticed for years.
The “there’s an app for that” world where small to medium sized teams can build a very iOS native app that takes advantage of the latest and greatest of the device is long gone.
It’s kinda weird how Apple doesn’t realize this and continues to build for that world. Maybe if they were willing to shift on their % for devs that do build that way but unless they did there just isn’t the audience buying apps outright and the only ways to profit are tricking people into abusive subscriptions or building on ads and their personal data.
Until then no idea why any dev would build just for the Apple ecosystem and not something agnostic.
It’s telling to me that the biggest tech apps of the last 2 years all ran web/desktop first.
Watson was an independent search application on Mac OS, until Apple basically photocopied it and named theirs Sherlock. Since then it's become a verb for when Apple takes your app and builds it into the OS.
Another blatant example was Dashboard, which copied Konfabulator, Night Shift is a copy of F.lux, etc.
When they demonstrated on-desktop widgets the other day, I thought to myself “there goes Konfabulator/Dashboard 2.0”, and then I thought to myself “you’re old enough to remember that and to be caustically cynical”.
Ah you're right, it was Sherlock 3 that copied the Watson features. Mostly related to searching the web for things like ebay listings, recipes, stocks, software.
So the Watson name was probably inspired by Apple's from previous versions, but the Sherlock 3 feature set definitely got cloned from Karelia's.
> Until then no idea why any dev would build just for the Apple ecosystem and not something agnostic.
The standard reason given is that iPhone users are much more valuable than Android users, in that they're a lot more likely to pay for things. If I'm creating a workout app with a fancy form-correction feature then I might well want to use Apple-platform things that make it quicker to develop, at the cost to me of only slightly restricting my actual market.
It's not just the machine learning stuff, they have a non-portable approach for everything, including the platform's primary programming language. They still seem to live in a world where a significant niche of developers targets Apple platforms and their bespoke APIs only.
The problem with that world view is that (a) everything with a network effect can't target a single platform anymore, and (b) the business model for old-school professional single-user apps was killed by the App Store.
They are relying on people looking into spending statistics by platform and realizing that if they want those sweet sweet $$$, they are forced to deal with apple and their walled garden.
I think they seem to be doing fairly well as a company, and part of that is not letting themselves be tethered by a standard to allow competitors equal access to their walled garden. Whether you like that or not, it’s the strategy they’ve taken. They would rather not have your app than distort their platform to accommodate its ability to run on another platform.
For developers the reason to adopt the apple ecosystem is fairly simple. People willing to pay for an apple device are likely willing to pay for a subscription. The apple model is essentially you buy a subscription to their hardware - they release at a regular clip, they anticipate most customers will refresh, there’s no meaningful upgrade path, etc. As a developer I prefer subscriptions over one time purchases because it incentives my maintenance and growth of features for existing customers rather than a never ending grab for new customers. As a consumer while my pocket book certainly prefer one time pay, I actually do see the benefit in incentivizing continuous improvements for existing customers. (I do however wish that apple didn’t hide the subscriptions management so deeply and made it very prominent, and until they do it falls into the abusive category IMO)
> It’s kinda weird how Apple doesn’t realize this and continues to build for that world.
If you’re a hardware manufacturer, I don’t think building for the common denominator of the web browser is a viable strategy. Looking at various of their competitors, it certainly brings in less money.
How many people would buy an iPhone that’s basically a “browser device” if, for 50% of its price, they could get something that’s 80% as good (percentages for illustration purposes)?
> It’s telling to me that the biggest tech apps of the last 2 years all ran web/desktop first.
What are the two apps that you are referring to? No snark just curious. Because the only thing I can think of are video calls or social media (which are arguably older than two years).
There a home gym setup called Tempo that claims to do this. I have the scaled down mobile app version so idk if the full setup is more informative, but it doesn't really give a lot of feedback at all. It basically just tracks the movement of the dumbbell plates that they send you to count your reps, and if you aren't moving the weight to the correct position your rep won't count. It definitely doesn't do anything like correct your form based on your body position like "hey straighten your back" or anything
I switched from the Move to the Studio (freestanding version) and the form feedback triggers __slightly__ more often but it's still not worth bothering about imo. the rep counting not working bothers me way more.
Hey, I'm the founder of Guru (https://getguru.ai/), a video AI dev platform. Developers are using our movement APIs to build some cool form feedback apps, including NFL coaches.
One of the reasons that I write native, is so that I can access stuff like this. It usually takes quite a while for hybrid platforms to catch up.
That said, I don’t have an immediate need for this particular SDK, in the project I’m developing. I just like to have the option to integrate stuff like this.
Also, I’m not a “bleeding edge” developer. I’m still using UIKit/AppKit/WatchKit (as opposed to SwiftUI), and my software supports one OS version back, upon release.
I'm part owner of a company (Halterix) that used pose detection (alternately via smartwatch accelerometer) and machine learning to quantify how well exercises are performed.
We built a demo app for use in physiotherapy to improve outcomes and ran a few clinical studies. The detection accuracy was excellent and patient reception was warm.
There are a number of competitors, some with multi-sensor systems targetted to pros, some with vision systems, etc.
We met with all the big fitness app makers and found generally while they weree somewhat interested in pose detection/accuracy assessment and feedback, it's not at the top of their list of priorities to implement (even to incorporate our 3rd party service).
For a more immediate and quite practical solution, I've had good results from simply taking videos of various movements and watching them right afterwards.
This comment seems to be unrelated to the subject. Is there a specific reason you mention it (along with the down vote I take is also from you, given your passive aggressive tone)?
The latent point is that most applications don't need specific APIs for their value propositions, why it would not make sense to write them in a native framework that enables these APIs.
> This comment seems to be unrelated to the subject.
Isn't it obvious? It's tech companies and mobile ecosystem. Everything that can be used for ads and surveillance, will be used for ads and surveillance.
Pose detection can provide a lot of insights about the user's overall health. All the unscrupulous players now need is some bullshit reason to convince the user to a) install their app, and b) use the feature. Like, idk, high-fidelity dancing AR avatars to use as stickers on social media (a real thing, btw.).
Apple may or may not make this hard, but it's a real thing to be concerned about. Arguably, it's the most obvious use case, given the state of this industry.
> Is there a specific reason you mention it (along with the down vote I take is also from you, given your passive aggressive tone)?
Yes, the reason is I considered it funny. Sorry if that didn't work.
Also, I didn't downvote your comment, but I just upvoted it to compensate for someone who downvoted you, seemingly without a good reason – at least I agree with it.
thus im equally sceptical of seeing these apis used. it seems developers are mostly porting web apps to all platforms ignoring neat but platform specific apis like this.
please prove me wrong and link some awesome apps that use pose detection.