On: Qwen3.8-27B on the iPhone 18 Neural Engine at 15–70 tok/s (1.5 bpw)

[simonw]

My current favorite is this one: https://apps.apple.com/us/app/locally-ai-by-lm-studio/id6741...

It's free, not annoying, and has a bunch of model options.

[layer8 → simonw]

Non-app-store link: https://locallyai.app/

However this uses the GPU, not the Neural Engine like the submission.

[jdgoesmarching → simonw]

Note that the model discussed in this article isn’t available in this app running on the phone. Still, it has some good local options like Gemma 4 and can link to the desktop LM Studio app which is cool.

[analog_daddy → simonw]

Wow! This app is pretty cool!