On: Apple's On-Device and Server Foundation Models

[jjtheblunt]

How would you know none of the apple apps use the neural engine? Is the key in the statement “LLM”?

[woadwarrior01 → jjtheblunt]

Yes, I specifically meant autoregressive LLMs. BERT style encoder only models, ViTs and CNNs ran perfectly fine. Yesterday's coremltools update[1] changes that.

[1]: https://github.com/apple/coremltools/pull/2232