On: Bonsai 2 27B: Near-Lossless Compression in a 9x Smaller Footprint

[SXX]

Sadly crashing on Pixel 9 Pro, but I guess phone GPU with 16GB RAM total wouldnt be enough anyway.

[14u2c → SXX]

Runs on my 16GB M2 Air (firefox). ~7 tok/s

[jeroenhd → SXX]

3/16 GiB of RAM on the P9Pro is dedicated towards on-device AI models, which WebGPU probably can't access.

[throwa356262 → SXX]

To be fair, the GPU in pixel 9 and 10 is pretty bad.

I really hope they stop using PowerVR in pixel 11.

[trvz → SXX]

It should be though.

[SXX → trvz]

Even native AI gallery uses smaller Gemma models. I guess whatever Chrome is using for WebGPU compute on Android is just adding too much overhead.