bhamm-lab's comments

On: Self-Hosted LLM Upgrade on AMD: Kimi Linear 48B, Qwen3 Coder Next, and Q2_K_XL

[bhamm-lab]

Wrote up some notes from upgrading my local llama.cpp setup on AMD Strix Halo hardware and testing a batch of newer open models.

Main findings:

- Kimi Linear 48B works well as a generalist on my hardware (fast, consistent).

- Qwen3 Coder Next is my new default for coding tasks.

- Aggressive quants (Q2_K_XL) on massive models (200B+) can still be useful for long-running/background tasks, even if they aren't interactive.

Happy to answer questions about the dual AI Max+ 395 setup or how I run models in kubernetes.