On: Step 5 Preview, a 1M-context MoE from StepFun, shows up on OpenRouter

[Iolaum]

Qwen-3.8-flash-next @Q4

[UncleOxidant → Iolaum]

Yes, this on Halogen. I can get ~45tok/sec which is quite usable.

[throwa356262 → Iolaum]

Not sure why this is downvoted.

It is fully possible to run the 180B Owen model on strix halo using strata and even get reasonable speed out of it.

https://github.com/Niko1221/Strata

[Iolaum → throwa356262]

Indeed - Even with llama.cpp you can store n-grams to disk and use it. I m using unsloth's llama fork atm but a new llama release is out that may be better - hadn't had time to test it yet.