On: Speculative Decoding in vLLM on AMD GPUs

[nicce]

You also want prompt/prefix cache. Otherwise there is a lot of duplicate prefill processing if you fork conversation, have a different chat window or anything like that. It therefore also makes subagents much faster.

[latentsea → nicce]

I'm getting 99% cache hit in DeepSeek Harness?