Worth noting as well that the weights come with an MTP layer that seems particularly accurate, to the point it can give you up to 6-8 correctly predicted tokens consistently enough to be useful. Which obviously boosts its speed enormously.
I find it difficult to believe how good this model is. It feels like it's lagging heavyweight frontier models by less than a year, and it runs on your PC.
Worth noting as well that the weights come with an MTP layer that seems particularly accurate, to the point it can give you up to 6-8 correctly predicted tokens consistently enough to be useful. Which obviously boosts its speed enormously.
I find it difficult to believe how good this model is. It feels like it's lagging heavyweight frontier models by less than a year, and it runs on your PC.