On: Degraded performance for multiple models

[brcmthrowaway]

Aren't the model weights frozen?

[mceachen → brcmthrowaway]

Model competence is an interaction of weights, system prompt, and harness.

[actsasbuffoon → mceachen]

Don’t forget reasoning effort. We get labels like “low,” “high,” and “max.” That doesn’t mean that the numbers associated with those don’t get remapped on the backend.

[Evidlo → brcmthrowaway]

I think there are other knobs that can be turned without retraining.