On: Gemini Distillation Service

[nerdsniper]

Fine-tuning is done against a dataset. Distillation is done against a model.

[verdverm → nerdsniper]

Distillation is used to build part of a data set for fine-tuning (loosely interpreted). Advanced model traces are useless if you don't have a base model that is good enough to be improved by them.