On: Mistral 7B

The point is that iteration is before training. You don't iterate post starting the training and presumably this person doesn't want to train the model themselves. If they do, they can do that but they need to get the training data and a huge amount of GPUs.

Once again, yes, the training cycle is more like a compile step.

Design iteration of the shape of the model absolutely occurs with partially trained weights however.