On: GPT-4 is getting worse over time, not better

[imtringued]

Continuous training is the holy grail. Yes you can just retrain with the updated dataset but guess what, you will run into catastrophic forgetting or end up in a different local minima.