I saw Musk saying a couple of days ago that we've "hit the limit of peak data" for training AI. My immediate reaction was no, surely you have not trained on every copyrighted textbook on every subject ever written. You hit the peak of easily accessible internet data that you could quickly steal to train your models.