Some more pressing questions are:
* What new capabilities emerge as the models get better and better at predicting (i.e. loss goes down)?
* How much will it cost to train increasingly large models? And to run inference on them?
* How difficult will it be to find or generate more and more high quality data?
what are they then? I thought everyone was firmly in the "let's train with more data and see what happens" camp
https://arxiv.org/abs/2001.08361
How loss translates into higher-level capabilities is anyone's guess.
Some more pressing questions are:
* What new capabilities emerge as the models get better and better at predicting (i.e. loss goes down)?
* How much will it cost to train increasingly large models? And to run inference on them?
* How difficult will it be to find or generate more and more high quality data?