1) There is not enough data in the entire world to train a trillion parameters large language model. And there is limited use for synthetic data, yet (we need an internal adversarial/evolutionary model like AlphaZero for that, which I think will be developed soon, but we are not there yet).
2) Models are not linearly scalable in terms of their performance. There are 8B parameter models (Zephyr-Mistral) routinely outperforming 180B parameter models (Falcon). Skill issue, like they say.
3) Even if you are an evil genius, you need money and computing power. Contrarily to what you might be reading in the media, there isn't much capital available to evil geniuses on a short notice (longer term arrangements might still be possible). And even if you have money, you can't just buy a truckload of H100s. Their sales are already highly regulated.
2) Models are not linearly scalable in terms of their performance. There are 8B parameter models (Zephyr-Mistral) routinely outperforming 180B parameter models (Falcon). Skill issue, like they say.
3) Even if you are an evil genius, you need money and computing power. Contrarily to what you might be reading in the media, there isn't much capital available to evil geniuses on a short notice (longer term arrangements might still be possible). And even if you have money, you can't just buy a truckload of H100s. Their sales are already highly regulated.