There isn't a curve at all. It's discrete steps. Which is why so many people freak out when there's a step change, because they extrapolate the existence of a curve where there isn't one.
Prompt: is the curve of improvement in LLMs linear or more exponential?
ChatGPT: The improvement in language models has generally followed an exponential trend. With the advent of deep learning and the increasing availability of large amounts of data, the performance of language models has improved dramatically in recent years, leading to breakthroughs in various NLP tasks. However, it is important to note that the improvement curve of language models is influenced by multiple factors, including the amount of training data, the size and complexity of the model, and the computational resources available. The improvement curve of language models may also become more linear as the state of the art approaches certain limits.
> The improvement curve of language models may also become more linear as the state of the art approaches certain limits.
This part of ChatGPT's response is the crux of my original question.
It might be notable that I was able to focus and possibly learn from the response to my question via interaction with an LLM, and not via interaction with a human.
The question in my mind is: is the curve just linear or is it getting closer to exponential at this point?