This is a great read, although it suffers from the shortsightedness common to people who seem to lack the minimum mathematical maturity necessary for understanding basic concepts of modern AI, such as how and why it's possible for a hidden vector's state to represent an abstract concept, like, say, an emotion or a belief.
Let's call the inability to understand basic concepts of modern AI "The Poets' Delusion."
In my experience, people suffering from The Poets' Delusion don't understand the scaling hypothesis or its implications.
They don't understand that as we have increased the number of parameters in AI models like LLMs, we're finding these AI models are learning to represent and act on increasingly abstract, subtle concepts, and no one yet knows the limits to such improvements in learning resulting from scaling up model size.
Let's call the inability to understand basic concepts of modern AI "The Poets' Delusion."
In my experience, people suffering from The Poets' Delusion don't understand the scaling hypothesis or its implications.
They don't understand that as we have increased the number of parameters in AI models like LLMs, we're finding these AI models are learning to represent and act on increasingly abstract, subtle concepts, and no one yet knows the limits to such improvements in learning resulting from scaling up model size.