>> Gradient descent is a search in model space for one that best models "meaning" relations in the data, among other things.
Not at all. Gradient optimisation "searches" for a set of parameter values that optimise (maximise or minimise) the gradient of a function. In machine learning, the function is some measure of error over a set of training data, that is to be minimised. There isn't anything in gradient optimisation that has to do with "meaning" at all.
What I think you're saying is that by training on a lot of text, a language model somehow learns to minimise its error on the meaning of the text. If you squint a bit, you'll see there's an obvious gap, there, between "text" and "meaning", and it seems to me you're trying to close that gap by alluding to how humans close it. But that's just comparing apples and oranges. As Yann LeCun pointed out recently (there was a video of a discussion with Andrew Ng posted on HN a few days ago) for a human to "train" on all the text that large language models have trained on, the human would have to read several books a day for some thousands of years. So whatever humans can do to learn how to bridge the gap between text and meaning, that's not how language models do it, and drawing an analogy between the two obscures more than it elucidates.
>> This is incredibly sophisticated mechanical pattern recognition with many layers.
It's incredibly complex. I don't know what "sophisticated" means. Is a jumbo jet "sophisticated"? I'd say it's a very complex machine, but that doesn't mean it has abilities other than the ones it was manufactured to have. Complexity is not magick, you know. We have maths for it and all. Those are even foundational maths of computer science. If we started imagining "meaning" everytime complexity cropped up in computer systems, we'd be up to our knees in it.
> There isn't anything in gradient optimisation that has to do with "meaning" at all.
That seems like a disingenuous interpretation of what I was saying. It should have been obvious and implied that I was referring to a loss function that was designed and engineered to choose a model that optimally captures meaning relations in the data (among other things). Gradient descent is directed by its loss function and that is not some random arbitrary function. For GPT it was specifically engineered for the purpose of selecting a model that captures meaning etc. These models are also trained with reinforcement learning which is a topic I don't know much about yet.
> it seems to me you're trying to close that gap by alluding to how humans close it.
I am talking about deep neural networks, not humans. GPT is a type of deep neural network. We do know something about how they work.
> It's incredibly complex. I don't know what "sophisticated" means.
I don't have time or patience for this pseudo-intellectual nitpicking. Just look it up in a dictionary. Basic English.
"Sophisticated (adjective) - (of a machine, system, or technique) developed to a high degree of complexity. E.g.
"highly sophisticated computer systems""
>> That seems like a disingenuous interpretation of what I was saying.
I hope not. It was not at all obvious to me what you were trying to say. It really doesn't help communication to use metaphors and analogies when discussing technical matters, especially when there is simple and commonly accepted terminology. Trust me, I've learned that the hard way (i.e. repeated rejections of my research articles with very frustrated comments by reviewers).
Besides, what you say is not right. Language models are not trained to optimise "a loss function that was designed and engineered to choose a model that optimally captures meaning relations in the data". Language models of the GPT line, specifically, are trained to maximise the following probability:
P(tᵢ|tᵢ₋ₖ, ..., tᵢ₋₁)
Where tᵢ is some token in the training corpus and tᵢ₋ₖ, ..., tᵢ₋₁ are the tokens in a "context window" of length k. That is actually the pre-training objective. The same probability is maximised in the supervised fine-tuning phase, but where tᵢ is replaced by a label and tᵢ₋ₖ, ..., tᵢ₋₁ is a sequence of tokens representing an instance with the given label.
You can find this information in the first GPT paper, here:
See equations (1) and (4) on page 3 (with more careful notation than mine).
Note that the above probabilities do not have anything to do with "meaning". They represent the ability of the model to, as people say, "predict the next token in a sequence". That is the only thing that GPT language models are designed and engineered to do.
You can search the entire GPT article above and you will not find any mention of "meaning". Nobody knows how to represent meaning and how to train a neural network on examples of it. It's much easier to train a neural net on text, of which there are no end of examples. That's what OpenAI has done.
>> I don't have time or patience for this pseudo-intellectual nitpicking.
Sorry, I meant that "sophisticated" is a loaded term. It's the kind of thing I hear often associated with socialites and gourmets. I find that it has no place in conversations about technology. It's the kind of expression people use when they try too hard to convince of the value of some technological artifact whose value should be clear without such exaggerations. So for example, you can talk about the complexity of large language models without having to say anything about their "sophistication", and that works well because one is measurable while the other is only a matter of opinion.
It's in the tile of the paper. "Improving Language Understanding"
So they call it "understanding". No problem. Understanding natural language requires building a representation of the meaning encoded in language. Otherwise what exactly do you think they are trying to "understand" about natural language here? GPT is not a spell-checker. It has a semantic model of language.
The stated goal is to optimize for "natural language understanding tasks", to use the exact phrasing in the article.
The article refers to "semantic similarity" which is precisely about modelling the meaning of language, by definition of the term.
Here's a definition of "semantic similarity"
"In the context of Natural Language Processing (NLP), "semantic similarity" refers to the measure of how closely related the meanings of two linguistic units, such as words, phrases, sentences, or documents, are to each other. The goal of assessing semantic similarity is to quantify the degree to which the meanings of these linguistic units overlap or share common conceptual features.
Semantic similarity plays a crucial role in various NLP tasks, including information retrieval, text summarization, document clustering, machine translation, and sentiment analysis. By understanding and quantifying the semantic relationships between linguistic units, NLP systems can provide more accurate results and better handle the complexities of natural language."
Yes, the word "meaning" is not explicitly used in the article, but it is implied. Ironically you are nitpicking over semantics that GPT can already comprehend, since it contains a model of the meaning of the word "meaning".
> They represent the ability of the model to, as people say, "predict the next token in a sequence". That is the only thing that GPT language models are designed and engineered to do.
GPT language models are designed and engineered to perform natural language understanding tasks. This is discussed in the paper that you linked to.
In order to predict or generate the next token, it uses a very super sophisticated semantic model of natural language and the previous tokens. You could also argue that a human is just "predicting the next token" when they generate a sequence of words while doing the tasks that GPT is designed to do. That misses a lot of important details. The devil is in those details.
>> It's in the tile of the paper. "Improving Language Understanding"
I have to confess I'm a little bit disappointed. I pointed you to the
OpenAI paper that has the technical details of how their models are trained,
and you chose to concentrate on the title, and the claims made about the
abilities of the model, rather than the way it actually works. The technical
details about the model's training make it clear that it is a language model,
trained to maximise the probability of a sequence of tokens, but you keep
believing that it can "understand"? Or represent meaning? I'm not even sure
what you believe.
The point is that you shouldn't believe. The OpenAI paper is not gospel. It's
a technical report. You should even be able to implement a system like the one
they describe, though at smaller scale. Maybe that will help you understand
how it really works? I don't know.
I think you don't want to know how things work. You just want to believe. And I
can't deal with faith. Faith makes no sense when you have a technical specification.
Sorry.
OK now I see your problem. You currently lack a fundamental understanding of neural networks and machine learning in general, and it seems that you didn't actually read the entire paper.
The loss function is involved in the training of a neural network (transformer).
THE LOSS FUNCTION IS NOT THE MODEL ITSELF.
The loss function is not a neural network. The loss function is not the transformer model. It is only part of the training step.
This is a major blind spot in your current technical understanding of this paper and machine learning in general.
The loss function (in GPT-3) is optimizing a 100 layer neural network with 175 billion parameters, and that is where the semantic relations emerge, in the model itself. The neural network is inferring these relations from the training data. Obviously there is no "meaning" defined in the loss function, and it makes absolutely no sense to look for it there. The structure to be learned is in the training data. Semantics are not explicitly coded into the model. The paper does not give you all of the technical details. The authors assume you have the pre-requisite knowledge of NLP, deep learning, and the transformer architecture.
"Semantic similarity" is not gospel. This is a standard NLP term and it is discussed in the paper, as it is one of the training objectives.
This emergent properties of transformers and other deep learning models is a fascinating thing to witness and to study. You should go back to basics in machine learning to properly understand how the loss function is only directing the training of a model, but is not the model itself, which can be incomprehensibly vast as in the case of GPT-3. You could not fit a full technical description of the actual trained GPT-3 model into any paper. The emergent properties of neural networks is a new field of research and we don't yet have a complete understanding of all of these properties or how to investigate them.
No I haven't taken anything personally. I am trying to explain how this stuff works. Go back and read your recent replies to me then see how condescending and ridiculous you were (accusing me of having blind faith, not wanting to know how things work, etc). I truly don't care. I'm just trying to explain something. Too bad if I have to be so blunt. No offence taken. It's actually more funny than offensive.
I have a degree in Applied Mathematics and Computer Science. But so what? You don't need a degree to understand the basics.
You're looking for an explicit semantic coding inside a loss function, which doesn't make any sense. Of course you won't find any coding for semantics described in the paper, because the entire point of machine learning is that it figures that out automatically.
How the hell can you have a PhD in machine learning without understanding that?
> Nobody knows how to represent meaning and how to train a neural network on examples of it.
We represent meaning with language models, and GPT has been trained on examples of it (semantic similarity). It learns how to represent meaning through the training process.
The meaning relations are an emergent property of the trained neural network, not explicitly programmed. The meaning relations are automatically learned by the machine. This is the magic of machine learning and neural networks. (I don't mean literal magic. These large language models are simply too large and opaque. You can't easily identify the specific neural activation pattern that encodes meaning of any given concept within a 100 layer deep 175 billion parameter neural network.)
"In essence, the "meaning" referred to in LLMs is not the same as the meaning humans typically ascribe to language, which is derived from direct experiences and real-world knowledge. Instead, it is a form of abstract, statistical understanding that emerges from the relationships between words, their contexts, and the patterns in the data. The model essentially learns a high-dimensional representation of these relationships, which can be thought of as an approximation of meaning within the context of the model's training data."
This might help get you on the right track.
Question:
Do neural networks have emergent properties?
Answer (from GPT-4):
Yes, neural networks can exhibit emergent properties. Emergent properties are characteristics or behaviors that arise from the collective interactions of the individual components in a system, which are not directly attributable to any single component. In the case of neural networks, these individual components are the neurons (or nodes) and their connections (or weights).
As neural networks learn from data through a training process, they adjust the weights between neurons, which can lead to complex patterns of activation across the network. The network's ability to recognize patterns, make predictions, and perform tasks can be seen as an emergent property that arises from the collective behavior of its interconnected neurons.
For example, in the case of language models like GPT-3, the ability to generate coherent and contextually relevant text can be considered an emergent property. This capability is not explicitly programmed into the model but arises from the learned relationships between words and phrases through the training process.
It's important to note that while neural networks can exhibit emergent properties, these properties might not always be easily explainable or interpretable, as they result from the complex interactions within the network.
Not at all. Gradient optimisation "searches" for a set of parameter values that optimise (maximise or minimise) the gradient of a function. In machine learning, the function is some measure of error over a set of training data, that is to be minimised. There isn't anything in gradient optimisation that has to do with "meaning" at all.
What I think you're saying is that by training on a lot of text, a language model somehow learns to minimise its error on the meaning of the text. If you squint a bit, you'll see there's an obvious gap, there, between "text" and "meaning", and it seems to me you're trying to close that gap by alluding to how humans close it. But that's just comparing apples and oranges. As Yann LeCun pointed out recently (there was a video of a discussion with Andrew Ng posted on HN a few days ago) for a human to "train" on all the text that large language models have trained on, the human would have to read several books a day for some thousands of years. So whatever humans can do to learn how to bridge the gap between text and meaning, that's not how language models do it, and drawing an analogy between the two obscures more than it elucidates.
>> This is incredibly sophisticated mechanical pattern recognition with many layers.
It's incredibly complex. I don't know what "sophisticated" means. Is a jumbo jet "sophisticated"? I'd say it's a very complex machine, but that doesn't mean it has abilities other than the ones it was manufactured to have. Complexity is not magick, you know. We have maths for it and all. Those are even foundational maths of computer science. If we started imagining "meaning" everytime complexity cropped up in computer systems, we'd be up to our knees in it.