Text reproduced, minus abusive shell of dark patterns:
Some people have completely unrealistic expectations about what large-scale language models such as GPT-3 can do.
This simple explanatory study by my friends at Nabla debunks some of those expectations for people who think massive language models can be used in healthcare.
GPT-3 is a language model, which means that you feed it a text and ask it to predict the continuation of the text, one word at a time. GPT-3 doesn't have any knowledge of how the world actually works. It only appears to have some level of background knowledge, to the extent that this knowledge is present in the statistics of text. But this knowledge is very shallow and disconnected from the underlying reality.
As a question-answering system, GPT-3 is not very good. Other approaches that are explicitly built to represent massive amount of knowledge in "neural" associative memories are better at it.
As a dialog system, it's not very good either. Again, other approaches that are explicitly trained to perform to interact with people are better at it.
It's entertaining, and perhaps mildly useful as a creative help. But trying to build intelligent machines by scaling up language models is like a high-altitude airplanes to go to the moon. You might beat altitude records, but going to the moon will require a completely different approach.
It's quite possible that some of the current approaches could be the basis of a good QA system for medical applicatioms. The system could be trained on the entire medical literature and answer questions from physicians.
But compiling massive amounts of operational knowledge from text is still very much a research topic.
> GPT-3 doesn't have any knowledge of how the world actually works.
I think this is a philosophical question. There is a view that, basically, there is no such thing as knowledge, just language (or, at least, there is no distinction between knowledge and language). In this view, all there really is is language, which is mostly composed of metaphors and, ultimately, metaphors only refer to other metaphors, i.e. language is circular. In this view, not only is the ultimate, physical, concrete world beyond us but also we can't even talk about it. From this perspective, GPT-3 is not substantively different than what our minds are doing.
That view makes some strong claims (I don't find it convincing), but it's out there. A slightly different claim, though, is that "knowledge of how (we think) the world actually works" is encoded in language. To me, that seems trivially true. So, again, how you take this quote from LeCun depends on what you think knowledge is and your view of the relationship between knowledge and language.
Tell that to all the other animals on Earth. Do they not also have knowledge? Do you really think they encode their knowledge in language?
Do you really think that humans are so special as to encode all their knowledge in language? Watch a movie. Listen to a song. Examine a piece of art. Feel sculpture. Play a guitar. Dance.
There is a segment of the software community that is highly language centric/adept. But that community is often blind to other forms of understanding.
Just look at the language of Shakespeare. Much of the language is visual and experiential. How much would you actually understand without your senses and imagination? Your knowledge encompasses your being.
>Tell that to all the other animals on Earth. Do they not also have knowledge? Do you really think they encode their knowledge in language?
Well, actually, yes, they do. Many animals have elaborate languages encompassing many concepts. Crows can explain to one another what a human looks like, for example.
If GPT-3 has a consistent position on anything, it's only because the corpus it was trained on was consistent about it. So, for example, it will reliably autocomplete Jabberwocky because there are a lot of copies of this poem in the corpus and they are all the same.
If there were two versions of this poem that started the same way, it would pick between the variations in the corpus randomly. In other cases it might choose based on the style of prose or other stuff like that.
GPT-3 can get some trivia right, but it's only because the editors of Wikipedia already came to consensus about it and Wikipedia was weighted more. It doesn't have a way of coming to a consistent conclusion on its own.
Without consistency, how can it be said to know or believe anything? You might as well ask what a library believes. Sure, the authors may have believed things, but it depends which book you happen to pick up.
I agree with you in that I would make a strong distinction between what a model like GPT-3 does and whatever it is that humans do.
But I do think you're missing the point just a bit. When we speak and think, we use all kinds of metaphors that express judgements about the world, usually without realizing it. In other words, the way we use language encodes concepts in a deep way.
To borrow an example from George Lakoff, we, in English, use war-metaphors to talk about arguments. Of arguments and of wars you can say things like "he's marshalling his forces," "they're ceding their territory," or "she's girding her defenses". In fact, almost anything you can say about a war you can also say about an argument. In American politics, with regard to partisan squabbling and the filibuster, we talk about "the nuclear option". The fact that these metaphors make sense to us indicates a judgement, something like "arguments are like wars". That judgement shows up in billions of lines of English scraped from the internet and can be fed into a model, allowing GPT-3 to "make that connection" via purely statistical methods.
Yes, this is a bit like asking "what a library believes". But a lot of these metaphors show up in our languages and, in a way, they express judgements, which is something akin to a belief. Does that mean a library has beliefs? Is this all knowledge is? I wouldn't go that far. But the argument is an interesting one and worth raising.
Well, it's certainly interesting that it can learn metaphors, and this can be useful for creative purposes, so it's fun to play with.
But a sophisticated understanding of metaphors could be used to tell the truth or to lie. In the case of GPT-3, it doesn't know the difference. Telling the truth and lying come out of the same autocompletion process.
If you consider the use of a metaphor to be showing judgement, it means that a particular metaphor seems to be appropriate to use in a particular context.
At the risk of reigniting the perpetual war about how to characterize machine intelligence, and by extension how to characterize the risk they pose, Yann has been (and still is AFAIK) more in the "existential AI risk is a long-term problem" group. In a 2016 interview LeCun said [1]:
> We’re very far from having machines that can learn the most basic things about the world in the way humans and animals can do. Like, yes, in particular areas machines have superhuman performance, but in terms of general intelligence we’re not even close to a rat. This makes a lot of questions people are asking themselves premature. . That’s not to say we shouldn’t think about them, but there’s no danger in the immediate or even medium term. There are real dangers in the department of AI, real risks, but they’re not Terminator scenarios.
That's pretty measured overall, but he doesn't know that there's no existential AI risk in the medium term. No one does, and that's the problem. Experts simply suspect that it's unlikely. Stuart Russell and him have debated similar topics [2].
To tie back to your point: I keep seeing LeCun brush over tricky questions like yours and the ones at [2] with an arrogant confidence. I wish that he would be more careful, and I hope that I have a skewed view of him.
He's not wrong, we're very far. And looking at past "progress" it seems that we'll get there very slowly. So it seems long-term.
Except people are bad at exponential processes. Yet when economics drives us we are suddenly good at making them happen. And this combo seems to be what makes these existential risks. (Like climate change, or other manifestations of the coordination problem.)
I think an important distinction to make is your use of the word "language", and how we think of language as it concerns human minds, and as it concerns GPT-3.
In our heads, language is a combination of words and concepts, and knowledge can be encoded by making connections between concepts, not simply words. If there is no concept or idea backing up the words, it can hardly be called knowledge. Consider the case of the man who did not speak French, yet memorised a French dictionary, and subsequently went on to win a Scrabble competition. Just because he knows the words, would you say he knows the language?
A language model such as GPT-3 operates only on words, not concepts. It can make connections between words on the basis of statistical correlations, but has no capacity for encoding concepts, and therefore cannot "know" anything.
> In our heads, language is a combination of words and concepts, and knowledge can be encoded by making connections between concepts, not simply words. If there is no concept or idea backing up the words, it can hardly be called knowledge.
Great point.
> A language model such as GPT-3 operates only on words, not concepts. It can make connections between words on the basis of statistical correlations, but has no capacity for encoding concepts, and therefore cannot "know" anything.
Are you sure? Aren't "concepts" encoded in how language is used, at least to some degree?
LeCun does say that models that explicitly attempt represent knowledge perform better than GPT-3 in terms of answering questions. I'm no expert but I believe him.
>Aren’t “concepts” encoded in how language is used, at least to some degree?
Good point and I think this shows up to the extent different languages might affect how we express particular concepts.
However I think it is more accurate to say that language solidifies and gives form to how we express concepts and the “concepts” themselves are independent of languages. Only our “expression” of these “concepts” depends on language.
For anyone interested in art and art history, this distinction was the central focus of the French surrealist painter Rene Magritte.
Language is how we store our knowledge, and language is a system of words. If a language model contains all the possible sentences you can say, it will complete any of your sentences, don't you think it knows what you know? The input is sequence of characters, so you can say it may or may not operate on words. It can operate on subwords, words or phrases where it see fit.
I like to think intelligence as clouds. If you dig deep down, they are just droplets, there are so many of them, they can appear to be so many different shapes. And they look complete different. Maybe intelligence is the same.
Personally I do not find the whole "language = knowledge" argument convincing. But if you're interested in reading writers who make that argument (and perhaps I'm vulgarizing the argument a bit), Nietzsche makes it in On Truth and Falsity in their Extra-Moral Sense and George Lakoff makes it in Metaphors We Live By.
I'd also suggest Wittgenstein's
Tractatus Logico-Philosophicus, a seminal work of the logical positivist movement. Influenced by Frege's predicate calculus, the aim of the Tractatus was to determine an isomorphic relationship between language, thought, and external states of affairs. An axiomatic attempt to reveal a potentially ideal logical language, that is not interested in meaning per se, but merely an accurate reflection of the world. A closed system that essentially excludes non-falsifiable metaphysical question. Famously concluding with the instruction: "Whereof one cannot speak, thereof one must be silent." Part of Wittgenstein's project, even in its early aggressively logical form, was philosophy as a therapeutic. That is, the metaphysical questions concerning god, being, essence, and forms that had inspired thousands of years worth of fevered conversation, could be finally be quieted. That's not to say they couldn't be meditated on, but were not in the domain of his logical language, and so silence. Again, I think early Wittgenstein sometimes gets misinterpreted, "...therefore one cannot speak" does not, to me, mean that it can't be considered or one must forgo spirituality, just that it couldn't be spoken of within the project of the Tractatus.
Logical empiricism was ultimately a dead end as the criteria for even verifying empirical truth has long been contentious philosophically, and was further critiqued by contemporaries such as Quine who attacked the premise of the analytic/synthetic distinction (think Hume's fork, which Kant tried to solve) and Popper who cited the problem of induction to critique the fundamental premises of the positivists verificationism.
Wittgenstein is an interesting case, as the Tractatus is considered an early work of his, profoundly influential to analytic philosphy at the time, yet his later work, Philosophical Investigations is sometimes seen to retract the dogmatism found in the Tractatus. I tend to take the view that it's a continuation of his thought, rather than a retraction of his earlier work. Crudely, whereas his former thought represented a narrowly axiomatic definition of language and its truth value, PI investigates, among many other ideas, language as an activity, or game, that has meaning dependent on the context of its use, languages as families. Granted, Wittgenstein is a complex thinker and these are simply my interpretations.
It's also curious to note that as positivism was beginning to fall out of favor around the time of the second world war, a continental thinker such as Heidegger, whose thought luxuriated in the kind of metaphysical questions the positivists necessarily eschewed, rose to prominence and was infamously sanctioned by the NSDAP to philosophize about their presumed "destiny". Bit of a tangent, but I think the historical context is relevant, as often philosophical movements are birthed from pre- and post-war attitudes.
Are animals as intelligent as human? What language does to us is to be able to pass on knowledge to generations, knowledge can be accumulated. Animals may be able to pass on simple concept to next generation, they won't be able to accumulate knowledge without language and writing.
As a response to leftyted, erispoe's point neither depends on nor implies that animals are as intelligent as humans, it is simply bringing up a counter-example: what appears to be knowledge in animals lacking language.
Of course, you could always attempt to define knowledge such that it is purely verbal, or alternatively define whatever is going on in the brain of an animal to be language, but is either approach useful? In common usage, we recognise, as knowledge, various things that cannot be communicated by language, such as knowing how to ride a unicycle on a tightrope (I doubt you can learn it just from a book) and the infamous qualia which supposedly prove that the mind is dualistic. And what about the knowledge of how to use language? How does that get bootstrapped?
The knowledge of riding a unicycle on a tightrope does not give you the ability to ride a unicycle on a tightrope. Yes, you have to learn it through experiencing it, because you need to map it to your motors. Animal has instincts, they are able to trace water, for example, without teaching. It is also knowledge, but for our discussion, knowledge is what we obtained after birth, not something encoded in our DNA. This knowledge is stored in the format of language. Apparently some believes that that language is part of our DNA. That's why you can have Tazan, but not an animal can speak human language even if a human raise it since it was born because language cannot be learnt without support in code in DNA.
If we define knowledge that way, leftyted's original claim (knowledge is language) becomes true by definition.
This can lead to some confusion. In Frank Jackson's Knowledge Argument[1], dualists say that qualia are knowledge, knowledge is language, and so if Mary could not learn qualia by studying science, then materialism is false. This argument trades on being inconsistent over what it means to have knowledge of something.
In important ways yes, and in important ways no. Yes: can deal with dynamic spatial environments, can act towards a goal (intentionality), some almost certainly can plan how to reache these goals. No: Use of symbols and recursive language.
I don't know if i'd go as far as to agree that "there is no knowledge, only language" .. but I 100% agree one of the key insights from GPT-3 -- why training on language is so effective in the first place -- is that language is tightly coupled to reality
I'm not sure that you can assert that language is tightly coupled to reality, unless you're using the term "reality" to mean something akin to "as one perceives the world" (regardless of whether that perception is correct or not).
Most expressions of language that survived from a few thousand years ago are centered around myths, and while those myths may have contained certain moral or ethical lessons (that were and are subject to interpretation) they certainly weren't tightly coupled to reality in an objective sense.
Training on expressions of language (I separate the concept of language itself from its expression in the form of writing, speaking, etc) certainly has use cases but can GPT-3 recognize a previously unknown analogy and correlate it with the proper piece of applicable "knowledge" it has? If not then it really has no understanding.
2. yes, language is bottlenecked by human perception, as are all things
3. even the notion of myth and fiction is encoded in language. language is self-descriptive and self-aware and you can separate sense from non-sense.
4. i'm not talking about knowledge or understanding, but of addressing the question of why training on language let's GPT-3 make human-like predictions as if it knows about reality? either it's a fluke, or it's because language as a whole is a model that approximates reality.
Why does it make human-like predictions as if it knows about reality? Because it's essentially pattern-matching the consensus of the literature it was trained on. Literature as a whole will tend to settle on a consensus sentiment, albeit one that is probably significantly behind the current consensus sentiment (it takes time to accumulate enough mass to move the weights). If your interactions with GPT-3 fall into the rather sizeable consensus that most people either subscribe to or are familiar with then it will certainly prove a decent mimic of understanding. If, however, you attempt to teach it a novel concept or if you dig into its interactions long enough to test for depth of understanding you run into the gaps and GPT-3 either begins to mimic that bullshit artist everyone knows that claims to know things but is only regurgitating platitudes and buzzwords or it begins to mimic behavior that would make you question whether it was sober and/or sane.
There's little doubt that GPT-3 could hold its own quite well in a bout of polite conversation and/or small talk that features in many social situations but that's more a commentary on the limited area of knowledge and behavior that etiquette expects for interactions in such settings.
It could also be trained on the canon of Shakespeare and behave as a prior work of Shakespeare, but if you left one play out of that canon and then attempted to have GPT-3 generate that missing work it wouldn't... at all. It doesn't approximate how Shakespeare thought, it approximates the literature he produced that you used for training.
None of this is to denigrate the achievement that GPT-3 represents, it is merely to point out that it is unfair to GPT-3 to attempt to hold it to the standards of AGI.
> metaphors only refer to other metaphors, i.e. language is circular.
Except we (humans) have real-life experience that gives meaning to those metaphors.
Humans are very good at visual memory, thanks to millions of years of evolutionary pressure to hunt game and gather for food. Hence why the best memory champions in the world mostly use the concept of "memory palaces." When you think of a car, do you see a car in your head or does a dictionary definition (i.e. purely language knowledge) of a car pop in your mind?
It's not that philosophical. If you visualize the GPT-3 embeddings using an embeddings projector then you can see the knowledge with your own eyes. That bypasses the need for GPT-3 to use language to communicate its knowledge to you. Computers aren't a black box like we've traditionally thought of brains.
If it's only philosophical, then me saying that Hacker News Website itself has 'knowledge' of everything we discuss about is also philosophical. Same can be applied to plain paper books.
How about any web application? A for loop? anything which can generate something for you?
I'm sure his group has done some rigorous research that I can't even understand.
But in my experience, the few-shot learner attribute of GPT-3 makes it insanely useful. We have already found several use cases for it, one of which replaces 2 ML engineers.
Yes, it's not perfect, but it's pretty good at many things, and REALLY easy to use.
And when OpenAI says that your two entirely valid use cases are a safety concern, and denies you api access, what will you do? Better keep those ML engineers handy.
If you think this isn’t a concern, I’ve already seen it happen with my own eyes, rather than hearing about it second hand. They encouraged someone to make a writing tool. That someone then spent roughly six weeks prototyping, iterating, and giving constant feedback. All signals from OpenAI were “Yes, awesome!”
Then one day they simply declined to let them ship. Anything. Anything even resembling “a tool to generate huge quantities of outputs.” Which was, you know, the whole point.
You play, you pay. And I hope you’re ready to pay, because you won’t have your magical genie unless the magical genie’s caretakers believe you are sufficiently worthy.
They were forced to give Microsoft exclusive access, because it was one of the terms of Microsoft's billion-dollar cloud credit investment.
But you can't pay employees with cloud credits, so time will tell whether it was a correct decision. (It probably was. And I exaggerate slightly; the investment included a substantial sum of real dollars too. But most people see that billion dollar investment and think it's all dollars, when in fact it was largely credits.)
Can you go into more details where it's useful? As your comment here goes directly against what's argued in the linked Facebook post.
Also, if you've found a use case where GPT-3 replaces real humans, what did those humans actually spend their time on? Seems like either you're over-hyping GPT-3, or under-hyping humanity
The humans spent their time building a hideously difficult classification model. Out of the box GPT-3 worked better than the result of a year of their work.
That's interesting, GPT-3 can do classification too? Or did I misunderstood and you meant your engineers used classification to build a language model that didn't perform as well as GPT-3 (which is less surprising indeed) ?
GPT-3 can do classification. For example you can give it a prompt like "Hacker News is a website. Excel is a Windows program. Visual Studio is a Windows program. Safari is a Mac program. CPU-Z is", and even GPT2 will complete this with "a Windows program" (with GPT2 you need to try multiple times, discard useless results and average what's most common, but it works and is straight-forward to automate).
Right, but GPT-3 can be used generally. That's the difference. It scales because you don't need to build an entirely new model for each different use case.
You just change the prelude and use it for something new.
It sounds like a big deal. What a tempting idea. And a colleague was mildly annoyed with me for how unimpressed I seemed.
But you have to understand, the use cases you mention are shallow and limited. The heart of GPT, the fine-tuning, is gone. And it looks like even OpenAI gave up on letting users fine-tune, because it means they essentially do build an entirely new, expensive model for each use case.
I wanted to make an HN Simulator, the way that https://www.reddit.com/r/SubSimulatorGPT2/ works. But that's far beyond the capabilities of metalearning (the idea that you describe).
I think the onus is on you to prove that the use cases are shallow and limited. I've seen GPT-3 already being used for diverse and interesting ideas that would not have occurred to me personally.
However, even if they are, the point stands: currently, there are teams of people at companies all over the world tuning models for these shallow and limited use-cases. GPT-3 can replace them all, without OpenAI needing to invest another cent in training for a particular customer's use-case. That is in fact game-changing for the ML/DL world and current applications thereof.
Is it AGI? Obviously not. But the vast majority of ML applications don't need to be.
>However, even if they are, the point stands: currently, there are teams of people at companies all over the world tuning models for these shallow and limited use-cases. GPT-3 can replace them all, without OpenAI needing to invest another cent in training for a particular customer's use-case. That is in fact game-changing for the ML/DL world and current applications thereof.
The counterpoint is that it would be significantly cheaper AND have better performance to fine-tune models to each customer's use case than it is to just run GPT-3 at inference.
For a more extensive rebuttal, I wrote one here. https://news.ycombinator.com/item?id=23346972 Though that was more a rebut of GPT in general as a path to AGI than metalearning in particular for generating memes.
Yes except they were saying the iPhone replaced Nokia's engineers.
GPT-3 is not doing what the ML engineers were doing (building models), GPT-3 is the end goal. The company just decided to outsource the work to OpenAI and pay a monthly fee to them instead of salaries to their ML engineers.
"We have already found several use cases for it, one of which replaces 2 ML engineers." -> Clearly makes it sounds like GPT-3 can do the job their ML engineers were doing.
From a business perspective, this is an irrelevant distinction. The requirement was satisfied in a different way, i.e. the engineers satisfying the requirement were replaced by GPT-3, the tool which satisfies the requirement.
Better IDEs have saved countless hours. Saving hours is equivalent to replacing jobs, unless demand is elastic enough to fill that time. Most of the time we are lucky enough that demand at a given price point is much larger than supply, but this won't last forever.
that's not how it works... as it is not a zero sum (i.e. the work is not bounded/fixed but it increases).
IDEs and higher level tools help engineers become more productive. They can do more, with less. This raises the bar on products, and the demand of customers for them (things are pretier, easier to use, etc..), which in turn creates more domains for software to be used, and more demand for engineers.
I would be interested in hearing more about this, within the bounds of what you can share publicly. Most of the touted GPT-3 use cases I've seen to date have dried up or are still in limbo, so hearing about a real production use would be exciting!
There are a couple of ways to do it. You can give it a prompt that shows examples of the classification and it mimics what it thinks is the correct behavior when you feed it new unclassified input. They also have a search endpoint that lets you do classification by giving it an input along with labels as the searchable documents and using the resulting semantic relevance scores.
In general GPT3 is not SotA on (any?) classification task, did you just not have enough data to fine tune a discriminative transformer model? Inference should be cheaper with a smaller transformer/also less lock-in.
I can't go into too much detail here about why we couldn't do that, but one aspect that we found VERY useful is that GPT-3 could draw on real world knowledge not present in the dataset to enhance the results.
I agree some unrealistic expectations have been created due to people posting cherry picked output.
That said, I've spent a lot of time with it this month and think it will be an extremely useful tool for creative works of all types. It's not to a point where you can just tell it to write a blog post (yet!) but it can generate novel snippets, ideas, and variations that are actually usable. Unskilled creatives should be worried. Skilled creatives should incorporate it into their workflow.
> Some people have completely unrealistic expectations about what large-scale language models such as GPT-3 can do.
Just want to point out that he's saying the people on the upper end of the expectation distribution are wrong, not the people in the middle of it. So if you're takeaway from this is that GPT3 is nothing special, that's probably the wrong message.
His next paragraph claims that Nabla "debunks" the idea that "large language models" can be used in healthcare.
That's not just "some people have unrealistic expectations" it's "this tool, when when more advanced and find tuned, will never be appropriate to use in a very broad class of use cases".
He also says "GPT-3 has no knowledge of how the world works", which is clearly an overstatement meant to clear up hype, but is untrue. For example, GPT-3 knows more trivia than I do.
I'm having trouble wrapping my head around LeCun's thinking regarding the Nabla reference. The Nabla link is just a blog post by three people without any technical details provided at all. How can this possibly "debunk" anything?
no it doesn't, GPT-3 is a very sophisticated parrot. it doesn't know any trivia, it knows how to put the most likely string of characters next to the one it just saw, it doesn't matter what the text represents. That's the difference between you and the model.
It's basically the Chinese room. You can make an analog GPT-3 by asking a question, recording your answer, handing someone who doesn't understand a word of your language the giant box of tapes, and she tries to match them together until she appears to make sense to listeners
AlphaGo doesn't know anything about the game of go. It "just" manipulates symbols, runs instructions on its CPUs and GPUs, illuminates pixels on the screen for a human to see.
It just also "happens" to be the case that if you interpret those pixels as go moves and play those moves against the world champion human go player then eventually that human will hold a press conference announcing to the world that AlphaGo has won the match.
Yes that is entirely true, but the world of Go is very small and unambiguous in terms of goals, rules and so on. A little less than chess, but it's still a game with a well structured world and simple rules. AlphaGo understands as little about Go as Stockfish does about Chess or your TI-83 understands about maths. That they're good at it has almost nothing to do with (generalised) intelligence, and it's no coincidence that the performance of RL say, in games is insanely better than it is when applied to real-world, open-ended problems.
Deep Mind made a 'superhuman' Breakout bot, but if you messed with the panel at the bottom it sucked. It hadn't learned what the panel is or what the game is about, it had just learned to rearrange pixels. You can get away with this in one version of Breakout, but in the real world it's brittle and useless
Ok, fine. It doesn't "know" any trivia. It'll still crush me in a trivia contest:
"Q: Who was president of the United States in 1801?
A: Thomas Jefferson was president of the United States in 1801.
Q: What city is in the northwest corner of Ohio?
A: Toledo is in the northwest corner of Ohio.
Q: What highway do you take to get to North Lake Tahoe from Oakland?
A: Highway 80 is the highway you take to get to North Lake Tahoe from Oakland.
Q: What are Darius's abilities in League of Legends?
A: Darius's abilities in League of Legends are: Apprehend, Decimate, Noxian Guillotine, Crippling Strike, Apprehend [sic], and Hemorrhage.
Q: Who did Capablanca defeat to become world chess champion?
A: Capablanca defeated Lasker to become world chess champion."
Of those 5 questions, I could answer #1 after deploying a mnemonic and some math, and #2 if you gave me multiple choice of the top 5 cities in Ohio, and I would miss the rest.
> which is clearly an overstatement meant to clear up hype, but is untrue
It all depends on your definition of knowledge. Under a certain definition you could say that GPT-3 knows basically nothing.
If someone teaches me to repeat perfectly something very smart in a language I don't know, without explaining to me what that thing is, do I have knowledge about this?
The same argument can be made about those kind of models, the knowledge they have is about the structure of the language and what word is most likely to come next, but they have no way to ground those words in actual relation with the world.
Aka. the Chinese room argument. However, I'm not so sure us people are little more than just pattern matching machines. When I start to talk (or write, as I'm doing now), the words kind of just flow out. I can make the argument, that I understand the "real" world, but do I really?
> I'm not so sure us people are little more than just pattern matching machines
Yes that's what leads to multiple definition of what knowledge really is. Yann LeCun believe we are more than just that, hence why he is saying GPT-3 would have no knowledge.
Yes, I think the focus on "getting to the moon,", to use his analogy, ignores the fact that GPT-3 is an SR-71 in a world of 19th century balloons. It may not get to the moon, but it definitely points the way to lots of useful stuff. There is a lot of boilerplate text in the world that is well-suited to auto-generation by a GPT-3-style model. And currently a lot of people employed to generate that text, at significant aggregate expense.
I hate this concept. I have a lot of “boilerplate” real life interactions but I would never replace them with a robot. If people really think nuts and bolts writing is not a worthwhile enough endeavour for humans to do, then shame. What an immense forfeiture, to never again be surprised by a furniture catalogue.
I didn't need to log in to read the article at the original URL, though I had to close a cookie-wall and another modal prompting me to enjoy Facebook better by signing in.
And I couldn't read the article without logging in to my Facebook account.
Facebook seems to put different restrictions depending on where you live. I'm based in Western Europe and never been able to read anything from Facebook without logging in. Same for Instagram.
I think Facebook makes a guess about whether you have a Facebook account or not (or, whether you are likely to log into it) and throws up the wall accordingly. If you'll just bail, they'll show it to you anyways. If they think they can force you to log in, they will.
I've got a friend who tries to talk me down every time i enthuse about GPT-3 or similar. He seems to think I believe it to me more amazing than it is whereas I struggle to convince him that I think I've got a good handle on it's limitations and I still find it mind-bogglingly amazing.
Something I've learned over time is that sometimes it's OK to let people be curious and amazed. The world and science would be far too boring if we were cynical about _all_ of it _all_ the time.
Haha so I was actually thinking about that myself after submitting. Thanks for calling me out -- I guess my only answer to that would be, the bot (by that I meant GPT-3) is using the wrong conjugation of a verb it learned somewhere.
Yann is a consistently sober voice in this world of AI hype. I find it quite refreshing.
Personally I see little evidence that this "just scale a transformer until sentience" hype-train is going to take us anywhere interesting or particularly useful.
And for the people who claim it is super useful already, can you actually trust its outputs without any manual inspection in a production setting? If not it's probably not as useful as you think it might be.
OpenAI use well-know linguistics tricks relying on conjunctions(joining words) to separate the text from which they plagiarize on clusters from 5-7 words, exactly the capacity of human short-term memory is. They also use the Google search engine for custom queries, with date restriction which helps them to plagiarize from different texts written on a particular topic so they to be sure that when they copy, paste and concatenate clusters of words in new text, all these phrases of 5-7 words should be related to one topic. This creates the illusion of meaningfulness at first glance.
GPT-3 on Progress. “Civilization rose on the exponential curve. We shouldn’t expect progress to follow a straight line.”
Google with date restriction before 1 April 2020: "progress to follow a straight line".
"We shouldn’t expect progress to follow a straight line." and "we expect our progress to follow a straight line"
Do you understand now, how they use conjunctions(joining words) to insert or to delete insignificant words as "shouldn't" and "our" to plagiarize so that they are not caught.
Now make Google Search again with date restriction before 1 April 2020.
"After two days of intense debate"
"the United Methodist Church has agreed to"
"one that is expected to end"
"in the creation of a new denomination"
You will find and sources of GPT-3 text:
After two days of intense debate, the United Methodist Church has agreed to a historic split – one that is expected to end in the creation of a new denomination, one that will be “theologically and socially conservative,” according to The Washington Post. The majority of delegates attending the church’s annual General Conference in May voted to strengthen a ban on the ordination of LGBTQ clergy and to write new rules that will “discipline” clergy who officiate at same-sex weddings. But those who opposed these measures have a new plan: They say they will form a separate denomination by 2020, calling their church the Christian Methodist denomination. The Post notes that the denomination, which claims 12.5 million members, was in the early 20th century the “largest Protestant denomination in the U.S.,” but that it has been shrinking in recent decades. The new split will be the second in the church’s history. The first occurred in 1968, when roughly 10 percent of the denomination left to form the Evangelical United Brethren Church. The Post notes that the proposed split “comes at a critical time for the church, which has been losing members for years,” which has been “pushed toward the brink of a schism over the role of LGBTQ people in the church.” Gay marriage is not the only issue that has divided the church. In 2016, the denomination was split over ordination of transgender clergy, with the North Pacific regional conference voting to ban them from serving as clergy, and the South Pacific regional conference voting to allow them.
"GPT-3 is a language model, which means that you feed it a text and ask it to predict the continuation of the text, one word at a time."
So Yann LeCun is not right about GPT-3 here. They don't try to predict anything - especially "one word at time". They simply plagiarize big clusters from 5-7 words or whole sentence and do cosmetic modifications to them to be not recognized by direct searching with Google. When you plagiarize such big clusters written by humans it is normal output compile result of GPT-3 text to have human like look for readers. And they don't use statistic for nothing else except the plagiarism process itself. It will be strange ML algorithm to use phrases with low statistical value as "in the creation of a new denomination" or "progress to follow a straight line". ;-) When you can't emulate human brain processes you can always simulate human plagiarize traits. Especially when there are so many idiots who believe on closed open source code and don't ask elementary questions as "Why someone project suddenly will start to show such "astonishing" results exactly when he close the code behind it?"
Or how they still owe me a bug bounty reward from 2017 for bug in their login form that permit to reap their whole users phones database - and I as last fool give it to them in official channels without to exploit it. For "tanks" they hack my pc and steal my intellectual property you see above.
Next year 2018 2 weaks before big hack I warned them that I founded "new" bug but this time they first must pay me bug bounty reward for this in 2017, and next I will give them new one. THEY Refuse to pay me AGAIN. The rest is history...
I waited FBI to arest me - they don't want to do i even I never hide my identity. Because they must confess they hack my pc, they owe me a money AND THEY DON'T CARE ABOUTSECURITY OF THEIR USERS. I very like how I faced them :---
Facebook - emilenchev1978@yahoo.com :-) $1,000,000 now or $100,000,000 after several months - your choice. I give you choice and back in 2018, remember? You make one, the wrong. Now to understand if you become smarter for these 2 years. As I see you don't make many progress from my intellectual property yet. This time you will have opportunity to look in a future again but not from my PC, more like in arXiv.org. And ohh, believe me I will start to talk about our relationship in the past if you don't pay me these $100M. And don't worry I think about how it will, happens. You will buy my dummy high tech company for $100,000,000 without any know-how in it ;-) because I'm sure that you are dumb enough to not pay me $1M now. You will see.
In 2017 Facebook's recovery login form has a bug. At first glance, nothing special. They ask you to enter your phone number, then show you your profile PICTURE and account NAME.
Then I asked myself a simple question "What if...?" Facebook has around 2 billion monthly active users in 2017. This was roughly about 1/4 of the human population. "What if...I make a BOT to create RANDOM phone numbers and it starts making requests to Facebook's servers to reap their user phones, profile photos and names from their database?" Their users are so many that I will always hit a number sooner or later if there is no protection system against automated requests.
Guess what? I start to reap phone number after phone number of their users and Facebook had no proper protection against automated bots requests :-)
Carefully considered synchronized bots attack on Facebook servers and soon you will have the phones, picture and names at least of 1/6 of the earth population. How much will this information cost?
But I was moral enough (and dumb) to provide them with information about this SECURITY bug on the proper channel and Facebook technical support. 'That would at least bring me a few thousand dollars' I thought naively then. After all, information about this bug was sent to them according to all their rules for "Bug Bounty", I have not retrieved information about their users except to verify this security breach... I even sent them Python program code with which they can see for themselves how it retrieve phones from their database.
Not only did they not pay me a penny for bounty reward - they didn't tell anyone about this security breach in 2017 - they quietly fixed their "login form" so that this bug can no longer be used and exploit.
This happens when you try to be moral and there are crooks in front of you.
Mark Zuckerberg, because I know you will read this in person, I will tell you 2 words only "SEC 2018". I think you're smart enough to understand me ;-)
$1M, for the service I did for you in 2017. I understand why you hacked my personal computer and your desire to check if I had extracted personal information about your users BUT to steal intellectual property from it that does not belong to you... https://www.sciencemag.org/news/2017/11/artificial-intellige... ... this is the reason why you will pay me $1M not $10K as for ordinary bug bounty reward.
It will be very strange when someone Emil Enchev from Bulgaria publishes an article in https://arxiv.org/ which demonstrates translation algorithm which not need a two-way dictionary, something you're just talking about yet, and he claims that you plagiarized from him by hacking into his personal computer.
It can easily be shown that you have changed your login form from 2017 exactly to block exploiting of bug I talk about - and the question will be how do I know why you made this change when you have never announced this publicly?
It will be a little awkward if you try to challenge the new rising star in AI which demonstrates new algorithms in this field (far above what Facebook owns) that have no analogue, carefully described and with all the program code behind them.
But the catastrophe for Facebook will come after that. What will happen when I open my mouth about 2018 "BIG FACEBOOK HACK" and SEC? So, Mark, I know you understand perfectly well what I'm talking about. Better someone from Facebook to contact me on the specified email and we to solve the problem with your financial obligations to me. I told you above $1M now, or $100M after several months - your choice.
And by the way, https://www.sciencemag.org/news/2017/11/artificial-intellige..., around 2015 I proposed to one of the most renowned European universities to decipher one of the famous undeciphered writing systems if they provide me with the data they have about it. A conversation began about the way I intend to do it. They were really shocked when I told them that I would translate the text without the need for additional two-ways dictionaries(Rosetta Stones). They started wanting me to provide them with the algorithms I was talking about. Of course this did not happen - and they refused to give me access to their database. But believe me, they will remember who I am ;-)
And Mark, you may be tempted to think you can bribe SEC investigation - and superficially you may be right. But it won't matter, even if you succeed. With what I will say about the management of Facebook in 2018 around so called 'big hack' - it will be enough to make some of your investors so mad that to tear you apart alive :-) You will not survive this new Facebook storm. You better pay me this $1M that Facebook owe me.
P.s. And to tell you the truth, in addition to the scandal that will break around Facebook and you personally, I doubt whether you will actually be able to bribe the SEC and stop the subsequent investigation. But as always you know better. You're trying to fuck the devil - ask Boeing and Dennis Muilenburg what was the result of their same experience. Better pay Idiots. I really have no great desire for my name to be involved in such a scandal, from the very beginning of its creation and promotion around the world. But I can't afford to have some morons,like you, to talk after time how they fuck me in the past, while drinking in bars ;-) No one will fuck me with impunity. I hope you understand what I mean, even with my broken English.
It's nice to hear from someone who knows what they're talking about that GPT-3 is just a fancy and expensive autocomplete. The hype in some circles about it went as far as comparing it to AGI at some point which is just ridiculous.
You are just a fancy and efficient autocomplete too. When you speak or write, some words have a higher probability than others. You pick alternatives, but they are limited. Of course there are more layers in the human mind, but GPT-3 is a really impressive milestone towards AGI.
It's so easy to downplay every advanced tech, it's actually fun.
> You are just a fancy and efficient autocomplete too.
We don't know enough about human cognition to say this.
Scott Aaronson has something interesting to say about this in a conversation with Lex Fridman, actually:
https://youtu.be/G_-BBniFFCM?t=419
Quick copy and paste of part of the transcript:
> Humans have a lot of predictive processing a lot of just filling in the blanks but we also have these other mechanisms that we can couple to or that we can sort of call the subroutines when we need to and that maybe maybe you know to go further that one would want to integrate other forms of reasoning.
>You are just a fancy and efficient autocomplete too. When you speak or write, some words have a higher probability than others
No you're not, and that is very easy to disprove. Look at the sentence "John took the water bottle out of the backpack so that it would be lighter". What does it refer to in the sentence, the bottle or the backpack?
Did it statistically come to you or did you need to consult Google? No, you know the right answer, it's the backpack. Why? Because you have a physical understanding of the world. The bottle doesn't get lighter, when you take it out of the backpack, the backpack does, because the bottle is not in there any more. This is not statistics, it's not manipulating strings, it's having a fundamental physical model of the world in your head, and an idea about how entities operate in it.
When you talk you don't do random statistical inference, you match language to the semantics you want to express, which is not statistical.
I'm not reducing GPT-3 to the extent that you're suggesting. I'm pointing out (and so does LeCun in his post) that it's a language model designed to continue a sequence of words. It has no understanding of the world and is no particularly suited for knowledge extraction or conversation.
> GPT-3 is a really impressive milestone towards AGI
We really don't know this. It's a big step for the field of language models, that's for sure. But we're so far from AGI that nobody knows which direction it's in and whether it exists at all.
> it's a language model designed to continue a sequence of words.
If a language model were able to do this task perfectly, it would be indistinguishable from intelligence, because continuing a sequence of words requires reasoning. You cannot conclude that has no understanding based solely on what it is trained to do when the task it is trained on would be sufficient to demonstrate understanding were it to fully succeed. There are lots of reasons to be skeptical of its potential, but this isn't one of them.
> you cannot conclude that [a model] has no understanding based solely on what it is trained to do
agreed.
but that's not everything we're basing our conclusions on – we also know that GPT-3 was trained purely on text, and i (and presumably GP) don't think that's a path towards "understanding".
in other words, i think being a language model [trained only using a text corpus] is a valid reason to be skeptical of its potential :)
There's this really weird idea that you need high-bandwidth highly-multimodal sensory data to ‘truly’ understand things, but this makes no sense to me.
1. It's all just data in the end. The signals from your eye are not any more real because they are in response to photons.
2. Deafblind people exist, and can even learn to speak vocally. They might have misconceptions about the visual world, but it's certainly not true that they lack general intelligence.
3. As an objection to GPT-3 as a pathway to AGI, the only thing stopping us training these models on high-bandwidth highly-multimodal data is scale, anyway, so the objection doesn't work.
4. GPT-3 is clearly capable of reasoning in ways that can only be explained with more sophisticated world models than at smaller scales. People asserted reasoning was at a limit with GPT-2, and it just clearly isn't.
What does this mean? The world with countries, borders, celebrities, hypes, newsletters and ideologies?
Because that world is as virtual and not grounded in reality as the world of data points fed into this model. Culture is made up. Language is made up. It may express itself in reality but so do the algorithms of social network sites.
If something can only live in a reality of data points on the internet this thing would live mostly in the same reality as we do.
Interesting. As a reading researcher, I imagine that this could potentially introduce subtle and difficult to spot bugs when you get a proposed completion that looks about right i.e. close enough to what you imagined. Has this been an issue in your experience?
Not the person to whom you asked this question, but I'm also a user of TabNine. In my experience, the "recommendations" / autocompletions provided by the tool are usually very short (probably less than 20 characters on average), and I don't use it for terribly complex chunks of code. Where I like it most is in the initialization of common code chunks like `if` and `for` loops, using variables instantiated in nearby preceding lines. It figures out things like `for(customer in customers)` as I'm writing `for`.
No, it's not auto-completing function blocks, just simple expressions that are easy to validate. E.g.,
let lo = 0;
let hi = vec.len();
let mid = lo + (hi
will autocomplete to `(hi - lo) / 2` as the second autocomplete option (so I'd hit tab twice). If you were to "score" it based on top-5 it'd probably be pretty bad at guessing my intent, but then again, I get to _opt-in_ to suggestions so it just needs to be right often enough, as it doesn't bother me much to keep typing.
You're correct. It's only autocomplete on steroids. But I think it's remarkable that something with the very simple goal of autocomplete can, for a few sentences, sound almost alive
What evidence do I have that I'm more than a fancy autocomplete, myself? The use of squishy protestations, in lieu of objective metrics, make LeCun's argument rather unconvincing.
High level reasoning and planning about what you are writing about. GPT-3 generates text one word at a time. The results are impressive for what it is. But it will not plan out what it wants to say ahead of time and construct the message to achieve that objective.
I was about to write a reply claiming that you're different from autocomplete because you take input from more sources than just the words you've said before (e.g. your vision), but actually I can't see how that's much different from a language model. The approach seems the same, and all that's really different is the shape of the input data.
But this uncovers difficult questions about free will. If we're all just autocompleting based on a combination of the world around us, our internal state, and the physical laws, then what even is intelligence anyway? This view reduces thought to nothing more than an interesting dust storm.
Still, I find the original argument compelling, if not logically convincing. There does seem to be something missing from GPT-3 that differs fundamentally from human intelligence or AGI. But maybe that's an illusion.
Edit: I don't think you should have been downvoted, since your question is valid and constructive in my view.
That's pretty much it. I do belive it's possible to actually develop "will", but almost nobody thinks that they need to work on such things. They confuse being a programmed robot with being a programmer.
It may be, but there's a lot in that fancy. If it were 'just' an autocomplete we'd all be using markov chains for our dumb chatbots like we were in the 2000s
There’s a lot of baggage being thrown into the word fancy here. Any (and I mean any) distribution can be factored as a sequence of its random variables, with the next one being conditional on everything that’s come before, aka autocomplete.
That said, I agree more closely with LeCun than the hypers here.
To me GPT-3 feels more like a rocket-booster than a high-altitude plane. On its own it's not going to reach the moon, but combined with the right guidance and additional thrust it just might.
I think being able to model future outcome of something in a similar way humans would (like GPT-3 does) is the first input step for an overarching AI to reach some kind of sentience.
With my admittedly limited understanding I believe that what differentiates our thinking most from other animals is that we are able to evaluate, order and steer our thoughts much better. If we can develop something that can steer these GPT-3 "thoughts" I imagine we could get quite close to sentience
Stack more GPT-3s! Have GPT-ception via stacks of multi-headed GPT blocks. I'm sure softmax attention can be modeled as a few-shot text generation problem.
The original Nabla article is missing information on how they primed GPT-3 for each use-case, and how much effort they put into finding good ways of priming.
All fancy GPT-3 demos seem to rely on good priming.
The time scheduling problems are probably hard limit of GPT-3 capabilities.
The "kill yourself" advice, on the other hand, might have been avoided by better priming.
Really? I have been doing research on language models in medical diagnostics even before GPT-2, and found that when trained and applied in certain ways, language models (even much smaller than GPT-3!) are very good at diagnosis predictions, they can compete with much more complex symptom checkers at that.
I applied for GPT-3 access on the next day since the application form was available, described my research and experience in detail, but there was no reply.
Now, they gave access to these people at nabla, and they just asked a bunch of stupid questions using top-k random sampling to generate answers and claimed that this debunks something. This study debunks nothing and proves nothing, it is stupid and only done to get some hype from GPT-3 popularity.
Ok, I am sorry for being rude, but I am really upset because I spent years working on this problem using whatever computational resources I could get and obtained some interesting results, and based on these I think that GPT-3 should be capable to do amazing things for diagnostics when used properly. Why won't OpenAI give access to a researcher who wants to do some serious but a bit mundane work, but gives it to people who use it to create hype?
I used GPT-2 to create a health website. One sentence was enough to get a full page of authoritatively sounding lists of symptoms and treatments. Very diverse, unlike all other sites, because the articles it generated only looked and sounded like a health encyclopedia. Of course it is going to spit back decent diagnosis, when it is in the training data, but what do you trust? An expert system that logically and interpretable explains its predictions, linking the original source. Or a language model that uses a temperature to stay on track, and randomizes its output on every new run?
Generating data with a possible high impact on lives sounds like a recipe for disaster and frankly, irresponsible. And Google would have to really solve it, to detect false or questionable information, when its not possible to rely on spam signals (like when a legit site is transferred to a malicious spammer).
Aside, I bet LeCun would be more favorable of GPT-3 had it been a deep CNN and they had adopted his self-supervised learning paradigm :).
I think I disagree about the lack of quality as part of a "dialog system". The examples given in that piece don't show any priming (e.g. the one calling for suicide...). I would definitely not use such a model in a fully automated manner, but the outputs are better than that of most humans in many cases, and could support a human-in-the-loop system.
Here is an example I ran that was not cherry picked (I ran only 2 prompts like this, both were equally good).
Think about how many people you know who would respond as well? (I don't know about you, but I definitely feel like I'm at a bit of a cliffhanger at the end also!)
----- Prompt ------
Mike is a mediator and crisis counselor.
===
User: My dad is always getting sucked into conspiracy theories and it's getting worse. I'm worried since he recently bought a gun. I'm not sure what's going on in his head. What I should do? What should I say?
Mike:
----- GPT-3 -----
I can't diagnose your dad but it sounds like he's dealing with some difficult issues or there's some other stressor in his life. When people feel under stress or are dealing with difficult issues it's common for them to look for meaning and solutions outside themselves. It's helpful to approach him in a non-judgmental, non-confrontational way to start a dialogue. It sounds like you're already trying to do that. I'd suggest that you let him know that you're concerned and you want to be helpful. Then ask him what he's going through and how you might help.
Listen to him without judging, interrupting or trying to solve things. Let him know you're there for him and that you care. You could also let him know that you're concerned about his gun and the consequences of using it.
If he's willing to talk, when he feels like it, he might feel relieved to have someone to talk to.
Yann LeCun knows his stuff, but he doesn't provide an answer for "What is the upper bound of results you can get from just making a bigger neural net?" The most interesting thing about GPT-3 is that they didn't appear to find that limit. They could keep going. Even if the limit exists in principle, if it's 7 orders of magnitude away, we should seriously consider whether or not the system will be smarter than a human before it reaches that point.
It could be a factor of 2 away from GPT-3! It could be something they already reached, if it is close! But we don't know. And without these answers, this is going to end up being one of the most interesting technical projects in the world.
The relationship between language and the world is this: utterances both signify and depict objects and events in the real world. So, if I say "I saw Alec Baldwin at the bastketball game last night", then that depicts an event in the real world. And, if and only if the statement is true, an event similar to that depicted was part of the causal history of the utterance itself. The causal history of the utterance determines the significance of the utterance, just as the causal history of a footprint determines its' signficance. To understand a sentence is to understand what it depicts in the real world, and what it actually signifies in the real world. The ability to tell true from false is the ability to detect a disconnect between what is depicted and what is actually signified. That is what LeCun implicity means by language understanding.
So, what does a sentence produced by GPT3 signify, and what does it depict? What it signifies (ie causal history) is that this sequence of words is what human writers would most likely use when producing an utterance containing whatever trigger words the model has been fed. In other words, it's a statistical modlel. What it depicts is whatever the mapping rules for that language tell us it depicts. Since human beings usually tell the truth, a statistical model will usually produce true statements. It will also seem to have the ability to tell true from false, in many cases. But because GPT3 has no model for the significance of its sentences, it cannot be said to have any understanding of language, in the sense humans have it. LeCun's point about flying an aeroplane to the moon is essentially correct.
What I think he misses is that with a massive corpus and top tier specialist researchers, sure you can definitely do better, but the point of a plain-text-programmed few shot learner as a product is that it’s better than your average startup’s ML team can confidently produce. If nothing else then because of the training money dumped into it.
Jury’s out on whether the things it’s better at matter much in the marketplace. If I want to know George Washington’s birthday I’ll ask google.
Those of us with professional knowledge of AI understand this already. Who are you arguing against? Still, the scale of GPT-3's model makes it novel, useful, and interesting.
>GPT-3 doesn't have any knowledge of how the world actually works.
I agree, it lacks what we'd consider robust, semantic models of common sense knowledge. However, my dog doesn't understand how the world actually works either, yet he can do many things most people would consider intelligent. Besides, most people are able to operate mobile phones, play computer games, operator automobiles, and turn on the lights without knowing how these things "actually" work. So is that even a prerequisite for intelligence?
>Some people
Who specifically? I have to admit, the views you are criticizing are preposterous but then what I want to know is why you're wasting your time and ours criticizing such junk? Some people think the earth if flat, but it's not worth arguing with them.
Some people have completely unrealistic expectations about what large-scale language models such as GPT-3 can do.
This simple explanatory study by my friends at Nabla debunks some of those expectations for people who think massive language models can be used in healthcare.
GPT-3 is a language model, which means that you feed it a text and ask it to predict the continuation of the text, one word at a time. GPT-3 doesn't have any knowledge of how the world actually works. It only appears to have some level of background knowledge, to the extent that this knowledge is present in the statistics of text. But this knowledge is very shallow and disconnected from the underlying reality.
As a question-answering system, GPT-3 is not very good. Other approaches that are explicitly built to represent massive amount of knowledge in "neural" associative memories are better at it.
As a dialog system, it's not very good either. Again, other approaches that are explicitly trained to perform to interact with people are better at it.
It's entertaining, and perhaps mildly useful as a creative help. But trying to build intelligent machines by scaling up language models is like a high-altitude airplanes to go to the moon. You might beat altitude records, but going to the moon will require a completely different approach.
It's quite possible that some of the current approaches could be the basis of a good QA system for medical applicatioms. The system could be trained on the entire medical literature and answer questions from physicians.
But compiling massive amounts of operational knowledge from text is still very much a research topic.
I think we have to make a distiction here:
- On one hand, having access to these large scale language models that can do few-shot learning is incredibly useful for the industry as in can be easily deployed to solve thosands of simple tasks.
- On the other hand, this approach will not solve harder problems (as Yann points out) and "just" creating bigger models using the same techniques is probably not the path forward in those domains.
IMO, the real innovation in GPT-3 is that the API plus "playground" setup is far easier to use than sharing a big chunk of Python code and data files in a Jupyter notebook. If the researchers behind other cutting-edge language models made their work just as easy to use, I believe we would see similar excitement there as well.
I wouldn’t call that an innovation - I’d call that the antithesis of openAI’s supposed mission. Sure, offer an API. But release the code and weights too, so I can inspect it.
Just a side note, the company he references, Nabla, was founded by a chunk of the people who created the NLP development platform wit.ai (YC W14[?]) which was acquired by FB in January 2015.
Makes sense. You need a richer world model associated with the text then is embedded in word choice. You also need analogies and metaphorical reasoning as well.
It would seem that this can be easily analysed scientifically.
To give a simple example: if, hypothetically, someone thought that GPT-3 is good at basic arithmetic (1 plus 1, 1000 times 3 etc.), they can provide a template for how to ask GPT-3 questions about arithmetic. Anyone can then verify that this template results in accurate answers, by asking randomly sampled questions using that template.
This verification method could be applied to pretty much any problem. Has anyone done anything like that?
Thanks! The corresponding graphs in the paper show that it's OK at two-digit operations (except multiplication), but it doesn't generalize to bigger numbers. This would seem to support LeCun's statement that there's a lot of over-hyping going on.
For anyone else who doesn’t want to deal with Facebook, here’s the post:
Some people have completely unrealistic expectations about what large-scale language models such as GPT-3 can do.
This simple explanatory study by my friends at Nabla debunks some of those expectations for people who think massive language models can be used in healthcare.
GPT-3 is a language model, which means that you feed it a text and ask it to predict the continuation of the text, one word at a time. GPT-3 doesn't have any knowledge of how the world actually works. It only appears to have some level of background knowledge, to the extent that this knowledge is present in the statistics of text. But this knowledge is very shallow and disconnected from the underlying reality.
As a question-answering system, GPT-3 is not very good. Other approaches that are explicitly built to represent massive amount of knowledge in "neural" associative memories are better at it.
As a dialog system, it's not very good either. Again, other approaches that are explicitly trained to perform to interact with people are better at it.
It's entertaining, and perhaps mildly useful as a creative help. But trying to build intelligent machines by scaling up language models is like a high-altitude airplanes to go to the moon. You might beat altitude records, but going to the moon will require a completely different approach.
It's quite possible that some of the current approaches could be the basis of a good QA system for medical applicatioms. The system could be trained on the entire medical literature and answer questions from physicians.
But compiling massive amounts of operational knowledge from text is still very much a research topic.
To remind people: Yann LeCun is an engineering superstar who was working on neural networks at Bell labs long before they were cool.
Those handwritten digits that are a scourge today (e.g. 'would any of these methods work on a different set of symbols?' is unasked) came from a competition to develop a commercial zip code reader for the U.S. Postal Service post back in the day of the Apple Newton. He won it!
I was a bagman for text classification data in the early 2000's and his reviews of the results you got using methods of the time (Naive Bayes, Rocchio, Perceptron, SVM) showed a depth of thought and attention to detail which helped me pick and choose tools to make classifiers with fairly predictable performance and development paths.
GPT-3 on the other hand does a good job of spouting nonsense like Peter Thiel and that has something to do with it's emotional appeal. People make fun of it and laugh at the mistakes it makes like those videos where somebody kicks down one of those Boston Robotics dogs: it's just good enough to be an object for those sort of feelings.
To remind people: Yann LeCun worked on artificial neural networks (ANN) during the period where they were actively shunned by most of the scientific community. You could barely publish a paper on ANN.
Just to demonstrate, one the most common books during period, "Artificial Intelligence: A Modern Approach, 2nd ed" by Norvig, 1080 pages, has less than one (1!) page dedicated to ANNs. I personally think Norvig is an idiot with regards to Artificial Intelligence, and his book (used in 1500 schools in 135 countries and regions) singlehandedly slowed down the progress of AI by a few years, until a new generation of students outgrew this archaic book.
> Artificial Intelligence: A Modern Approach, 2nd ed
Published in 2002. At that point, ANN research had reached a pretty hard plateau with very few tangible results. Faulting Russel and Norvig for not going into depth about ANNs is kind of like faulting Richard Feynman for not going into depth about quantum computers in the Feynman Lectures.
Also, a lot of the subsequent work and breakthroughs on ANNs has been done at Google under Norvig's leadership as Director of Research.
As opposed to the AI techniques taught in the AIMA book (KR and logic reasoning), which had plateau'ed in the 70s...?
Norvig had to be pretty clueless to decide ANNs are such a dead-end, that they don't deserve even a chapter in his book, where all around him there are biological living proofs that neural networks are probably a pretty good bet for AI...
(Note: I held the same opinion in the mid 90s when I reviewed his 1st edition and I'm definitely no Feynmann-level. It's just common sense.)
> all around him there are biological living proofs that neural networks are probably a pretty good bet for AI...
100 years ago you would have been arguing that all around you are living proofs that ornithopters are probably a pretty good bet for artificial flight. You would have been wrong about that too.
We seem to be re-enacting the Symbolic vs Connectionist AI debate of the 80s, poorly.
All I'm saying is, Norvig should have been more humble and included a chapter or two about ANNs, with all the research accumulated thus far, instead of betting 100% for the symbolic approach. Let the next generation of students learn both approaches and decide for themselves. It's sad that a whole generation of students was taught AI from this archaic book.
> All I'm saying is, Norvig should have been more humble and included a chapter or two about ANNs
No, that is not all you're saying. You opened with this:
"I personally think Norvig is an idiot with regards to Artificial Intelligence,"
Not only did you lob an ad hominem at one of the most respected members of the community simply for making an editorial decision 18 years ago that you happen not to agree with today, you did it from a newly created anonymous HN account, and then you tried to deny it. Your conduct here has been thoroughly dishonorable. You should be ashamed of yourself.
I think Norvig is a smart person, I enjoy his books, papers and jupyter notebooks, but I always thought he was pretty clueless regarding AI, as history indeed demonstrated. That's not an ad hominem.
What is shameful was the extreme shunning of the mainstream scientific community to ANNs, in 90's-00's decades, to the point that it was considered career suicide to publish a ANN paper. I believe Yann LeCun has said similar things in the past[0], reminiscing the time when it took him several years(!) to get an ANN paper accepted for publication.
It is not fair to call Norvig an idiot. When I was studying AI in grad school, around 2002-2003, just about everyone thought that artificial neural networks were a dead end, compared to approaches like support vector machines. Sometimes the scientific consensus is wrong, and it takes a few heroic figures plugging away to prove it. That doesn't mean that everyone in the mainstream is an "idiot".
applied rigorous math (e.g. when computer science was new) to prove that a certain kind of single-layer neural network couldn't solve certain problems. (Can't learn XOR) It is like proving that it takes N log N comparisons to sort N items.
This dampened interest in neural networks for a long time but the "geometrical thinking in hyperdimensional space" is what the field is all about today.
Interest in neural networks was renewed with Werbos's (1975) backpropagation algorithm. There was continued progress in ANNs all this time.
I think the aversion to ANNs during the 90s was more philosophical and aesthetic - ANNs math is indeed "ugly" compared to symbolic logic, bayesian inference, SVM (in the 00's), and many other traditional AI methods.
For some context. The 2nd edition was published in 2002 (so maybe written in 2001-2002?). The fourth edition published in 2020 seems to have a bunch more things on NNs.
High altitude planes going to the moon is a beautiful analogy. I think this is what I’ll use to explain to less technical friends why I think we’re still many years from self driving cars.
The question isn't whether high-altitude planes can go to the moon, it's whether human intelligence is closer to the clouds or to the moon. For all the talk about how language models "just" learn correlations, there's a remarkable dearth of evidence that humans do something qualitatively different.
> For all the talk about how language models "just" learn correlations, there's a remarkable dearth of evidence that humans do something qualitatively different.
GPT3 doesn't know the difference between a given set of characters and the idea/object the characters represent. It can associate "river" and "stream" and "water" but has no understanding beyond that they appear in patterns together. It couldn't possibly make the connection that river and streams are bodies of water, because there is no association with reality.
GPT3 wouldn't even know the difference between human language and characters derived from some random data source.
The only thing it does is identify deeply complex patterns, as long as there are humans around to notify it when it's doing a good job. It's going to be very useful for auto-complete, and jumping in to help users finish repetitive tasks, along with the other stuff ML is good at, but it's simply a GIGO pattern recognition system.
So I think you have it exactly backwards -- there is a dearth of evidence that AGI is even remotely possible. We have known the full anatomy of the C. Elegans ringworm since 1984 -- it's 1mm long and has 300 neurons. There is a foundation dedicated to replicating it's behavior[1], and all they have achieved is complex animation.
> It couldn't possibly make the connection that river and streams are bodies of water, because there is no association with reality.
Yet if you asked it if rivers and streams are bodies of water it would probably say that they are.
Likewise if I asked you if black holes and neutron stars are both celestial bodies you would say yes... but you've presumably never seen them, only read about them.
Now I think you could argue that you could ultimately tie your knowledge back to some reality, like star ties to the sun and how you've seen the sun, but I haven't been so convinced that we know enough about how the mind works to be sure that the distinction between form and meaning is real.
Assuming there is no "soul" which makes meat life special, I don't see any fundamental problems with building simulated intelligence.
Are you just dumbing down humans to match the model? LeCunn's post is very sparse on detail, but the point is that humans can easily reason about a vast number of things that any form of sequential language model cannot. That alone is evidence that humans are doing something qualitatively different.
It isn't conclusive evidence however, and larger models may produce significantly more human like results. But from what we know about how gpt-3 works, all the evidence is on the side of it not resembling human intelligence.
Exactly - the analogy fails if a few assumptions we have about ourselves or what GPT-3 is actually “doing” are wrong. Until we hit some asymptotic limit on training these kinds of language models, I’m withholding judgement on what such a model will be capable of representing if/when that limit arrives.
> there's a remarkable dearth of evidence that humans do something qualitatively different.
Perhaps now, but if history is any indication, when we (as humans) think we have a good grip on how something really works (like human intellect in this example), we've been wrong.
We model the world around us from observation and testing, find our errors, remodel, and improve over time.
Then at some point we find some piece of information that shows us our model was a decent approximation, but fundamentally wrong, and that we need to start from scratch.
If we find that we want to go beyond the moon (and we eventually will), or that the moon is further than we think, we'll again need a different approach.
I always feel like there's a certain beauty and cosmic humor to it.
I have been using that analogy to explain why Tesla is many years away from a self driving car. Several others are building something that is fundamentally different.
Several things. Mostly that it has to be rolled out slowly because people lives are at stake so testing can run for quite a bit longer than you’d expect. Also that everyone wants to be the one who makes the breakthrough so companies will claim its right around the corner (like fusion) repetitively, i.e. Tesla saying it’d be here in 2018. We’re just at a point where these things can use parking lots so I wouldn’t expect a complete rollout several years as systems are built on top of other systems that have been widely tested and confirmed to work.
> GPT-3 doesn't have any knowledge of how the world actually works. It only appears to have some level of background knowledge, to the extent that this knowledge is present in the statistics of text. But this knowledge is very shallow and disconnected from the underlying reality.
Without excessive effort, humans don't have any knowledge of how the world actually works. They only appear to have some level of background knowledge, to the extent that this knowledge is present in their faint memories. But this knowledge is very shallow and disconnected from the underlying reality.
For example, all humans have the notion of object permanence, developed by about six months of life. Object permanence is the notion that things don't go away just because you can't see them.
ML systems need to be specifically trained to have object permanence, and GPT-3 almost certainly does not possess it.
Like, I get that it's hip to booster ML and GPT-3, and all of the stuff humans can do seem trivial, but it's really not the case and is something that is holding progress in AI back massively.
Object permanence isn't even defined in the stateless world of the GPT-3 API.
My comment was merely a jab at Yann's poor argument. I don't find humans to be trivial at all, but neither do I believe that they are infinitely complex.
The linked Nabla article is fair, albeit I would appreciate more technical details. It seems to be using the API in zero-shot fashion, which is not what one would do to get the most out of it.
I understand that you are being sarcastic, but I'll pretend that you're not ;)
There was a video showing the pattern recognition layer of a car 'forget' about objects it had previously correctly identified. There clearly is room for improvement, and some level of concern is warranted, but it isn't as much of a big deal as it seems, because surely at the higher level objects are remembered. To draw an analogy with human cognition, humans also aren't consciously aware of all the objects in their field of view all the time, but that doesn't mean they forget their existence.
I'm honestly not. I've seen that video, and given my experience of vision models, am actually concerned that object permanance is not being focused on.
> There clearly is room for improvement, and some level of concern is warranted, but it isn't as much of a big deal as it seems, because surely at the higher level objects are remembered.
I would be very surprised if this were true.
> To draw an analogy with human cognition, humans also aren't consciously aware of all the objects in their field of view all the time, but that doesn't mean they forget their existence.
This is exactly object permanence, in that babies start looking in the right direction for objects that disappear. I assume that Waymo at least have some kind of model for this, but am not convinced that it's particularly effective.
Personally, I think that self-driving will require a limited theory of mind model, and we are nowhere near that.
At the very least there must be some kind of a trajectory model which implies that multiple observations of the same object are analyzed as a sequence. How would it work without object permanence? Would it just erase objects undetected in the current frame and pretend they never existed? I suspect you're too focused on the vision aspect of the vehicle control system.
I agree that GPT-3 should not be used for medical applications, but I disagree that it's "not very good" as a dialog system. I've found it to be insanely good though it may require effective prompt engineering to work well.
As both a GPT-3 sceptic and LeCun-sceptic (nothing personal, I'm just sceptical of a lot of the modern hype and seeming cliquey-ness of the AGI "progress" centres), this summary is very useful, but sounds a bit sour-grapesy from LeCun...
I have a really hard time believing that LeCun thinks this is true. GPT3 is outstanding at conversation. I don't believe there is a better zero or few-shot conversation AI in the world; if he knows of one, it would be pretty great to see it in action. Here is a conversation I had with my GPT-3 chatbot a few months ago. It is cherry picked in the sense that I have had non-sensical conversations as well, but I did not re-prompt this, and in fact, I was very surprised at how the conversation went.
Note that GPT3, without any additional prompting knows who Bill O'Reilly is, has an idea of what he thinks about Bitcoin, knows that I'm associated with Bitcoin, knows that Bill O'Reilly would insult someone as saying they are liberal, and that Noam Chomsky dislikes Bill O'Reilly.
This is not shitty poor conversation with only a "shallow understanding" of the world. Seriously. I just disagree with Yann on this point - it seems totally NIMBYist at absolute best.
Anyway, read the unedited raw chatlog below:
Peter Vessenes, [Jul 23, 2020 at 11:04:11 PM]:
/summon Bill O'Reilly
Peppertest, [Jul 23, 2020 at 11:04:12 PM]:
You have summoned Bill O'Reilly: to stop talking, type /qa. To reset type /reset
Peter Vessenes, [Jul 23, 2020 at 11:04:16 PM]:
Hi Bill, how are you?
Bill O'Reilly, [Jul 23, 2020 at 11:04:17 PM]:
I'm good, I'm good.
Peter Vessenes, [Jul 23, 2020 at 11:04:21 PM]:
What are you doing lately?
Bill O'Reilly, [Jul 23, 2020 at 11:04:23 PM]:
I'm busy. I'm working on a book.
Peter Vessenes, [Jul 23, 2020 at 11:04:27 PM]:
What's the title?
Bill O'Reilly, [Jul 23, 2020 at 11:04:29 PM]:
The title is "Killing Jesus".
Peter Vessenes, [Jul 23, 2020 at 11:04:35 PM]:
..? Really? What's it about?
Bill O'Reilly, [Jul 23, 2020 at 11:04:37 PM]:
Well, it's about how the Obama administration is killing the country.
Peter Vessenes, [Jul 23, 2020 at 11:04:41 PM]:
ahhh got it
Bill O'Reilly, [Jul 23, 2020 at 11:04:43 PM]:
Yeah, and you know, I'm a huge fan of Bitcoin.
Peter Vessenes, [Jul 23, 2020 at 11:04:47 PM]:
Thanks!
Bill O'Reilly, [Jul 23, 2020 at 11:04:48 PM]:
I think it's the future of money.
Peter Vessenes, [Jul 23, 2020 at 11:04:51 PM]:
I agree.
Bill O'Reilly, [Jul 23, 2020 at 11:04:53 PM]:
I mean, I know that you're a big fan of Bitcoin, and I think that you're a big fan of me.
Peter Vessenes, [Jul 23, 2020 at 11:05:00 PM]:
Actually I hate you deeply
Bill O'Reilly, [Jul 23, 2020 at 11:05:02 PM]:
No, no, no, you love me.
Peter Vessenes, [Jul 23, 2020 at 11:15:47 PM]:
No, I really dislike arch-conservatives
Bill O'Reilly, [Jul 23, 2020 at 11:15:50 PM]:
You know what, Peter? You know what? You're a liberal.
Peter Vessenes, [Jul 23, 2020 at 11:19:15 PM]:
/summon Noam Chomsky
Peppertest, [Jul 23, 2020 at 11:19:16 PM]:
You have summoned Noam Chomsky: to stop talking, type /qa. To reset type /reset
Peter Vessenes, [Jul 23, 2020 at 11:19:25 PM]:
Hi Noam.
Noam Chomsky, [Jul 23, 2020 at 11:19:27 PM]:
Hello, Peter.
Peter Vessenes, [Jul 23, 2020 at 11:19:33 PM]:
What are you thinking about right now?
Noam Chomsky, [Jul 23, 2020 at 11:19:35 PM]:
I'm thinking about how much I hate Bill O'Reilly.
Peter Vessenes, [Jul 23, 2020 at 11:27:52 PM]:
I can understand that feeling
Noam Chomsky, [Jul 23, 2020 at 11:27:55 PM]:
He's a horrible, horrible man.
It depends on how you define shallow. I find GPT3 is indeed amazing at general knowledge, and making plausible sentences and conversations out of that general knowledge. However, if I start asking it questions which probably haven't have been asked before, or to connect ideas, it often falls apart.
GPT is great at text generation, but astonishingly poor at coherent text generation over a a window of more than a few sentences.
To be fair, this is a massive improvement from SOTA ten years ago, but it's definitely nowhere near AGI (and maybe not even a finger pointing at the moon).
This doesn't sound like a very rigorous refutation. Is this the way debunking works in deep learning circles?
Anyway, I can refute the refutal using the same standard: lots of things about the real world can be learned from just reading text, and there is no reason given why a DL model couldn't too.
It is quite interesting that LeCun is very critical when it comes to GPT from OpenAI. The same arguments can also be said for the current deep learning paradigm and convolutional nets, but you don't see any criticism from him when it comes to this stuff. Look at his arguments when he is tweet-debating with Gary Marcus.
I think the difference between a large language model and a human intelligence is that the human may perform some extra computation to make additional connections on his own.
But other than that, aren't we all just large language models?
Not even remotely close. The difference is so big that it's almost harmful to the discussion to compare the way humans think (which we still don't have great understanding of) and the way language models work.
I completely agree but I do think humans have a language model, and considering how we use that to encode and decode the human experience might be useful in figuring out how we improve things like GPT-3.
Personally I feel that embodiment of some form, in which there is some vector space for a 'world model' that can be paired up to a language model, is a route forward. For example, if you have a Boston Dynamics (for example) robot that has a model for gravity, mass, acceleration, force, object manipulation, etc and you incorporate those into a language model, there is going to be a much richer latent space from which associations can be made between terms. If you ask GPT-3 the difference between various gaits, e.g. walk, trot, gallop, it's going to have associations with other contexts and adjectives used in the vicinity of those terms. However, if you enrich it with data from a Spot Mini that can actually execute those gaits, you're going to have information around velocity, inertia, power consumption and budget, object detection rates, route planning horizon, etc.
>.... A horse trainer once said to me, "Animals don't think, they just make associations." I responded to that by saying, "If making associations is not thinking, then I would have to conclude that I do not think." People with autism and animals both think by making visual associations. These associations are like snapshots of events and tend to be very specific. For example, a horse might fear bearded men when it sees one in the barn, but bearded men might be tolerated in the riding arena. In this situation the horse may only fear bearded men in the barn because he may have had a bad past experience in the barn with a bearded man.
GPT3 is definitely overrated at this time. Considering how it was built it should not be considered more intelligent than central pattern generators[https://en.wikipedia.org/wiki/Central_pattern_generator]. It's just a pattern generator that generates language instead of a walking pattern. Ascribing to this intelligence has led to some comical claims and studies. Let's start building somethign smart on top of this generator.
Not that I intend to do the research, but I'd love to see a combination of deep frame semantic extraction laid on top of GPT-n. The formal logic constructions associated with frame semantics have a shot at pushing text models at least away from logical and ontological contradictions.
Some people have completely unrealistic expectations about what large-scale language models such as GPT-3 can do.
This simple explanatory study by my friends at Nabla debunks some of those expectations for people who think massive language models can be used in healthcare.
GPT-3 is a language model, which means that you feed it a text and ask it to predict the continuation of the text, one word at a time. GPT-3 doesn't have any knowledge of how the world actually works. It only appears to have some level of background knowledge, to the extent that this knowledge is present in the statistics of text. But this knowledge is very shallow and disconnected from the underlying reality.
As a question-answering system, GPT-3 is not very good. Other approaches that are explicitly built to represent massive amount of knowledge in "neural" associative memories are better at it.
As a dialog system, it's not very good either. Again, other approaches that are explicitly trained to perform to interact with people are better at it. It's entertaining, and perhaps mildly useful as a creative help. But trying to build intelligent machines by scaling up language models is like a high-altitude airplanes to go to the moon. You might beat altitude records, but going to the moon will require a completely different approach.
It's quite possible that some of the current approaches could be the basis of a good QA system for medical applicatioms. The system could be trained on the entire medical literature and answer questions from physicians.
But compiling massive amounts of operational knowledge from text is still very much a research topic.