Read it to the end. The end is quite mind blowing. I'm not sure how someone can finish this article and not realize that chatGPT actually completely understands what you're telling it.
In some ways it's amazing, in other ways it's like saying that if I gave my mother a cue card saying "NVIDIA-SMI has failed because it couldn't communicate with the NVIDIA driver" and she read that back to me when I said "nvidia-smi" then she must be running a virtual machine in her head and querying it for a GPU. Or if she read "ping reply 14ms" off a notepad she must be simulating ICMP echo and understand networking.
She clearly wouldn't be. ChatGPT has zillions of web pages as cue cards. It will have many examples of ping and reply, of first ten prime numbers in Python, of Docker output. Was it actually building a Docker file inside it in any way?
I asked it a similar prompt, I'm amazed at how much it can get right, I've never seen any other program or computer do what it does. At the same time, it's not implementing the things behind the scenes that we're asking it to and more complicated prompts makes that visible - it's not actually a CPython runtime or a Linux shell or curl'ing a web page; this is the disconnected lever fallacy I mentioned earlier. "If I copy the interface to a warp drive, that will give me a warp drive, if I copy the interface to a Linux machine, that's the same as being a Linux machine".
It's also akin to Searle's Chinese Room - at some point it may be good enough to be indistinguishable from a Linux machine and that would render this argument irrelevant.
It sounds like you’re making the same mistake that people often make with the second part of the Chinese room argument where a real English-speaking human is following a bunch of instructions to manually compute the Chinese room algorithm. That human doesn’t understand Chinese, just like your mom doesn’t understand Unix systems or command line-interpreters. In both cases, the humans are just functioning as hardware executing a computer program.
I’ve given it much more complicated prompts to create a specific Python script with very specific requirements and it did pretty well. I’ve even given it a Python script I wrote that took in a JSON file And asked it to make sone changes to the script.
It does far more than just mindlessly return results.
There's no example cue cards of itself. In, the ending of that article, chatGPT curled the internet from the virtual machine. It specifically curled the chatGPT api and talked to itself.
Then it asked itself to create another virtual machine, and the virtual chatGPT on the virtual internet on the virtual machine created another virtual machine.... The end...
It's not actually creating any of these things. It's imitating these things indicating it Understands what it is.
It imitated itself. Indicating an awareness of self. aka self awareness.
Now, after you run the first command, chatgpt will sometimes run one kernel version and sometimes another (according to uname -a). Sometimes there will be network access, and sometimes there won't. You can even trick chatgpt into believing there is internet access, after which other "cue card" contents will be returned.
There really isn't any understanding. The responses are equivalent to googling the internet, where sometimes people get articles about how to run ls on ubuntu and sometimes get articles about how to run ls on redhat.
You can even cause chatgpt to prune after a portion of the conversation, after which completely different responses get generated. It's not understanding but may contain more entropy than many people's passwords...
I can't reply to the OP because it's banned or something but their lack of understanding about this area actually makes me think someone is using ChatGPT to troll this thread with confident sounding BS.
It's not a ban. On HN they recently put a thread limit on depth.
Either way your post is rude.i have a contrary opinion can I not express it without being accused of trolling? I assure you I am not.
If anything, the majority shares there opinion with you. They are generic and harping on the same tired tropes of llms being just statistical text completion machines without ever remarking about the obvious black box nature of neural nets. It's more likely the majority and generic opinions that you all share are the ones trolling with chatGPT.
In fact ask chatGPT about the topic. Are you self aware? And it will give you the same generic opinion you and everyone else on this thread. If you're right about chatgpt being a statistical parrot, then ironically my opinion is more likely to NOT be generated by an LLM given how contrary it is to the majority opinion (aka training data).
A generic opinion doesn't mean it doesn't understand. Think about it. It means it's either giving us the most statistically likely text completion OR it actually has a generic understanding and a generic opinion on the post. It doesn't bend the needle either way.
>I vouched for several of your dead posts.
Thank you. Again, I assure you none of my posts are trolls.
In your dead reply you say "Inconsistent and wrong answers does not negate the fact that it understands you.". If you ask a schoolkid to multiply 5x5 and they say 25 and then you ask them to multiply 300x10 and they say 310 you don't say "they understand multiplication and wrong or inconsistent answers can't convince me otherwise", you say "they memorised the twelve times table but don't understand multiplication".
The other way that you say doesn't exist is the way it actually was built to work - it's trained to find repeated text patterns in its training data, and then substitute in other text into them using them as templates. Yes there is no Google result for "Bash script to put jokes in a file" but there are patterns of jokes, patterns of Bash scripts putting text into file and examples of filesystem behaviour. That it can identify those patterns and substitute them inside each other is what makes it work. You say "ping bbc.co.uk" it says "ping reply NN milliseconds where NN=14" because there are many blog posts showing many examples of pings to different addresses and it can pull out what's consistent between them and what changes. You say "{common prime number code}" it replies "{prime numbers}".
Ask it to "write an APL program to split a string" and it says "{complete nonsense}"[1] because there aren't enough examples for it to have any idea what to do, and suddenly it's clear that it isn't understanding - it doesn't understand the goal of splitting a string or whether it's achieved the goal, it doesn't understand the APL functions and how they fit together so that it can solve for the goal, it can't emulate an APL interpreter and try things until it gets closer to its goal even though it has enough computing power that it potentially could, it can't pause for thought or ask someone else for help, it can't apologise for not knowing enough about APL to be able to answer because it doesn't know that it doesn't know. Your 17th Century man also couldn't write an APL line to split a string, but he would shrug his shoulders and say "sorry, can't help you". The internet it was trained on has a lot of Python/Docker/Prime numbers/Linux shell basics and a lot of Wikipedia / SEO blogspam because they are written about millions of times over, and they contain much less about APL.
Pareidolia is the effect where humans see two blobs and a line and 'see' a face. People see a bag with buckles and folded cloth looks like a scowling face, our mirror neurons project a mind into the bag and we feel "the bag's feelings" and say the bag is angry, and laugh because it's a silly idea. When ChatGPT parrots some text which looks correct, we project an intelligence behind the text, when it parrots some text which is wrong we excuse it and forgive it, because that's what we do with other humans.
A human who speaks Spanish from a phrasebook is stumped as soon as the conversation deviates from the book. Even if they have excellent pronounciation. A human who understands Spanish isn't. ChatGPT is a very big phrasebook.
[1] https://i.imgur.com/7z4LB9W.png - plausible at a glance, the APL comment character is used correctly, so is variable assignment. But you can't split a string using catenate (,) and replicate-down (⌿) it has effectively done a Google search for APL and randomly shuffled the resulting words and glyphs into a programming style example. What the code does is throw a DOMAIN ERROR. You can say "there is understanding but it made a mistake", but it's the kind of mistake that makes me say "there is no understanding".
Think of it this way. I ask the child 31234 * 2847374623. The child gives me exactly the correct answer. Then I ask the child 1 * 0 and the child gives me 2.
I would say the child on a certain level understands the concept of multiplication by virtue of being able to calculate the complex answer despite his secondary answer being incorrect.
You're wrong. It is trained with text from the internet as an llm but on top of that it is also trained on good and bad answers. This is likely another layer on top of the LLM that is regulating output. Look it up if you don't believe me. There was an article about how openai outsourced a bunch of Kenyans to do training work.
The apl example doesn't prove your point imo. I'm not saying chatGPT understands everything. I'm saying it can understand many things. The thing with apl is that it has incorrect understanding of it due to limited data.
I don't think I'm biased. My opinion is so contrary to what's on this thread it's more likely your the one excessively projecting a lack of intelligence behind chatGPT. Pareidolia is you, because you're the one following the common trope. It takes extra thinking and an extra step to go beyond that.
It is true that chatGPT has a big phrasebook. However. It is also true that the example I mentioned is NOT from the phrasebook. It is obviously a composition of multiple entries in that phrasebook put together to form a unique construction. My claim is that there are a many number of ways that those phrases can be composed but chatGPT chose the right way because it has enough training data to understand how to compose them.
Clearly for apl it understands it incorrectly. The composition of an incorrect result means incorrect understanding and the composition of a highly improbable correct result means true understanding which is inline with what I am saying that both understanding and incorrect understanding can exist in parallel.
> "it's more likely your the one excessively projecting a lack of intelligence behind chatGPT. Pareidolia is you, because you're the one following the common trope. It takes extra thinking and an extra step to go beyond that."
It takes no thinking at all to hug a stuffed toy, or to anthropomorphise a cat or dog and attribute human motivations and intelligence to them. It's generally frowned upon to suggest that the horse licking the hand is looking for the taste of salt and not showing love and affection of its owner. Humans see intelligence everywhere especially where it's scary - an animal screech in the woods or out at sea and bam, witches, sirens, werewolf shapeshifters, Bigfoot, the Banshee, aliens, Sagan's "Demon Haunted World" - human level malevolent intelligence projected into a couple of noises.
People were fooled by the Mechanical Turk[1], people are fooled by conjouring trick magic with only a couple of movements made non-obvious, in recent years an Eliza quality basic chatbot passed a Turing test at British The Royal Society[2] largely by exploiting this effect pretending to be a Ukranian 13 year old so the testers would give it the benefit of the doubt for poor quality answers and poor use of English.
(The other side of that is that if I could only pass a Turing Test in English, I couldn't pass one in Ukranian, so maybe I'm not sentient in Ukranian?)
That is, I think it's better to err on being hard to convince, rather than to err by being convinced too easily.
> "I'm not saying chatGPT understands everything. I'm saying it can understand many things."
If we see understanding not as a boolean toggle, but as a scale, and different levels in different areas, I think I'm coming round to agreeing with you, it has non-zero understanding in some areas, it has raised the understanding bar off the ground in some areas, there is some glow of understanding to it.
The more I try to argue it, the more I come round to "a human should be able to X" which ChatGPT can actually do. Multiplication - a pocket calculator can do it quickly and accurately but does not understand the pattern. Why doesn't it understand? Because it can't explain the pattern, can't transfer the pattern to anything else. A human can't do mental arithmetic as fast or as accurately as a pocket calculator but can talk about the pattern, can explain it, can transfer the pattern and reuse it in different ways demonstrating that the pattern exists separate from the for-loop that does calculating. See this ChatGPT example: https://i.imgur.com/jc58Fqu.png it has transferred the pattern of multiplication from arabic numerals to tallied symbols and then with minimal prompting, to different symbols. I carried on, prompted it with a new operation called blerp and gave three examples of blerp 10 = 15, blerp 20 = 30, blerp 100 = 150 and asked what was blerp of {stone stone stone stone}. ChatGPT inferred that the pattern was multiply by 1.5, and transferred it to stones, back to numerals, gave me the right answer, in a way that a pocket calculator could never do, but a human could easily do. That's a pattern for multiplying separate from a for-loop in an evaluator, right?
If I say that a human speaking from a Spanish phrasebook does not understand Spanish, and someone who does understands it a little can go off phrasebook a bit, someone who understands it a lot can go as far as you like. ChatGPT can go off phrasebook in English extremely well and very far, and make text which is coherent, more or less on topic, novel, can give word-play examples and speculate on words that don't actually exist.
Does it have 'true understanding', whatever that is, as a Boolean yes/no? No.
Does it have 'more understanding' than an AI decision tree of yesteryear, than a pocket calculator, than an Eliza chatbot, than a Spanish phrasebook, than Stockfish chess minmaxer? ...yes? yes.
Humans don't even have "true understanding" if you define it as understanding everything.
There are things we understand and things we don't.
Same with chatGPT. This is new. Because prior to this, a computer literally had, in your words, zero understanding.
I think the thing that throws everyone off is the fact that things that are obviously understood by humans are in many cases not understood by chatGPT. But the things chatgpt does understand are real and have never been seen before till now.
The virtual machine example I posted is just one aspect of something it does understand and imo, it's not a far leap to get it to understand more.
> I'm not sure how someone can finish this article and not realize that chatGPT actually completely understands what you're telling it.
ChatGPT is a predictive language model. It understands nothing. It simply tries to mimic its training data. It produces output that mimics the output of someone who understands, but does not understand itself. To clarify, there is no understanding happening.
That is why language models hallucinate so convincingly. Because they are able to create convincing output without understanding.
Or maybe they hallucinate so convincingly because they do understand, but they don't understand much? What is this distinction you make "output that mimics the output of someone who understands, but does not understand itself." ?
Imagine you learning a foreign language, the Common European Framework of Reference for Languages (CEFR) grades people at their skill from A1 (beginner) through A2, B1, B2, C1, to C2 (fluent). At the start you are repeating phrases you don't understand imitating someone who does, but you cannot change the phrases at all because you don't know more words and cannot change the grammar because you don't understand it. Call this a chatbot with hard coded strings it can print.
After a while, you can fit some other words in the basic sentences. Call this a chatbot which has template strings and lists of nouns it can substitute in, Eliza style "tell me about {X}" where X is a fixed list of {your mother, your childhood, yourself}. After a bit longer you can make basic use of grammar. If you get to fluent you can make arbitrary sentences with arbitrary words and see new words you have never seen before and guess how they will conjugate, whether they are polite or slang from the context, what they might mean from other languages, and use them probably correctly.
ChatGPT can make new sentences in English, new words, it can make plausible explanations of words it has never seen before - make up a word like "carstool" and it can say something like that word does not exist but if it did it could be a compound word of 'car' and 'stool' like 'carseat', a car with a stool for a seat. Ask it to make up new compound words and it can say English does not have compound words made of four words, but if it had some examples might be trainticketprintingmachine (a machine for printing train tickets). Something that a complaete beginner in a foreign language could never do until they gained some understanding. Something that an Eliza chatbot could never do.
> Or maybe they hallucinate so convincingly because they do understand, but they don't understand much? What is this distinction you make "output that mimics the output of someone who understands, but does not understand itself." ?
ChatGPT is a language model and therefore generates text exactly from start to end, linearly, with each successive token being picked from a pool of probabilities.
It does not form a mental model or understanding of what you feed into it. It is a mathematical model that outputs token probabilities, and then some form of sampling picks the next token (I forget exactly how).
It re-uses the communication of understanding in its training data but never forms new understanding. It can fabricate new words and such because tokens don't represent entire words but rather bits and pieces of them. It sees the past however many tokens for each new token that it outputs so it can mimic nearly every instance of a real human reflecting on what they have already said.
> Something that a complaete beginner in a foreign language could never do until they gained some understanding. Something that an Eliza chatbot could never do.
Because they aren't language models trained on terabytes/petabytes of data. They haven't memorized every pattern on the open Internet and integrated it into a coherent mathematical model.
ChatGPT is extremely impressive as a language model but it does not understand in the same way a human or an AGI could.
Not in the way a human or AGI could, but it does understand some things in some way. Yes it's trained on TB/PB of data, maybe that's why it can. Maybe it's a mathematical model that outputs token probabilities, and that's why it can.
It seems like you're arguing that because it functions in some way, it can't show intelligence or understanding. Arguing that it may look like a duck, quack like a duck, but it's really just a pile of meat and feathers so it can never be a true duck. What am I doing when I learn "idiomatic Python" or "design patterns" or what "rude words" are except being trained on patterns and mimicing other people? I can transfer patterns from one domain to another, so can ChatGPT. I can give an explanation of the pattern I followed, so can ChatGPT. I can notice someone using a pattern wrong and correct them, so can ChatGPT. I can misuse a pattern, have someone explain it to me, and correct myself. So can ChatGPT. I can draw inferences from context from things unsaid or obliquely referenced, so can ChatGPT.
> "It re-uses the communication of understanding in its training data but never forms new understanding."
It gave the wrong answer, I explained in English how to get the right answer, it corrected itself and gave the right answer. That new understanding at least in the short term. If that's "just mimicing understanding" then maybe all I'm doing when I hear an explanation is mimicing understanding.
A trivial Markov chain can't generate anything like ChatGPT can, and that's a difference worth attention.
https://www.engraved.blog/building-a-virtual-machine-inside/
Read it to the end. The end is quite mind blowing. I'm not sure how someone can finish this article and not realize that chatGPT actually completely understands what you're telling it.