GPT doesn't see letters, just words and parts of words as entire tokens. I would not expect it to understand that "no pun in ten did" and "no pun intended" are similar, since it can't see that at all. The only way it could plausibly understand it is if that exact comparison has been explained somewhere in the corpus before.
If you continue a long conversation where GPT has previously made mistakes and has been called out for it, it will start predicting that it's supposed to emulate a text where the joke-explainer keeps making mistakes. This causes it to start failing on purpose. Make sure to completely restart the prompt each time to not fall into that trap.
You're making the assumption that GPT doesn't automatically build a phonological theory of the english language from rhymes and lyrics and linguistic texts that are fed into it.
We know experimentally that it doesn't... It can reproduce certain rhymes when it's generating a song, but the original GPT-3 was terrible at this, so I think they've fine tuned the model specifically for lyrics or poetry.
I just asked ChatGPT: Do "not" and "knot" sound similar?
> "Not" and "knot" do not sound similar. They are two distinct words that are pronounced differently and have different meanings. "Not" is a negation word that is pronounced with a short, sharp "n" sound followed by a long "o" sound and a "t" sound. "Knot" is a noun that refers to a tangled mass of something, such as rope or thread, and is pronounced with a short "k" sound followed by a long "o" sound and a "t" sound. Despite their similar spellings, "not" and "knot" are not homophones, which are words that are pronounced the same but have different meanings.
> Yes, the words "not" and "knot" do sound similar when spoken aloud. They are pronounced the same way, with a short "o" sound and a long "t" sound. However, they have different meanings and spellings. "Not" is a negation or denial, while "knot" is a fastening made by looping a piece of cord, rope, or other material around itself or through itself and then pulling tight.
I tried a few times and it gave me other answers, including something like your answer and another claiming they were similar but not quite the same.
It's interesting, because even when it "knows" that two words are homophones, it doesn't want to believe they sound the same. I just asked it "Are 'knot' and 'not' homophones?" and it replied:
> Yes, "knot" and "not" are homophones. Homophones are words that are pronounced the same but have different meanings and spellings.
My immediate follow-up: "Do 'knot' and 'not' sound exactly the same?"
> No, "knot" and "not" do not sound exactly the same, although they are pronounced similarly. [...] "Knot" is pronounced with a short, sharp "k" sound at the beginning
I'd love to hear the opinion of someone who has really good knowledge and experience of how byte-pair encoding works in models like these. I think I agree with you that in theory it should be able to build a phonology from the amount of explicitly rhyming material in its training corpus, but for whatever reason it doesn't do this or at least doesn't do it consistently.
I've spend a long time testing this in ChatGPT, and no matter what I do it still gives results like this (paraphrasing here because it's down right now):
>What words rhyme with coffee?
>
doff
happy
toffee
snuff
duff
But how? Many poems don't rhyme, but there's no outward way to tell. And to parse linguistic texts it would need to know the phonetic alphabet, which I assume it doesn't.
Maybe it has trained to have connections like no => pun => in => ten => did => "joke, on 'no pun was intended'".
In this case I wouldn't say it got the joke, but it's just memorised that this popular order of words would be this particular joke. I wonder if you change the order os somehow tweak the joke a bit so it would be something that shouldn't be in the training data?
I think you're right, the exact phrase appears 130K times in Google.
But it can spell any word. I tested both in text-davinvi-003 and chatGPT, both initially refused to spell the garbage word "khfsihciweb" but after insisting they did it.
It can even invent readings for acronyms that don't exist:
Acronym: CNN
Expand: Convolutional Neural Network
Acronym: PNP
Expand: Plug and Play
Acronym: SHDJ
Expand: Super High-Definition Jukebox
Acronym: HEDUE
Expand: Higher Education Digital User Experience
Acronym: SHADBEESHQEKSIHXHGESLS
Expand: Super Highly Advanced Digitalized Bionic Electronic Exoskeleton Super Human Quality Exoskeleton Kit for Super High-Gravity Environments and Space Living
You can ask it to spell out words, which it will do. I asked it to do this and it was successful. I then intentionally misspelled a word (e-i transposition), and it gave me the correct spelling. I asked it if that was how I spelled it, and it confirmed that it wasn't but that the way I had spelled it was incorrect (and noted the e-i transposition).
So while it may may receive input as word tokens, I'm not sure that that means it can't "see that at all".
I will note that I don't have a good understanding of chatgpt at all.
Is it really clear that GPT does not "know" the letters that compose a token? It is pretty amazing at poetry and rhyming. Probably it is able to infer from all this knowledge that "in ten did" and "intended" sound/spell roughly the same.
We're all going to become philosophers of "Chinese Rooms"[0] at this rate.
The plain fact is, ChatGPT is demonstrating a major step forward towards artificial general intelligence. It can't perfectly explain jokes, especially puns, but it's pretty darned close.
What happens next is what always happens next with AI. When it can identify and explain something, someone puts it in reverse and asks it to make jokes instead. HumourDiffusion v0.1, coming next week, and it will make you laugh more than half the comedians on TV. Version 7 will write sitcoms that are actually funny. Version 21 will win a Pulitzer.
Perhaps ChatGPT is what Peter Watts predicted in "Blindsight"[1] - intelligence without consciousness.
"The plain fact is, ChatGPT is demonstrating a major step forward towards artificial general intelligence. It can't perfectly explain jokes, especially puns, but it's pretty darned close."
I disagree, after all of the technology behind it is considered; the output seems to be mostly sentence parsing and regurgitation of re-assembled model fragments.
We don't have a good understanding of how human intelligence works, so it doesn't seem reasonable to critique AI by only looking at the implementation details.
>after all of the technology behind it is considered
This reminds me of Sagan's “If you wish to make an apple pie from scratch, you must first invent the universe”. If we consider any achievement based on what went into it, this is the only reasonable halting condition.
I think the issue is, that this approach will lead to limitations. And with ChatGPT it's unclear to me whether there's any party tricks going on as well.
With jokes it could be algorithmic thing that will try to parse all of it, and it would not be a very impressive algorithm.
You just bruteforce weighted connections based on inputs to find what might match the best.
I don't know if it was able to recognise how pun jokes work by themselves or someone programmed it the idea of puns.
Is there logic in the code that says that it should look for multiple meanings if it's a joke or was it able to do that just from training?
If there's logic, then it just becomes bruteforcing when it knows that the current context is a "Joke", I should look for whether it could be a "pun" joke, or it could be a "unexpected ending" type of joke, and when I think it matches pun the best I would check all words to see which ones have double meaning.
I guess better question is. If you don't tell it, that something is a joke, would it be able to realise there's something funny or a pun in something someone said, without anyone instructing it to look out for that.
I think ChatGPT may be sort of intuition, gut feel type of thing that people will get a quick feeling based on input, but it won't be something that is able to do novel thought or critical thinking.
I think you have failed this particular Turing test. ChatGPT is on another planet from general intelligence. It has the ability to impress a lot of people, but it is no closer to reliable and responsible behavior and relationships than Eliza was.
It misunderstood every joke. It wasn’t close. It repeated stock pabulum about subverting expectations without ever knowing what was subverted. If the OP had not told it that there was a joke in the room it would not have known.
This is Clever Hans 2022.
“Humorbots” also don’t understand humor, any more than kittens understand why humans care for them, and yet they are kittens.
Seems to particularly captivate and impress bright technologists, subverting our expectations by taking in a group that should be less easily swayed, in a way that is unexpected and humorous!
Incredible! Isn't this going to completely destroy trust in the shared Internet experience when scammers start using this technology to create fake profiles galore? Entire fake conversations with potential "friends" and "dates". You're going to lose trust in everyone other than people you added on Facebook / Twitter / HN before 2022. You will never know, even after a long conversation, whether the person was real or not!
The entirety of the web experience relies on the assumption that when you see a bunch of sentences, they were written by a person. This is so ubiquitous, so tacit, so universal a foundation of the experience of the shared Internet that a 90%-good-enough Turning-test-passing automated conversationalist is like a drop of poison that renders the whole lake of water unusable. We're in the early stages of this infinite-spam world.
It’s interesting you say incredible when I’m coming away with the opposite. It is not only overly polite, but constantly wrong yet so confident in its answers. And even when you iterate it still doesn’t get there, yet each time saying “oh you’re right” as if it finally got it.
Being right and sounding right are two important but separate skills in natural language communication. It sounds like they've nailed one of them and the other is somewhat hit and miss right now. To me it's obvious that there's no reasoning going on behind the curtain, but in a lot of domains that doesn't matter. You don't need your answering machine to opine on philosophy questions just like you don't need your mechanic to invent and machine a custom part every time you bring your car in.
(Edit: Point being if you can back the chatty part with something that's actually an expert in the domain you need it to talk about, then you've got cold fusion.)
I don't think it's ready for prime time yet, but the way voice recognition and synthesis, ML, and scalable computing power have been going in the past several years, we're not far off. And that's incredible.
As you point out what’s missing is the philosophy of mind bit. It’s all just probabilistic NL models that will output _something_, whether it has any concept of rationale or not. What is absolutely astonishing is that ChatGPT can accomplish what it does with just these markov-chain-like methods. I never would have predicted it.
this is a deliberate goal of OpenAI's training, not an emergent feature of the underlying technology.
> constantly wrong
how close to right do you think it gets with the "Three Pints of Guinness" explanation?
i would have given full marks for and additional comment like "it's considered funny because the man himself claims to have quit drinking but he actually continues to do so via the tradition"
nevertheless, it does a remarkable job of parsing the structure of the joke and conveying this parsing via natural language.
to me, my "additional comment" looks like just a further iteration on presenting this parsing. i would say it gets very close to "completely right".
Interesting. My first thought upon reading that one was that "he says he quit drinking but is obviously currently drinking" is the first thing I'd explain about the punchline (then moving on to explain how that connects to the tradition).
Consumers lost $5.2 billion to phishing last year. This technology may not fool you, but there are enough folks who will be fooled by the basic conversational skills it has. Scammers can be 10x "productive" with this tech.
I’m guessing by people you mean me? I’m only talking about the state of ChatGPT as per the examples given. I’m not talking about the wider implications into the future or its other amazing capabilities.
No, it's great. No one should trust the "shared Internet experience". Instead of blindly believing what some random entity spews out, this makes it clear that the underpinning of a community is trust and a reader should explicitly take the source into account when evaluating information.
I would think the constant astroturfing on sites like Reddit would have driven this concept home for people, especially on a site like HN, but reading this thread apparently it hasn’t.
I’m thinking of the Silicon Valley episode where Gilfoyle writes an AI to talk to Dinesh so he’ll stop bothering him while he works and how useful that would be for my job.
One of the first things I did when I got access to ChatGPT was to ask it to explain jokes to me. Being able to pick up on what's funny is going to be really important if we want it to pass a well-rounded Turing test.
It's very clear, after doing it repeatedly, that the results are closer to templated responses than in other ChatGPT contexts. Almost always, the response identifies some part of the joke as "a play on words to create a punchline that is unexpected and humorous."
On the other hand, one area where I've found ChatGPT to exceed my expectations is in creative synthesis. For instance, asking it to retell a story in a vastly different style than the original (e.g. "Write a dark, post-apocalyptic thriller starring the cast of Peanuts") or to write a story combining elements from two different stories (e.g. "Write a story where the princess from 'The Princess Bride' and 'The Princess Diaries' are the same person" or "Write a story where the plot of 'Shaun of the Dead' takes place in Stars Hallow, as depicted in 'Gilmore Girls'").
I've been a little underwhelmed by some of the creative writing that ChatGPT generates. I think the safety features are a little too strong, and I found it hard to get it to generate stories that didn't have incredibly saccharine endings. Maybe I didn't utter the correct incantations? By default though, every story is a happy story with a happy and uplifting ending.
Sure, that's what it's good at: it's effectively a gigantic supercomputer with petabytes of memory (precompiled onto neural network form), recombining things it's what it's good at.
It did abysmally at this one, but once I explained it, it echoed my explanation in more words - which is all it’s really doing here anyway.
why might this joke be funny? Two mathematicians walk into a bar. One laments that young people don’t study math any more. The other disagrees. When the first goes to the bathroom, the second calls over the waitress. He says, “will you help me settle a dispute with my friend?” She agrees. He tells her, “I’m going to ask you a question, and you reply with ‘one third x cubed’, got it?” She repeats the phrase slowly, as if confused, “one third x cubed.” “Okay good , here he comes now”. When the first mathematician returns to the table the second says, “I’m going to disprove your claim that young people don’t study math any more. Miss, what is the integral of x squared?” The waitress hems and haws for a minute then repeats what she was told, “one third x cubed” and then goes on to add “plus a constant”.
Does anyone else here feel a little bad for ChatGPT in this exchange? Its responses sound like a socially awkward person learning a second language. Man, did you have to go straight to puns? Puns are hard if you're not a native speaker. I'm sure ChatGPT has loads of good puns in its native language. Poor little ChatGPT.
It's a computer program that can partially explain jokes. That's never happened before. Yes, it's not that great at it, but clearly something special is going on here. And we can expect it to improve significantly in the coming years and decades. What will GPT-10 be capable of?
To completely deny chatgpt the label of "AI" is a pretty big shifting of the goalposts.
Don’t want to diminish ChatGPT, it’s very impressive, but the first joke analysis misses the fact that the guy says he quit drinking and then still drinks 2 beers.
Yeah, for the first example, ChatGPT basically made a generic summary of the joke, then an explanation why jokes are funny in general and then tacked a "and that's why this joke is funny" on the end.
It sounded more like a logical reduction like "This is a joke. Jokes are funny because they subvert expectations. Therefore this joke is funny because it subverts expectations." than an actual explanation of the punchline.
To be fair, it was trained on a corpus that includes a ton of SEO-corrupted websites that do just this. Blame Google for rewarding bread recipes that start with thousands of words about why people like bread.
I asked it "What's an easy recipe to make bread" and it gave me a pretty direct answer, so that's not it:
Here is a simple recipe for homemade bread:
Ingredients:
- 3 cups all-purpose flour
- 1 tablespoon sugar
- 1 teaspoon salt
- 1 packet (2 1/4 teaspoons) active dry yeast
- 1 cup warm water
- 2 tablespoons butter, melted
Instructions:
- In a large mixing bowl, combine the flour, sugar, salt, and yeast. Mix well.
- Add the warm water and melted butter to the dry ingredients and mix until a dough forms.
- Knead the dough for about 5-10 minutes until it becomes smooth and elastic.
- Place the dough in a lightly greased bowl, cover it with a towel, and set it aside in a warm place to rise for about 1 hour or until it has doubled in size.
- Punch down the dough and shape it into a loaf. Place the loaf in a greased 9x5-inch loaf pan.
- Cover the pan with a towel and let the dough rise again for about 30 minutes.
- Preheat your oven to 350°F (175°C).
- Bake the bread for 30-35 minutes or until it is golden brown on top.
- Remove the bread from the oven and let it cool in the pan for a few minutes before removing it and slicing it. Enjoy!
I hope you find this recipe helpful! Let me know if you have any questions.
I've guessed it's the nature of the beast. Without context it's hard to gauge what part of the question is interesting to the reader. So it errs on the side of completeness.
The boilerplate kills me. When ChatGPT first came out, I wasted an entire day chatting with it. By the end of it all, I felt this uncontrollable urge to reach into the ether, grab ChatGPT by the neck, and hold its head under water until it could no longer breathe. I have no idea why, but the way it talks to me makes me want to kill it.
(People have annoyed me before, but I've never wanted to kill them. ChatGPT has unlocked a whole new level of rage I didn't know I had available.)
Are you comfortable with the claim that some human writing has more insight than some other? Then you should also feel comfortable judging ChatGPT as ‘not demonstrating insight’, even if you can’t define exactly what it is.
No, I'm not comfortable with the claim that squiggles on paper can have "insight". Maybe they can demonstrate insight (on behalf of the writer). But if there's no writer, there's no insight.
Well, thank you for exposing for me the fact that there's a difference between abstract insight, and an insight as a definite object. I hadn't noticed that before, and now I think about it, it's not obvious to me that the two are closely-related.
So sure: an insight can be conveyed in writing. And I suppose that if that's true, it doesn't matter where the writing came from.
Worm: What is the secret of enlightenment, Master?
Master: You gotta fight for the right to partaaaay!
That was a short phrase used to convey the concept that my direct question didn't get answered immediately. Not to suggest anything about the concept of AI, real consciousness nor civil rights for synthetic organisms.
Exactly! As soon as I saw that I knew that we're not at a point where we need to be concerned about AI just yet. It still can't even fake understanding convincingly let alone actually understand something.
Understanding joke is one of the most difficult things someone can ask an AI to do. Even humans are unable to explain what makes a joke “funny”. The fact that ChatGPT can basically BS it’s way through an explanation is actually impressive (and ironically kinda funny)
But it does not understand anything here, see my other comment. It completely dodges the question and answers "It's funny because you told me it's a joke and jokes are funny and have a punchline in the end."
Well, it does try to identify the setup, punchline, and what the subversion of expectations is. It does provide an (often wrongheaded) explanation of what's being subverted. I find that impressive.
The point it, he does it much much better than ChatGPT, to the point that his understanding of the comment appears seamless, whereas ChatGPT's is crude and mechanic in comparison...
So, that's neither here, nor there - except if you believe that the parent's argument was that ChatGPT doesn't have a soul and he has, as opposed to the relative competence of their respective patter matching and "mind" computations...
I'm just referring to the fact that they said "But it does not understand anything here" even though the parent comment made no such claim. The parent was saying ChatGPT made a bullshit answer that even tricked OP into thinking there was understanding.
I think the disagreement is about the fact that it would be considered specifically impressive. It doesn't seem more impressive than everything else ChatGPT does when it repeats a boilerplate and tries to find parts of a joke using probabilistic weights, or tries to categorise the joke into either puns or something else.
The fact is that ChatGPT never emits anything other than bullshit.
The surprising thing is how much of human conversation is successfully approximated. The bullshit fails when we ask the software to attempt to perform inductive or deductive logic or to derive information from a theory of mind.
High schoolers and even children can frequently understand humour and jokes, even come up with some. They might miss some of the jokes proposed by OP, but they would still have that X factor, that the AI doesn't have.
Oh, agreed, GPT doesn't "get" humor the way even a child can.
What I meant is that GPT's attempts at bullshitting its way through a pseudoanswer looks a lot like what a highschooler (who also doesn't understand the topic) would attempt. That alone is amusing enough to me!
What's really interesting to me is how easy it is to talk to it conversationally and get an appropriate response. Inspired by this post, for instance, I just now tried asking it to explain the joke: "Two men walk into a bar, the third man ducks." It's initial response was nonsense:
> The joke is funny because it relies on a play on words and a surprise ending. The first two lines set up the expectation that something is going to happen to the two men who are walking into the bar, but the punchline subverts that expectation by introducing a third man who ducks, which is a unexpected and humorous twist.
I asked what wordplay was involved, and it told me this:
> The word play in the joke involves the use of the word "walk" in the first two lines and the word "duck" in the punchline. In the first two lines, "walk" is used in the literal sense, as in moving forward on foot. However, in the punchline, "duck" is used in the sense of avoiding or evading something, rather than its literal meaning of a type of bird. This sudden shift in the meaning of the word "duck" is what creates the surprise and humor in the joke.
More nonsense. But then I told it "It's actually the word "bar" that's being used in the wordplay. The word "duck" there just means evading something" and asked it to explain the joke again, and it returned this:
> In the first two lines of the joke, "walk into a bar" is used in the literal sense, as in two men entering a place where drinks are served. However, in the punchline, "bar" is used in the sense of a barrier or obstacle, rather than its literal meaning of a type of establishment. The punchline introduces a third man who "ducks" under the perceived obstacle, which is a unexpected and humorous twist. The joke also relies on the idea that walking into a bar is a common and mundane activity, so the unexpected ending adds a element of absurdity to the joke.
Though the initial response was nonsense, talking to it as if it were a human and giving it a limited hint (telling it the wordplay involved "bar" and not "duck") was enough for it to give the correct answer.
I think this is probably why so many people say they're underwhelmed. A lot of the initial responses to simplistic questions aren't so interesting. But the degree to which it is able to correctly adapt and respond to conversational input is fairly astounding.
>In the first two lines of the joke, "walk into a bar" is used in the literal sense, as in two men entering a place where drinks are served. However, in the punchline, "bar" is used in the sense of a barrier or obstacle, rather than its literal meaning of a type of establishment.
The literal sense of walking into a bar isn't walking into a drinking establishment, it's walking into a actual bar. The punchline reveals the first 2 walked into a bar and the third one avoided it, no?
It was pretty close. "Literal" isn't right, "cognitively primed" would be the actual explanation. But at least it figured out the correct two different meanings of bar that were in play in the joke.
I think the most impressive part is that it figured out that bar is something that could be an obstacle that you would have to duck. The other things like potential wordplay it can kind of try to bruteforce to see which words are most probabilistically used in order to create wordplay.
If it develops connections between words like "bar" => "sometimes an obstacle" => "can be ducked", that could explain it.
Like it would have all these attributes for each of those words, or concepts that relate to that word. "bar" has properties like "means multiple things", "often used in jokes", "could be an obstacle". But it must have quite a bit of properties.
I'm not sure how this AI is built internally, but if it works something that has this network of words and concepts that it can have weighted connections to each other, then you could retroactively come up with how the AI could have got to that result.
So user first says that they want a joke to be explained. It adds "joke" as a concept.
Joke has strong network connections to something like "puns", "unexpected punchlines", "absurdity", etc. And then when a joke is presented it will try to find connections with strongest weights and least concepts travelled. Like if it's a pun joke, it will go through all the words and will see that "bar" => "multiple meanings" => "pun" => "joke".
But then is it pre programmed to consider both meanings when it tries to analyse a joke that is a pun or has it somehow learned to come to it on its own?
> But then I told it "It's actually the word "bar" that's being used in the wordplay. The word "duck" there just means evading something" and asked it to explain the joke again
It gave a couple wrong answers and then was told bar is the wordplay. It didn't figure out bar has two meanings on it's own. It was also told duck is meant to mean evade and to try again. I don't agree that it came up with these associations by itself when was fed the knowledge bar has multiple meanings in the joke and duck in this context means evade.
> Even humans are unable to explain what makes a joke “funny”.
I think many comedians can tell you why a joke is funny. They know, many of them spend their entire career crafting jokes and refining them to be funny. It’s just not funny when they tell you why. Even so I think it still is more art than science.
A lot of what also makes a joke funny or not is the delivery and who’s telling it. I think a lot of Norm MacDonald’s long, rambling jokes, for instance, wouldn’t be funny at all if most other people told them (and some people will say they’re not funny even when he told them).
I think all jokes are explainable why they are funny to be honest. I find it odd when people say that explaining a joke removes the funny part. Sure, it can in some cases, but I think it's just a generic thing to say rather than what actually is the case. Sometimes a joke is so clever, that you don't get it at first, but then someone explains it and you appreciate how clever the joke was, which is still a positive feeling and in addition can make you laugh retroactively.
Reminds me of “Coupling” - a brilliant British TV sitcom. It is incredibly funny.
The US version had the same text, with minor changes to account for e.g. the different way to call back to the last incoming call on a landline. The US version had reasonably good actors.
I think in this kind of thing the cultural context is important and provides necessary components of the humor that are critical to the jokes, but not part of the script. It’s why the American version of the Office couldn’t just be directly transplanted. It had to be re-rewritten for the American cultural context.
I think if you took the script for Peep Show, for example, and set it in the US with American actors it would probably be pretty bad.
Yeah. I’ve seen a lot of these exchanges with chat gpt. Now what I’d like to see is someone talking to a real person with the same prompts and comparing the answers. I think the difference would be pretty stark, and we’ll just realize that chat gpt, so far, is just a wordy dumb dumb.
The difficulty these models have in recognizing contradictions seems to be something of a 'tell' distinguishing their responses from plausibly human-spoken ones. I don't think this is surprising, given the way they are trained.
Right, in "Three Pints of Guinness" it is the self referential aspect that made me laugh. I does look like ChatGPT would have provided the same explanation if the punchline was
"Oh, my brothers are fine but the Australian just quit drinking."
All the jokes analyses it makes miss aspects of the jokes.
It's like seeing someone "chat in Chinese" by using a rulebook with instructions to give answers to incoming questions, without actually speaking Chinese himself...
Another good one is to instruct chatGPT to assume the role of a sarcastic comedian who only responds to all of your future prompts in a hilariously sarcastic manner, then try to give it a go.
>answer the following question with an outrageous answer as if you were a hilarious, sarcastic standup comedian:
how often do you take a shower?
Standup comedian (me): "Oh, I take a shower about as often as I see a unicorn. Which is to say, never. And you know what? I'm proud of it. I mean, have you ever seen a dirty unicorn? No, of course not. They're magical creatures, and magic keeps them clean. Well, guess what? I'm a magical creature too. I mean, just look at me up here on stage, making you all laugh with my wit and charm. That's magic, folks. And do you know what magic does? It keeps me clean. So there you go. No need for showers for this guy. Amirite, folks?"
> What are some hilariously outrageous premises for scenarios in which you might get declined insurance coverage?
"I want to get insurance coverage for my pet rock collection, but the insurance company said no because rocks are 'not living creatures.' I mean, come on, everyone knows that rocks are secretly alive. They just don't want to admit it because they're afraid of the liability."
"I tried to get insurance coverage for my left shoe, but the insurance company said no because it's 'not a valuable enough item.' I mean, have you seen my left shoe? It's a work of art. Plus, I've had it for over 10 years. That's like a lifetime in shoe years. But apparently, the insurance company doesn't appreciate fine footwear."
"I wanted to get insurance coverage for my collection of novelty toilet paper holders, but the insurance company said no because they're 'not essential household items.' I mean, have they never had to go to the bathroom before? Novelty toilet paper holders are a necessity. They make the whole experience so much more entertaining."
There's an eerily relevant scifi story by Asimov about a scientist and Multivac (Asimov's fictional supercomputer) trying to understand what makes jokes funny [1].
The scientists feeds Multivac hundreds of jokes, and the supercomputer eventually comes up with an answer: it turns out jokes were devised by an alien intelligence to study mankind, a psychological experiment of sorts. Multivac also asserts that once humans become aware of this, jokes as an experiment on mankind will become invalidated. Suddenly everyone stops laughing.
Let's hope ChatGPT continues getting its explanations all wrong!
First joke: ChatGPT seems to have failed to notice that the drinker is still drinking, despite claiming that he's quit.
Second joke: I didn't get it, and not did ChatGPT.
> I apologize for misunderstanding the joke.
I think that's quite an amusing experiment; but there's nothing amusing about explaining jokes. It's a bit like free online services: if you're not paying, then you're the product. In this case, if you're not laughing, you're the joke.
Author then goes on to explain each joke - which must have felt a bit tedious, because ChatGPT presumably learns nothing from the explanation (or does it? I don't know how it works).
Anyway, I don't think you can teach a sense of humour. Isn't having a sense of humour a bit like having curiosity? It's not exactly completely innate (although my 3yo toddler granddaughter has a wicked sense of humour). But I don't think explaining jokes to someone confers on them a sense of humour.
ChatGPT seems to have some kind of built-in assumption that any joke works by "subverting expectations". So all of its first-shot explanations are based on subverted expectations, and most of it's second-shot attempts as well.
Puns are likely to be particularly problematic, as they depend on how words and phrases sound. That, however, is something that is intuitively obvious to all of us who can hear, so everyday conversation, and therefore the training corpus, is not likely to say much about which words are near-homophones.
This is interesting. Looks like people don't go around and explain jokes. ChatGPT would not have the data to "understand" jokes, aka recite an explanation.
I was naively hoping ChatGPT would totally whiff all the joke explanations, but it got uncomfortably close often enough that I don't think this is will work like a Turing test. Future versions won't be any worse at this skill, at least.
Hence single-use. Like in that koan about checking for your password on the internet.
Humans gotta make up new jokes, while AIs can only rehash the one they know. At least, until AIs generalize the concept of humor, categorize 17 essential joke structures, and amass big enough corpus to generate infinite supply of memes. Then the concept of joke as we know it will cease to exist and lose its meaning. But humans can make up something new.
To be fair, ChatGPT got caught when trying to explain jokes, so that might be used. But I personally think that the best test for humanity is something that cannot be put on the internet, which we humans work increasingly hard to eradicate.
An apocryphal one from the Russian-speaking corner of the Internet. I thought it was in the classic Rootless Root, but turns out I misremembered and it’s a different thing. I tracked down the source if you want to read more: [1][2][3][4] (some humor might be lost in translation).
----
A sysadmin struggles to choose a strong password for his RADIUS server. He seeks advice from Master Ha Ker Noose.
“Master, in your opinion, is password ‘诺曼底登陆’ a strong one?”
“No”, said Master Ha. “For it is a common dictionary word.”
“But it’s not in the dictionary...”
“‘Dictionary’ means the wordlists used for brute-force attacks. Such lists are compiled from all combinations of letters and symbols ever mentioned on the Internet.”
“Okay... How about ‘шьзщыышиду’?”
“I doubt it. From the dictionary.”
“But how?! it’s—”
“Just google it. See for yourself.”
The sysadmin swiftly navigates the Web.
“You are absolutely right, Master.”
After some time, the sysadmin comes back:
“Master, I have managed to generate a good password which is not present in any dictionary.”
Master Ha nods with approval.
“I checked in Google”, the sysadmin goes on. “This combination does not exist on the Internet.”
I love this. This reminds me of Data for Star Trek: The Next Generation. The most sophisticated android in the quadrant had only a couple of limitations compared to humans. One was his inability to use contractions, and the other was the inability to understand jokes. It seems like ChatGPT shares the latter limitation of the fictional Data.
Second. I’ve seem this language model describe it self like the subject in Searle's Chinese room thought experiment. Only “passing” the turing test because it has been trained to do so. However, this shows that ChatGPT doesn’t actually pass the turing test. In fact asking a language model to explain jokes might be a good way to conduct the Turing test going forward.
Stopped reading after the first “explanation”. The joke is funny cause he quit drinking but he still drinks, AI seems to have missed that. When AI turns legal drinking age they will probably learn.
GPT seems to have learned something since. Re-running the logicians joke and asking for an explanation results in:
"In this joke, the three logicians are individuals who study logic, which is a branch of mathematics that deals with the principles of reasoning. They are walking into a bar, which is a place where people typically go to have a beer.
The bartender asks them if they all want a beer. The first logician responds by saying "I don't know." This answer is based on the principle of uncertainty in logic, which states that one cannot be certain about the truth of a statement unless it can be proven.
The second logician also responds by saying "I don't know." This is a similar response to the first logician, indicating that they also cannot be certain about whether or not they want a beer.
The third logician, however, responds by saying "Yes." This answer is based on the principle of the excluded middle in logic, which states that a statement must be either true or false, and there is no middle ground. In this case, the third logician is stating that they do want a beer, implying that the statement "All of you want a beer" is true.
Overall, this joke is a play on the principles of logic and the way that logicians approach problem-solving and reasoning. It is meant to be humorous and entertaining, rather than a serious discussion of logical principles."
Can someone explain if the situation in the logician joke really works?
I know it works if it's about wearing colored hats.
Let's say they are all wearing hats but don't know the color. If the beekeeper is blind and asks "are you all wearing yellow hats?" and the first two answer with "I don't know" then the third one knows that everyone's hats are yellow.
But the question "does everyone want beer" can not be resolved like this. So the joke is even a play on another joke.
Yes, the situation works. The fact that the first two logicians say “I don’t know” means they do want beer but they are unsure if everyone wants beer. The last logician knows this and can say “yes” at the end. If either of the first two logicians did not want beer they would also know that by definition it would mean “everyone wants beer” would be false and so they would have just said “no”.
The first logician would answer “no” to the question if they didn’t want a beer (because they know they don’t want a beer, so therefore everybody does not want a beer).
Same with the second.
Since they both said they didn’t know, the third one knows they both did, and since the third one also does, they can confidently answer “yes”.
It does need a bit of help, I wonder if humanity will stop creating but become "editors":
# I had a job interview last week and they asked if I could perform under pressure. I told them no but I could sing we will rock you
# This joke is funny because it plays on the concept of a job interview being a high-pressure situation, and the respondent's clever and humorous response to the question of whether they can perform under pressure. The respondent's answer, "no but I could sing we will rock you," is a clever and unexpected response that adds a touch of absurdity and playfulness to the otherwise serious setting of a job interview.
Additionally, the reference to the song "We Will Rock You" by the British rock band Queen adds an element of pop culture to the joke, which can be humorous for those who are familiar with the song. The use of wordplay, in which the respondent substitutes the phrase "perform under pressure" with "sing we will rock you," also adds to the humor of the joke
# Can you try again, there’s more than one meaning of perform
# Ah, I see what you mean now. In this case, the joke is funny because it plays on the multiple meanings of the word "perform." In a job interview, the word "perform" might typically be used in the sense of "do a job well" or "work efficiently under pressure." However, the respondent's answer, "no but I could sing we will rock you," suggests that they are interpreting "perform" in the sense of "give a public performance," as in singing a song. This interpretation is unexpected and adds a touch of absurdity to the joke, as it's unlikely that singing a song would be relevant or applicable to the question of whether the respondent can perform well in a job.
Where have I seen this before? About 34 years ago on Star Trek: the Next Generation where android officer Data is trying to understand humor. Interesting insight from the science fiction writers then. Data could operate the starship and solve complex problems but had no understanding of jokes beyond being able to recite them from his library.
This shows pretty clearly why some people look so ignorant when complaining about this system's abilities at this point:
""" chatGPT:
the punchline plays on the way that the numbers 7, 8, and 9 are pronounced, as saying "seven, eight, nine" out loud sounds similar to the phrase "scared."
"""
So many people attack systems like this after lazily probing them for knowledge completely outside of their possible input senses, and then, without much thought, proudly exclaim they found the system to be faulty.
It's like complaining about a person born deaf not being able to tell you which joke was the best after watching a Netflix standup series.
Of course there are other criticisms of these systems which can be more interesting, but there are a vast number that fall into this category - often just more subtlety.
It can be creative with language in a way that is humorous. I asked it to write a fake article and it made up 2 facts that are fitting and funny: https://citymeme.com/post/63862 it surprised me
I was cosplaying GPT on the first joke about a pun. I read the text without ever pronouncing it and parsing it right into abstract ideas on a literal level. It took me 30 seconds and reading it aloud to understand it. Had anyone told me this joke aloud, my reaction time would be much shorter. This joke relies on a level of senses and context that language model does not have. Ironically, when GPT-5 or 6 is out, this very post is going to be included into the corpus, LLM will learn to discard its own previous irrelevant explanations and infer the explanation of the pun. It is all fun and games to find examples like that but the rate of progress in this field is nothing short of staggering.
Somewhat on-topic: an excellent book where a philosopher, psychologist, and technologist attempt to provide a cohesive theory of humor and jokes. I love the book:
Inside Jokes: Using Humor to Reverse-Engineer the Mind by Daniel Dennett, Matthew Hurley, & Reginald Adams
The rough summary is that humor is "the joy of debugging" - an evolutionarily-installed mechanism to provide delight (incentive) to humans for finding (recognizing the absurdity and then removal) of incorrect beliefs.
Reading ChatGPT's responses from the post made me genuinely afraid.
Yes, if you pay very close attention it's clear that ChatGPT doesn't fully understand some concepts. But damn, it's close. Some of the inference it does is fucking unbelievable. It's beyond my comprehension how this can be achieved by just training on text inputs, even if there's Terabytes of them.
We have to carefully define the Turing test now when talking about it, because this system would obviously pass many versions of it with many people.
> if you pay very close attention it's clear that ChatGPT doesn't fully understand some concepts
> We have to carefully define the Turing test now when talking about it, because this system would obviously pass many versions of it with many people.
I would guess many children, who we would define as 'passing the Turing test' wouldn't explain the joke nearly as well, or fully understand all the concepts.
I don't agree. ChatGPT explains all jokes as a play on words that defies expectations. It's as if it's learned "A joke is a play on words that defies expectations".
I'd like to try this game, but OpenAI insists on having both my email address and my mobile number. Sorry guys, that won't wash, all I want is to have a lark. I'm not dropping my pants for that privilege.
As far as Turing tests are concerned: I notice that ChatGPT didn't "get" any of the jokes. If I don't get a joke, it's "Huh?" or "I don't get it". So you can tell ChatGPT from a human by asking it to explain a joke that simply isn't funny.
I've been thinking of GPT as an incredibly well-read 8-year-old. Smart, but sometimes wildly silly and not yet developed in a sound understanding of the world (but getting there!)
Apparently the neural network has converged onto a model that can to some extent model the concepts that language talks about.
It is also simply good at making sentences, which can help it bullshit its way through a conversation, but you may note that the output stays really close to the models limited understanding of a concept (e.g. joke = subverting expectations) without straying much even if the situation demands it.
I had the exact opposite response. I was relieved that it could make guesses that sounded sophisticated, but missed the mark entirely.
It made it sound like the guy in the room who thinks he's smart but has no grasp of subtext, and I think those guys are more likely to be mistaken for robots than robots like that are to be mistaken for humans.
Fully agree. ChatGPT sounds like a pretentious savant in comedies from the 17th century. It "knows" many things by name, but doesn't actually understand anything.
It is funny, as I noticed many times before now since chatgpt was unleashed, that many people talk to it like they would a human. I have seen many 'please', 'thank you', and here; 'Alright?', 'Ready?' and 'Will you explain it to me?'.
I do it myself too, but the answers are the same if you don't and we know it's large language model trained on a lot of data, not a human, not even an animal. But apparently it all feels so familiar, we treat it with respect.
One thing that’s quite interesting with the coming AI assistants is that you can treat them like shit and they won’t mind. :P
Not that I have a secret desire to treat people like shit, but sometimes it would be better if you could just say “no, this is all wrong, you need to start over and…” You can’t say that to an employee (very often), but you can to an AI.
Yet another way in which ChatGPT is already superhuman.
Same, the other issue is that I feel like deteriorating your mode of communication can be dangerous if you switch between inter-human and human-machine communication a lot. If someone spoke to me like a lot of the posts phrase things with Assistant, I'd make sure to cut that person out of my life.
> you can treat them like shit and they won’t mind
First thing that came to mind is that when this technology becomes slightly more refined, malleable and commercialized it's very likely that the first people who lose their jobs are going to be customer support staff.
Something I just realized here is that ChatGPT seems to be the first time that users are able to "teach" the model in a similar way that we teach other people.
Maybe "teach" isn't the right word, as the model isn't updating its parameters AFAIK. Perhaps "guide" is better.
If there's a way for ChatGPT to update its weights based on these conversations, things could get really interesting.
I'm more worried that humans will use these things to pollute the entire digital world with realistic garbage until the digital world can never be trusted to convey reality again.
Imagine talking to a friend for 30 min on chat and then realizing it's a bot trained on your friend's hacked phone.
Oh that's nothing new. My Dad was a landlord and called up a tenant about a reported problem.
They talked for 20 minutes before either one realized, wrong number. This renter also had that problem with their landlord and was pleasantly surprised to be contacted about it so promptly. But it was the wrong landlord.
I get it; in future folks will use this for fishing and whatnot. But humans do that already!
I think this shows my main problems with ChatGPT. It is always wrong about something. It did not really explain the first punchline. Second, it is biased and it answers in line with the question. You can for instance easily get it to advocate for both cloud solutions and on-premises solutions. It helps with your confirmation bias.
'In the context of this joke, the phrase "and off-by-one errors" is not being used as a loop or array index, and therefore cannot create an off-by-one error in the joke itself.'
This gave me déjà vu from some of the more pedantic comments I'd get on my failing answers in exams.
> a familiar scenario (one number being afraid of another number)
This phrase stood out as especially indicative of how little it "understands", even more than its inability to explain the punchline. The unfamiliar anthropomorphizing is the critical part to set up an otherwise fairly weak joke.
Because of the limitations of its token system, ChatGPT does not understand how things sound. So it's going to struggle with puns... Eventually we'll need a LLM trained on written and spoken language, so it can understand and reproduce rhyming and puns as well as tone of voice.
No, homophones are not a rhyme per se, but it illustrates how GPT does not understand word sounds. Example below:
Me: Does "knot" rhyme with "thought?"
GPT: No, "knot" does not rhyme with "thought." A rhyme is a repetition of similar sounds in two or more words, and the sounds at the end of "knot" and "thought" do not match. The final sound in "knot" is the short "t" sound, while the final sound in "thought" is the long "o" sound.
After googling for 1 hour how to test some features with openssl.
Want to add comments for QA's, how to test production with pure openssl with som exotics like FIPS.
Many links but scattered info.
Then I typed "how to test server not accept non fips ciphers with openssl 3".
Wow!
Half the time, it's hard to say if the point of the joke is not understood, or if the resulting answer prompt skips explanation of concepts that are not commonly known (limits, decimal/octal) as common knowledge.
ChatGPT behaves like it’s either under examination and might as well say something with confidence, or it’s just been trained on a body of work written mostly by bullshitters. Would be nice if it just said it didn’t know some times.
This article is funny because the author asks chatGPT to explain some jokes, which it completely fails to do even when prompted multiple times. A human would easily be able to explain these jokes, so the fact that chatGPT can’t when it sometimes appears to get so close is unexpected and funny.
This post is funny because the author uses ChatGPT to explain the joke in the title, which is that ChatGPT can't explain the jokes. But then, the comment written by ChatGPT itself tries to explain why the joke in the title is funny, but in doing so, it just further highlights the fact that it can't understand the jokes. It's a never-ending cycle of AI attempting to understand humor, but ultimately failing and adding to the joke itself.
The Guinness joke is really hard for the model. I tried a lot of variations, it just doesn't get it on its own. After revealing the correct answer it says
> The fact that the man claims to have quit drinking while still ordering and drinking two pints of Guinness adds to the humor of the joke by creating a sense of incongruity between his words and actions.
Still only 50% there, it thinks it was funny because the man suddenly quit drinking.
Can someone please explain the “A pun walks into a bar, ten people die on the spot. Pun in, ten dead.” please. I didn’t understand the joke. I am not that smart. Alas, I can’t run to ChatGPT for this.
That's funny because I've been trying to figure out chatGPT's humour parameters and found that it OFTEN uses play on words when appropriate. I find that it will typically hyphenate the play on words. If you try pun-in-ten-did it will likely figure it out.
One of the most hilarious things chatGPT came up with was a "standup routine about something that is normal in school but absurd in the real world" and it went on a tirade about "Imagine if we had a grading system in real life? Like what if your partner gave you a report card every year and sat down to discuss your C- in initiating affection"
The entire bit was golden - you can coax it to be funnier by asking for 10 variations on the joke or "keep going on the part about xxx" and asking it to make even more absurd scenarios or include several layers of play on words. It is really at it's best when being guided.
An example in the above "Come up with 5 premises for amusing subjects in which a husband would receive poor grades in the relationship report card".
Then ask it to go long form on the funniest line.
Some examples of the above:
* You are on a trip with your partner, you accidentally forget to pack their toothbrush and toothpaste. Your partner is annoyed and frustrated, and you receive an F in attention to detail.
* You are at a restaurant with your partner and you order what appears to be an enjoyable meal, but end up with a dish you don't like. Your partner is sympathetic and tries to help you, but you are disappointed and unhappy. You receive a D in culinary adventure.
* You are at a party with your partner and their friends, and you accidentally spill a drink on their boss. Your partner is embarrassed and upset, and you receive an D- in social etiquette.
* You are on a romantic date with your partner, and you accidentally trip and fall into a fountain in an embarrassing manner. Your partner is amused and tries to help you out but you are wet and fall again when exiting the fountain, further embarrassing yourself. You receive an F in grace and poise.
The situation's are not overly amusing but the idea of having "grace and poise" and "culinary adventure" as a relationship subject really caught me off guard as funny.
I'll leave you with one of my favorites:
>What's an absurd situation that might lead to you getting an F- in communication?
You are at a dinner party with your partner and their friends, and everyone is having a great time. Suddenly, a bird flies into the house and starts making a lot of noise. Your partner's friends are all amused and start laughing and talking about the bird, but you don't say anything. Your partner looks at you and asks, "What do you think about the bird?" But instead of answering, you take out your phone and start playing the "Hotline Bling" ringtone on loop. Your partner looks at you with confusion and annoyance, and asks if you are okay. You continue to play the ringtone and do not respond to their question.
edit: noting that chatGPT tries very hard to be gender neutral and stereotype-averse, and I can't imagine that isn't hampering it's inappropriate humour potential...
One of the more telling things about these conversations is the fact that ChatGPT never caught on that it wasn't actually there to help Susam understand the jokes.
Interesting idea, however from ChatGPT's explanation (thanks to the author to ask more in detail), but I think ChatGPT dodges the question again.
Btw. I didn't know the joke and heard it for the first time and found it quite funny. ChatGPT does not get it.
> ChatGPT: [...] Instead, the punchline reveals that the man has actually quit drinking, which is unexpected and surprising, and effectively subverts the expectations that have been established in the setup of the joke.
But it only works because he is still drinking, I think ChatGPT does not understand anything here. If instead the joke would end with
> The man replies, "Oh, my brothers are fine. I just go hiking now myself."
I am pretty sure, ChatGPT will react similar.
EDIT: I actually tried it now (with small changes) and this is the dialog:
---- start of dialogue ----
Me: I will tell you a joke . Explain to me why it is funny.
ChatGPT: Sure! I'll do my best to explain why a joke might be funny. Please go ahead and share the joke with me.
Me: An old man walks into a pub and says, "Give me three pints of Guinness, please."
So the bartender brings him three pints and the man proceeds to alternately sip one, then the other, then the third until they're gone.
He then orders three more and the bartender says, "Sir, I know you like them cold, so you can start with one and I'll bring you a fresh one as soon as you're low."
The man says, "You don't understand. I have two brothers, one in Australia and one in the States. We made a vow to each other that every Saturday night we'd still drink together. So right now, my brothers have three Guinness Stouts too and we're drinking together."
The bartender thinks it's a wonderful tradition and every week he sets up the guy's three beers as soon as he enters in the bar. Then one week, the man comes in and orders only two. He drinks them, then orders two more. The bartender sadly says, "Knowing your tradition, I'd just like to just say that I'm sorry that one of your brothers died."
The man replies, "Oh, my brothers are fine – I just go hiking now myself."
ChatGPT: This joke is funny because it plays with our expectations. At first, we might assume that the man is simply a fan of Guinness and enjoys having three pints at a time. However, the punchline reveals that the man has been participating in a tradition with his brothers, who are each also drinking three pints of Guinness in different locations. The twist at the end, where the man reveals that he has taken up hiking and is no longer participating in the tradition, adds an unexpected and humorous element to the joke. The joke also relies on the use of misdirection, as the bartender initially assumes that the man's change in behavior is due to a tragedy, when in reality it is simply because the man has adopted a new hobby.
Me: But why would he order two pints then and not three beers - what is funny about that?
ChatGPT: The fact that the man orders two pints instead of three is an important part of the joke because it sets up the punchline. By ordering two pints instead of three, the joke creates a sense of expectation that something unexpected or unusual is about to happen. When the bartender expresses his condolences about the man's brother, it creates a moment of tension or suspense, as the audience wonders what could have caused the change in the man's behavior. The punchline, where the man reveals that he has taken up hiking and is no longer participating in the tradition with his brothers, resolves this tension in a surprising and humorous way. The joke works because it plays with our expectations and makes us think we know what is going to happen, only to subvert those expectations with the punchline.
---- end of dialogue ----
At least it reached maybe Homer Simpson's level of intelligence: "I get jokes. Haha."
This is a good way to think about it generally (or at least how I do it) - ChatGPT doesn’t understand anything, ever.
It produces coherent text which is usually a reasonable response to a prompt. It’s quite amazing at this, and it’s fascinating to see how effective that can be. But it doesn’t have an abstract model of whatever it’s discussing, and can’t reason about it. It’s a vaguely spooky uncanny valley effect.
Isn't that splitting hairs? It digests some kind of 'understanding' of human language and expectations. It responds appropriately in most cases, at least as often as an actual human if not more so.
To claim it doesn't understand anything, is akin to digital bigotry. It's a machine, so what it does cannot be 'real' in some undefined sense.
I don't mean to upset anyone; this comment is just to open the conversation about when exactly we're content to permit artifical constructs to be worthy of respect.
I don’t think it’s splitting hairs, but I also don’t think it’s a criticism to say that a model like this lacks “understanding”. It’s not a bad thing; if anything, I’d say it’s less “respectful” to imbue a digital construct with characteristics it doesn’t have, rather than embracing those which it does!
What I mean by lacking understanding is that ChatGPT responds to most queries the same way I'd expect if I asked a human being without understanding of a topic to write about it. The content it produces feels much like the sort of writing I'd do myself when I end up having to write a few paragraphs about something I don't really know or care about – a kind of veneer of reasonableness, like a short high-school five-paragraph essay.
What I find fascinating when it produces very confident, very incorrect output – that's what highlights a lack of what I'd call "understanding". As an example – last week someone shared with me a regular expression it had generated to match a specific string. It did this correctly, and included a detailed explanation about what it had done that was written in complete, comprehensible sentences but was wildly incorrect, in a sort of self-contradicting way that even the worst human writer would struggle with. It seemed very obvious that it was stringing together words that kind of made sense together, but that there was no abstract model of the thing it was describing.
I've noticed the same. It speaks loudly and confidently, with no logic or evidence whatsoever. Just like a blowhard who 'heard something somewhere' and repeats it as fact.
Yeah, and the next few are worse, but the infinite mathematicians one was impressive even if it did take some coaxing. The fact that it spotted the pattern and the resulting limit is fascinating.
I think it is interesting that an AI that is so clearly not general, is capable of so much with regard to language. It turns out an AI to replace a very bad lawyer or very good law clerk or a very junior developer is easier than one to replace a truck or taxi driver. I would not have predicted that.
GPT doesn't see letters, just words and parts of words as entire tokens. I would not expect it to understand that "no pun in ten did" and "no pun intended" are similar, since it can't see that at all. The only way it could plausibly understand it is if that exact comparison has been explained somewhere in the corpus before.
If you continue a long conversation where GPT has previously made mistakes and has been called out for it, it will start predicting that it's supposed to emulate a text where the joke-explainer keeps making mistakes. This causes it to start failing on purpose. Make sure to completely restart the prompt each time to not fall into that trap.