
Like an AI Could Ever Spot Sarcasm - bcaulfield
https://blogs.nvidia.com/blog/2018/01/31/ai-detect-sarcasm/
======
dmreedy
This is neat.

Sarcasm is hard because it's a violation of the cooperative principle. I think
describing it as simply incongruity is understating the difficulty of the
problem (that said, good enough for 80% accuracy on the task is nothing to
sniff at).

In communication, you've got two systems attempting to convey their internal,
unobservable states to eachother. This happens across a number of different
channels, each noisier and more imperfect than the next. Natural Language is
lossy enough compression as it is. Now add idiolect, prosody, body language,
context. And on top of that, the internal states of the two systems may only
be alignable along the coarsest axes to begin with (do we share the same
qualia?). The only way we can have any hope of understanding eachother is when
we know both sides are trying their best to make themselves understood, as per
Grice's maxims.

Sarcasm is a form of dissembly, a deliberate subversion of the cooperative
principle, that words mean things, for other means. Sometimes to communicate
to others, who are not the target. Sometimes for the communicator's own
personal satisfaction. Sometimes it's meant to be understood, but the drier
the sarcasm, the fewer hints are given.

And at the same time, in some cases, a sarcastic utterance is the most concise
way of conveying information, if it is correctly received. Because acting in
violation of typically observed rules of conversation carries its own set of
semantics. It's another mode of communicating, and all of these different
modes capture different lower-dimensional slices of the hyper-thing of mind-
state and intent that they're purporting to convey.

All that to say, that elsewhere in these comments, the problem has been
described as AI Complete. I'd go a step further, and say that it's telepath-
complete, to know the intent of another, and even then, knowing the accurate
truth of the thing by oracle might end you up with less Information than if
you'd just gotten the joke in the first place.

~~~
perl4ever
That's a very good summary. I would say based on my experience on the
internet, that spotting sarcasm is frequently beyond human capability, so it's
not realistic to do it with an AI any time soon.

Besides communicating with a recipient, subverting the cooperative principle
can also be used to selectively communicate with one recipient out of many, or
to mislead some or all.

A lot of things people want to calculate these days seem like absurd hubris to
me, but I suppose that's also how progress has always occurred, by trying to
do things that seem out of reach.

~~~
mirimir
> I would say based on my experience on the internet, that spotting sarcasm is
> frequently beyond human capability ...

In a 1983 Usenet FAQ by Jerry Schwarz: "Avoid sarcasm and facetious remarks."

More recently, aka Poe's Law.

------
jboggan
Last year we were working on a rubber ducky debugging chatbot and relying a
bit too heavily on StackOverflow data for conversational training data. It
frequently gave sarcastic answers, especially if you asked for help concerning
regular expressions.

~~~
colejohnson66
Did it respond with Zalgo if you asked about Regex and HTML?
[https://stackoverflow.com/questions/1732348/regex-match-
open...](https://stackoverflow.com/questions/1732348/regex-match-open-tags-
except-xhtml-self-contained-tags)

~~~
joshu
He comes.

------
qntmfred
I often recall this post from Andrej Karpathy when I'm noticing the current
limitations of AI [http://karpathy.github.io/2012/10/22/state-of-computer-
visio...](http://karpathy.github.io/2012/10/22/state-of-computer-vision/)

------
cabalamat
Sarcasm is almost certainly AI-complete. Consider this post:
[https://cabalamat.wordpress.com/2018/01/31/royal-mail-
announ...](https://cabalamat.wordpress.com/2018/01/31/royal-mail-announces-
commemorative-brexit-stamps/) which is pretty easy for a human to tell is
sarcastic, but would be difficult for an AI unless it actually understood it.

~~~
gowld
Why? It has a few tell-tale word choices, and image recognition would indicate
that the stamp images are "wrong" and/or correlated to humor memes.

~~~
cabalamat
Go and build an AI that detects sarcasm, then I will agree with you.

------
hx2a
If anyone is interested in studying sarcasm themselves, there is a dataset
available:

A Large Self-Annotated Corpus for Sarcasm
[https://arxiv.org/abs/1704.05579](https://arxiv.org/abs/1704.05579)

1.3 million sarcastic statements!

------
woliveirajr
> So now, computers can be as sarcastic as the rest of us. That sounds just
> great.

I love how sarcasm has so many subtle ways to be use to break tensions in some
meetings, and how it can quickly derail any light conversation into some
almost-nuclear mood.

But computers generating sarcasm, well... that will put a whole new level to
automatic chat bots.

------
ACow_Adonis
To be perfectly fair, human beings can't pick up on sarcasm reliably: or at
the very least it seems to be cultural/contextual.

Australians/brits consistently bemoan the inability of Americans to understand
deadpan and sarcasm.

And I'm even reminded of one of my European friends (from Austria) bemoaning
conversations with Germans along the lines of:

Austrian: "could you be any more condescending!?" German: "yes I could be more
condescending."...blank stare.

~~~
interfixus
That is very much a statistically significant thing, yes. I'm Danish. Get
along fine with Brits and Aussies, deadpans, understatements and poisonous
sarcastic barbs translating well enough both ways. With Americans present,
experience has taught me to shift into a far more cautious mode. And often
these days, what Americans describe as irony, I fail to recognize as such.

A very broad generalization, of course, but has served me well enough as
survival guide.

~~~
perl4ever
In some parts, "irony police" are a running joke - people will explain why any
given example of irony is not _really_ ironic.

------
mhd
Can't wait for the first next gen AI programming language to support an "as
if" statement.

------
sbarre
> “We found lots of tweets, especially in politics, to be sarcastic,” he said,
> in what may be the biggest understatement so far of 2018.

------
ryacko
The Secret Service did want a sarcasm detector.

[https://www.thedailybeast.com/secret-service-wants-a-
sarcasm...](https://www.thedailybeast.com/secret-service-wants-a-sarcasm-
detector)

------
chocolateboy
(The Groucho Marx joke isn't sarcasm.)

------
laretluval
The point of sarcasm is to hide your true intent except for people who are
sufficiently aligned with you to be able to decode it correctly.

If computers can decode sarcasm then it is not effective. Then sarcasm will
evolve.

~~~
gowld
I trust many more details of the secrets of my life to computers than people.
Computers are smarter than people in many areas, so sarcasm that only a
computer recognizes is not a problem for sarcasm.

------
matte_black
Yes, this is exactly what we need.

~~~
Declanomous
Side note, is there a name for a type of humor where you are being facetious
and serious at the same time?

That's kind of my go-to, and I'd say about 20% of people overall don't get it.
I hear "I can't tell if you are joking or not" all the time. I personally
think both is a reasonable answer, but if we aren't able to teach people how
such a sense of humor works, I don't have much hope for training an AI.

~~~
perl4ever
If I was trying to explain humor to an alien, I would say it's a pleasurable
sensation created by the resolution of ambiguity or uncertainty in the
communications between two or more people. This is achieved by performing
similar mental operations and reaching the same result as each other.

So typically, you say something, and someone understands that it is not
serious, and they laugh at the point at which they realize you two understand
each other.

Your style of humor is more evolved, in that it adds an additional step, where
first they think you are not serious, and then they understand you are, and if
they get it, they laugh because they have completed the path to a shared
understanding. It may not be appreciated so much, because since the journey is
more difficult, the listener may infer that you are trying to put down people
who are not sufficiently quick witted.

------
mozumder
If humans can’t identify sarcasm, why would you expect AI to?

It’s like expecting AI to read handwriting, when people can’t even do that
100% of the time.

~~~
brisance
People didn’t expect to be humbled by a machine at the game of Go either. Yet
here we are.

~~~
sidlls
Apples and oranges comparison. Go is much closer to chess than it is to
recognizing sarcasm.

~~~
umanwizard
I’m not convinced.

Good computer chess play is based on a deterministic tree walking algorithm,
something that’s been clearly in computers’ wheelhouse since forever.

Good computer go play is based on neural pattern recognition, which is also
(probably) what human sarcasm detection is based on.

~~~
Declanomous
On the other hand, go operates in a much more finite universe than
conversation does. You have 2 pieces, and a number of places to put them. You
have a goal.

Conversation doesn't work like that, and takes a vast amount of information to
understand whether someone is being sarcastic.

~~~
brisance
So you’re saying that the people cited in the article accomplished something
more significant than the AlphaGo team…?

~~~
TuringTest
Probably not, because the approach taken in the paper is identifying a small
set of patterns that are present in sarcasm, not fully understanding the
meaning and context of why the sentence is sarcastic. It's a useful tool with
pragmatic applications in online discourse analysis, but not a solved problem.

------
civilian
Sarcasm is a great tool. But there are a couple of people I know who never
turn off their "Sarcastic tone", and it becomes impossible to tell what's
genuine and what's bullshit. I try to tell them that their noise-to-signal
ratio is bad, but they um... react defensively.

~~~
freeflight
It can be a coping mechanism for depressive people just like cynism, at least
I know that it serves that function for me: Can't be straight up negative, for
dragging the mood down, so I'll settle for some sarcasm.

~~~
gowld
You are likely not fooling anyone with your sarcasm.

~~~
freeflight
It's not about fooling anybody, I consider it more like gallows humor.

------
fellellor
I have to note that often humans fail to detect sarcasm, and police forums
with religious fervor demanding the application of those ridiculous "\s" tags
wherever they be deemed necessary. Small wonder, why it would be difficult to
catch in a program.

------
stochastic_monk
Inherently ambiguous medium of hierarchical, ambiguous data requiring personal
and situational context difficult to interpret.

Surprise! NLP is still hard... which is good news for anyone interested in
understanding language, neural networks, o the human condition.

------
JoeAltmaier
It's more than incongruity. Context is king. "I like being ignored" is
sarcasm. But followed up with "No, really, I like being ignored. Its like a
super power! I can ..." is serious (sort of).

~~~
ACow_Adonis
And then you get people like me, where "I like being ignored" is not sarcasm,
but a commentary/personality preference related to disliking a cultures
obsession with fame and popularity and struggling to think of anything worse
than being recognised/constantly under surveillance.

As you said, context is everything. It is literally impossible to detect
sarcasm reliably with only the sentence in which sarcasm is contained.

~~~
Retra
There's also a blurry line between sarcasm and lies. I'll often say things
that are patently untrue to people with the expectation that they know the
truth (and me) well enough to get that I'm not serious.

For instance, to my sister:

"I hate you." \- sarcasm. "I've already told you many times that I hate you."
\- obvious lie, maybe sarcasm?

------
knowThySelfx
Computers are getting smarter. They can do many stuff faster than humans. They
can even detect sarcasm. But do they "understand" it? Can Computers understand
a subtle joke? Maybe we could train a Computer to detect jokes and make it
laugh, but can it really understand a joke? Interesting topic.

~~~
MaxBarraclough
What does 'understand' mean, if not something functional? You mean
consciousness? Tough question to answer.

Are you familiar with Daniel Dennett's 'Competence without comprehension'
ideas? He points out for instance that even a very simple computer is far
better at division and exponentiation than even the smartest human. It does
this without any comprehension at all - a pocket calculator is far too simple
to have any meaningful understanding of anything, but that has zero impact on
its functional usefulness.

Another favourite example of his is evolution by natural selection.

Annoyingly I couldn't find a brief online article on the topic by the man
himself.

~~~
knowThySelfx
Yeah I had consciousness in mind.

------
ateesdalejr
Yes! I need alexa to be sarcastic.

~~~
posterboy
That's irony, not sarcasm.

------
blattimwind
> That was especially true for tweets that include numbers; their accuracy
> rate of 80 percent more than triples previous efforts.

Which is to say that previous efforts had only half of the true positives a
random number generator would have had. Neat.

~~~
notahacker
Sounds like previous efforts might even have been outperformed by a naive
algorithm that purely looked for one word sentences like "right", "totally"
and "neat" :-)

------
kfe
Looking forward to Deep Sarcasm...

------
espe
claims too much. and the task definition seems off: sarcasm is more like a
speaker attitude, while irony is the linguistic phenomenon. another case of
"lets throw an ANN onto anything".

------
Stratoscope
Like a human being could ever read that text.

If you're having as much trouble reading it as I was, open the developer tools
and on the body style, remove "DINPro" from the font-family, turn off the
font-weight: 300 or change it to 400, and increase the font-size to 16px or
whatever size you like.

If the AIs ever do take over, my one request is that they use plenty of
sarcasm when they talk about designers who think unreadable text with
substandard font-weight is a good thing.

Of course, being AIs, maybe they could care less.

~~~
com2kid
It actually looks quite nice in Firefox, it is hard to read in Chrome though.
Maybe they have the one last web design team that primarily tests in FF? :D

~~~
Stratoscope
I'm curious which OS you're using. On Windows, it looks pretty much equally
bad to my eyes in Chrome, Firefox, and Edge.

~~~
marcosdumay
It's legible in my computer. Debian 8, with Firefox, no-script, and ublocker,
with 24" screen and a forced minimum font size of something that I don't even
remember.

Anything of those could be making it legible. I'd rather trust the opinion of
somebody with a more standard system, and I doubt the GP is that somebody
either.

~~~
icebraining
It's the no-script. Allowing scripts loads a crappy font. Hurray for NoScript!

