
Are we in an AI Overhang? - andyljones
https://www.lesswrong.com/posts/N6vZEnCn6A95Xn39p/are-we-in-an-ai-overhang
======
polytely
I've been wondering about something: (I only know the basics of AI, this might
be kinda incoherent)

Right now if you look at GTP-3's output it seems like it's approaching a
convincing approximation of a bluffing college student writing a bad paper,
correct sentences and stuff but very 'cocky'. It cannot tell right from wrong,
and it will just make up convincing rubbish, 'hoping' to fool the reader. (I
know I'm anthropomorphizing but bear with me).

Current models are being trained on a huge amount of internet text. As smarmy
denizens of hackernews we know that people are very often wrong (or 'not even
wrong') on the internet. It seems to me that anything trained on internet data
is kinda doomed to poison itself on the high ratio of garbage floating around
here?

We've seen with a lot of machine-learning stuff that biased data will create
biased models, so you have to be really careful what you train it on. The
dataset on which GTP-n has to be trained has to be pretty huge(?); and
moderation is hard(?) and doesn't scale; it's easier to generate falsehood
than truth; and the further we go along the more of internet data will be
(weaponized?) output of GTP-(n-1); So won't the arrival of AGI just be
sabotaged by the arrival of AGI?

Has anyone written something about the process of building AGI that deals with
this?

~~~
lordnacho
I've been wondering about something similar to you, but I read one of Pearl's
causality books recently and thought that might be the missing piece.

It's certainly impressive what GPT-3 can do, but it boggles the mind how much
data went into it. By contrast a well-educated renaissance man might have read
a book every month or so from age 15 to 30? That doesn't seem to be anywhere
near what GPT could swallow in a few seconds.

When you look at how GPT answers things, it kinda feels like someone who has
heard the keywords and can spout some things that at least obscure whether it
has ever studied a given subject, and this is impressive. What I wonder is
whether it can do reasoning of the causality kind: what if X hadn't happened,
what evidence do we need to collect to know if theory Z is falsified, which
data W is confounding?

To me it seems that sort of thing is what smart people are able to work out,
with a lot of reading, but not quite the mountain that GPT reads.

~~~
skulk
> By contrast a well-educated renaissance man might have read a book every
> month or so from age 15 to 30? That doesn't seem to be anywhere near what
> GPT could swallow in a few seconds.

You're ignoring the insane amount of sensory information a human gets in 30
years. I think that absolutely dwarfs the amount of information that GPT-3
eats in a training run.

~~~
PaulDavisThe1st
But that sensory information includes very few written words. GPT(n) isn't
being trained on "worldly audio data", or "worldly tactile data", or in fact
any sensory data at all.

So the two training sets are completely orthogonal, and the well educated
renaissance man is somehow able to take a very small exposure to written words
and do at least as well as GPT(n) in processing them and responding.

~~~
MauranKilom
And the renaissance man has tons of structure encoded in his brain on birth
already. Just like GPT-3 does before you give it a prompt. I'm not saying this
is fully equivalent (clearly a baby can't spout correct Latex just by seeing
three samples), but you simply cannot just handwave away thousands of years of
human evolution and millions of years of general evolution before that.

The renaissance man is very obviously not working solely based on a few years
of reading books (or learning to speak/write).

~~~
PaulDavisThe1st
A person who is never taught to read will never be able to respond to written
text. So the renaissance-era man is working "solely" based on their lived
experience with text, which compared to GPT(n) is tiny.

Ah! you cry. Don't humans have some sort of hard-wiring for speech and
language? Perhaps. But it is clearly completely incapable of enabling an
untrained human to deal with written text. Does it give the human a head start
in learning to deal with written text? Perhaps (maybe even probably). It
demonstrably takes much less training than GPT(n) does.

But that is sort of the point of the comment at the top of this chain.

------
gillesjacobs
I largely agree with the arguments made, but the following assertion is plain
bogus

> GPT-3 is the first NLP system that has obvious, immediate, substantial
> economic value.

Text mining (relation extraction, named entity recognition, terminology
mining) and sentiment analysis are billion dollar industries and are being
directly applied right now in marketing, finance, law, search, automotive,
basically every industry. Machine translation is another huge industry of its
own. Chat bots were all the hype a few years ago. Let's not reduce the whole
field of NLP to language generation.

~~~
tambourine_man
I think he means “ obvious, immediate, substantial economic value” to non
technical people. It take little effort to imagine how to monetize it even for
regular folks.

~~~
riffraff
really? I am very impressed by GPT-3 but I still don't see any way to make
money "obviously" out of it.

Maybe it can be an adjuvant to human in some tasks but then so could existing
technologies too, I guess?

~~~
tambourine_man
Do you want a computer that can reliably understand you and give you the best
possible answer 99% of the time?

I think most people would go: yes! How much does it cost?

~~~
riffraff
If you're running a search engine maybe.

But everyday people are used to getting 80% answers by search engines, I don't
think many would pay for something that is "like google, but a bit better".

This seems the be current issue for many things that we used to pay for
(dictionaries, encyclopaedias, newspapers, etc), and I'm not sure this would
be different.

~~~
rytill
I would easily pay money for something that is like Google, but a bit better.

~~~
cambalache
I wont

------
blueyes
Both DeepMind and OpenAI were founded on the premise that we are in an AI
overhang. OpenAI, in particular, believes in scale. Scale will get us there
based on the algorithms we have, such as the Transformer. With each new
release, they add evidence that they were correct.

The call for legislation neglects that there exists a global arms race to make
this technology succeed. Legislation in one nation will simply handicap that
nation. Against that backdrop, legislation is probably unlikely among the
nations already leading in AI.

~~~
hyperbovine
> With each new release, they add evidence that they were correct.

Is it though? If the goal is human-level AI, or hell, even rat-level AI, the
evidence is pretty convincing that you should be able to train and deploy it
without requiring enough energy to sail a loaded container ship across the
Pacific Ocean. Our brains draw about 20 watts, remember. This suggests to me
that no, in fact, scale will not get us "there".

[https://www.forbes.com/sites/robtoews/2020/06/17/deep-
learni...](https://www.forbes.com/sites/robtoews/2020/06/17/deep-learnings-
climate-change-problem/)

~~~
andyljones
Training an AI in 2020 is best thought of as a capital investment. Like
digging a mine or building a wind farm, the initial investment is very large
but the operating costs are much lower, and in the long run you expect to get
a lot more money out - a lot more value out - than you put in.

Training GPT-3 cost $5m; running it costs .04c per page of output.

~~~
ac29
If it is scaled up by 1000x as the article proposes, does that mean it will
cost $40/page of output? Or does the additional cost just go into training the
model?

~~~
andyljones
If it's 100x from increased investment and 10x from short-term efficiency
gains, yeah you'd expect $4/page. Model compression or some other tech might
make it more efficient in the long-run.

------
simonw
I've been pandemic-rewatching Person of Interest. It's really quite shocking
how much more relevant it feels today compared to when it aired just a few
years ago.

It's fun looking at things like GPT-3 and imagining how they could be used to
build the surveillance AI at the heart of Person of Interest.

(If you haven't watched Person of Interest yet, here's my pitch for it: it's a
CBS procedural where the hook is that an engineer built a secret, surveillance
feed tracking AI for the government after 9/11 - but he cared about civil
liberties, so he built it as an impenetrable black box. All it does is kick
out the SSN of someone who is about to be either the victim or the perpetrator
of a terrorist attack - which means government agents still have to
investigate what's going on rather than taking the AI's word for it. "The
Machine" also sees victims/perpetrators of violent crimes - but the government
don't care about those. Finch, the machine's inventor, does - so he fakes his
own death, hooks into a backdoor into the machine that gives him those SSNs
and sets up a private vigilante squad to help stop the violent crimes from
happening. So that gives you the "case of the week". Only it's actually an
extremely deep piece of philosophical science fiction disguised as a case-of-
the-week procedural, and as time goes on the plots become much more about AI,
the machine, attempts to build rival machines, AI ethics and so on. It's the
best fictional version of AI I've ever seen. The creative team later worked on
Westworld.)

~~~
Barrin92
I honestly don't think the show has much to do with AI at all and is more like
a retelling of the Greek classics in a sci-fi wrapping, which is actually
something that comes up in the show at several points excplicitly.

The AIs in the show very quickly turn into godlike characters with
antropomorphic personalities and the real world issues of AI such as
surveillance, economics and so on are all dealt with in very shallow fashion.
I had the same issues with Westworld too. It turns from an AI premise into a
classical Christian morality tale very fast. ("we need to suffer to become
conscious").

~~~
simonw
One of the things I loved about the show is that different characters have
different philosophies concerning AI, and they argue about them. Nathan v.s.
Fitch. Fitch vs. Root. Control, Greer - for the most part the show tried to
give some depth and background to their thinking around the implications of
what they were responsible for.

Way smarter than you would expect from a CBS procedural!

~~~
ShamelessC
That's Jonah Nolan's show right?

~~~
simonw
Yup, he did it before Westworld.

------
GrantS
I thought the overhang was going to be along the lines of the following,
whether realistic or not:

-GPT-3, as is, should be the inner loop of a continuously running process which generates 1000s+ of ideas for "how to respond next" to any query, with a separate network on top of it as the filter which cherry-picks the best responses (as humans are already doing with the examples they are posting)

-Since GPT-3, as is, can already predict both sides of a conversation, it can steer a conversation toward a goal state just like AlphaGo does by evaluating 1000s+ of potential moves, lots of potential responses and counter-responses until it finds the best thing to say in order to get _you_ to say what it "wants" you to say.

It seems ready to go as the initial attempt at the inner loop of both of these
tasks (and more) without modification or retraining of the core network
itself, no?

~~~
lambdatronics
I'd love to see what could be done with GPT-3 as part of a GAN. Text
compression/summary, maybe?

------
lacker
_GPT-3 is the first AI system that has obvious, immediate, transformative
economic value._

I think the jury is still out on this one. It certainly seems powerful, it's
doing interesting things, and it's better in many ways than any system that
has come before. But there's a different between exciting demos and
transformative economic value.

It's too soon to be sure, but to me, the most interesting question is whether
any valuable startups will be built on top of GPT-3. Some leading indicators
before that are whether useful products are built on GPT-3, and whether early-
stage startups built on GPT-3 get seed investment. I'm not aware of any of
these yet but maybe latitude.io counts as one.

------
f00zz
In Kernighan and Pike's "The Practice of Programming" there's a chapter that
covers the implementation in different languages of a random text generator
using markov chains. It's a nice exercise and a lot of fun to play with.

I'm guessing that not many people have read that book, because I'm seeing here
and elsewhere even technical people talking about GPT-3 as if it's heralding
the imminent advent of SkyNet. I get that transformers have a somewhat longer
attention span than markov chains, but it's still a statistical language
model. It can't even do the kind of planning or reasoning that early AI demos
like SHRDLU could.

~~~
Veedrac
It's ridiculous to equate this to a traditional Markov chain language model.
Here's something a Markov chain certainly cannot do:

Human: I want to test your creativity. Please invent a new word and give its
meaning.

GPT-3: Ok. Um... Tana means to hit someone with the intention to wound them.

Human: Please use the word tana in a sentence.

GPT-3: You are about to tana the man attacking you.

Human: Speak like a dwarf.

GPT-3: I ain't talkin' like a dwarf.

[https://www.reddit.com/r/MachineLearning/comments/hvssqn/d_g...](https://www.reddit.com/r/MachineLearning/comments/hvssqn/d_gpt3_demos/fyylreb/)

~~~
f00zz
Right, I admit that I don't know the first thing about ML, so I tried an
experiment.

Consider a language with the tokens "{[()]}" and the following grammar:

S := S S | '{' S '}' | '[' S ']' | '(' S ')' | <empty>

That is, "[()]" and "[]()" are valid sequences, but "[(])" or "))))" aren't. A
child would quickly figure out the grammar if presented some valid sequences.

I generated all 73206 valid sequences with 10 tokens and used it as input to
the RNN text generator code at [http://karpathy.github.io/2015/05/21/rnn-
effectiveness/](http://karpathy.github.io/2015/05/21/rnn-effectiveness/).
After 500,000 iterations I'm still getting invalid sequences.

Am I doing something stupid, or is a RNN text generator weaker than a child
(or a pushdown automaton)? Is GPT fundamentally more powerful than this?

~~~
Veedrac
GPT-3 can generate well-formed programs, so yes, it does things well beyond
this complexity.

> After 500,000 iterations I'm still getting invalid sequences.

How frequently? If it's only the occasional issue it might be down to the
temperature-based sampling that code uses, which means it will, with some
small probability, return arbitrarily unlikely outputs.

------
legulere
Nah we’re in for the next AI winter. GPT-3 shows how much energy is needed to
perform a nice trick with current technology. We mostly have reached the
limits of the technology. Investing more compute power for a few percentage
points more Precision is not going to bring the technology forward.

~~~
Veedrac
2017 SOTA on Penn Treebank was 47.69 perplexity. GPT-3 is at 20.5. AI has
already been productized on consumer devices through Siri, Google Assistant,
speech detection, speech generation, textual photo library search, similar
data augmentations for web search, Google Translate, recommendation
algorithms, phone cameras, server cooling optimization, phone touch screens'
touch detection, video game upscaling, noise reduction in web calls, file
prefetching, Google Maps, OCR, and more. DLSS _alone_ justified continued
investment by NVIDIA. NVIDIA Ampere will be ~6x as fast at running consumer-
targeted models as Turing, given raw throughput increases compounded with
sparsity and int8 hardware. A huge number of research threads around AI have
direct applicability to large tech companies.

~~~
legulere
I'm not arguing that current machine learning technologies are not useful. I'm
just arguing that progress is based on increasing some metric, usually
depending on a trade-off of computation. This can even make ML-techniques
applicable to some new fields, but it's not what is holding back autonomous
driving, the often touted parade example which also brings in a lot of
employment for machine learning.

This article clearly sits on the peak of inflated expectations in the hype
cycle.

[https://en.wikipedia.org/wiki/Hype_cycle](https://en.wikipedia.org/wiki/Hype_cycle)

~~~
Veedrac
It's not just that you're not arguing it isn't useful; as far as I can tell,
neither of your comments contain an argument against ML _at all_. I have
nothing to meaningfully argue against.

ML is undergoing a Cambrian explosion of use-cases (see my prior comment),
almost all of this over an incredibly small time period, progress is
accelerating, and many of these use-cases are incredibly high value. Scale is
not proving a major stopper; Google's MoE experiments show that huge models
are productizable, and small models work plenty fine too in restricted places,
to the point where they're literally used to parse touch screen sense data in
phones.

If you want to claim we're in for another AI winter, you need a vastly
stronger argument than ‘something something hype cycle’.

------
datameta
I wonder if at some point the amount of extra data necessary to achieve an
n-fold improvement will outstrip what we can provide.

I think the time for AI legislation is now - _before_ FAAMG deploys something
like the next-gen of GPT-3. Of course with the legislative lag that exists
even for decade-old tech I don't have the highest confidence in this being
achieved by a federal government in the state it is in now.

~~~
inetsee
As blueeyes has already pointed out "Legislation in one nation will simply
handicap that nation." I don't have a lot of faith in our legislators ability
to legislate safety without relegating us to an AI backwater.

~~~
datameta
You're right. I think ideally the legislation should be international. Maybe
something like the Washington Naval Treaty that set an upper limit on the
tonnage and armament of new battleships. Or perhaps more aptly something akin
to SALT I & II where older models are taken offline to avoid derelict AI
systems from falling into malicious hands and to keep the number from growing
out of control. Although this parallel is somewhat weak considering the
capabilities of one advanced model are more valuable than 10x models of the
last generation.

Theoretical wishful thinking, I suppose, but I strongly believe that corp/govt
scale ML research should be treated like advanced weaponry because it isn't a
matter of if but _when_ AI will be weaponized (whether the flavor of warfare
is physical or informational).

Although of course as with weapons treaties - the major powers would likely
tend to be selective in what they commit to limiting themselves in.

~~~
Jach
The world couldn't even come together on controlling 3D printed weaponry,
there's no hope for an arms treaty for AI right now. The "it's not feasible to
regulate even if you tried" stance applies too -- you can restrict central
actors without much difficulty, and that would work for AI just as well as it
works for battleships, but there's a _lot_ of distributed compute whereas
there's not a lot of distributed shipyards. Like, you just have to follow
what's been done with anime image nets to see that something like GPT-3 is
possible for a distributed worldwide group to achieve and is not limited to
firms or governments.

Maybe when we have a disaster directly attributable to AI, nations can get on-
board with something like the BWC and CWC. Until then, be even more
pessimistic. (If you want a fun if rather dry book to read on material
technology developments that were in the pipeline a couple decades ago, some
of which have come to fruition, as well as some policy recommendations for the
technologies that aren't generally good, check out Jürgen Altmann's Military
Nanotechnology.)

~~~
sbierwagen
>The world couldn't even come together on controlling 3D printed weaponry

Beg pardon? Plastic guns have been banned in the US since 1988
[https://en.wikipedia.org/wiki/Undetectable_Firearms_Act](https://en.wikipedia.org/wiki/Undetectable_Firearms_Act)

I assume other countries have similar bans.

~~~
Jach
As a small amount of metal can be added at the end to make the weapon 'legal',
that act does little to address the numerous¹ problems beyond being able to
sneak a gun past airport security. Hardly an important milestone in
controlling anything. It didn't even affect any gun in existence at its time.

But more generally, as we all know, a ban without provisions for enforcement
is useless. Compare to the CWC (Chemical Weapons Convention) which I point to
as one of the best pieces of international "coming together" via treaty. It
includes requirements that member countries submit to inspections from its
enforcement body (OPCW) and furthermore that countries can request the OPCW
inspects another member country if they suspect non-compliance. It also
includes restrictions on transfer of various chemicals in order to incentivize
non-member countries to become members so they can purchase chemicals for
industrial purposes from other members.

¹ and bigger, if you're modeling this from assumptions where it's a problem at
all -- not everyone thinks it is, "an armed society is a polite society" etc.

------
01100011
One thing I don't hear a lot of people talking about are ML/AI systems in the
hands of government agencies. We know that the military and NSA are often
ahead in many technologies but when it comes to AI the assumption seems to be
that the industry is moving faster than the government. Is that really a safe
assumption?

The goverment is openly using autonomous systems to pilot drones, but what
else are they leveraging AI for? Threat analysis? Logistics? Weapons
optimization? PsyOps?

The DoE is openly a very large consumer of GPUs. What about the military?

~~~
confeit
You can get a glimpse by scrolling websites like:
[https://www.darpa.mil/opencatalog?ppl=view200&sort=title&ocF...](https://www.darpa.mil/opencatalog?ppl=view200&sort=title&ocFilter=software)
[.mil] and looking at DARPA and Office of Naval Research sponsored ML/AI
research. The military has been deeply involved with ML/AI research since its
inception, and it is near impossible to avoid first - or second degree
involvement, if active in ML/AI.

The military wants: automated chat agents/web users that can be sent to dark
web markets and hacker IRC channels and report back intelligence. Common sense
inference from security and drone footage: predict who the killer is when
watching a movie. Author deanonimization and cross-device tracking. Global-
scale 99.9%+ accurate face detection.

The Dutch Intelligence Agency organizes a yearly competition with difficult
codes to crack. [1] It is rare for someone to answer all questions correctly.
The answers require logic, creativity, common sense, linguistics, causal
inference, spatial reasoning, expertise, analysis, and systematic thinking. I
bet the military would be mighty interested in an automated problem solver for
that. And mighty scared some other country gets there first.

[1] [https://www.aivd.nl/onderwerpen/aivd-
kerstpuzzel](https://www.aivd.nl/onderwerpen/aivd-kerstpuzzel)

------
dougmwne
Let me flip this argument on its head. Consider this: About 5 years ago
several key SV people including Sam Altman, Peter Thiel and Elon Musk became
suddenly very concerned about AI ethics and started OpenAI. What if they, with
this insider status, had already seen a GPT-3 like system at Google, Facebook,
Baidu or wherever and its capabilities for political and social manipulation
so concerned them that they started OpenAI in an effort to bring this tech out
of the shadows and into the sunlight so we could debate it and regulate it.
GPT-3 might not be a state of the art breakthrough. It could be just catching
up with where the big tech companies were 5 years ago so that we can finally
see what they are capable of. Corporate secrets are a normal part of doing
business and maybe the tech companies didn't like the PR they would have
gotten from publicizing something like this. Remember the blowback from
Google's project that called business for their store hours? They already
struggle with regulators across the world as it is. Do we really believe that
little OpenAI is so much farther ahead of Google like the posted article
posits?

~~~
stjo
Sounds like a nice conspiracy, but realistically, how could they hide
something like that? Presumably there are hundreds or more employees working
on this. If Elon Musk et al. heard about it 5 years ago, this must be one of
the best kept secrets in recent history.

~~~
ColanR
> how could they hide something like that?

Very carefully. I mean, that's not much of an argument. Lots of stuff is
successfully kept secret. The US managed to keep a lid on their surveilance
for decades (iirc) before the lid got blown on that, and people used to give
the same argument you are in that context, too.

What's the alternative? Do you think megacorps never keep illicit things under
wraps for extended periods of time?

~~~
newen
True. Many thousands of people have security clearances in the US. The penalty
for breaking security clearance is harsh and many have done so but still,
there have to be tons of secrets kept by the state. Not a big stretch to
imagine companies convincing people to keep secrets.

~~~
dougmwne
Most companies default to secrecy. What's the recipe for Kentucky fried
chicken? What problem is holding Waymo back specifically, right now? How much
advertising business, in dollars, does Facebook take from political PACs. Who
will be Biden's VP pick? What will Apple's next iPhone look like? What new
streaming show is Disney about to reveal? Capitalism runs on information
asymmetry.

------
rvz
ʸᵉˢ

Explainability in AI is really overlooked and often skipped over as there is
little progress in this area. GPT-3 is essentially GPT-2 + tons of data,
compute and parameters and yet it still cannot explain itself as to why it can
generate 'human-level' text, much like how AlphaGo can't explain why it
performed move 37. Not discrediting these achievements, but explainability is
just as important in these AI models.

Once you have an AI-based 'auto-pilot' in any vehicle, the importance of AI
explainability will haunt manufacturers when the regulators would want them to
explain why this 'AI' took this decision and they're unable to explain this.

I hope GPT-4 isn't just going to be GPT-3 + 1000x the data. Otherwise nothing
would have changed here other than the parameters and data.

~~~
lambdatronics
The easy solution is to copy what the human brain does: just make up something
plausible. There's pretty good evidence that we don't have great introspective
access to much of our own internal processing. We just paper it over as
"intuition" or "judgement."

------
g_airborne
> The current hardware floor is nearer to the RTX 2080 TI's $1k/unit for 125
> tensor-core TFLOPS, and that gives you $25/pflops-d.

It's definitely true that the RTX 2080 Ti would be more efficient money-wise,
but the Tensor Cores are not going to get you the advertised speedup. Those
speedups can only be reached in ideal circumstances.

Nevertheless, the article as a whole makes a very good point. The thing that
is most scary about this is that it would become very hard for new players to
enter the space. Large incumbents would be the only ones able to make the
investments necessary to build competitive AI. Because of that, I really hope
the author isn't right - unfortunately they probably are.

~~~
tomp
OpenAI is kind-of a new player. Well-funded, but still - there's a lot of
money available for this kind of exponential opportunities.

------
mrfusion
> GPT-3 is the first AI system that has obvious, immediate, transformative
> economic value.

What is it’s economic value? What does it transform? I’ve been trying to
figure that out since I heard about it.

Anyone have any ideas?

~~~
crowbahr
Theory based on some output I've read:

It's good enough to actually start replacing a lot of customer service jobs.
Not just being a shitty annoyance like current bots but being _useful_ in that
it will be as flexible as a human, directing you to good help via vague terms,
potentially being smart enough to refer you higher up if necessary.

Getting rid of all those screening call center employees is potentially very
lucrative.

------
PaulHoule
It think it is all right except for the A.I. part.

GPT-3 is taking a graph-structured object ("language" inclusive of syntax and
semantics) over a variable-length discrete domain and crushing it into a high-
dimensional vector in a continuous euclidean space. That's like fitting the
3-d spherical earth onto a 2-d map; any way you do it you do violence to the
map.

I think systems like GPT-3 are approaching an asymptote. You could put 10x the
resources in and get 10% better results, another 10x and get 1% better
results, something like that.

You might do better with multi-task learning oriented towards specific useful
functions (e.g. "is this period the end of a sentence?") but the training
problem for GPT-3 is by no means sufficient for text understanding.

GPT-3 fascinates people for various reasons, one of them being almost good
enough at language, lacking understanding, faking it, and being the butt of a
joke.

If GPT-3 were a person with similar language skills and people blogged about
that person, mocking it's output, the way we do with GPT-3, people would find
that cringeworthy. Neurotypicals welcome it as one of their own, and aspies
envy it because it can pass better than they can.

At $2 a page it can replace richmansplainers such as Graham and Thiel who
never listen. It's not a solution for folks like like Phillip Greenspun who
read the comments on their blogs.

For that matter, it may very well model the mindlessness of corporate America:
if you accept GPT-3 you prove you will see the Emperor's clothes no matter how
buck naked he is. AT&T executives had a perfectly good mobile phone business:
what possessed them to buy a failing satellite TV business? Could GPT-3
replace that "thinking" at $2 a page? Such a bargain.

~~~
SpicyLemonZest
Why do you think that systems like GPT-3 are approaching an asymptote? Most
people I've talked to say the opposite; they wouldn't have expected GPT-3
could be so much better than GPT-2 with no major additional breakthroughs.

~~~
PaulHoule
It's just structurally wrong for the domain.

For instance, understanding language requires some of the capabilities of a
SAT solver. This was something everybody believed in 1972, but today is
denied.

Fundamentally "understanding" problems require the ability to consider
multiple alternative interpretations of a situation, often choose one or work
with the incomplete knowledge you have.

Back in the 1970s we had intellectually honest people like Richard Dreyfus
writing books like "Things Computers Can't Do" that describe many specific
ways the architecture at the time fall short. People on GPT-3 are working in a
way that is academically valid (able to make results that are meaningful to a
community) but from engineering it is like building a bridge with one end or a
tall tower that carries no load.

GPT-3 has a structural mismatch with the domain it works in. Unlike early
medical diagnosis systems like MYCIN, it is never a doctor, it just plays one
on TV and it does the "passing for neurotypical" terrifyingly well.

The secret of GPT-3 is that people want to believe in it. Somebody will have
it generate 100 text snippets and they will show you the three best. Your mind
makes up meaning to fill up for its mindlessness. When this was going on with
ELIZA in 1965 people quickly understood that ELIZA was hijacking our instinct
to make meaning.

For some reason people don't seem to have that insight today, and it bothers
me why that is. Back in the 1980s they had a lot of fear about compressing
medical images because it could lead to a wrong diagnosis. Today you see
articles in the press that are completely unquestioning that a neural network
that has been trained to hallucinate healthy and cancerous tissues will always
hallucinate the right thing when you are looking at a patient.

~~~
simiones
> People on GPT-3 are working in a way that is academically valid (able to
> make results that are meaningful to a community) but from engineering it is
> like building a bridge with one end or a tall tower that carries no load.

To me it seemed like the opposite. They are essentially working without any
hypothesis of how their model actually works, without any model of the way it
actually learns or the way it produces the results that it does, and instead
placing blind trust in various metrics that are improving.

They are treating this as an engineering problem - how can we make the best
human-sounding text generator - and not like a traditional research problem.
GPT-3 has not taught us anything about anything except "how to generate text
that seems human-like to humans". We have no firm definition of what that
means, we have no idea of why it works, we have no idea of any systematic
failures in its model, we know next to nothing about it, other than its
results on some metrics.

Imagine the same applied to physics - if instead of inventing QM and
Relativity or Mechanics, physicists got it in their head to try to feed raw
data into a black box and see how well it predicts some observed movements.

In fact, this would be a pretty interesting experiment: how large would a deep
learning model that could accurately predict what mechanics predicts get,
given only raw data (object positions, velocities, masses, colors, surface
roughness, shape, taste etc.)? Unfortunately, I don't think anyone has been
interested in this type of experiment, because it is not useful from an
engineering (or profit) perspective.

~~~
gld773
>Imagine the same applied to physics - if instead of inventing QM and
Relativity or Mechanics, physicists got it in their head to try to feed raw
data into a black box and see how well it predicts some observed movements.

In fact, this would be a pretty interesting experiment: how large would a deep
learning model that could accurately predict what mechanics predicts get,
given only raw data (object positions, velocities, masses, colors, surface
roughness, shape, taste etc.)? Unfortunately, I don't think anyone has been
interested in this type of experiment, because it is not useful from an
engineering (or profit) perspective.

Isn't that what googles alphafold is doing pretty much?

[https://deepmind.com/blog/article/AlphaFold-Using-AI-for-
sci...](https://deepmind.com/blog/article/AlphaFold-Using-AI-for-scientific-
discovery)

and it seems GPT-3 formed concepts related words together without being asked,
its not picking the next best word strictly as a matter of statistic
probability. So why wouldn't that apply to physics simulations / chemistry
etc?

feed it chemical formulas and balancing equations from old chem 101 textbooks
and it will fill in the blanks and start teaching itself how those things
relate just by being corrected enough, then you can see if it has any
predictive value.

~~~
simiones
I think both of your points are solving different problems that what I was
suggesting.

My point is that an interesting scientific question is: "is the huge size of
the GPT-3 model intrinsic to the problem of NLP, or is it an artifact of our
current algorithms?"

One way to answer that is to apply the same algorithms and methods to
mechanics data generated from, let's say, classical mechanics; and compare the
generated model size with the size of the classical mechanics description. If
the model ends up needing roughly the same amount of parameters as classical
mechanics, then that would be a strong suggestion that NLP may intrinsically
require a huge model as well. Otherwise, it would leave open the hope that and
understanding can be modeled with fewer parameters than GPT-3 requires.

Your examples are still in this realm of engineering - trying to apply the
black box model to see what we can get, instead of studying the model itself
to try to understand it and how it maps to the problem it's trying to solve.

------
earthboundkid
GPT-3 is a search engine pretending to be an AI.

------
mijail
Can anyone provide the background or framework of how I should see the
business value of gpt3.

Are there businesses that have a tremendous needs for the possibilities it
provides?

I've seen use cases that some NLG companies provide like sports and stock
summaries but what world should I imagine where this is transformative?

------
maerF0x0
> so dropping $1bn or more on scaling GPT up by another factor of 100x is
> entirely plausible right now.

I'd note it's rare that cost of scaling a computing project is a linear growth
function.

100x-ing an AI project could be 1100x cost.

~~~
lacker
For AI on this scale, there are two important costs. The cost of compute, and
the cost of engineer salaries. Scaling the compute is probably a bit above
linear due to various overheads, but the cost of engineer salaries is far less
than linear, so I would expect the total cost of scaling large-scale AI to be
sublinear.

~~~
gwern
I would expect sublinear scaling in compute cost. Learning curve effects are
strong in DL:
[https://cdn.openai.com/papers/ai_and_efficiency.pdf](https://cdn.openai.com/papers/ai_and_efficiency.pdf)
The more runs you do, the cheaper they get. Plus, OA is no longer running on
rented V100s, they have their own MS Azure supercomputer, remember (in fact,
GPT-3's evaluation was interrupted because they moved to it), so they remove
the enormous cloud margins.

------
cl42
This is a fascinating discussion -- I'm curious what people thing is the next
step with AI? The post and several commenters here talk about how the tech in
GPT-3 is "dumb" in that it's a big network but the network architecture is a
fairly standard approach.

I'm curious what people think are the next stages of AI research that
companies are working on... Is it Probabilistic Graphical Models? Is it
Probabilistic Programming? Is it knowledge graph extraction from text? Is it
something else? Curious what people think...

~~~
ilaksh
I think its about automatically building accurate and well-factored world
models online that ultimately integrate not only high-dimensional sense data
(such as visual information) but also language. This involves effectively
solving the symbol grounding problem among other things. There is some serious
effort in this direction in deep learning.

There are also other efforts using different types of probabilistic
programming as well as symbolic and neural net combinations.

There's another link on one of the first few HN pages right now about
dreaming. I think that dreaming gives one a lucid demonstration of some of the
capabilities that we need to emulate if we are going to have human-like
intelligence. AI will need to be able to visualize new situations, basically
like on-demand, flexible simulations of mashed-up possibilities, involving
things like physics and psychology etc.

I think we almost need the AI to have something like a 3d gaming engine with
physics, but also it can effortlessly conjure up AI agents in this simulation,
but also, many of the physics rules and behaviors of the AI agents are
automatically learned with only a few examples. This is the type of capability
that allows humans (and some other animals) to adjust so readily to new
situations.

I speculate that there may be some representation or type of computation that
has not been invented yet which facilitates both the simulation-type data and
also the abstractions over it, all the way up to language, in a more seamless
way than has so far been described. I saw a paper talking about the symbol
grounding problem in terms of everything being categories, but really in the
end it was broken down into something kind of like Lisp + probabilistic
programming, and it seemed to not really have sufficient granularity to really
do justice or properly integrate sense data. Certainly not in a seamless or
truly unified way in my opinion. Although I guess I don't really understand
category theory.

------
andrewnc
> GPT-3 is the first AI system that has obvious, immediate, transformative
> economic value.

Seriously? No other piece of machine learning has had economic value? How
short sighted.

------
SeanLuke
> GPT-3 is the first AI system that has obvious, immediate, transformative
> economic value.

This statement is unbelievably ignorant of history. Just picking one random
example out of a hat: planning and scheduling systems have had a profound
impact on the manufacturing and shipping industries for many decades now.

------
haolez
If I had the money and the dataset for training a model like GPT (that's a big
if), is the code to implement such a thing trivial? Or is it a valuable
proprietary asset of OpenAI as well as their instance per se?

~~~
zitterbewegung
You would have to also reproduce their code which is the largest cost in any
software development project .

The code is non trivial but if you wait someone reimplements it .

The dataset is also nontrivial because they probably cleaned the data which.

It’s a valuable asset but it’s not like someone couldn’t reproduce it.

------
cs02rm0
I fear Betteridge's law of headlines applies here.

A CS lecturer of mine told us that when he was a student he had a lecturer who
advised him to be sceptical of AI revolutions. That was nearly 20 years ago.
I've no doubt we'll see further steps but I'm not going to hold my breath for
something transformative.

~~~
jqgatsby
I’d like to go on the record as being a GPT-3 skeptic. Yes, it’s a massive
improvement over markov models, and yes, it will be used for propaganda. But
the AI effect is very strong, and in a year or two people will be used to it
and you’ll see more writing to the effect “why GPT-3 wasn’t such a big deal
after all”.

Personally, my guess is that it’s actually just plagiarizing the training set
in a way that most researchers will come to view as a kind of cheating. What I
mean by that is, if you take some plagiarism detection software and run it on
GPT-3’s output, it will ring like crazy.

I say this both because I believe it and because if it’s not the case, if we
really have a proto-AGI on our hands, then being wrong won’t matter. I
sincerely hope that we are a thousand years away from that, because otherwise
we are plainly doomed.

~~~
armitron
Strange logic.

We're doomed regardless. We don't have a thousand years. Maybe not even 100.

~~~
jqgatsby
Are you referring to environmental collapse? I agree but I'd like to try a bit
harder before calling it. :)

------
eternalban
Question:

Is a collapse in learning time a possible breakthrough for future, or do we
have definitive ~information theoretic bounds for says number of dimensions,
etc.

------
YeGoblynQueenne
>> GPT-3 is the first AI system that has obvious, immediate, transformative
economic value.

To say the least, it is not immediately clear where that "transformative
economic value" lies.

From what I've seen so far GPT-3 can generate structurally smooth but
completely incoherent text and despite claims to the contrary cannot perform
anything close to "reasoning" [1]. It can also perform some side-tasks like
machine translation and question answering, though with nowhere near good
enough accuracy for it to be used as a commercial solution for these tasks.

All this is not very useful or even interesting. Text generation is a fun
passtime but unless one can control the generation to very precise
specifications, to generate good quality text that makes sense on a particular
subject, text generation is nothing but a toy with no commercial value (and
even its scientific value is not very clear). And GPT-3's generation cannot be
controlled to such precise specifications.

We've had AI software that could interact intelligently with a user since the
1970's, with Terry Winograd's SHRDLU [2] and that never led to "immediate,
transformative economic value", even though it was every bit the sci-fi-like
AI program that could be directed by natural language to perform specific
tasks with competence, albeit in a restricted enviroment (a "blocks world").
GPT-3 is not even capable of doing anything like that (nor are any other
modern systems). How is a language model that is likely to respond with "blue
offerings to the green god of mad square frogs" to a request to "place the
blue pyramid on the red sphere" bring "transfomative" value?

In fact, we've had systems capable of generating much more coherent (and still
grammatically corret) text for some time [3] and even those have not caused a
dramatic upheaval of "transformative economic value".

I'm sorry but I'm afraid that, with GPT-3, we're again in a spiralling peak of
hype, just as we were a few years ago with all the claims about sef driving
cars "next year" etc. I think we all know how those panned out.

In any case, you don't have to take my word for it. As with self-driving cars,
all we have to do is wait a few years. Say, until 2024. We'll have a good idea
about GPT-3's "transformative value" by then.

__________________

[1] Unless of course one insists on Procrusteanising the definition of
"reasoning" sufficently to cover essentially random guessing.

[2]
[https://en.wikipedia.org/wiki/SHRDLU](https://en.wikipedia.org/wiki/SHRDLU)

[3] I'll need to dig up some references if you ask, but in the meantime search
for "story generation".

------
sgt101
Any one know how to test this yet?

------
akyu
I'm very heavily inclined to believe that yes we are.

And my money is still on DeepMind.

~~~
ghostbrainalpha
Can you link to the critical paper you are referencing?

------
amelius
Perhaps I missed it but what are some useful applications of GTP-3?

~~~
ALittleLight
For it "as is" \- i.e. without imagining any new things, I would say you could
make money from AI Dungeon, chat bot, and selling API access.

I know that I badly want to play with the AI and would pay some amount per
month to get some number of queries.

~~~
ilaksh
AI Dungeon has a monthly fee now to upgrade to GPT-3 and a new much better
engine. It works pretty well.

------
Pamar
You know what? There are two people I'd really would like to interview about
GTP-3 (and what an hypothetical GTP-4 or 5 could achieve).

One is Hofstadter. The other is Ted Chang.

