
Researchers: Are we on the cusp of an ‘AI winter’? - m-i-l
https://www.bbc.co.uk/news/technology-51064369
======
rococode
I think there's an interesting disconnect right now between research and
practice. Cutting-edge research does feel like it's reaching a plateau -
across most AI fields even "major" breakthroughs are only gaining a couple
percentage points and we're probably starting to hit the limits of what
current approaches can achieve. When the state-of-the-art is 97% on a task,
there's only so much room for improvement. Yoav Goldberg posted a tweet about
Facebook's RoBERTa model that summed it up pretty well: "oh wow seems like
this boring public hyperparameter search is going to take a while" [1].
There's a vague feeling of "What's next?" now that all the benchmarks are
fairly well-solved but AI in general clearly doesn't feel solved.

However, state-of-the-art models aren't really used in production yet. I think
the trend of "use AI/ML to solve X" has only started to pick up in the past 2
years, and it'll continue well into the 2020s. The process of taking research
models and putting them into production is not standardized yet, and many
models don't even really work in production - if your model takes a second to
do an inference step that's fine for research but maybe not for a real
product.

I think in the next decade, on the research side, benchmarks will be beaten
less often, and instead there will be more focus on trying out radically new
things, understanding weaknesses in current techniques, and finding new
measurements that assess those weaknesses. On the industry side, there will
still be lots of cool and exciting new achievements as already-known
techniques are applied to old problems that haven't been addressed by AI yet.

As an aside, this was the first time in my life that I read the phrase "10s"
referring to the 2010-2019. Kind of an odd-feeling moment!

[1]
[https://twitter.com/yoavgo/status/1151977499259219968](https://twitter.com/yoavgo/status/1151977499259219968)

~~~
Veedrac
> Cutting-edge research does feel like it's reaching a plateau

It's really not. The second half of last year alone had MuZero and Megatron-
LM, to name just a couple that most scream to me that we are actually
progressing towards AGI.

You say ‘When the state-of-the-art is 97% on a task’, but solved tasks are the
least interesting tasks.

~~~
Piskvorrr
Do we have any other indicators that it's actually progressing somewhere,
besides screaming? Even such a triviality as "how do we recognize that we got
there"? The research is still in very early phases, IMNSHO: impressive
practical applications appear, but they're side effects of what appears as
random flailing: "build it bigger, see if it helps. Build it sideways, see of
it helps. Build it at full moon, see if it helps".

That suggests that the applications are the low-hanging fruit, with far more
interesting results still to be discovered.

~~~
Veedrac
MuZero is in some sense the proto-holy grail, in that it implements learning
and planning into unstructured tasks over purely internal models. While there
is an obvious chasm between it and the end point, this is still something that
has only recently become more than an abstract goal, at least to any effect.

Being able to perform planning over ‘simple’ domains like Atari games and Go
(and not even in the same trained model!) might not seem very comparable to
the real thing, but evolutionary history spent the bulk of its time building
up the basics—most animals fail most cognitive tasks—so I don't think this is
indicative of the progress being misguided, especially given networks-on-GPUs
is literally a 10 year old field.

I think MuZero is a clear example of building by principles over random
flailing. I get why there does also seem to be the latter, but it's certainly
not the whole of it, and anyhow it worked for evolution ;P.

~~~
Piskvorrr
Sure, that looks promising. I'm not holding my breath for The Holy Grail: in
an environment where it's been just-around-the-corner for as long as the field
exists, there's always another unexpected corner;)

------
cbanek
> While AGI isn't going to be created any time soon, machines have learned how
> to master complex tasks like:

> translating text into practically every language

Note: they said they have mastered these tasks.

Yeah... I'm not sure a lot of native speakers would agree. Here's a great
example of using Google Translate to automatically translate a video game.

[https://www.youtube.com/watch?v=_uNkubEHfQU](https://www.youtube.com/watch?v=_uNkubEHfQU)

> Driving cars

I'm not so sure about that one either.

I think we're in the valley that AI can do a lot of things, but are hitting
limits in accuracy where humans are still better at some of these things
sometimes. That is, the AI isn't always better than humans, even at a
specific, non-general task.

Now don't get me wrong, we've made a lot of progress, but I wonder if we can
get these things to a place better than humans before the next economic
recession. I think the biggest risk to AI is having the money dry up. Right
now the hype is strong and the money is (nearly) free. If one of those
changes, we could put this back on the shelf for another decade. If we go into
a recession, labor will be cheap, so why bother automating with AI?

For example, we had self driving freeway cars back in the 90's.[0][1] Here's
one of the lessons learned:

> In 1987, some UK Universities expressed concern that the industrial focus on
> the project neglected import traffic safety issues such as pedestrian
> protection.

And who doesn't remember the brilliant Dr. Sbaitso, my childhood therapist.
[2]

[0]
[https://en.wikipedia.org/wiki/Eureka_Prometheus_Project](https://en.wikipedia.org/wiki/Eureka_Prometheus_Project)
[1]
[https://www.youtube.com/watch?v=I39sxwYKlEE](https://www.youtube.com/watch?v=I39sxwYKlEE)
[2]
[https://en.wikipedia.org/wiki/Dr._Sbaitso](https://en.wikipedia.org/wiki/Dr._Sbaitso)

~~~
AlexCoventry
> Now don't get me wrong, we've made a lot of progress, but I wonder if we can
> get these things to a place better than humans before the next economic
> recession.

I don't think it's necessary to completely solve superhuman performance to
achieve automation of great economic value. Some of the most famous AI
achievements leverage a fairly modest intelligence improvement with massive
amounts of classic automation. E.g., the AlphaGo policy/value networks,
combined with MCTS.

It may also be possible for automation to reliably determine when it's
encountering a situation it's going to fare badly in, and then hand off to
human telepresence control. It wouldn't surprise me if the first self-driving
systems worked that way.

~~~
philipov
> and then hand off to human telepresence control. It wouldn't surprise me if
> the first self-driving systems worked that way.

The issue is that humans fare badly in situations where they don't have to pay
attention until they suddenly have a short time to react before disaster, and
that's exactly the problem that "supervised" self-driving systems have been
suffering from when they crash.

And if you have to pay constant attention to the road anyway, I'd rather be
driving myself. What's going to win won't be self-driving tech, it will be
driver assist, like advanced anti-lock brakes, or self-adjusting cruise
control.

------
allovernow
I'm not sure about the pace of progress in research, but as an ML engineer at
a startup who has been following developments, even if AI research stalls out
completely, we've been given a huge set of amazing tools to apply to all kinds
of technical problems for years to come.

I also think that even in the absence of massive breakthroughs, there's still
plenty of work to be done during a "winter" in filling in the gaps of
understanding between various SOTA advances. I think it may be the nature of
science to advance in a sort of unbalanced tree of breakthroughs, where we
drill down on certain popular and lucrative branches for a while before coming
back to fill out and balance the width of the tree, if that analogy makes
sense.

Just between transformers/autoencoders, GANs, classic classifiers, and
combinations thereof I think we are already poised to see neural networks
change society in the next ten or so years in a way similar to the influence
of the internet. Especially if hardware and cloud computing continues to
scale.

~~~
mattnewport
Can you give some examples of applications that you think will have big
impacts? I see places where current AI techniques can make incremental
improvements but I just don't see any applications that really seem game
changing. The ones that come closest tend to be dystopian unfortunately, like
most applications of facial recognition.

~~~
crubier
My startup is using AI in many forms in order to build an accurate digital
twin of the world cheaply, and extract valuable insights from it.

In a few years we will have an accurate digital twin of the world, almost
indistinguishable from the real world. this would have been impossible or way
too expensive without massive automation with AI

~~~
pauljurczak
> In a few years we will have an accurate digital twin of the world

No, you will not, unless you redefine what "accurate" means.

~~~
crubier
Google has 3D models of cities nowadays. 20years ago we only had 2D maps. Why
do you think this trend will not continue?

~~~
pauljurczak
Google Street View is full of artifacts, it's not even close to being
accurate. The same goes for satellite imagery of rugged mountains. I'm not
even mentioning the vegetation, snow cover, river levels, etc.

~~~
crubier
Accuracy is relative to the need. By accurate I mean accurate enough that we
can extract actionable information from it.

For power plant critical structures we want 0.5mm, updated every 6months. For
forest management we want 5m, updated every 2 years.

But this increase in spatial and temporal accuracy will keep on going. At some
point in the future a small swarm of insect-sized drones will be able to
capture a whole forest in a day for a super low cost. And a few people walking
with basic smartphones for will be enough to map a whole city.

In 1980 you would have said that google maps 3D and google street view would
never exist...

~~~
pauljurczak
Well, you just redefined accurate to "accurate enough that we can extract
actionable information from it". Your statement is true now.

BTW, I never say never.

------
mindcrime
No. Maybe an "AI Fall", but I doubt there will ever be another true "AI
Winter". The AI we have today is too good, and creates too much value... at
this point, there is no longer any question as to whether or not there is
value in continuing to research and invest in AI.

What will happen, almost without doubt, is that particular niches within the
overall rubric of "AI" will go in and out of vogue, and investment in
particular segments will fluctuate. For example, the steam will run out of the
"deep learning revolution" at some point, as people realize that DL alone is
not enough to make the leap to systems that employ common sense reasoning,
have a grasp of intuitive physics, have an intuitive metaphysics, and have
other such attributes that will be needed to come close to approximating human
intelligence.

Disclaimer: credit for the observation about "intuitive physics" and
"intuitive metaphysics" goes to Melanie Mitchell, via her recent AI Podcast
interview with Lex Fridman.

One other observation... while we still don't know how far away AGI is (much
less ASI), or even if it's possible, the important thing is that we don't need
AGI to do many amazing and valuable things. I also doubt many people are
actually all that disillusioned that we aren't yet living in The Matrix (or
are we???).

~~~
_bxg1
We could still have the bottom fall out of the term "AI", since there's a big
gap between the present reality - no matter how useful - and the aspirational
nature of the phrase "Artificial Intelligence". Take any business that brands
itself as an "AI startup", any quote from Mark Zuckerberg about solving
Facebook's content problem with AI, etc., and replace "AI" with "statistical
algorithms" and it just doesn't have nearly the same ring to it. That alone
means we're due for some kind of big correction.

------
zelly
Reminder: Most of the seminal accomplishments of this era's AI wave were
actually developed in the 70s-90s. Yes, even GANs and RL. This industry has
been riding on the NVIDIA welfare program for the past 10 yrs. How long until
the hardware gets maxed out?

[http://people.idsia.ch/~juergen/deep-learning-miraculous-
yea...](http://people.idsia.ch/~juergen/deep-learning-miraculous-
year-1990-1991.html)

~~~
nabla9
We need yet another set of big breakthroughs because just adding more
computing capacity is not going to carry the boom.

Maybe the correct way to measure advance in AI is Turing Awards.

~~~
zelly
Yeah, my opinion, neural networks alone aren't going to cut it. We need a
better primitive.

------
thrower123
I've been waiting for the hype and marketing to collapse for a few years.

Probably the fastest way to tarnish public perception of AI would be to keep
pushing "AI-enhanced" products in front of the consumer as has been done.
These things tend to demo well and have a nice cool factor for the first
fifteen minutes or so, but after any kind of prolonged usage the limitations
and rough-edges come up quick.

~~~
allovernow
This is brand new technology. It's going to take a few years to reliably
productionize - and most of the applied solutions will look nothing like the
research. Many real world problems are going to combine multiple neural nets
into systems with specific applications and there's a lot of detail to work
out.

The hype may collapse in the short term, but that's only because many of the
first movers are stereotypical tech startups who overpromise without truly
understanding the problem or solution spaces and therefore underdeliver.

But, speaking from personal experience, some of the tech has already been
proven - one example is massively accelerated modeling as an alternative to
slow finite difference/finite element simulation with 99% accuracy, which will
in the next 6-12 months totally change the approach to a wide range of
modeling problems, and enable a totally new form of work where instead of
setting up a model and waiting days or weeks, one may iterate effectively in
real time. There are emerging solutions to knowledge management and
"intelligent" data harvesting, where ML outputs are being manipulated in a
rudimentary form of reasoning. Think specialized industries like petroleum,
mechanical engineering, EM engineering - plenty of "layman" related features
like recommendation engines are going to flop, but the cat is out of the bag
for heavy industrial knowledge work. Just give it some time - we are on the
cusp of a monumental leap in R&D across the spectrum of human endeavor. Very
exciting times.

~~~
metalens
I used to do electromagnetic modeling using finite element methods (though now
a product manager for AI software infra) and it would to take me on the order
of hours to days or weeks to model wave interaction with real-world objects.

A machine learning model trained to understand Maxwell's Equations can in
principle be used perform said simulations, resulting in probably an order or
more of magnitude increase in simulation speed. Getting this to work well will
reduce the time (and cost) it takes to design optical sensors, radar for
autonomous vehicles, smartphone antennas, MRI machines, and more.

Having said that, it would require a lot of heaving lifting to pull this off
to achieve near-physical accuracy for real-world physics problems.

A cursory search on Google for "arxiv deep learning electromagnetics" returns
results of proofs of concept in this direction.

~~~
YeGoblynQueenne
Were would the speedup come from? I don't understand.

If I understand your comment correctly, essentially you have a hand-crafted
simulator for some physical process and then you train a neural net model to
approximate the simulator. Why would the approximated simulator have "an order
or more of magnitude increase in simulation speed"? Unless the approximation
has massive losses in accuracy, of course.

Honestly asking and really interested to know what you mean.

~~~
allovernow
It's all about precision heuristics, derived from joint probabilities of
inputs and outputs. That, by and large, is how I am increasingly coming to
understand the power of neural networks.

Imagine you are given a picture of a candle, overlaid with a grid, and asked
to fill in, with colored pencils, colors for the air surrounding the candle
representing relative temperature. Of course a human utilizes intuition to
rapidly assign high temperature to the flame and decreasing temperature with
increasing distance.

A "dumb" finite method would need, even for such a relatively simple problem
(for a human), to perform calculations for a series of time steps in each grid
until some steady state condition to arrive at a much more precise but still
overall similar coloring of the grid cells. You can do the same task much more
quickly because you have developed intuition of the physics, which is to say
you have learned heuristics which capture the general trends of the problem
(air is hot close to a flame and cold far away).

Neural nets take the best of both worlds - by effectively learning probability
relationships between input and output pixels, they internalize heuristic
approaches to produce outputs approaching finite method accuracies at a
fraction of the computation. There's a lot of waste that can be optimized out
of finite computation by hardcoding rules (heuristics), but doing so for real
problems is impractical. Neural nets learn these rules through training - a
far simpler task is organizing the data to teach the net the right trends;
much like designing lessons for a child to teach a predictive ability.

~~~
YeGoblynQueenne
I'm skeptical of the claim that it's easier to train a neural net than to
hand-code a set of heuristics _when the heuristics are already known_. For the
time being, optimal results with neural nets need more data and more computing
power ("more" because it's never enough) and are primarily useful when a hand-
coded solution is not possible.

I also don't understand how it is possible for a neural net (or any
approximator, really) to approximate a "precision heuristic" faster than a
hand-coded heuristic and without a gross loss of well, precision in the order
that would make the results unusable for engineering or scientific tasks.
Could you elaborate?

~~~
richk449
I’m also skeptical, but after reading the explanation above, I am intrigued.

Say I have a cube with 100 x 100 x 100 mesh cells inside, and ports on
opposing faces. Given enough time, I can literally run through every possible
combination of PEC and air for every cell and solve the FD form of maxwells
equations, then save the results. Now, a user can ask my solver for any of
those cases, and I simply pull the presolved result, and give the user the
answer with orders of magnitude reduction in time.

Obviously, the presolving approach doesn’t scale. More materials, more mesh
cells, eventually it is impractical to presolve every case. But the beauty of
neural networks is that they can be very good at generalizing from a partial
sample of the problem space. In effect, they can give results close enough to
the presolve solution with drastically reduced numbers of computations.

~~~
YeGoblynQueenne
>> But the beauty of neural networks is that they can be very good at
generalizing from a partial sample of the problem space.

That is really not the case. Neural nets generalise very poorly, hence the
need for ever larger amounts of data: to overcome their lack of generalisation
by attempting to cover as many "cases" as possible.

Edit: when this subject comes up I cite the following article, by François
Chollet, maintainer of Keras:

 _The limitations of deep learning_

[https://blog.keras.io/the-limitations-of-deep-
learning.html](https://blog.keras.io/the-limitations-of-deep-learning.html)

I quote from the article:

 _This stands in sharp contrast with what deep nets do, which I would call
"local generalization": the mapping from inputs to outputs performed by deep
nets quickly stops making sense if new inputs differ even slightly from what
they saw at training time. Consider, for instance, the problem of learning the
appropriate launch parameters to get a rocket to land on the moon. If you were
to use a deep net for this task, whether training using supervised learning or
reinforcement learning, you would need to feed it with thousands or even
millions of launch trials, i.e. you would need to expose it to a dense
sampling of the input space, in order to learn a reliable mapping from input
space to output space._

~~~
allovernow
Well...I think that take is a little overly cynical, and I disagree
particularly with this:

>the mapping from inputs to outputs performed by deep nets quickly stops
making sense if new inputs differ even slightly from what they saw at training
time

In my experience that isn't really true, if you have an appropriately designed
net, training data which appropriately samples the problem space, and the net
is not overtrained (overfit).

You can think of training data as representing points in high dimensional
space. Like any interpolation problem, if you sample the space with the right
density, you can get accurate interpolation results - and neural nets have
another huge advantage, in that they learn highly nonlinear interpolation in
these high d spaces. So the net may be unlikely to generalize to points
outside of the sampled space - although now that I think of it I'm not sure of
how nets handle extrapolation - but when you're dealing with space with
thousands of dimensions (like each pixel in an image) you can still derive a
ton of utility from the interpolation which effectively replaces hardcoded
rules about the problem you're solving.

~~~
YeGoblynQueenne
I may be jumping the gun a little because I was thinking about this in the
context of another thread, but a practical problem with machine learning in
general is that, for a learned model to generalise well to unseen data, the
training dataset (all the data that you have available, regardless of how you
partition it to training, testing and validation) must be drawn from the same
distribution as the "real world" data.

The actual problem is that this is very difficult, if not impossible, to know
before training begins. Most of the time, the best that can be achieved is to
train a model on whatever data you have and then painstakingly test it at
length and at some cost, on the real-world inputs the trained model has to
operate on.

Basically, it's very hard to know your sampling error.

Regarding interpolation and dense sampling etc, the larger the dimensionality
of the problem the harder it gets to ensure your data is "dense", let alone
that it covers an adequate region of the instance space. For example, the
pixels in one image are a tiny, tiny subset of all pixels in all possible
images- which is what you really want to represent. Come to that, the pixels
in many hundred thousands of images are still a tiny, tiny subset of all
pixels in all possible images. I find Chollet's criticism not cynical, but
pragmatic and very useful. It's important to understand the limitations of
whatever tool you're using.

>> although now that I think of it I'm not sure of how nets handle
extrapolation

They don't. It's the gradient optimisation. Gets stuck to local minima, always
has, always will. Maybe a new training method will come along at some point.
Until then don't expect exrapolation.

------
mellosouls
In the sense of AGI, it's _all_ been hype. We are in an ML summer and have
been for the past few years.

But "deep learning" is nothing more than that, nothing to do with AGI, we're
not approaching an AGI winter except for people who were daft enough to fall
for the hype.

There have been no advances in AGI in decades, it's already winter, and we've
_long_ been in it.

~~~
zelly
> it's already winter, and we've long been in it

In terms of research and innovation, yes, but in economic terms, it has not
even begun. There is still huuuge VC and government money being pumped into
anything with AI on it. The last AI winter started when the financiers
discovered the disconnect between the money they put in and delivery on
promises.

AI went from an obscure hard CS field that only a few graybeards at MIT knew
anything about, to this worldwide meme. Before the default thing your
grandmother would tell you to study in college was business. Now your
grandmother would tell you to study AI. I'm seeing a lot of people enter this
space with the vague goal of getting rich quick. This is the same cohort that
jumped into tech in the late 90s and the real estate market in the mid-2000s.
It's not the AI of the Norvig and Marvin Minsky days.

I couldn't be more bearish about AI. I still love it though. I won't stop
studying it when it becomes not cool anymore.

~~~
mellosouls
Agree, me too. It's one of the most fascinating problems in the world, and
I'll be delighted when "they" decide it's dead and the spotlight moves on.

------
mark_l_watson
I have worked in the field since 1982, so I have experienced “the need to work
on other things for a while” to earn a living.

My prediction is that we are going to see a small revolution in cost
reduction: hardware for deep learning will get cheaper; great educational
materials like fast.ai and Andrew Ng’s lessons will increase the hiring pool
of people who know enough to be useful; the large AI companies will continue
to share technology and trained models to help their hiring funnel and general
PR; programmer less modeling will really start to be a real thing.

~~~
sgt101
Alot of the cost in AI projects now are in training or education, but instead
in problem solving and plumbing. AI/ML free projects using things like Kafka
and Flink are not cheap.

Coding up a CNN or MLP is not a big deal, but it never really was - it was
work to build a c back propagation implementation but if I did it in 1995 then
anyone could. The question and real differentiator is in answering three
problems :

\- what's the problem? \- how can we get the data to the system? \- how do we
frame the data and output in terms of (any) AI technology?

All of these steps are closely coupled and require expertise.

On the programmer less modelling; I still have not seen a tool that is better
than code for expressing a model precisely and testably, and my experience is
that until we have some running code we don't really know that we understand
the system.

------
ddragon
The AI summer/winter cycle feels to me like something similar to a search
algorithm, we have a phase of exploration, which seem to have slower, if any,
progress and no one is very sure what's the next big thing so they perhaps
start trying many things (the winter "skepticism") and eventually someone
finds some breakthrough and gets everyone to an exploitation phase in which
everyone knows where to invest and comparatively small effort is required to
create progress (the summer "hype"). And eventually all low-hanging fruits are
over and the search seems to converge to a local maximum and again larger
exploration is required.

So maybe the winter is just as important as the summer. Each winter lead to a
summer with different focus points (specialist systems and logic followed by
neural networks, bayesian models and SVMs and finally deep learning). And
after each cycle we have more and more tools, each more useful than the last.
And also maybe the key to avoid this strict cycle would be to encourage more
exploration during the exploitation phase, giving full support to both
incremental ideals that improve on the state of the art and (potentially)
revolutionary ideas that give poor immediate results but create new venues to
investigate.

Of course that's a simplification and there are many aspects to it, including
data availability, hardware and tooling that can easily prevent brilliant
ideas that were had too soon.

------
catoc
Can't comment on every industry, but in medicine - especially the 'pattern-
recognition-specialties' such as formost pathology and radiology - the actual
implementation/usefulness/impact of "AI" (ML/DL) has not yet taken a foothold.

Yes it's hyped, but the match between even the current state of DL and what is
needed and possible in these specialties is so close to being perfect, and the
gain is so close. What is holding us back is regulatory issues and technical
implementation issues that have nothing to with the state of DL, just basic IT
problems, lack of standards.

Investments may fall back and companies may stop adversing it as "AI", but the
impact of ML/DL in medicine will not fall back.

The "AI" we see today is already effective, just not applied at scale.

~~~
glial
There is a business opportunity here.

~~~
catoc
Oh yes - and it's seeing massive investments. Last RSNA - largest Radiology
conference - saw >100 AI startups showcasing their products. (half of which
will probably be gone by December this year, bankrupt, taken over or merged)

------
richk449
Why would there be an AI winter? Was there a car winter after cars became a
growing product? Was there a processor winter after microprocessors became a
growing product? ERP software?

Didn’t the previous AI winter happen because the hardware wasn’t advanced
enough to make the technology useful to most people? Since that is no longer
the case, why this consistent belief that there will be another winter?

~~~
soVeryTired
2001 wasn't exactly great for the semiconductor industry...

~~~
richk449
Sure, there will be ups and downs like any industry. But a “winter” implies a
decade at least without substantial industry or research, not just a bad year.

------
m_ke
Research progress might slow down for a bit in some areas of machine learning
but the commercialization of existing technology will keep us busy for the
next 10 years.

Unlike the past two winters, deep learning is actually enabling a ton of
applications that wouldn't have been possible otherwise and we now live in a
world with a lot more data and computers to apply it to.

~~~
catoc
Exactly this

------
dr_dshiv
We don't want AI, we want systems that work better autonomously. We have lots
of autonomous systems, mostly run by people (a shop keeper for a shop owner,
for instance). Now that we have reached certain limits of pure digital
systems, more innovations (ie, changes leading to better system outcomes) will
happen due to human involvement in the data understanding and automation. It's
just going to look more like people going to work.

The idea that AI (ML models) would be designed once is silly. The tuning and
application always involves human judgment over time. We just hide the human
contributions to AI/ML systems because it gets too complicated. But really,
all good/practicable/in-the-wild AI systems involve a lot of people-in-the-
loop!

------
m0zg
IMO, no. Unlike the last time, things actually work this time. Perceptual
things especially. People in the thread seem to be dismissive of those "single
digit percentage point" gains that are being made nearly every month in some
important tasks, but those last few percentage points often decide whether the
system is garbage or useful. Compare e.g. Siri and Google Assistant, for
example. Likely relatively small difference on metrics which results in a
_huge_ difference in usability.

Another mistake people make is they look at model performance on academic
dataset and make unsubstantiated conclusions about usefulness of models. Guess
what, practical tasks _do not_ involve academic datasets. Some of academic
datasets are _stupid hard_ on purpose (e.g. ImageNet, which forces your net to
recognize _dog breeds_ that few humans can recognize). If your problem is more
constrained, and the dataset is large enough and clean enough, you can often
get very good results on practical problems, even with models that do not do
all that well in published research.

------
zmmmmm
I actually think that basic deep learning is well on its way into the plateau
of productivity. It's not going to be used strictly as AI though, just a more
robust type of model fitting than traditional ML which required cleaner data
and better extracted features.

------
WilTimSon
While the expectation vs. reality dichotomy is very real, the cost vs. return
is just as vital and, ultimately, more easily solvable in the years to come.
Curbing the expectations of the money-givers in regards to what they might get
out of these ventures is always tough but using tech is going to be cheaper
because, well, the price trends for tech have been downward for a while.

Personally, hoping to see more shifts toward trying new things rather than
attempting to perfect the already existing models. This would, well, not solve
but circumvent the need to try and improve something when the tools are not
there yet. This way, a broader groundwork will be laid.

------
KKKKkkkk1
I heard that the top tech research labs are already experiencing cutbacks and
hiring freezes. Can anyone confirm?

------
etaioinshrdlu
I think the total economic impact of AI will be greatest for tasks that output
high-dimensional data, such as GANs. For the simple reason that it can replace
a lot more human labor. A great many jobs could be augmented with such tech.

Furthermore, I think the results from GPT-2 and similar language models show
that researchers have found a scalable technique for sequence understanding.
They are likely to just work better and better as you throw more data and
training time at them. Imagine what GPT-2 could do if trained on 1000x more
data and had 1000x more parameters. It would probably show deep understanding
in a great variety of ideas and if prompted properly would probably pass a lot
of Turing tests. There is evidence that this type of model learns somewhat
generally, that is, structures it learns in one domain do help it learn faster
in other domains. I am not sure exactly what would be possible with such a
model, but I suspect it would be extremely impressive and meaningful
economically.

I think we are likely to see that type of progress in the next year or two,
and for there to be no AI winter.

~~~
ForHackernews
Does GPT-2 really "understand" anything? I feel like this is pretty quickly
going to devolve into a semantic argument, but having interacted with some
trained GPT-2 models, it seems to produce only what Orwell would have called
duckspeak[0]. There's very clearly no mind behind the words, so it's hard for
me to credit it with understanding.

[0]
[http://www.orwelltoday.com/duckspeak.shtml](http://www.orwelltoday.com/duckspeak.shtml)

~~~
etaioinshrdlu
I think the only time a system can be truly be said to understand something is
when its answers are derived from logic (such as old school symbolic AI). No
matter how good current statistical approaches get, they won't meet that bar.

However, I do believe we see evidence of approximate logical reasoning in
these models, as well as the concept of abstraction.

Furthermore we can take statements generated with statistical techniques and
validate them mechanically with older techniques. This is basically what
recent work in automated theorem proving using deep learning is about.

Generating logical statements using heuristics and then validating them
mechanically also sounds like a reasonable approximation of what a human often
does, speaking as a human.

~~~
ForHackernews
> Generating logical statements using heuristics and then validating them
> mechanically also sounds like a reasonable approximation of what a human
> often does, speaking as a human.

I think I agree with that, but I might add that humans who understand a topic
well can also make novel connections and uncover further implications that
might seem illogical at first glance. This process of "insight" seems poorly
understood by everyone, but I think it goes beyond validating heuristic
intuition.

------
tomrod
AI winter came because the best in-market was Clippy and Naive Bayes spam
detectors.

------
blackrock
Perhaps Artificial Intelligence needs to be rebranded?

Maybe call it:

Cybernetic Research (CR)

Computational Cognition (CC)

Statistical Reasoning (SR)

Computational Reasoning (CR)

------
bitL
There might be another A(G)I winter, but there won't be an ML winter...

------
hooande
never use the phrase "AI Winter" again. never.

