It's some combination of reasoning and memory. But whatever it is, the thing that we call intelligent has to perform better than most people at relevant tasks.
It's interesting because some people are clearly better at thinking about very specific things. Like, is a chess champion the same kind of intelligent as a mathematician? I don't know, probably not.
If it was really straightforward to pin it down, people wouldn't be struggling with things like comparative intelligence tests.
Since the beginning of time.
What if it's not really a "thing" at all, no wonder you can't put a finger on it.
Maybe just a lack of stupidity.[0]
There's lots of other intelligent life around here besides human beings, a whole lot of them cold-blooded creatures. Some even have memory banks that are right up there, OTOH others having minuscule brains can be amazing what they accomplish, they really must be taking maximum advantage of what storage & programming they do have to work with.
Whether the behavior arises from instinct or training, or any combination, it all ends up as zillions of autonomous agents who are capable of self-learning to some degree or another.
It goes along with the territory that things like more memory, more complex data manipulation, quicker-witted actions, and adaptive responses are perceived as more intelligent. I don't know who could argue against that.
These are some things that are shared with lower life forms [1], but also programmable machines.
The stronger these characteristics, and especially when one or more such features are relatively excessive, the more remarkable the behavior seems to be across the board. Depending on the inclination, it can be quite anthropomorphic at times.
For machine learning, upping the ante on all this stuff to excess can't help but be incredibly impressive, and it may seem like the gains could never end.
With more hardware resources the remarkability does go up whether it is inherently built in, or gained through training or self-education.
All that seems like a lot of work, but that's basically just throwing hardware and megawatts at the problem.
When you look at it like hardware, for AI you need all that blank memory to learn with. When you start out, the intelligent code only takes up an insignificant amount of memory.
I say NI has "understandable" code in the mind where a bit may be inherent, but you were trained with more which you try to understand, and/or wrote more yourself which may be the most meaningful to you. This is built up slowly and carefully and virtually takes up an equal amount of memory compared to that being used for data storage. All supposedly understandable, built layer upon layer. More or less. In real life some people are always going to be more understandable than others, some very difficult to understand indeed ;)
For AI it's not that smart until it has built up its own intelligent code from the little seed it started with, and once again it's this very complex and massive code (rivaling an extensive database itself) that determines the behavior. And similar to NI, working with limited stored data (but as vast as possible) of variable accuracy (can be: vastness > quality, hardware > software) saved in the remaining available memory banks.
Ideally, somebody would be able to go over all this behavioral code to familiarize themselves well enough to root out even the most minor stupid shit.
Sounds like a tough job, but how hard can it be, think of all the work that was saved not having to write all that code to begin with ;)
If you can't outcode the machine in quantity, you better be able to outcode it in quality.
When it needs a human's touch there's nothing like the real thing. Just think how realistic it could be if there were enough brilliant brains on the task for their brain cells to be burning as many megawatts as some of the electrical hardware.
OTOH people themselves may present a fairly low bar for stupid, but that's the baseline and below that people may not buy it. Or they may not pay the kind of price they would if nothing stupid could be noticed by anybody.
Maybe someday experts will be able to unequivocally gauge intelligence, but it's going to be more work than they ever dreamed of to curtail residual stupidity down to such low levels that it will not become prominent under extreme leverage. There's more than enough to go around already.
Some degree of stupidity has never been able to be completely overcome by any life form, so there is not any other kind of autonomous training example.
People are going to need to do a lot more pinpointing of creeping stupidity, when they can't even accurately agree about intelligence itself.
.
[0] Which may be the more elusive thing some people are gifted with more than others, over a much broader spectrum.
[1] There may never have been agreement about a true measure of intelligence, but there's always been agreement that some life forms are more intelligent than others. Still nothing definitive about whether dogs are more intelligent than cats though, and they've been working on that since the cave man days. But you can take it from there.
Edit: Turned out more serious than I thought, oh well, my head's a blank.
It's interesting because some people are clearly better at thinking about very specific things. Like, is a chess champion the same kind of intelligent as a mathematician? I don't know, probably not.