I studied AI at the Batchelors level and have from time to time read up on the discoveries. I think the problem is still the same as a decade ago despite all of the sparkling discoveries made in the meantime. We can't define the problem. We can make a really broad and concise description of what it is supposed to do, but that's not the same as defining the problem. Maybe that's not as relevant as I felt it would be (I was of the opinion back then and still am that AGI is not arriving in our or our children's lifetime). Perhaps we will stumble upon it. That is at least how we arrived at our faculties. Nature tried a billion different combinations and we are the current incarnation of matter trying to figure itself out.
There is none. We are all just bunch of programmable monkeys outside of their original regulation loop.
AI might get to the same level (as we can already see with GPT-3, as it slowly accrues wisdom), but then it will need to get a digital notebook, a calculator and a drawing board.
The advantage it will have over us is that it won't have to sleep, eat, will reproduce at the factory production rate and most importantly, it won't have emotions that would hurt when people are shit.
It won't be coherent super intelligence for quite some time. And if it becomes one, it will be slow. About the same latency as humans on the planetary level. Maybe even slower than our ~100ms.
Till then, there will be squabbling. Prepare for a literal digital ecosystem.
> The advantage it will have over us is that it won't have to sleep, eat, will reproduce at the factory production rate and most importantly, it won't have emotions that would hurt when people are shit.
That's the fate of all AI efforts: whenever we understand something well enough, it ceases to be seen as AI.
As a historic example, the A* algorithm hails from a time when searching through a graph was still seen as AI.