Are there works that expose this limitation of MLPs more formally?
>not everything good can be made More Neural.
Neural networks are universal function approximators, so you probably mean not everything good can be made with MLPs trained by gradient descent?
>It's the lack of [...] the ability to perform inferences over discrete spaces.
How would you judge the extent to which AlphaGo has learned to react to single discrete changes in the input. It seems that it learned very well to react very sharply to whether a single stone is placed at a strategically significant position.
Are there works that expose this limitation of MLPs more formally?
>not everything good can be made More Neural.
Neural networks are universal function approximators, so you probably mean not everything good can be made with MLPs trained by gradient descent?
>It's the lack of [...] the ability to perform inferences over discrete spaces.
How would you judge the extent to which AlphaGo has learned to react to single discrete changes in the input. It seems that it learned very well to react very sharply to whether a single stone is placed at a strategically significant position.