The Big question is: WHERE is the Complexity? If Complexity is Fixed in a System, it must be Somewhere. (you can see this at play any time you look at a large software system.) Do you have simple 'blocks' and many of them? OR, do you have more complicated blocks (requiring more computons), but fewer of them? I think this is an exciting research area right now.
Biological neurons are not easily modeled as any neat mathematical system. They seem to perform significant computational tasks even when alone - as do many other cells in the body, btw. Neuron linking is also more complex than the simplistic weighted connection model. Also, in biological neural networks, you have multiple kinds of neurotransmitters, not just a single signal, and beyond the quantity of neurotransmitters and their electrical properties being transmitted, the frequency of neuron firing is also known to be a significant factor that affects computation; and beyond neurotransmitters, hormones secreted in the brain and other places have some clear effects on cognition. And these are just pieces we know - there are certainly all sorts of other effects that we don't know, some chemical, some structural, some even physical perhaps (consider how all of the moving charges in one part of the brain might interact with other parts through electromagnetic fields).
We have no idea how much of this complexity is fundamental and how much is incidental, of course. But it is certain that every part of the brain is way more complex than the ultra-simplistic ANNs, and replacing the sigmoid function with some ODE will not move that needle significantly.
The Big question is: WHERE is the Complexity? If Complexity is Fixed in a System, it must be Somewhere. (you can see this at play any time you look at a large software system.) Do you have simple 'blocks' and many of them? OR, do you have more complicated blocks (requiring more computons), but fewer of them? I think this is an exciting research area right now.