Neural networks are a relatively simple mathematical model. They don't actually "think" or have a conscience. Neural networks are also regularly fed books, in order to model some properties of natural language.
Neurons are also relatively simple, at least in comparison to the mind. I don't think the simplicity or complexity of the underlying model has much bearing on the higher-level properties of the network.
Now, this isn't to say that the kinds of neural networks we build today are conscious, but I don't think that's because they're based on a simple mathematical model; I think that's because they don't have the network-level properties that conscious humans do, for example, a self-representation.
Here's a good introduction: http://colah.github.io/posts/2014-07-NLP-RNNs-Representation...