illustrates the one way nature of neural nets and indicates that that these may be much much harder to reason about and debug than in classic cases of concurrency with shared state.
I'm the author of the blog post.
I think you're right in saying that neural nets are hard to reason about. It's probably impossible to put into words exactly how a deep neural network achieves its results. There are two ways to interpret this:
1) This is a bad thing, because we don't understand exactly how the neural net works. We can't really be sure there won't be an unexpected failure case.
2) This is a good thing, because we don't HAVE to understand how the neural network works. We had no idea how to solve the problem, but we just threw loads of data at a neural network, and it solved the problem.
The "correct" interpretation probably depends on the situation/problem. That being said, my former colleagues and myself made some attempts, with limited success, in understanding neural networks trained for some specific problems.
And in this paper
http://arxiv.org/pdf/1211.1552.pdf
starting in section 4, we provide a much more in depth analysis regarding neural networks trained for image denoising.
Utlimately, our analyses "fail" in the sense that we cannot put into simple words how the neural networks work. However, we are successful in gaining limited insight into how the neural networks operate.
