Another virtue of non-fancy methods is that they can provide a basic explanation of their reasoning, which is very important in the medical field. I think one of the main drawbacks of deep learning is its opacity.
You can do k-NN in the feature space of the network (activation of one of the last layers) to generate partial "explanations" of the neural network decisions.
