Harder operations and you need to do a lot more of them. Far more suited to having a single massive training system then send out the information just for inference.
Another thing that can be done is to train a large neural net then figure out which bits you can cut out without sacrificing much accuracy. The newer, smaller net is then faster to run and more likely to actually fit neatly into the RAM on your phone.
> Can the pre-trained brain (the one in the phone) flip to training mode? Can you teach it something and upload that new training result to the original?
Technically you probably could, but practically the answer is no for the types of nets used in this kind of thing. You'd want to be training the net on millions of images, and even if it were as fast as the inference on the phones that'd still take way too long.
[edit - interestingly this is not only technically possible but pretty much what is often done but on more powerful machines. You can start with a pre-trained network or model and then "fine tune" it with your own data: http://cs231n.github.io/transfer-learning/]
> Or for things it doesn't recognise, do you need to add the images and classification to the training data and create a 'new brain' and download it to the phone?
This is generally the approach, yes. It has other advantages though, the performance can be checked and compared once then re-used lots of times.
> Is there one super organism (cloud based learning) that gives birth to millions of mini-minds. Each mini-mind asking it's parent to help it with things it doesn't understand. In 20 years time what will this say about consciousness? Where would it live? Is this a new way to think about minds, those that are distributed in many physicals devices?
In many ways, sounds similar to delegating work to more junior / less well trained staff.