That uses MTP which the OP already discounted as being "a mess". Frankly my experience matches his and thus I actively avoid MTP whenever I can.
> I think the problem is if its too easy then people will be copying files off of each others phones without permission.
You usually need to unlock the phone to use MTP. If an attacker has access to unlock your phone then it makes no difference if they copying the files via USB, Dropbox, email or whatever - your attacker already has the permission they need to do so.