A great starting point is asking how many general purpose computers are on your phone. My count is currently 2-4. Your primary OS, the one you interact with. The radio baseband, which you don't have control over. Your SIM card which can speak to the baseband without your primary OS. Lastly depending on your storage, (SD Card) may have it's own cpu, but this (I don't believe in general) can be accessed by the baseband or sim card without passing through the primary os.
I've been thinking lately about a way to address this: get a "phone" with no cell hardware, and pair it with a cellular/WiFi bridge such as MiFi. Get internet-based replacement services for voice calls and texts. Now you have one less undocumented interface or super-privileged CPU to worry about.
There's a lot of practicality questions in terms of battery life, expense, convenience to carry around, but for certain security scenarios it seems like a step forward.
"And they call me paranoid."
Glad my phone, and my messages are encrypted. Filevault on my laptop.