We absolutely are missing parts of the story. Negotiations and contracts at that scale are going to be massively complex, I can only imagine. Even my small shop's contracts with other businesses are multi-tiered and fairly dense.
Surely there is far more to this than simply "throwing weight" around by Apple, unfortunately. As well, it hasn't been that long since the carriers ruled everything about our experience with our phone. I'm sure there's plenty of old bureaucracy left for getting them to say "yes" to what all of us would want.
Perhaps, even, the things that frustrate you were some of the things Apple had to give up to give us the other awesomeness :P
Your reaction was my initial reaction as well, and definitely one I sympathize with (which made me actually comment). But, after thinking about my own basic experiences with contracts (and if you've ever signed a mortgage, for instance; a daunting amount of contractual repercussions, heh), I can only imagine the war room that negotiations must be with those carriers.
But all that, of course, is conjecture... Apple could just be assholes :P
Maybe I'm displaying some extreme naivete here, but I thought Apple did a spectacular job during the original iPhone launch proving to the carriers (and to the world) that they really didn't need them in order to move phones. Despite the fact that the original iPhone didn't even have any subsidy and yet STILL required you to sign a 2-year contract with AT&T (what!), Apple moved units like CRAZY via their own direct sales channels (Apple retail stores and store.apple.com).
It seems to me that given this fact, there really is no reason why Apple has to continue to kowtow to the carriers. Apple should stop sucking up to them and push direct sales of unlocked phones harder than ever. People will still buy them. They should then show a complete and fearless disregard for carriers by discarding the concessions they have made to them in the hardware and software design. Just throw stuff like carrier profile restriction features completely out. Sell iPhones to and for your end-users again.
If Apple decided to actually do this, what could any of the carriers really do about it? Nothing, that's what. Ban the iPhone from their networks? They wouldn't dare. (Or at least the GSM ones wouldn't...IS-95/CDMA2K is a slightly stickier subject, I'll admit.) There always seems to be a lot of talk (especially State-side) about carriers being open-access and what that means (remember the rider on the 700MHz auctions that Google convinced the FCC to sneak in which Verizon is now bound to as the winner of that spectrum block?), but GSM carriers have always mostly been "open" with respect to devices. Part of the promise of the SIM "personality module" is that it brings to the wireless industry what modular plugs in the home brought to the wireline industry following events such as the FCC's Carterphone ruling: a decoupling of the carrier and the end-user's device. I want to see the wireless industry move in that exact same direction. Sure, AT&T could decide to block iPhones from their network, but then they'd be turning down revenue from potential customers solely out of spite, and they aren't going to do that.
So far, the only company to have the balls to pursue a strategy like this, surprisingly, is Google. For some time now, you've been able to buy an unlocked Galaxy Nexus phone from them at a very reasonable price, and the Nexus 4 continues that tradition and even goes a step further: the ONLY sales channel (at least at this time) for the Nexus 4 is direct from Google. There are no subsidized or carrier-locked versions of this phone sold through carriers. There's a single SKU (well two, really, if you count the two fixed storage capacity options) for an unlocked GSM/UMTS phone. Google may have consulted with AT&T and T-Mobile here in the States out of courtesy before releasing the phone; I don't know. But I don't think that there's anything that says they had to get their approval to make and sell the phone, and there's nothing that says a user can't buy one without consulting their carrier first and put their own SIM card in it once it arrives.
Some may point out that Google isn't moving very many phones by themselves at the moment with this sales system, so doesn't that prove that this is not a viable business plan? I would counter that Google doesn't have quite the cachet that Apple does in this market at this time for reasons completely unrelated to how phones are sold, and that Apple, unlike Google, is in a position right now where it hold all of the cards. And for the sake of the consumer, Apple should take advantage of that position. That they don't appear to be ready and willing to do this (and in fact seem to be trending in the opposite direction) worries me.
I definitely salute Google for their move, and wish other phone manufacturers would follow in their footsteps. C'mon guys: take your destiny into your own hands rather than leave it in the trust of the carriers. Seriously.