Really? I ported an Atlanta number from T-Mobile to Sprint without a single problem here in Seattle. They did ask me for an Atlanta zip code during the process, but it never came up as an issue.
Phone number portability (one carrier to another) is mandated by law. They gave you no problems because they mustn't. GV style portability (a virtual number that works with any carrier) enjoys no such privilege, so they can, and in Apple/AT&T's case, have, blocked it.
It's not "a virtual number that works with any carrier." I can assure you that the number is (1) natively homed or (2) ported - to a particular physical, facilities-based carrier (generally a CLEC) at any time. In a lot of places, Google Voice is using bandwidth.com's new CLEC license holdings (after they stopped being a pure Level3 reseller), and probably the usual national-level multibillion dollar folks that are interconnected to everyone, everywhere and have a lot of direct end-office trunking: Global Crossing, XO, Level3, etc.
For example, here in Atlanta, my friend has a Google Voice number out of 404-939. If you take a look at the pooling and/or NANPA assignment information for that block, you'll find it's pooled and held by bandwidth.com's new CLEC:
