Case in point: we have an iPad here that is still on the iOS 5 generation it came with. It has been widely reported that iOS 6 was slower (as well as breaking things like Google Maps), iOS 7 was slower still (as well as making various visual styling changes we don't like) and both iOS 6 and iOS 7 have suffered at least one severe security flaw that iOS 5 did not have. In short, there appears to be no rational reason we would "upgrade" this device as we are repeatedly prompted to do.
Recently, Apple appears to have changed the rules for app developers so new things going into the app store have to aim primarily at iOS 7. At some point we upgraded an app we'd bought a while ago to get some much-plugged new feature that someone thought was interesting. Aside from the fact that the new feature itself turned out to be paywalled anyway, so the upgrade invitation for the app was basically just spam abusing the app store mechanism, the "upgraded" app also had a horribly glitchy UI where previously it had been smooth as silk drawing exactly the same screens.
Can we "downgrade" that app back to something that works, which we already had before, via the app store or otherwise? No, it doesn't seem so.
If we "upgraded" iOS itself, as suggested to fix the app (though we've seen no reports that doing so does in fact fix this app), could we then "downgrade" again if we didn't like it? Also no, as far as we can tell.
As it happens, we need to have Apple devices around for testing our web stuff, so we're stuck with spending some of our company money on new gear now and then for as long as our customers are doing the same, and we're stuck with using more recent versions of iOS on some devices for the same reason. But at this point, nothing would ever convince me to spend my own money on an iSomething. I was always suspicious of the lock-in and kindergarten UI, and seeing it in action, the results are at least as bad as I expected.
What is possibly more interesting is that it seems average users are getting fed up as well. Not so long ago, mobile visits to a site I run that isn't particularly geek-related were utterly dominated by iOS devices. Today, while iOS still represents a clear majority, it is a much closer balance between iOS and Android, and the trend is strongly in the latter's direction. I can't help wondering whether this is people who've now been not-upgraded to more recent versions one time too many, or who've been forced to dump expensive and otherwise working hardware for artificial software reasons, starting to vote with their wallets.
As far as development and having to spend money on hardware, though, I'd say Android is far worse than Apple in that regard. Android still has nearly half of devices running 2.x and the screen sizes are all over the map. We write and support native apps for both Apple and Android and we have far more Android devices laying around for testing than we do Apple devices.
I think it is deeply regrettable that certain parts of the industry have moved in that direction, and Android is a fine example (as is almost anything else Google makes). That said, it's still preferable to Apple, who move fast, break stuff, and then won't even let you fix it by moving back again.
Yes, if you plan ahead. Keep the .ipa files, those can be loaded onto the phone via iTunes.
If you've upgraded iOS, you can also downgrade that. It's a pain, you need to get the .ipsw file and store some sort of activation code.
Is this more or less of a pain than on the desktop? Well, if you want to downgrade an app on the desktop, you'll need to keep the old installer around. Same thing with an OS, you'll need to wipe and reinstall everything from the old OS installer.
It's more unexpected on iOS, because generally you never need to deal with installers, but I don't think it's more painful. It's just more painful by contrast, since upgrading is so nice on iOS.