Yes, if you plan ahead. Keep the .ipa files, those can be loaded onto the phone via iTunes.
If you've upgraded iOS, you can also downgrade that. It's a pain, you need to get the .ipsw file and store some sort of activation code.
Is this more or less of a pain than on the desktop? Well, if you want to downgrade an app on the desktop, you'll need to keep the old installer around. Same thing with an OS, you'll need to wipe and reinstall everything from the old OS installer.
It's more unexpected on iOS, because generally you never need to deal with installers, but I don't think it's more painful. It's just more painful by contrast, since upgrading is so nice on iOS.