If you're displacing groundwater based desert agriculture, that's one thing, but it's worth keeping in mind that in general, desert != wasteland. Deserts are fragile ecosystems with many rare and specialized plants and animals.
Absolutely, but then again (almost) everything is a fragile ecosystem. And if man-made climate change is real (I'm inclined to believe that), then it's less of an absolute and more of a trade-off ... a tricky one certainly, but already we're seeing deserts expand (https://en.wikipedia.org/wiki/Desertification) and there are efforts to prevent it (https://en.wikipedia.org/wiki/Great_Green_Wall) so it could potentially be another tool.
Deserts are fragile ecosystems with many rare and specialized plants and animals.
Climate change may push the temperature of some of those deserts too high, wiping out the specialized plants and animals that live there. Perhaps we could use a solar installation to provide shade in areas that are getting too hot.