When you travel to a different timezones you just have to adjust your clock and, without asking, you know that shops open at 8~10am until 4~8pm with or without a midday stop for lunch (12~3). You have dinner at 18~22, and breakfast at 8~10. All the divergences depend on the country you are in but you can get good aproximations that work for 90% of them (breakfast 9, open shops 10, lunch 13, close 17, dinner 20).
But, if you go for an universal timezone, you've got to learn the different times for the different activities again and again and again. You change a one step correction (change time on your watch) to a constant struggle.
With universal timezone: You wake up when travelling, it's 17:30 and you don't remember the country unless you do the mental effort to wake up, it's time to get up or not? You start to calculate and... too late to decide, you are already woken and could not get back to sleep even if you wanted.
With different timezones: You wake up when travelling, it's 2:30 and you don't remember the country unless you do the mental effort to wake up, it's time to get up or not? No, time to sleep some more.
I was in charge of a big supply database server on an aircraft carrier, and every time we changed timezones I had to update the timezone on the server. Heading from East to West was no problem, but when we went from West to East I had to shut the server down for an hour when changing timezones, to prevent timestamps from overlapping.
Admittedly, this was partially due to bad programming, but it's just a small real-world example of problems caused by the existence of timezones.
Given that servers frequently changing timezones can be sorted out by a modicum of thought by programmers and is actually pretty rare in the grand scheme of things, I don't think it's too heavy a price to pay.
Or UNIX time. While not monotonically increasing (it has issues with UTC leap seconds), it's pretty good.
You basically have to reinvent timezones every time you make any kind of calculation involving people in another timezone.
Similarly, you are on holiday and someone tells you a museum is open till 04:30. Can you go to dinner first?
I guess that a main question is what kind of query is more frequent: "What time is it now in X", or "At what time is X where I am now?". I thought the former was way more frequent, but now, I do not know anymore.
Maybe the best thing to do is to drop absolute references altogether. IMO, "Shall we meet in 3 hours" is easier for handling across timezone discussions, and will work equally well for "in a different timezone than the one I am used to".
Of course, your SMS/ping/twitter/email client would have to automatically count down such timers for you.
For someone who travels a lot, a single time zone may well be more inconvenient, hard to say.
Times relative to now would work well but could be unwieldy. How do you do it if you're e-mailing or texting? If you want to meet next Monday afternoon, do you say "in 2 days and 5 hours"?