You must not watch many videos in your browsers then outside the odd one on Youtube. The reality out there is that Flash is still the main choice for delivering video. Flash is a necessary evil. It may be dying but for many it's still necessary and its removal from Firefox removes significant usability for its users.
I watch plenty of videos and I've never had a problem with the big streaming sites delivering h264 versions. The only flash videos I still encounter are on older, lightly, maintained sites or sites that just download flv files.
And if iPad sales are any indication I don't think it's as big a deal as you make it out to be.
The iPad includes H264 support, as does Chrome. Firefox doesn't (and neither does Chromium, does it?). So this does screw anyone over who wants to watch H264, which really is the majority of video on the web.
People won't serve you HTML 5 video unless you impersonate a mobile device. It's still a problem if you don't want to change your user-agent every time you visit a website and just cross your fingers that they think you're mobile and therefore serve up HTML 5 (and as others have pointed out, often in a format that non-Chrome won't play anyway).
Right, YT is one among a handful of exceptions that serve HTML 5 to a desktop browser. That doesn't really change the general argument -- almost all sites are not going to give HTML 5 video as an option to desktop clients.