I don't know your specific situation, but most people use password-based sites that have an "I forgot my password" feature that authenticates based on email. Thus most people have a single point of failure already: their email account.
If there's a security question I use a string of random characters as the answer and don't record it (which effectively disables the feature on most sites).
You're right about email being central to authentication on the Web. This makes it important to protect.
Do you use different passwords on every site? I have three tiers of passwords I use depending on how much I care about the site in question. I can do the same thing with OpenIDs, although most of the sites that support OpenID in the first place fall into the lowest tier of necessary security.
The flip side is that putting all your auth in one basket makes it worth spending money to protect it. Unfortunately, I don't see any OpenID providers other than VeriSign doing this.
