I've been doing this for a while using the Google Authenticator PAM plugin[1]. I use the same module to add two factor authentication to both SSH and OpenVPN connections.
Better yet, use passwords plus client side certificates. It prevents these scenarios without requiring you to trust a third party with your two-factor authentication tokens.
Or, if you want to do the same thing without trusting the security of your VPN to a third party, and you're willing to do a little more fiddling with configuration files, you can just use libpam-google-authenticator: http://www.howtoforge.com/securing-openvpn-with-a-one-time-p...
You are only trusting the half of the security which is better IMHO. The evil person would need to hack/steal both you and the (likely more careful) third party in order to access your accounts. In your scenario it would only need to hack you to get everything.
[1] http://code.google.com/p/google-authenticator/