I just started using gpg-agent to do my ssh authentication via my crypto-stick (An OpenPGP smartcard with the reader built in.) I'm totally sold on it. It's got a hardware lockout so it can't be brute forced. The actual private key never touches the client machine's memory. And its reasonably cheap.
Basically they offload your private keys to a dedicated device, completely in hardware, that's isolated from the machine's memory, so they can't be compromised via a software attack. The only attacks I've ever seen used to retrieve the primary key involve cracking the case and trying to open the chips up physically.
The cards are OpenPGP, and have room for three keys, up to 3072 bit RSA. The first two are pretty standard PGP keys, a signing key and and encryption key.
The third doesn't get used as much. It's a 'certification' key. If you have a certification key, you can use gpg-agent as a drop-in replacement for ssh-agent. And then you don't need an id_rsa file or anything like that stored on your computer. It's all on the card.
When you ssh into a box that has your public key setup, gpg-agent asks for your passphrase. Once that's done, the session key and that stuff gets setup on the smartcard. Since all that happens on the card, it doesn't matter if your system is compromised, the key can't be retrieved. (It also invalidates that old XKCD joke about getting your password with a rubber hose. If you destroy the device, your password is useless.)
The cards also lock out after three invalid passwords, so they can't be brute forced. If you do lock it out, there's an admin password to remove the lock. If someone gets that wrong three times, it actually erases the keys and turns the device into a brick.
It's really way more secure than I need, but it's a neat little gadget.
Anyway, I've seen dozens of articles over the years talking about securing a server's ssh config. But I've never seen one that addressed securing the clients...
