PS: Banking websites have their own issues. But because they tend to use multitiple forms of authentication the are significantly harder to break into on the client side.
For example, Bitcoins could be stored on a smartcard having a flexible e-paper display, flexible built-in keypad, and flexible LiPo battery . Withdrawing coins from the card could require a user typing in an amount and a pin code, and then using a smartphone to scan a QR code shown on the e-paper display (or sliding the card in the merchant's payment terminal, which would scan the QR code). The QR code would represent a signed Bitcoin transaction to a pre-programmed address whose private key sits on some online server, which is only used as an intermediary step before forwarding the coins to the final merchant. The smartcard would effectively never connect to an online device during its entire life, making it un-hackable without having physical access to it. Smartcards could also be manufactured in pairs, or triplets, etc, to have clones of them in order to have redundant backups of the Bitcoins in case of a loss of one of the cards. If you know about the Bitcoin blockchain, you might ask how the smartcard can sign transactions without access to the current blockchain. Well it is mathematically possible, because a transaction just consists of ECC-signing a few bytes representing the destination addresses.
Don't discard a technology because you are unable to comprehend it enough to think of solutions to address some of its flaws. (I do agree that addressing the security of Bitcoin wallets is of utmost importance.)
 The technology for this already exists. I own one of those: http://gallery.drfaulken.com/d/8752-1/IMG_1466.JPG
Not sure what you mean. By 'device' I didn't mean some special hardware developed by some special company, where the government can then regulate that industry. I just meant any computer. I'm saying that signing a transaction can be done offline on devices that are never connected to the Internet, such as an old laptop, or yes even a special device. There's no fundamental requirement to have the keys on your virus-ridden home PC at any time. This doesn't remove any of Bitcoin's advantages from what I can see.
And multi-signature transactions will allow for multi-factor authentication at a protocol level.
PS: Your also describing an adhock solution. As soon as you want to mass produce them to allow significant and convenient adoption you get into regulation issues. And by 'device' I am including just the software to manage your account from a cheap netbook.
Things like the BitcoinArmory client  + upcoming multi-sig transactions should make it secure enough relative to traditional currencies. Use a a *nix instead of Windows (much easier for the mainstream these days with OS X/iOS/Android) + secure wallet.dat backup like SpiderOak or Tarsnap and you're in good shape security-wise.
To secure your 'real' bank account, you need your computer that you do your online banking to be secure, your need the computer in the card reader at the store to be secure, you need the computer in the POS to be secure, you need the stores back office system to be secure, you need the computers at the credit card processing company to be secure (yes you VISA) and you need your bank to be secure.
To secure bitcoin all you need is your bitcoin wallet to be on a USB key in your pocket.
Now, plug in that bit-coin wallet into a unsecured computer and within 5 seconds your account could be drained and there is no way for you to ever recover your money. Your PC and wallet might be secure, but you have literally no way of knowing that. Worse yet as soon as large numbers of people start having a few thousand $ worth of bitcoins zero day attacks are going to take on a hole new meaning.
PS: I don't do online banking or use a debit card, the entire system is horribly and fundamentally insecure. But, I only need to pay off my CC every month and suddenly I have near total safety. Or, I can walk up to any ATM and suddenly have total anonymity at the cost of some risk.
PS: I still think Bitcoins are an interesting idea. I am just describing why their adoption has been so slow. There is simply no compelling reason for significant legal transactions to use Bitcoins, which covers for their inherent risks.
This is a baseless statement.
I, for one, don't want my bank controlling my money supply or telling me how I can and can't spend it.
And my claim is hardly "baseless". You can reject the orthodox views but please don't claim to be in the majority -- whether we use the polite term "heterodox" or the less polite "crank" the fringe nature of such views is apparent.
You could have a bank account where the bank claims no liability if your debit account was emptied by a hacked chip+pin reader - they just wouldn't have many customers!
Similarly a bank could decide to offer a bitcoin account where it will offer you the option of a chargeback. It will simply charge a commission on the transaction to cover itself.
Merchants would have never come up with that on their own. It was forced on them with credit cards by consumer protection laws. Debit cards don't have that legal requirement, but consumers have come to expect it.
PS: I am not going to sue if some random website fails to ship a 200$ graphics card. So reputation becomes even more important, but only because fraud will also become far easier.