This problem is not insolvable. As you pointed out, a secure device can be built to securely transact. Or use a "bank" (but I don't like it because any centralization is against the design principle of Bitcoin).
For example, Bitcoins could be stored on a smartcard having a flexible e-paper display, flexible built-in keypad, and flexible LiPo battery . Withdrawing coins from the card could require a user typing in an amount and a pin code, and then using a smartphone to scan a QR code shown on the e-paper display (or sliding the card in the merchant's payment terminal, which would scan the QR code). The QR code would represent a signed Bitcoin transaction to a pre-programmed address whose private key sits on some online server, which is only used as an intermediary step before forwarding the coins to the final merchant. The smartcard would effectively never connect to an online device during its entire life, making it un-hackable without having physical access to it. Smartcards could also be manufactured in pairs, or triplets, etc, to have clones of them in order to have redundant backups of the Bitcoins in case of a loss of one of the cards. If you know about the Bitcoin blockchain, you might ask how the smartcard can sign transactions without access to the current blockchain. Well it is mathematically possible, because a transaction just consists of ECC-signing a few bytes representing the destination addresses.
Don't discard a technology because you are unable to comprehend it enough to think of solutions to address some of its flaws. (I do agree that addressing the security of Bitcoin wallets is of utmost importance.)