I dont see how that matters. You still need a password + one file (on the phone). SSH keys with passphrase are two factors as well in my book. In fact, I would say that not having to transmit the weakest part (the mnemonic password/pin) over the wire is a plus.