I don't understand, why is that a bad thing? Maybe it might be an issue if you're trying to do something anonymously (but at that point, you shouldn't be using your regular machine anyway).
It's rather unexpected. Browsers carry so much baggage that you need a private mode, I wouldn't have expected something like ssh to have this sort of problem before I read about how it exposes your keys.