Yes, client will notice that server fingerprint changed. So the question is, how many people will ignore that notice and still enter their password? SSH is a good software in that regard, as it allows clients to notice that server changed, but still it's an attack vector, one you should not just dismiss.
