I always imagined this to have evolved from a long history of humans getting sick around rotting corpses. The logical move is to stay away from them, and thinking they're freaky-looking is a good driver for that. Though the idea of neandertals eliciting a similar reaction has always been interesting to me.
