I think it has to do with the observation that no matter in which directinn we look, galaxies are moving away. Its not like we look towards one direction and say, "aha, that must be the center of the universe". Its the same story in ALL directions - we wee galaxies moving away from us at increasing velocity.
