The caste system exists outside India. In Japan, people who were falconers, butchers, leather workers and other "ignoble" or "cruel" professions under Buddhism were branded as untouchables, burakumin in Japanese, sometimes also eta or hinin, literally meaning "subhuman". Somehow the lord who employed the falconer was not cruel or ignoble, funny how things work out...
The issue is that Americans with half-baked knowledge think that all Indians have a caste assigned to them, or that India has a national level caste allocation policy. For Americans who hate India, this gives them more ammo to hate us.