From what I have read (Im no expert on this) I think they basically have to mine dirt and then put the dirt in a slew of toxic chemicals that break down everything but the rare earth minerals. It leaves the issue of having large amounts of toxic chemicals that need to be disposed of. So any country with lax environmental laws don't have this huge cost (For example CHina just dumps it in the rivers where in the US you would need to spend large amounts of money disposing of it properly.
I've read in some places that it is normally found with Thorium, is that the main problem?