It's politically difficult, not statistically difficult. Statistically VAM does a great job.
Paying attention to the "good" students vs the difficult to teach ones is not enforced by every objective measurement system, it's purely a function of how you compute the teacher's score. There are many choices:
# focus on the best, ignore the rest
# focus on the worst, ignore the rest
# Focus on the cheapest improvements possible
# independent of whether best or worst
# Somewhere in between mean and max
# 1 < K < infinity