I remember a parent being "ejected" when I was younger. Technically, they didn't "have to leave", but social pressure is enough for them to recognize they should leave the situation.
* All of the parents already know who the abusive parent is
* The first warning was the coach talking to the "entire group" (at a natural stopping point). The group dialed it back (there will always be some controversial calls), but that one parent didn't get the message.
* Finally, the game stopped suddenly (ball was in play, but in a neutral position) and they we're directly addressed by the ref.
This caught the abusers attention. This was an abnormal event. They were being addressed directly. Whether they realized they were the problem, they certainly recognized the social pressure. They recognized how embarrassing it was (sadly, they probably didn't realize how embarrassing it was for their kid) and decided to take the ref's suggestion to cool down.
I doubt many refs want this responsibility, even if it were feasible. And I don't think it is feasible in most cases. For instance, how is the ref going to pick out the problem fans while also managing the game itself? And how is the ref supposed to enforce the removal?
Keep in mind these refs are often volunteers, working solo, have no official connection to the teams or field owners.
And there is a ref shortage virtually everywhere, at all levels and sports. Making it more difficult just isn't realistic.