Isn't the problem the binary consequence, complete removal or completely left alone?
Instead, a continuous incentive gradient can push people away from the line. For example, comments on HN and Ars Technica that are downvoted are displayed grayed-out so they're harder to read and more likely to be skipped over when skimming; on Ars Technica, if a comment is sufficiently downvoted, its contents are collapsed into a stub, and you have to click the expand button to read it.
Besides pushing people away from the line, another advantage of a continuous incentive gradient instead of a discrete punishment is that because the stakes are lower, subjective disagreements are less divisive. If a comment straddles the line of violating the rules, and what's at stake is whether the comment is removed or shown, then community members will argue much more vehemently with each other and with mods than if what's at stake is whether the comment should be more or less gray, or whether the comment should be completely removed or merely collapsed.
(Zuck has discussed similar ideas about "borderline content": https://www.facebook.com/notes/mark-zuckerberg/a-blueprint-f... )