The problem I see is - How do you know if a comment is negative or not? Surely, you can tell, but problematically that might be tough.
Maybe swap out the "post comment" button for 2 "commit" buttons 1 that says "Positive" the other that says "negative". (This could be done a myriad of ways, but you get the point)
You could then measure a users tenancy to post useful positive/negative comments based on the average votes that type of comment receives.
posters that consistently have low average scores for their comments could lose posting rights, for a period.