For a time, I tried modifying my behavior by hiding downvote links on via a Stylus CSS:
[title=downvote] {
display: none
}
I had an already high upvote:downvote ratio, but this made me even more conscious of the few times I bothered to downvote a comment (because it meant I had to take extra steps to disable that CSS). I mostly only downvote for flamebait or outrageously unproductive comments.