The problem with flags and downvotes isn't that they fail to kill the offending comment, or that they fail to cause a karma penalty. The first effect is localized to the thread and the second is very indirect.
The problem with downvotes is that they do not give meaningful feedback or encourage positive contributions. They are not even especially effective at discouraging negativity. When not lost in a sea of upvotes, downvotes only encourage defensiveness and complaining.
Flags should be visible by the commenter only, and the issue of whether to kill comments or penalize karma doesn't have to be decided immediately.
Downvotes get lost in a sea of upvotes. Flags are stronger, enough of them will kill the comment regardless of karma. It also lets you be more specific than downvoting.
The problem with downvotes is that they do not give meaningful feedback or encourage positive contributions. They are not even especially effective at discouraging negativity. When not lost in a sea of upvotes, downvotes only encourage defensiveness and complaining.
Flags should be visible by the commenter only, and the issue of whether to kill comments or penalize karma doesn't have to be decided immediately.