I'm just one of the naïve newbies speaking out here but if you don't like something, either enter the discussion with your reasons thus or ignore it and move on.
And just to prove my point, I could flag your post without thinking twice about it. But that's not in the spirit of discussion and is a negative influence.
I choose to comment before I upvote anything, that's how I roll, but on that basis I can argue that the former constitutes a contribution while re-appropriating another button to serve the purpose of a downvote contributes nothing. If the former is the means to encourage discussion then the latter is the means to discourage it.
Downvoting is precisely as valuable as upvoting. They're both ways of saying that some things are more worthy of discussion than other things. The fact that 100 people like something does not make it good or even popular. 100,000 people might think it's an awful topic, or an awful presentation of the topic. If they all agreed on a single article that is better, they could upvote that, but if they just feel the article is particularly bad relative to all other articles, they have no recourse but a mass-upvote, which (besides being burdensome) means they then can't upvote any articles they find especially useful.
To illustrate what I'm saying: If we all agreed never to bury an article that somebody voted for, 100 NASCAR fans (an insignificant number both as a percentage of HN users and a percentage of NASCAR fans) could completely transform HN into a NASCAR fansite by bloc-upvoting articles relevant to their interests. That would be a horrible outcome.
An article about Jimmie Johnson is less appropriate to HN than all the other articles on the front page right now, and the way people register that opinion is by flagging.
If you want to influence HN, you should submit and upvote stories you prefer like the site was designed for.