You have to view an individual comment to get the flag option. I'm not sure what the down votes are for. "This comment does not belong on HN", maybe.
I have no idea what the flag is for. "This content is so bad it needs to be removed, not just hidden", perhaps.
I agree with your post though! I'd be wary about having too many options for marking posts.
My observation is that downvotes also signify comments that have one of more errors of fact. Of course "fact" is also open to interpretation and not absolute.
You could say "you can only be President for two terms of 4 years each" but someone could dispute that because Roosevelt was President for longer or because in their country that is not the case. Etc.
A downvote because of inaccuracy without a comment isn't very helpful of course.