That's not quite what I meant by the phrase. It "does damage" because others in the majority accept that it does—for the same reasons that I am "in debt" when a number on a machine reads negative. I might agree or disagree with the conception of money as a fiscal instrument, but it is supported by the group in a way that prevents me from participating in the group unless I acknowledge it and treat it the same way that they do. To not acknowledge money (or karma) is, in effect, an act of self-alienation. You "reject yourself" from the group by doing it, because the game they are now playing is a competitive one—and if you do not acknowledge the table stakes, you cannot ante.
This is why it is important to do some research on social game design (or consult a game designer) before you code a game mechanic into your social webapp. Years of research into MUDs and MMOs show that different mechanics reward, and thus encourage, different types of player-styles, and thus shift the user-base in different directions.
Here's the most famous paper on the subject, just to get anyone interested started: http://www.mud.co.uk/richard/hcds.htm . It would be awesome if someone would write an "Interpreting Bartle for Web 2.0" article and submit it back here; I've been meaning to do it for months now but it's still a ways down on my to-do list.