For example, take dual n-back. Just with the materials in http://www.gwern.net/DNB%20FAQ I could construct a case that dual n-back is a fantastic intervention which will boost children's grades, help cure ADHD, help addicts kick drugs, increase your IQ, etc - or I could construct a case that it's a statistical artifact contributed to by financially compromised researchers which has failed replication and ought to be kicked to the trashheap of history.
Both would be true if you 'evaluate of my points on their own merit'.
When someone says that him believing in pseudoscience X reduces their confidence in any of the articles on topics Y or Z, they are quite correct. They have learned reasons to think that he left evidence out, or twisted it, or didn't look for counter-evidence, or engaged in any of the myriads of sins of omission or commission that lead a reader to falsehood.