It's very difficult to weigh each point on its 'own merit'; for example, if someone is writing an article on diet, it's relatively easy for them to construct a very impressive looking article, each citation of which is genuine and does indeed match the description, and which is logical - but also completely wrong because they cherrypicked the few dozen results which supported the thesis and ignored all the other studies and meta-analyses which reveal the thesis is complete bullshit.
For example, take dual n-back. Just with the materials in http://www.gwern.net/DNB%20FAQ I could construct a case that dual n-back is a fantastic intervention which will boost children's grades, help cure ADHD, help addicts kick drugs, increase your IQ, etc - or I could construct a case that it's a statistical artifact contributed to by financially compromised researchers which has failed replication and ought to be kicked to the trashheap of history.
Both would be true if you 'evaluate of my points on their own merit'.
When someone says that him believing in pseudoscience X reduces their confidence in any of the articles on topics Y or Z, they are quite correct. They have learned reasons to think that he left evidence out, or twisted it, or didn't look for counter-evidence, or engaged in any of the myriads of sins of omission or commission that lead a reader to falsehood.