It's a heuristic. If you haven't done a randomized placebo controlled double blind exhaustive study of the compound and all you have to go on is the layout of the molecule, is it a bad guess?
That doesn't change whether or not it's a good heuristic. Whether it's a good heuristic depends on how often "extra chlorine hanging off the side" is indicative of toxicity. He didn't make a claim about one atom changes in general, and even if you did you would need to show that on average across many instances from the sample population (substances we are likely actually to run into) it's misleading.
