In the second World War, bombers returning from missions over Germany were carefully examined for battle damage and the locations of bullet holes mapped. Armour plating is heavy and cuts into the planes' useful range and load---where should it be placed? Abraham Wald suggested putting armour plate over places where no bullet hole had ever been observed, because those were the vulnerable spots. Planes shot in those locations never returned to be counted.
It's called the problem of non-ignorable non-response.
The fallacy the author is thinking of is called argumentum e silentio, or the argument from silence. Academically, arguments of the form "well, X civilization is known for meticulous record-keeping and they make no mention of Y, therefore Y must not have happened" would be good examples of arguments from silence.
In this instance however, it'd be of the form, "if our product was defective, our customers would speak up. We haven't heard anything from our customers about it, therefore our product is not defective."
The author correctly points out that this formally follows the modus tollens format:
A: Our products are defective
B: Customers are complaining to us
P1. A → B
P2. ~B
------
C: ~A
This is a valid[1] argument: assuming the premises P1 and P2 are true, C necessarily follows. The fault in logic comes from P1 not being true: it is not necessarily the case that customers would complain to the company if the product was defective. Since P1 is false, the argument is unsound[2] and thus ~A (our products are not defective) is not necessarily true.
However, it's important to point out that one can still inductively[3] conclude ~A is likely to be true, just not necessarily true. That is:
P1. Based on past evidence, our products being defective likely
means customers would speak up about it.
P2. Our customers haven't complained to us
-----
C: Our products are probably not defective.
This is a perfectly fine inductive argument. The company, should they be claiming to make this argument, would have to be amenable to be proven wrong: while it may be unlikely that customers would remain silent if their product was defective, it's certainly possible.
I worked on the first Microsoft product to ship with the "sorry, the program has crashed, please report it" dialog box. (MSN Explorer, if you're curious.) Very soon I had thousands of reports of crashes in a system service that were clearly not the fault of our app. I took the reports to the other team to get the problem fixed, and their initial response was "no users have reported it, and it's never happened in our test lab, so we'll make it a low-priority bug". (!!!)
Thank goodness for the internet...your app/device can report its own problems rather than waiting for your users to do it.
(BTW: MS product teams did quickly regroup to prioritize actual crash reports over the test lab.)
Absence of evidence is evidence of absence (it's not proof of absence). If there is no evidence that anyone is unhappy with the product, it is strictly better than if there is evidence that some people are unhappy (obviously). So, the "evidence of absence" one isn't a fallacy, really.
Hell, the modus tollens is a valid argument form. If X, then Y. No Y, therefore no X (but this isn't strictly "evidence of absence").
It can be valid, in the case that iff X, then Y. Then the absence of Y implies the absence of X, but not otherwise.
And, no; absence of evidence is not evidence of absence. Absence of evidence is evidence of absence of evidence, and nothing more. "Whereof one cannot speak, thereof one must be silent."
EDIT: I'm speaking in terms of predicate logic here, not Bayesian logic. The absence of evidence proves nothing, though it can — and often does — indeed suggest something.
No, really and truly, absence of evidence is (usually) evidence of absence. It's just often extremely weak evidence.
For "no evidence of X" to be no evidence against X, it would need to be true that you're no more likely to see evidence of X if X than if not-X. That's a pretty weird situation. It might happen if X were totally unobservable, or if you were being systematically deceived. (I would then say that nothing is evidence of X.) But the usual case is that evidence of X is more likely when X than when not-X, and then absence of evidence of X is evidence of not-X.
More formally: I take "A is evidence for B" to mean "Pr(B|A) > Pr(B)". "Absence of evidence is evidence of absence" means that if Pr(B|A) > Pr(B) then Pr(~B|~A) > Pr(~B). Assuming that none of the events involved is outright impossible, we can equivalently write the condition for being evidence as Pr(B|A) > Pr(B|~A), and the question is whether this implies Pr(~B|~A) > Pr(~B|A).
Well, we can express everything here in terms of four probabilities which I'll call AB, Ab, aB, ab. "Ab" means Pr(A & ~B), etc.: capital for true, lowercase for false. Now Pr(B|A) > Pr(B|~A) is AB/(AB+Ab) > aB/(aB+ab), and Pr(~B|~A) > Pr(~B|A) is ab/(aB+ab) > Ab/(AB+Ab).
The first, on clearing denominators, becomes AB(aB+ab) > aB(AB+Ab); a term AB.aB on each side cancels, leaving AB.ab > aB.Ab.
The second, on clearing denominators, becomes ab(AB+Ab) > Ab(aB+ab); a term ab.Ab on each side cancels, leaving ab.AB > Ab.aB.
And, lo and behold, these two conditions are the same. In other words, barring pathological cases where some of the probabilities are zero, "A is evidence for B" is exactly equivalent to "Not-A is evidence for not-B".
But the strength of the evidence can be very different in the two cases, as in Hempel's paradox: let A be "the object I just observed is a white crow" and B be "crows are not all black"; clearly A is very good evidence for B; so not-A is evidence for not-B; but if what you just observed is (say) a red car then it's very very weak evidence that all crows are black. But it's evidence, none the less: you just eliminated one possibility for something that might have been a counterexample.
Or to put it another way, the world isn't deterministic. A and not-A aren't the only options; there's also "sometimes A, sometimes not-A, depending on where in the probability distribution each situation falls".
And here's where we need to distinguish knowledge of evidence from evidence itself; if you haven't collected any observations, then not knowing of any evidence doesn't imply that evidence is actually absent. In other words, "I don't know" isn't actually absence of evidence.
Most fallacies are like that: "Hey, X isn't actually ironclad, irrefutable proof of Y. (So you get to ignore X.)"
Of course, X usually is evidence of Y, and the above is far too high of standard anyway. Hence why I don't see fallacy guides as deeply insightful, nor any urgency in updating them to cover every special case.
Going from "nobody has complained" to "the customers are happy" does not seem too much of a stretch, particularly for an expensive unit. Granted, it doesn't logically follow, but I would class this as a reasonable deduction from evidence rather than a fallacy. Note also that it's self-correcting - once one person complains, the company can't use that defense any more.
Unfortunately one cannot assume the opposite either: that just because your unit broke, others have broken, or many others have broken. You buy thousands of things in your lifetime; odds are that at least one will suffer some kind of freak failure, or even repeated failures.
In this case the manufacturer is in a position to have more evidence, at least in principle, even if they might be inclined to ignore it.
In my own experience, if Google didn't turn up other instances of my problem, the issue was extremely rare or caused by some unique quirk in my environment.
I own a relatively expensive piece of electronic equipment that keeps breaking.
I corresponded with someone who works for the manufacturer. His response to me was (paraphrased), “we’ve sold thousands of these units and nobody else has complained. Therefore, our product must be good.”
I've never quite understood the logic behind this thinking on the part of product manufacturers. The product has already disappointed the customer on multiple occasions, but the customer is still willing to give the product another chance in the form of a repair or replacement; and the manufacturers first impulse is that rotten customer is stealing from us, treat him like some kind of thief.
They're about to lose that customer (and everyone he knows who asks him for a recommendation) forever. Send him a replacement. Send two. For bonus points, pay for him to ship the defective one back for detailed analysis of what was going wrong.
I was eating at a bar the other day and this guy next to me was telling his girlfriend that he wanted a new iPhone. He planned on giving his mother his current iPhone, but he wanted to make sure that she wouldn't have any problems with it so he intended to break it somehow and give her the replacement. He demonstrated a possible technique for doing so; repeatedly slamming it flat against the bar (firmly, but not violently).
I did not catch his girlfriend's reaction, but I was disgusted. I'm sure Apple plans for and can afford this type of loyal customer, but everyone ends up paying for it.
It depends on the margin. Excluding other concerns, if you're about to lose 10 customers (1+recommendations) and your margin is 5%, then it's better to keep the replacement; if your margin is 20% then it's better to keep the customers.
For many companies, the rate of defects and replacements determines the difference between profit and bankruptcy.
This calculation does not take into account the lifetime value of a customer. And his reviews. My grandfather told stories about a Ford lemon he got in '68 and how rotten the experience was until he died in his late 70's.
* Our Internal Revenue Collection website must be really good, because no one ever complains about it, even though we have a conspicuous link to the complaints form on every page.
* Why would anyone ever hesitate to make a complaint to the Ministry of Internal Revenue Collection if they felt they had something to say?
It's also possible (and perhaps more likely) that all the other customers received products that work OK, but there was some kind of rare manufacturing defect in the device the author bought (e.g., an out-of-tolerance part or a bad trace on a circuit board). Of course, the manufacturer should have recognized that as a possibility. This kind of problem shows up a lot with cars, where you can have a "lemon" that has recurring problems.
Yes, that was my immediate thought. I was a little surprised I could not find a named fallacy for it either, but the true fallacy is assuming different objects are the same. E.G. "no one else's soup has flies in it, therefore yours clearly does not either".
Something else that might apply here is Occam's Razor[1] (lex parsimoniae if you like Latin). The hypothesis that one device is broken is a simpler hypothesis than the one that all devices are broken.
Is it the same unit that keeps breaking, or do they keep sending replacements and they all keep breaking?
If it's the former, then the story seems to miss the rather critical point that even if they were 100% certain that every other user was happy, that's irrelevant to whether he's got a faulty unit.
However good your quality control is, eventually a faulty one is going to fall through the net, so the fact that a thousand or a million other people have ones that didn't go wrong does't help the one person who's unit did.
Of course, if he claims to have had multiple faulty units, and no one else has ever reported any, that's a different matter.
This seems to me to be more like Supportus Lazicus. Many support organizations are purposefully understaffed, so you have to mathematically prove that there's a problem to get through the noise. For small ticket items it's very hard to beat this. Many large ticket software firms have this problem too. That's why you need to negotiate an out in the contract, so you can call the salesman and say, "Your competitor is coming in for a demo in forty five minutes. I'm not sure if this is a bug or not, can you look into it for me?" You'll get a response within 30 mins.
"I own a relatively expensive piece of electronic equipment that keeps breaking."
Underconstrained problem set: no $ figure, class of product, mfg name, or indication of who repaired it. Without any specifics, it's difficult to analyze the manufacturer's response in its true context.
It's called the problem of non-ignorable non-response.