

Sonus silentii fallacy  - pud
http://pud.com/post/62758727564/sonus-silentii

======
itafroma
The fallacy the author is thinking of is called _argumentum e silentio_ , or
the argument from silence. Academically, arguments of the form "well, X
civilization is known for meticulous record-keeping and they make no mention
of Y, therefore Y must not have happened" would be good examples of arguments
from silence.

In this instance however, it'd be of the form, "if our product was defective,
our customers would speak up. We haven't heard anything from our customers
about it, therefore our product is not defective."

The author correctly points out that this formally follows the modus tollens
format:

    
    
        A: Our products are defective
        B: Customers are complaining to us
        
        P1. A → B
        P2. ~B
        ------
        C: ~A
    

This is a valid[1] argument: assuming the premises P1 and P2 are true, C
necessarily follows. The fault in logic comes from P1 not being true: it is
not necessarily the case that customers would complain to the company if the
product was defective. Since P1 is false, the argument is unsound[2] and thus
~A (our products are not defective) is not necessarily true.

However, it's important to point out that one can still inductively[3]
conclude ~A is _likely_ to be true, just not _necessarily_ true. That is:

    
    
       P1. Based on past evidence, our products being defective likely
           means customers would speak up about it.
       P2. Our customers haven't complained to us
       -----
       C: Our products are probably not defective.
    

This is a perfectly fine inductive argument. The company, should they be
claiming to make this argument, would have to be amenable to be proven wrong:
while it may be unlikely that customers would remain silent if their product
was defective, it's certainly possible.

[1]:
[https://en.wikipedia.org/wiki/Validity](https://en.wikipedia.org/wiki/Validity)

[2]:
[https://en.wikipedia.org/wiki/Soundness](https://en.wikipedia.org/wiki/Soundness)

[3]:
[https://en.wikipedia.org/wiki/Inductive_reasoning](https://en.wikipedia.org/wiki/Inductive_reasoning)

~~~
dajohnson89
Arguably, the fallacy in question could be _ad populum_. It's the appeal to
popularity: "Lots of people do x, therefore x must be good".

------
jloughry
In the second World War, bombers returning from missions over Germany were
carefully examined for battle damage and the locations of bullet holes mapped.
Armour plating is heavy and cuts into the planes' useful range and load---
where should it be placed? Abraham Wald suggested putting armour plate over
places where no bullet hole had ever been observed, because those were the
vulnerable spots. Planes shot in those locations never returned to be counted.

It's called the problem of non-ignorable non-response.

~~~
t0
It's like evolution. The bad designs just disappear. But it's hard to figure
out all of the other failures just by looking at the one that made it.

------
wrs
I worked on the first Microsoft product to ship with the "sorry, the program
has crashed, please report it" dialog box. (MSN Explorer, if you're curious.)
Very soon I had thousands of reports of crashes in a system service that were
clearly not the fault of our app. I took the reports to the other team to get
the problem fixed, and their initial response was "no users have reported it,
and it's never happened in our test lab, so we'll make it a low-priority bug".
(!!!)

Thank goodness for the internet...your app/device can report its own problems
rather than waiting for your users to do it.

(BTW: MS product teams did quickly regroup to prioritize actual crash reports
over the test lab.)

~~~
nfoz
> Thank goodness for the internet...your app/device can report its own
> problems rather than waiting for your users to do it.

Which removes convenience, privacy, and resources from your users to make up
for your shortage of proper design and testing.

/crank :)

------
StavrosK
Absence of evidence _is_ evidence of absence (it's not proof of absence). If
there is no evidence that anyone is unhappy with the product, it is strictly
better than if there _is_ evidence that some people are unhappy (obviously).
So, the "evidence of absence" one isn't a fallacy, really.

Hell, the modus tollens is a valid argument form. If X, then Y. No Y,
therefore no X (but this isn't strictly "evidence of absence").

~~~
rosser
It can be valid, in the case that _iff_ X, then Y. Then the absence of Y
implies the absence of X, but not otherwise.

And, no; absence of evidence _is not_ evidence of absence. Absence of evidence
is evidence of absence of evidence, and nothing more. "Whereof one cannot
speak, thereof one must be silent."

EDIT: I'm speaking in terms of _predicate_ logic here, not _Bayesian_ logic.
The absence of evidence _proves_ nothing, though it can — and often does —
indeed _suggest_ something.

~~~
gjm11
No, really and truly, absence of evidence _is_ (usually) evidence of absence.
It's just often extremely weak evidence.

For "no evidence of X" to be no evidence against X, it would need to be true
that you're _no more likely to see evidence of X_ if X than if not-X. That's a
pretty weird situation. It might happen if X were totally unobservable, or if
you were being systematically deceived. (I would then say that _nothing is
evidence of X_.) But the usual case is that evidence of X is more likely when
X than when not-X, and then absence of evidence of X is evidence of not-X.

More formally: I take "A is evidence for B" to mean "Pr(B|A) > Pr(B)".
"Absence of evidence is evidence of absence" means that if Pr(B|A) > Pr(B)
then Pr(~B|~A) > Pr(~B). Assuming that none of the events involved is outright
impossible, we can equivalently write the condition for being evidence as
Pr(B|A) > Pr(B|~A), and the question is whether this implies Pr(~B|~A) >
Pr(~B|A).

Well, we can express everything here in terms of four probabilities which I'll
call AB, Ab, aB, ab. "Ab" means Pr(A & ~B), etc.: capital for true, lowercase
for false. Now Pr(B|A) > Pr(B|~A) is AB/(AB+Ab) > aB/(aB+ab), and Pr(~B|~A) >
Pr(~B|A) is ab/(aB+ab) > Ab/(AB+Ab).

The first, on clearing denominators, becomes AB(aB+ab) > aB(AB+Ab); a term
AB.aB on each side cancels, leaving AB.ab > aB.Ab.

The second, on clearing denominators, becomes ab(AB+Ab) > Ab(aB+ab); a term
ab.Ab on each side cancels, leaving ab.AB > Ab.aB.

And, lo and behold, these two conditions are the same. In other words, barring
pathological cases where some of the probabilities are zero, "A is evidence
for B" is _exactly equivalent_ to "Not-A is evidence for not-B".

But the strength of the evidence can be very different in the two cases, as in
Hempel's paradox: let A be "the object I just observed is a white crow" and B
be "crows are not all black"; clearly A is very good evidence for B; so not-A
is evidence for not-B; but if what you just observed is (say) a red car then
it's _very very weak_ evidence that all crows are black. But it's evidence,
none the less: you just eliminated one possibility for something that might
have been a counterexample.

~~~
amalcon
If I haven't looked in my desk drawer, this is _not_ evidence that it's empty.
It's only evidence that it's empty if I open it and don't see anything.

Or, put another way: The real world isn't boolean. A and Not-A are not the
only options; there's also "I don't know."

The last case there is what's generally referred to when people say that
absence of evidence is not evidence of absence.

~~~
Gormo
Or to put it another way, the world isn't deterministic. A and not-A aren't
the only options; there's also "sometimes A, sometimes not-A, depending on
where in the probability distribution each situation falls".

And here's where we need to distinguish knowledge of evidence from evidence
itself; if you haven't collected any observations, then not _knowing_ of any
evidence doesn't imply that evidence is actually absent. In other words, "I
don't know" isn't actually absence of evidence.

------
erehweb
Going from "nobody has complained" to "the customers are happy" does not seem
too much of a stretch, particularly for an expensive unit. Granted, it doesn't
logically follow, but I would class this as a reasonable deduction from
evidence rather than a fallacy. Note also that it's self-correcting - once one
person complains, the company can't use that defense any more.

------
codex
Unfortunately one cannot assume the opposite either: that just because your
unit broke, others have broken, or many others have broken. You buy thousands
of things in your lifetime; odds are that at least one will suffer some kind
of freak failure, or even repeated failures.

In this case the manufacturer is in a position to have more evidence, at least
in principle, even if they might be inclined to ignore it.

In my own experience, if Google didn't turn up other instances of my problem,
the issue was extremely rare or caused by some unique quirk in my environment.

------
noonespecial
_I own a relatively expensive piece of electronic equipment that keeps
breaking.

I corresponded with someone who works for the manufacturer. His response to me
was (paraphrased), “we’ve sold thousands of these units and nobody else has
complained. Therefore, our product must be good.”_

I've never quite understood the logic behind this thinking on the part of
product manufacturers. The product has already disappointed the customer on
multiple occasions, but the customer is _still_ willing to give the product
another chance in the form of a repair or replacement; and the manufacturers
first impulse is _that rotten customer is stealing from us, treat him like
some kind of thief_.

They're about to lose that customer (and everyone he knows who asks him for a
recommendation) _forever_. Send him a replacement. Send two. For bonus points,
pay for him to ship the defective one back for detailed analysis of what was
going wrong.

~~~
PeterisP
It depends on the margin. Excluding other concerns, if you're about to lose 10
customers (1+recommendations) and your margin is 5%, then it's better to keep
the replacement; if your margin is 20% then it's better to keep the customers.

For many companies, the rate of defects and replacements determines the
difference between profit and bankruptcy.

~~~
noonespecial
This calculation does not take into account the lifetime value of a customer.
And his reviews. My grandfather told stories about a Ford lemon he got in '68
and how rotten the experience was _until he died in his late 70 's_.

~~~
PeterisP
Still, is the expected profit from lost sales really larger than the cost of a
new replacement car or two as suggested in the post above?

------
pjdorrell
A variation on this is:

* Our Internal Revenue Collection website must be really good, because no one ever complains about it, even though we have a conspicuous link to the complaints form on every page.

* Why would anyone ever hesitate to make a complaint to the Ministry of Internal Revenue Collection if they felt they had something to say?

------
greenyoda
It's also possible (and perhaps more likely) that all the other customers
received products that work OK, but there was some kind of rare manufacturing
defect in the device the author bought (e.g., an out-of-tolerance part or a
bad trace on a circuit board). Of course, the manufacturer should have
recognized that as a possibility. This kind of problem shows up a lot with
cars, where you can have a "lemon" that has recurring problems.

~~~
svachalek
Yes, that was my immediate thought. I was a little surprised I could not find
a named fallacy for it either, but the true fallacy is assuming different
objects are the same. E.G. "no one else's soup has flies in it, therefore
yours clearly does not either".

~~~
greenyoda
Something else that might apply here is Occam's Razor[1] ( _lex parsimoniae_
if you like Latin). The hypothesis that _one_ device is broken is a simpler
hypothesis than the one that _all_ devices are broken.

[1]
[https://en.wikipedia.org/wiki/Occam%27s_razor](https://en.wikipedia.org/wiki/Occam%27s_razor)

------
prof_hobart
Is it the same unit that keeps breaking, or do they keep sending replacements
and they all keep breaking?

If it's the former, then the story seems to miss the rather critical point
that even if they were 100% certain that every other user was happy, that's
irrelevant to whether he's got a faulty unit.

However good your quality control is, eventually a faulty one is going to fall
through the net, so the fact that a thousand or a million other people have
ones that didn't go wrong does't help the one person who's unit did.

Of course, if he claims to have had multiple faulty units, and no one else has
ever reported any, that's a different matter.

------
mathattack
This seems to me to be more like _Supportus Lazicus_. Many support
organizations are purposefully understaffed, so you have to mathematically
prove that there's a problem to get through the noise. For small ticket items
it's very hard to beat this. Many large ticket software firms have this
problem too. That's why you need to negotiate an out in the contract, so you
can call the salesman and say, "Your competitor is coming in for a demo in
forty five minutes. I'm not sure if this is a bug or not, can you look into it
for me?" You'll get a response within 30 mins.

------
gcb1
"but 100% of the access are using IE6. why should we add support for another
browser?"

------
kineticfocus
A Priori rationalization based on Argumentum ad Ignorantiam. Conflated with A
Fortiori of the evidence. (Just my guess)

edit: could also be labeled a non sequitur.

------
fluxon
_" I own a relatively expensive piece of electronic equipment that keeps
breaking."_

Underconstrained problem set: no $ figure, class of product, mfg name, or
indication of who repaired it. Without _any_ specifics, it's difficult to
analyze the manufacturer's response in its true context.

------
cstrat
Reading the title I attributed the faulty electronics as being a Sonos
product.

~~~
prof_hobart
I wouldn't be surprised if it was a carefully chosen name.

