
How to add a million bugs to a program, and why you might want to - moyix
http://moyix.blogspot.com/2016/06/how-to-add-a-million-bugs-to-a-program.html
======
mamurphy
Answer to Question 1: Use our program! Answer to Question 2: To see if your
automated bug-testing is finding all the bugs.

~~~
cperciva
I'm not impressed with their answer to the second question. Knowing if your
automated bug-finding tools manage to find all of the bugs _introduced by this
tool_ tells you very little about whether they will find bugs not introduced
by this tool unless we have some assurance that the bugs introduced by this
tool are somehow representative of bugs in general -- and I see no such
assurance.

~~~
geoelectric
Regardless, defect seeding is a standard (albeit not common) practice in
software quality. Nowadays it's more common to talk about it in the form of
mutation testing, which is what this basically is.

There are a number of types of bugs that are significantly more common than
others, so it's possible to draw equivalence classes around them and use them
as representatives of the whole. That's the basic technical skill around
testing in general so isn't terribly scary as part of a SQA process.

~~~
reitanqild
Those who are using the JVM platform can use the pitest maven plugin.

It runs the test suite, then introduce bugs and test and at the end compiles a
line-by-line summary of what bugs where injected, caught and missed.

------
GauntletWizard
I toyed with this idea a little while back; I wrote some very critical tooling
in bash, and wanted to make certain that no matter where in the program I
encountered an error, the second try would work (presuming I had diagnosed the
error).

My thought was: These scripts are running with set -euxo pipefail. I can
insert a 'false' line between any two lines, run the script, then remove it
and run it again and I should still get the correct results. I never finished
writing this test tool, but I still think it's a good idea I want to get back
to.

------
cprecioso
Didn't Netflix have some programs they'd call "monkeys" that randomly wreak
havoc in their codebase / infrastructure? Is that related to this?

~~~
ratbeard
Chaos monkey - [http://techblog.netflix.com/2012/07/chaos-monkey-released-
in...](http://techblog.netflix.com/2012/07/chaos-monkey-released-into-
wild.html)

Nope, not related.

------
signa11
there is a 'fault-injection-framework' available here:
[https://blitiri.com.ar/p/libfiu/](https://blitiri.com.ar/p/libfiu/)

haven't used it for anything realistic, so not sure how well it pans out in
practise.

