My point about metrics is why I think the research is inconclusive. It’s very difficult to get a metric that’s meaningful in this context. If you said: this code base on average has 1 bug per 100 lines of code, that doesn’t say anything meaningful. If that code is assembly, that’s not very good because of how terse the code is. Whereas, if that code is Python or Ruby, that’s much better because of how concise those languages are.
Because of this, I feel like the only way to truly measure whether or not static typing has a significant effect would be to create two equivalent projects. Say you created stack overflow in Python and in C#. Then you could compare the quantity of bugs and see if it differs. But even this has problems because who knows how many bugs haven’t been caught? Is the code truly equivalent? Did the people who wrote the two codebases have slightly different experience resulting in differing number of bugs?
There’s too many variables in an experiment like this to conclusively determine whether or not static typing reduces the bugs. But, I don’t think that means that we can’t infer that eliminating a whole class of bugs is helpful.
Edit: the more I try to think about my reasoning the more I’m thinking it’s flawed. I think the answer to whether or not static typing reduces bugs is unknowable, but I strongly believe that it helps. Maybe we’ll get a study that isolates this metric one day :)
Because of this, I feel like the only way to truly measure whether or not static typing has a significant effect would be to create two equivalent projects. Say you created stack overflow in Python and in C#. Then you could compare the quantity of bugs and see if it differs. But even this has problems because who knows how many bugs haven’t been caught? Is the code truly equivalent? Did the people who wrote the two codebases have slightly different experience resulting in differing number of bugs?
There’s too many variables in an experiment like this to conclusively determine whether or not static typing reduces the bugs. But, I don’t think that means that we can’t infer that eliminating a whole class of bugs is helpful.
Edit: the more I try to think about my reasoning the more I’m thinking it’s flawed. I think the answer to whether or not static typing reduces bugs is unknowable, but I strongly believe that it helps. Maybe we’ll get a study that isolates this metric one day :)