Random code changes - terminology and tools



  • Someone told me that the unit tests are good when you change something in the code under test, and the tests detect it (they fail). The changes can focus on introducing common errors, like off-by-ones, negation of conditions, and so on. If such change results in all test suite still passing, it means that a given piece is not covered. It seems to be much more powerful than calculating a naive code coverage because it's easy to get some line/branch covered, but it's not so easy to prepare for all the possibilities when a given branch can execute.

    Is there a name for such technique of injecting errors to the production code? I thought it's fuzzing, but it's about putting random data as input.

    I'm also wondering if there are any automated tools that can do it, for e. g. Java, Python or any other widely used technology. I'd like to experiment with it and see how it works in practice. While there's a risk of tests over-fitting, it could still pinpoint some issues in the production code.

    It's extremely hard to find something in the Internet when you don't know its name, hence this question.



  • Mutation testing (or mutation analysis or program mutation) is used to design new software tests and evaluate the quality of existing software tests. Mutation testing involves modifying a program in small ways

    https://en.wikipedia.org/wiki/Mutation_testing


Log in to reply
 

Suggested Topics

  • 2
  • 2
  • 2
  • 2
  • 2
  • 2
  • 2
  • 2