2008-11-25

Seven Principles of Software Testing


Bertrand Meyer, a frequent contributor to IEEE journals, and chief architect at Eiffel Software, published an article on software testing. He proposes the following principles based upon "experience studying software testing and developing automated tools, such as AutoTest." His test principles include:
  • Definition: To test a program is to try to make it fail.

  • Tests vs. Specs: Tests are no substitute for specifications.

  • Regression Testing: Any failed execution must yield a test case, to remain a permanent part of the project's test suite.

  • Applying Oracles: Determining success or failure of tests must be an automatic process

  • Contracts as Oracles: Oracles should be part of the program text, as contracts. Determining test success or failure should be an automatic process consisting of monitoring contract satisfaction during execution.

  • Manual and Automatic Test Cases: An effective testing process must include both manually and automatically produced test cases.

  • Empirical Assessment of Testing Strategies: Evaluate any testing strategy, however attractive in principle, through objective assessment using explicit criteria in a reproducible testing process.

  • Assessment Criteria: A testing strategy's most important property is the number of faults it uncovers as a function of time.

No comments: