Wednesday, October 1, 2008

Limitations of Coverage Testing

Overview:
As a continuation of the coverage testing for the Stack program, I will now look for flaws in this type of testing. Because coverage testing only checks to see if you tested every line, it does not fully test the logic behind each line. To build upon my 100% coverage testing on the Stack program, I will edit the testing or the Stack program to show a major bug while maintaining the 100% coverage. This new bug can be demonstrated with a new test I create.

Procedure:
As I think of ways to create a major bug that coverage testing can not detect, Philip Johnson gave us an example where if we set a limit on the array size, we can create a bug if we push to much. As I looked more into JUnit and Coverage testing, I can see that I just have to change the logic and return variable to show a bug. I decided to hard code the return string for toString in Stack.java to work around the coverage test. A test class "testLimit" has been created to show this bug in testStack.java.

Conclusion:
This exercise showed me that coverage testing is not enough to find all bugs in a program. There needs to be thorough testing instead of just aiming for 100% coverage. All coverage can tell you is if you missed a method, line, or block in testing.

Distribution file

No comments: