With recent increased expectations for quality, and the growth of agile processes and test driven development, developers are expected to do more and more effective unittesting. Yet, our knowledge of when to use the variousunit level test criteria is incomplete. The paper presentsresults from a comparison of four unit level software testing criteria. Mutation testing, prime path coverage, edgepair coverage, and all-uses testing were compared on twobases: the number of seeded faults found and the numberof tests needed to satisfy the criteria. The comparisonused a collection of Java classes taken from varioussources and hand-seeded faults. Tests were designed andgenerated mostly by hand with help from tools that computetest requirements and muJava. The findings are that mutationtests detected more faults and the other three criteriawere very similar. The paper also presents a secondarymeasure, a cost benefit ratio, computed as the number oftests needed to detect each fault. Surprisingly, mutation required the fewest number of tests. The paper also discusses some specific faults that were not found and presents analysis for why not.
['Nan Li', 'Upsorn Praphamontripong', 'Jeff Offutt']
An Experimental Comparison of Four Unit Test Criteria: Mutation, Edge-Pair, All-Uses and Prime Path Coverage