Encoding Test Requirements as Constraints for Test Suite Minimization

J. Campos and R. Abreu
International Conference on Information Technology: New Generations (ITNG), 2013

Abstract

Software (regression) testing is performed not only to detect errors as early as possible but also to guarantee that changes did not affect the system negatively. As test suites tend to grow over time, e.g., new test cases are added to test new features, (re-)executing the entire suite becomes prohibitive. We propose an approach, RZoltar, addressing this issue: it encodes the relation between a test case and its testing requirements (code statements in this paper) in a so-called coverage matrix; maps this matrix into a set of constraints; and computes a set of optimal solutions (maintaining the same coverage as the original suite) by leveraging a fast constraint solver. We show that RZoltar efficiently (0.68 seconds on average) finds a collection of test suites that significantly reduce the size of the original suite (61.12%), while greedy only finds one solution with a reduction of 56.58% in 6.92 seconds on average.

Bibtex

@inproceedings{Campos:2013:ETR:2496031.2497106,
  author = {Campos, Jose and Abreu, Rui},
  title = {Encoding Test Requirements as Constraints for Test Suite Minimization},
  booktitle = {Proceedings of the 10th International Conference on Information
  Technology: New Generations},
  series = {ITNG '13},
  year = {2013},
  isbn = {978-0-7695-4967-5},
  pages = {317--322},
  numpages = {6},
  url = {http://dx.doi.org/10.1109/ITNG.2013.51},
  doi = {10.1109/ITNG.2013.51},
  acmid = {2497106},
  publisher = {IEEE Computer Society},
  address = {Washington, DC, USA},
  keywords = {Constraint solver, empirical evaluation, regression testing, test
  suite reduction},
}