Leveraging a Constraint Solver for Minimizing Test Suites

J. Campos and R. Abreu
The Symposium on Engineering Test Harness (TSE-TH), co-located with the International Conference on Quality Software (QSIC), 2013


Software (regression) testing is performed to de- tect errors as early as possible and guarantee that changes did not affect the system negatively. As test suites tend to grow over time, (re-)executing the entire suite becomes prohibitive. We propose an approach, RZoltar, addressing this issue: it encodes the relation between a test case and its testing requirements (code statements in this paper) in a so-called coverage matrix; maps this matrix into a set of constraints; and computes a collection of optimal minimal sets (maintaining the same coverage as the original suite) by leveraging a fast constraint solver. We show that RZoltar efficiently (0.95 seconds on average) finds a collection of test suites that significantly reduce the size (64.88% on average) maintaining the same fault detection (as initial test suite), while the well-known greedy approach needs 11.23 seconds on average to find just one solution.


  author = {Campos, Jose and Abreu, Rui},
  title = {Leveraging a Constraint Solver for Minimizing Test Suites},
  booktitle = {Proceedings of the 13th International Conference on Quality Software},
  series = {QSIC '13},
  year = {2013},
  isbn = {},
  pages = {253-259},
  numpages = {6},
  url = {},
  doi = {10.1109/QSIC.2013.17},
  publisher = {IEEE Computer Society},
  address = {Washington, DC, USA},
  keywords = {Regression testing, constraint solver, empirical evaluation, fault detection,
  test suite reduction,