Continuous Test Generation: Enhancing Continuous Integration with Automated Test Generation

J. Campos and A. Arcuri and G. Fraser and R. Abreu
ACM/IEEE International Conference on Automated Software Engineering (ASE), 2014

Abstract

In object oriented software development, automated unit test generation tools typically target one class at a time. A class, however, is usually part of a software project consisting of more than one class, and these are subject to changes over time. This context of a class offers significant potential to improve test generation for individual classes. In this paper, we introduce Continuous Test Generation (CTG), which includes automated unit test generation during continuous integration (i.e., infrastructure that regularly builds and tests software projects). CTG offers several benefits: First, it answers the question of how much time to spend on each class in a project. Second, it helps to decide in which order to test them. Finally, it answers the question of which classes should be subjected to test generation in the first place. We have implemented CTG using the EvoSuite unit test generation tool, and performed experiments using eight of the most popular open source projects available on GitHub, ten randomly selected projects from the SF100 corpus, and five industrial projects. Our experiments demonstrate improvements of up to +58% for branch coverage and up to +69% for thrown undeclared exceptions, while reducing the time spent on test generation by up to +83%.

Bibtex

@inproceedings{Campos:2014,
  author = {Campos, Jos{\'e} and Arcuri, Andrea and Fraser, Gordon and Abreu, Rui},
  title = {Continuous Test Generation: Enhancing Continuous Integration with
  Automated Test Generation},
  booktitle = {Proceedings of the 29th ACM/IEEE International Conference on
  Automated Software Engineering},
  series = {ASE '14},
  year = {2014},
  isbn = {978-1-4503-3013-8},
  location = {Västerås, Sweden},
  pages = {55--66},
  numpages = {12},
  url = {http://doi.acm.org/10.1145/2642937.2643002},
  doi = {10.1145/2642937.2643002},
  acmid = {2643002},
  publisher = {ACM},
  address = {New York, NY, USA},
  keywords = {automated test generation, continuous integration, continuous testing,
  unit testing},
}