P. Stadler, R. Ploesch, R. Ramler: Applying a Genetic Algorithm for Test Suite Reduction in Industry, 15th International Conference Software Quality Days (SWQD), Munich, Germany, May 23-25, 2023, Springer Lecture Notes in Business Information Processing (LNBIP). talk, Doi: 10.1007/978-3-031-31488-9.
Time and cost of test execution increases when regression test suites grow over time. Techniques for test suite reduction have been proposed to streamline frequent test execution in continuous integration and to optimize the set of tests without sacrificing coverage and fault detection. In this paper we report on the design of a genetic algorithm to tackle the underlying optimization problem in context of an industry project from a software company developing tools for test automation. The prototypical implementation of the algorithm has been applied to the project’s test suite containing several hundred test cases. We achieved an optimal solution with a 28 % reduction of test cases. The evaluation of the reduced test suite using higher-level coverage and mutation analyses showed a minimal loss of coverage.
The results demonstrated that the genetic algorithm can be successfully applied in industry and the achieved results are able to satisfy the requirements of the studied project. Nevertheless, major challenges have been identified by applying the approach in industry. They are related to the reliable collection of test execution data from previous test runs and dealing with test suites containing tests exhibiting unpredictable side-effects and flakiness.