Hyper-parameter Tuning for Search-based Test Case Generation
search-based software engineering
software test generation
MetadataShow full item record
AbstractThere are different techniques used to automatically generate adequate test cases for a software program. Among all, search-based test case generation, which is the application of meta-heuristic search for generating test cases, has been studied a lot in the literature, lately. Since, in theory, the performance of meta-heuristic search methods is highly dependent on their hyper-parameters, there is a need to study hyper-parameter tuning in this domain. In this study, we first studied the effectiveness of hyper-parameter tuning for search-based test generation exhaustively analyze 1,200 hyper-parameter configurations of a well-known search-based test generation tool for Java (EvoSuite). Finding out that some Java classes do not respond to changing hyper-parameters in terms of coverage, we were motivated to allocate tuning budget to a subset of classes rather than the entire set. We proposed a new metric (``Tuning Gain''), which estimates how cost-effective tuning a particular class is. We then predict ``Tuning Gain'' using static features of source code classes. Finally, we prioritize classes for tuning, based on the estimated ``Tuning Gains'' and spend the tuning budget only on the highly-ranked classes. To evaluate our approach, we analyzed 250 classes of 19 projects from benchmarks such as SF110 and SBST 2018 tool competition. We used a tuning approach called Meta-GA and compared the tuning results with and without the proposed class prioritization. The results show that for a low tuning budget, prioritizing classes outperforms the alternatives in terms of extra covered branches (10 times more than a traditional global tuning). Moreover, we tried different approaches like changing subset size, shrinking search space size and giving high tuning budgets to examine the potentials of our tuning method.
CitationZamani, S. (2020). Hyper-parameter Tuning for Search-based Test Case Generation (Unpublished master's thesis). University of Calgary, Calgary, AB.
University of Calgary graduate students retain copyright ownership and moral rights for their thesis. You may use this material in any way that is permitted by the Copyright Act or through licensing that has been assigned to the document. For uses that are not allowable under copyright legislation or licensing, you are required to seek permission.