Improving Automated Android Test Generation
- Mobile apps are nowadays the preferred means to accomplish ubiquitous tasks like messaging, e-commerce and even playing games. Often, there exist multiple apps for the same purpose, and it is the choice of the end user to pick an appropriate app. Apps that behave unexpected, e.g., crash frequently, are sooner or later replaced, which isundesirable for the companies developing such apps. Thus, it is essential to tests apps properly before they are released onto the market. However, testing manually is often not only too cost-intensive but also too time-consuming in the short development phase, thus an automated solution is preferred. Testing mobile apps automatically received increased attention in the last decade from primarily people in academia, and several testing techniques evolved. One technique that yielded promising results, especially in different domains, is search-based software testing in which a metaheuristic, e.g., a genetic algorithm, is applied to solve an optimisation problem, e.g., test generation. A main objective ofMobile apps are nowadays the preferred means to accomplish ubiquitous tasks like messaging, e-commerce and even playing games. Often, there exist multiple apps for the same purpose, and it is the choice of the end user to pick an appropriate app. Apps that behave unexpected, e.g., crash frequently, are sooner or later replaced, which isundesirable for the companies developing such apps. Thus, it is essential to tests apps properly before they are released onto the market. However, testing manually is often not only too cost-intensive but also too time-consuming in the short development phase, thus an automated solution is preferred. Testing mobile apps automatically received increased attention in the last decade from primarily people in academia, and several testing techniques evolved. One technique that yielded promising results, especially in different domains, is search-based software testing in which a metaheuristic, e.g., a genetic algorithm, is applied to solve an optimisation problem, e.g., test generation. A main objective of test generation is to produce tests that reveal as many faults as possible.
This in turn requires the generation of tests that deeply explore the tested app. The core metric to quantify how much code tests cover is the measurement of code coverage, which can be computed at different levels of granularity ranging from determining the fraction of covered activities to a very fine-grained measurement that calculates the percentage of covered lines. This coverage information is then often used to guide the search of the employed metaheuristic.
However, current automated test generation approaches produce tests with a rather low code coverage. Thus, a substantial part of tested apps remains unexplored, which in turn misses revealing deeply residing faults. We identified three core issues that are directly related to the generation of low-coverage tests. First, the applicability of current test generators is often limited. This comprises the fact that current state-of-the-art code coverage tools are incapable of instrumenting a substantial number of apps and consequently, test generators cannot utilise detailed coverage information during exploration.
In addition, test generators are often only equipped with a primitive set of actions that are insufficient to simulate system events and complex user inputs. Second, the test execution is extremely time-consuming. This includes among other things the overhead associated with executing individual actions, intermediate restart operations as well as fitness evaluations. Since search-based algorithms require a substantial number of test executions to play out their strengths, the slow test execution impedes the effectiveness of the search. Third, the guidance offered by search-based algorithms is often hampered by applying inadequate fitness functions or by using non-representation-specific variation operators.
In this thesis we address the problem of low-coverage tests in the Android domain by proposing several enhancements for the three identified core issues. Concerning the applicability problem, we provide the implementation of a robust code coverage tool that is capable of measuring coverage at different levels of granularity and requires no access to the source code. We also propose to include actions that can simulate system events as well as complex user inputs. Regarding the performance issue, we suggest the integration of a surrogate model that is capable of predicting the outcome of individual actions or complete tests over time in order to reduce the overall test execution costs. With respect to the lack of guidance offered by traditional search-based algorithms, we suggest alternative search strategies. In the case of a deceptive fitness landscape, we propose using novelty search algorithms. Alternatively, we suggest utilising estimation of distribution algorithms that require no crossover or mutation perators to sample new tests. While all those enhancements had a positive impact on the Android test generation process, the individual empirical studies highlighted that further research is necessary to unleash the full power of the proposed search-based algorithms. In particular, exploring complex user interfaces meaningfully requires more attention whether by introducing additional actions or by extracting valuable hints to infer reasonable text inputs. In addition, the guidance offered by fitness functions is often limited because they are either designed too coarse at all or do not accurately reflect the search objectives.…


| Author: | Michael AuerORCiD |
|---|---|
| URN: | urn:nbn:de:bvb:739-opus4-14955 |
| Advisor: | Gordon Fraser, Jacques Klein |
| Document Type: | Doctoral Thesis |
| Language: | English |
| Year of Completion: | 2024 |
| Date of Publication (online): | 2024/09/30 |
| Date of first Publication: | 2024/09/30 |
| Publishing Institution: | Universität Passau |
| Granting Institution: | Universität Passau, Fakultät für Informatik und Mathematik |
| Date of final exam: | 2024/09/26 |
| Release Date: | 2024/09/30 |
| Page Number: | x, 140 Seiten |
| Institutes: | Fakultät für Informatik und Mathematik |
| Dewey Decimal Classification: | 0 Informatik, Informationswissenschaft, allgemeine Werke / 00 Informatik, Wissen, Systeme / 000 Informatik, Informationswissenschaft, allgemeine Werke |
| open_access (DINI-Set): | open_access |
| Licence (German): | Standardbedingung laut Einverständniserklärung |

