Vet: Identifying and Avoiding UI Exploration Tarpits

Abstract

Despite over a decade of research, it is still challenging for mobileUI testing tools to achieve satisfactory effectiveness, especially on industrial apps with rich features and large codebases. Our experiences suggest that existing mobile UI testing tools are prone to exploration tarpits, where the tools get stuck with a small fraction of app functionalities for an extensive amount of time. One ex-ample is that a tool logs out an app at early stages without being able to log back in, and since then gets stuck with exploring the app’s pre-login functionalities (i.e., exploration tarpits) instead of its main functionalities. While tool vendors/users can manually hardcode rules for the tools to avoid specific exploration tarpits, these rules can hardly generalize, being fragile in face of diverted testing environments, fast app iterations, and the demand of batch testing product lines. To identify and resolve exploration tarpits, we proposeVet, a general approach and its supporting system for the given specific Android UI testing tool on the given specific app under test (AUT). Vet runs the tool on the AUT for some time and records UI traces, based on which Vet identifies exploration tarpits by recognizing their patterns in the UI traces. Vet then pinpoints the actions (e.g., clicking logout) or the screens that lead to exploration tarpits. In subsequent test runs, Vet guides the testing tool to prevent or recover from exploration tarpits. From our evaluation with state-of-the-art Android UI testing tools on popular industrial apps, Vet identifies exploration tarpits that cost up to 98.6% testing time budget. These exploration tarpits reveal not only limitations inUI exploration strategies but also defects in tool implementations. Vet automatically addresses the identified exploration tarpits, enabling each evaluated tool to achieve higher code coverage and improved crash-triggering capabilities.

ACM SIGSOFT Distinguished Paper Award.

Publication
In The ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering.
Date