GLIB: Towards Automated Test Oracle for Graphically-Rich Applications


Mobile games are ubiquitous with attractive visual effects of Graphical User Interface (GUI) that offers a bridge between software applications and end users. However, various types of graphical glitches may arise from such GUI complexity and have become the main composition of game compatibility issues. Our study of abundant of real-world bug reports indicates that graphical glitches frequently occur during the game GUI rendering on different devices and severely degrade the game app usability, leading to poor user experience. Different from other common GUI glitches that most existing GUI testing work has focused on, game GUIs composed of 3D models typically have different manifestation of glitch issues. To solve this gap in existing techniques, we propose GLIB, a novel deep learning (DL) approach for detecting game GUI glitches and develop a code-based data augmentation technique via bug understanding for enhancing the modeling ability of our GLIB. The evaluation on 201 real-world game test cases shows that GLIB can achieve 100% precision and 99.5% recall in detecting game GUI glitches and that our code-based augmentation approach can generate more real-like GUI glitches than the existing heuristic-based approach. Our case study on 14 real-world games further demonstrates that GLIB can effectively uncover GUI glitches with most of them having been confirmed and fixed by the app developers.

* The first two authors contributed equally.

In The ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering.