Discovering UI Display Issues with Visual Understanding

GUI complexity posts a great challenge to the GUI implementation. According to our pilot study of crowdtesting bug reports, display issues such as text overlap, blurred screen, missing image always occur during GUI rendering on difference devices due to the software or hardware compatibility. They n...

Celý popis

Uložené v:
Podrobná bibliografia
Vydané v:2020 35th IEEE/ACM International Conference on Automated Software Engineering (ASE) s. 1373 - 1375
Hlavný autor: Liu, Zhe
Médium: Konferenčný príspevok..
Jazyk:English
Vydavateľské údaje: ACM 01.09.2020
Predmet:
Computer bugs
Deep Learning
Graphical user interfaces
Mobile App
Software
Software engineering
UI display
UI testing
Usability
User experience
Visualization
ISSN:2643-1572
On-line prístup:Získať plný text
Tagy: Pridať tag
Žiadne tagy, Buďte prvý, kto otaguje tento záznam!
Popis
Shrnutí:GUI complexity posts a great challenge to the GUI implementation. According to our pilot study of crowdtesting bug reports, display issues such as text overlap, blurred screen, missing image always occur during GUI rendering on difference devices due to the software or hardware compatibility. They negatively influence the app usability, resulting in poor user experience. To detect these issues, we propose a novel approach, OwlEye, based on deep learning for modelling visual information of the GUI screenshot.Therefore, OwlEye can detect GUIs with display issues and also locate the detailed region of the issue in the given GUI for guiding developers to fix the bug. We manually construct a large-scale labelled dataset with 4,470 GUI screenshots with UI display issues. We develop a heuristics-based data augmentation method and a GAN-based data augmentation method for boosting the performance of our OwlEye. At present, the evaluation demonstrates that our OwlEye can achieve 85% precision and 84% recall in detecting UI display issues, and 90% accuracy in localizing these issues.
ISSN:2643-1572
DOI:10.1145/3324884.3418917