Discovering UI Display Issues with Visual Understanding

GUI complexity posts a great challenge to the GUI implementation. According to our pilot study of crowdtesting bug reports, display issues such as text overlap, blurred screen, missing image always occur during GUI rendering on difference devices due to the software or hardware compatibility. They n...

Celý popis

Uloženo v:
Podrobná bibliografie
Vydáno v:2020 35th IEEE/ACM International Conference on Automated Software Engineering (ASE) s. 1373 - 1375
Hlavní autor: Liu, Zhe
Médium: Konferenční příspěvek
Jazyk:angličtina
Vydáno: ACM 01.09.2020
Témata:
Computer bugs
Deep Learning
Graphical user interfaces
Mobile App
Software
Software engineering
UI display
UI testing
Usability
User experience
Visualization
ISSN:2643-1572
On-line přístup:Získat plný text
Tagy: Přidat tag
Žádné tagy, Buďte první, kdo vytvoří štítek k tomuto záznamu!
Popis
Shrnutí:GUI complexity posts a great challenge to the GUI implementation. According to our pilot study of crowdtesting bug reports, display issues such as text overlap, blurred screen, missing image always occur during GUI rendering on difference devices due to the software or hardware compatibility. They negatively influence the app usability, resulting in poor user experience. To detect these issues, we propose a novel approach, OwlEye, based on deep learning for modelling visual information of the GUI screenshot.Therefore, OwlEye can detect GUIs with display issues and also locate the detailed region of the issue in the given GUI for guiding developers to fix the bug. We manually construct a large-scale labelled dataset with 4,470 GUI screenshots with UI display issues. We develop a heuristics-based data augmentation method and a GAN-based data augmentation method for boosting the performance of our OwlEye. At present, the evaluation demonstrates that our OwlEye can achieve 85% precision and 84% recall in detecting UI display issues, and 90% accuracy in localizing these issues.
ISSN:2643-1572
DOI:10.1145/3324884.3418917