An Empirical Study of Android Test Generation Tools in Industrial Cases

User Interface (UI) testing is a popular approach to ensure the quality of mobile apps. Numerous test generation tools have been developed to support UI testing on mobile apps, especially for Android apps. Previous work evaluates and compares different test generation tools using only relatively sim...

Celý popis

Uložené v:
Podrobná bibliografia
Vydané v:2018 33rd IEEE/ACM International Conference on Automated Software Engineering (ASE) s. 738 - 748
Hlavní autori: Wang, Wenyu, Li, Dengfeng, Yang, Wei, Cao, Yurui, Zhang, Zhenwen, Deng, Yuetang, Xie, Tao
Médium: Konferenčný príspevok..
Jazyk:English
Vydavateľské údaje: ACM 01.09.2018
Predmet:
Android UI testing
Codes
Computer crashes
empirical study
Internet
Mobile applications
Software engineering
test generation
Test pattern generators
Testing
User interfaces
ISSN:2643-1572
On-line prístup:Získať plný text
Tagy: Pridať tag
Žiadne tagy, Buďte prvý, kto otaguje tento záznam!
Popis
Shrnutí:User Interface (UI) testing is a popular approach to ensure the quality of mobile apps. Numerous test generation tools have been developed to support UI testing on mobile apps, especially for Android apps. Previous work evaluates and compares different test generation tools using only relatively simple open-source apps, while real-world industrial apps tend to have more complex functionalities and implementations. There is no direct comparison among test generation tools with regard to effectiveness and ease-of-use on these industrial apps. To address such limitation, we study existing state-of-the-art or state-of-the-practice test generation tools on 68 widely-used industrial apps. We directly compare the tools with regard to code coverage and fault-detection ability. According to our results, Monkey, a state-of-the-practice tool from Google, achieves the highest method coverage on 22 of 41 apps whose method coverage data can be obtained. Of all 68 apps under study, Monkey also achieves the highest activity coverage on 35 apps, while Stoat, a state-of-the-art tool, is able to trigger the highest number of unique crashes on 23 apps. By analyzing the experimental results, we provide suggestions for combining different test generation tools to achieve better performance. We also report our experience in applying these tools to industrial apps under study. Our study results give insights on how Android UI test generation tools could be improved to better handle complex industrial apps.
ISSN:2643-1572
DOI:10.1145/3238147.3240465