SeeAction: Towards Reverse Engineering How-What-Where of HCI Actions from Screencasts for UI Automation

UI automation is an useful technique for UI testing, bug reproduction and robotic process automation. Recording the user actions with an application assists rapid development of UI automation scripts, but existing recording techniques are intrusive, rely on OS or GUI framework accessibility support...

Full description

Bibliographic Details

Published in:	Proceedings / International Conference on Software Engineering pp. 463 - 475
Main Authors:	Zhao, Dehai, Xing, Zhenchang, Lu, Qinghua, Xu, Xiwei, Zhu, Liming
Format:	Conference Proceeding
Language:	English
Published:	IEEE 26.04.2025
Subjects:	Action Recognition Computer bugs Computer vision Deep learning Graphical user interfaces Intelligent automation Multi-task Learning Multitasking Recording Reverse engineering Software engineering Testing UI Automation UI Testing
ISSN:	1558-1225
Online Access:	Get full text
Tags:	Add Tag No Tags, Be the first to tag this record!