Programming cobots by voice: a pragmatic, web-based approach

This paper introduces a novel voice-based programming approach and software framework for collaborative robots (cobots) based on the Web Speech API, which is now supported by most modern browsers. The framework targets human programmable interfaces and human-machine interfaces, which can be used by...

Full description

Saved in:
Bibliographic Details
Published in:International journal of computer integrated manufacturing Vol. 36; no. 1; pp. 86 - 109
Main Authors: Ionescu, Tudor B., Schlund, Sebastian
Format: Journal Article
Language:English
Published: Taylor & Francis 02.01.2023
Subjects:
cobots
GUI automation
meta-programming
plugins
speech recognition
Voice-based programming
ISSN:0951-192X, 1362-3052
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:This paper introduces a novel voice-based programming approach and software framework for collaborative robots (cobots) based on the Web Speech API, which is now supported by most modern browsers. The framework targets human programmable interfaces and human-machine interfaces, which can be used by people with little or no programming experience. The framework follows a meta-programming approach by enabling users to program cobots by voice in addition to using a mouse, tablet, or keyboard. Upon a voice instruction, the framework automates the manual tasks required to manipulate the vendor-provided interfaces. The main advantages of this approach are simplified, guided programming, which only requires the knowledge of 5-10 voice instructions; increased programming speed compared to the manual approach; and the possibility of sharing programs as videos. The approach is generalized to other kinds of robots and robot programming tools using so-called meta-controllers, which leverage the power of graphical user interface automation tools and techniques.
ISSN:0951-192X
1362-3052
DOI:10.1080/0951192X.2022.2148754