Eye Movement-Based Human-Swarm Interaction for Coverage Control of Mobile Robots With Constraints

Inspired by search and rescue tasks in hazardous environments with mixed human and robots teams, we develop a human-swarm interaction scheme for coverage control of mobile robots in a non-convex and closed 2-D area. Motivated by releasing the manual operations of a human, we develop two eye movement...

Full description

Saved in:
Bibliographic Details
Published in:IEEE transactions on industrial electronics (1982) Vol. 72; no. 6; pp. 6454 - 6464
Main Authors: Fang, Ziyi, Qin, Jiahu, Qin, Jianmin, Ma, Qichao, Han, Ruitian, Liu, Qingchen
Format: Journal Article
Language:English
Published: New York IEEE 01.06.2025
The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Subjects:
Augmented reality
Constraints
Control systems design
Controllers
Density functional theory
Density functions
Eye movements
Gaze tracking
Hazardous areas
Human motion
human–swarm interaction
Measurement
Mobile robots
mobile robots coverage
Operators (mathematics)
Robot control
Robot kinematics
Robot learning
Robot sensing systems
Robots
Sensors
Service robots
Trajectory
Voronoi graphs
ISSN:0278-0046, 1557-9948
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Inspired by search and rescue tasks in hazardous environments with mixed human and robots teams, we develop a human-swarm interaction scheme for coverage control of mobile robots in a non-convex and closed 2-D area. Motivated by releasing the manual operations of a human, we develop two eye movement-based interaction modes, namely eye gazing mode and eye tracking mode, which use Gaussian functions to approximate signal density functions with respect to a human operator's focus on the specific area of interest. These density functions are then utilized to generate controllers for the robots to achieve the optimal coverage objective with human influence. The coverage controller is designed considering several practical constraints and requirements, including 1) the non-convex area; 2) the limited sensing radius; 3) the time-varying nature of the density function; and 4) the requirement for multiple sensors to monitor an area. To solve these challenges, we design a gradient-based controller by utilizing the visibility-limited higher order Voronoi partition technique, the geodesic distance metric and a well-designed performance criteria function. The implementations and experiments using a Microsoft Hololens 2 Augmented Reality headset and multiple turtlebot robots are presented.
Bibliography:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
ISSN:0278-0046
1557-9948
DOI:10.1109/TIE.2024.3497343