A Mixed-Reality-Augmented Deep Reinforcement Learning Approach for Multi-Robot Safe Motion Generation in Human-Robot Collaborative Manufacturing Cells - Li, Chengxi; Yin, Yue; Ye, Hantao; Zheng, Pai; Gupta, Satyandra K. (IEEE transactions on automation science and engineering, 2025)
Augmenting capabilities of human operators with multi-robot cells offers substantial advantages for increasing productivity in manufacturing applications. This synergy effectively combines the strengths of both robots and humans, maximizing operational efficiency and leveraging human capabilities. However, achieving these benefits requires real-time, reactive coordination of multi-robot motion generation in response to human motion. Current approaches face significant challenges, particularly in dealing with uncertainties in human motions. To address these issues, this paper introduces the Deep Reinforcement Learning (DRL) approach for end-to-end safe motion generation in human multi-robot collaborative workspaces. First, the DRL approach is augmented by adopting mixed-reality (MR) features to facilitate efficient state perception and representation of tasks, humans, robots, and scenes for enabling effective learning motion generation policy. Moreover, to better promote high-dimensional action generation of the multi-robot systems involving human, an advanced DRL approach is developed. The approach leverages memory-enhanced representation learning, intrinsic reward-guided exploration, and action space pruning to better address the motion generation challenges. Empirical testing demonstrates the effectiveness of the proposed system, with experiments showing high success rates across tasks with varying team sizes and difficulty levels, thereby demonstrating applicability in human-robot collaborative manufacturing tasks. Note to Practitioners-The work reported in this paper is expected to be useful in applications requiring complex multi-robot motion generation within human-robot collaborative manufacturing cells, particularly in scenarios where the number and layout of robots may change, for example, low-volume high-mix manufacturing cells. Challenges in HRC robot motion generation are often compounded by the uncertainties associated with human worker movements and the interactions among robots in the cells. The proposed DRL-based motion policy leverages MR-assisted features to enhance state extraction capabilities and incorporates algorithmic modules to improve representation learning and exploration capabilities, thereby enhancing reactive motion generation for safe collaboration. The system demonstrates high success rates across tasks of varying complexities and team sizes. Owing to the flexible policy generation settings and the generalization capabilities of the perception approach, this approach is applicable to a variety of robotic cells and other close-proximity human-robot/machine collaborative motion generation scenarios with minimal tuning.