LEMMA: Learning Language-Conditioned Multi-Robot Manipulation

Complex manipulation tasks often require robots with complementary capabilities to collaborate. We introduce a benchmark for L anguag E -Conditioned M ulti-robot MA nipulation ( LEMMA ) focused on task allocation and long-horizon object manipulation based on human language instructions in a tabletop...

Celý popis

Uloženo v:
Podrobná bibliografie
Vydáno v:IEEE robotics and automation letters Ročník 8; číslo 10; s. 6835 - 6842
Hlavní autoři: Gong, Ran, Gao, Xiaofeng, Gao, Qiaozi, Shakiah, Suhaila, Thattai, Govind, Sukhatme, Gaurav S.
Médium: Journal Article
Jazyk:angličtina
Vydáno: Piscataway IEEE 01.10.2023
The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Témata:
Benchmark testing
Benchmarks
Collaboration
Data Sets for Robot Learning
Multi-robot systems
Multiple robots
Multitasking
Natural Dialog for HRI
Planning
Robot kinematics
Robots
Task analysis
Task complexity
ISSN:2377-3766, 2377-3766
On-line přístup:Získat plný text
Tagy: Přidat tag
Žádné tagy, Buďte první, kdo vytvoří štítek k tomuto záznamu!
Popis
Shrnutí:Complex manipulation tasks often require robots with complementary capabilities to collaborate. We introduce a benchmark for L anguag E -Conditioned M ulti-robot MA nipulation ( LEMMA ) focused on task allocation and long-horizon object manipulation based on human language instructions in a tabletop setting. LEMMA features 8 types of procedurally generated tasks with varying degree of complexity, some of which require the robots to use tools and pass tools to each other. For each task, we provide 800 expert demonstrations and human instructions for training and evaluations. LEMMA poses greater challenges compared to existing benchmarks, as it requires the system to identify each manipulator's limitations and assign sub-tasks accordingly while also handling strong temporal dependencies in each task. To address these challenges, we propose a modular hierarchical planning approach as a baseline. Our results highlight the potential of LEMMA for developing future language-conditioned multi-robot systems.
Bibliografie:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
ISSN:2377-3766
2377-3766
DOI:10.1109/LRA.2023.3313058