Text2Motion: from natural language instructions to feasible plans

We propose Text2Motion, a language-based planning framework enabling robots to solve sequential manipulation tasks that require long-horizon reasoning. Given a natural language instruction, our framework constructs both a task- and motion-level plan that is verified to reach inferred symbolic goals....

Celý popis

Uloženo v:
Podrobná bibliografie
Vydáno v:Autonomous robots Ročník 47; číslo 8; s. 1345 - 1365
Hlavní autoři: Lin, Kevin, Agia, Christopher, Migimatsu, Toki, Pavone, Marco, Bohg, Jeannette
Médium: Journal Article
Jazyk:angličtina
Vydáno: New York Springer US 01.12.2023
Springer Nature B.V
Témata:
Artificial Intelligence
Computer Imaging
Control
Engineering
Feasibility
Language instruction
Large language models
Mechatronics
Natural language
Pattern Recognition and Graphics
Planning
Reasoning
Robotics
Robotics and Automation
Skills
Task planning (robotics)
Vision
Large language models
Long-horizon planning
Robot manipulation
ISSN:0929-5593, 1573-7527
On-line přístup:Získat plný text
Tagy: Přidat tag
Žádné tagy, Buďte první, kdo vytvoří štítek k tomuto záznamu!
Popis
Shrnutí:We propose Text2Motion, a language-based planning framework enabling robots to solve sequential manipulation tasks that require long-horizon reasoning. Given a natural language instruction, our framework constructs both a task- and motion-level plan that is verified to reach inferred symbolic goals. Text2Motion uses feasibility heuristics encoded in Q-functions of a library of skills to guide task planning with Large Language Models. Whereas previous language-based planners only consider the feasibility of individual skills, Text2Motion actively resolves geometric dependencies spanning skill sequences by performing geometric feasibility planning during its search. We evaluate our method on a suite of problems that require long-horizon reasoning, interpretation of abstract goals, and handling of partial affordance perception. Our experiments show that Text2Motion can solve these challenging problems with a success rate of 82%, while prior state-of-the-art language-based planning methods only achieve 13%. Text2Motion thus provides promising generalization characteristics to semantically diverse sequential manipulation tasks with geometric dependencies between skills. Qualitative results are made available at https://sites.google.com/stanford.edu/text2motion .
Bibliografie:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
ISSN:0929-5593
1573-7527
DOI:10.1007/s10514-023-10131-7