OVITA: Open-Vocabulary Interpretable Trajectory Adaptations

Adapting trajectories to dynamic situations and user preferences is crucial for robot operation in unstructured environments with non-expert users. Natural language enables users to express these adjustments in an interactive manner. We introduce OVITA, an interpretable, open-vocabulary, language-dr...

Celý popis

Uložené v:
Podrobná bibliografia
Vydané v:IEEE robotics and automation letters Ročník 10; číslo 11; s. 11054 - 11061
Hlavní autori: Maurya, Anurag, Ghosh, Tashmoy, Nguyen, Anh, Prakash, Ravi
Médium: Journal Article
Jazyk:English
Vydavateľské údaje: Piscataway IEEE 01.11.2025
The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Predmet:
big data in robotics and automation
Codes
Dynamics
Grounding
human-robot collaboration
Large language models
Motion and path planning
Motion planning
Planning
Robot arms
Robot sensing systems
Robots
Service robots
Training
Trajectories
Trajectory
Translation
ISSN:2377-3766, 2377-3766
On-line prístup:Získať plný text
Tagy: Pridať tag
Žiadne tagy, Buďte prvý, kto otaguje tento záznam!
Abstract Adapting trajectories to dynamic situations and user preferences is crucial for robot operation in unstructured environments with non-expert users. Natural language enables users to express these adjustments in an interactive manner. We introduce OVITA, an interpretable, open-vocabulary, language-driven framework designed for adapting robot trajectories in dynamic and novel situations based on human instructions. OVITA leverages multiple pre-trained Large Language Models (LLMs) to integrate user commands into trajectories generated by motion planners or those learned through demonstrations. OVITA employs code as an adaptation policy generated by an LLM, enabling users to adjust individual waypoints, thus providing flexible control. Another LLM, which acts as a code explainer, removes the need for expert users, enabling intuitive interactions. The efficacy and significance of the proposed OVITA framework is demonstrated through extensive simulations and real-world environments with diverse tasks involving spatiotemporal variations on heterogeneous robotic platforms such as a KUKA IIWA robot manipulator, Clearpath Jackal ground robot, and CrazyFlie drone.
AbstractList Adapting trajectories to dynamic situations and user preferences is crucial for robot operation in unstructured environments with non-expert users. Natural language enables users to express these adjustments in an interactive manner. We introduce OVITA, an interpretable, open-vocabulary, language-driven framework designed for adapting robot trajectories in dynamic and novel situations based on human instructions. OVITA leverages multiple pre-trained Large Language Models (LLMs) to integrate user commands into trajectories generated by motion planners or those learned through demonstrations. OVITA employs code as an adaptation policy generated by an LLM, enabling users to adjust individual waypoints, thus providing flexible control. Another LLM, which acts as a code explainer, removes the need for expert users, enabling intuitive interactions. The efficacy and significance of the proposed OVITA framework is demonstrated through extensive simulations and real-world environments with diverse tasks involving spatiotemporal variations on heterogeneous robotic platforms such as a KUKA IIWA robot manipulator, Clearpath Jackal ground robot, and CrazyFlie drone.
Author Ghosh, Tashmoy
Prakash, Ravi
Nguyen, Anh
Maurya, Anurag
Author_xml – sequence: 1
  givenname: Anurag
  orcidid: 0009-0003-2651-1757
  surname: Maurya
  fullname: Maurya, Anurag
  email: anuragm1@iisc.ac.in
  organization: Human-interactive Robotics (HiRo) Lab, Cyber-Physical Systems, Indian Institute of Science, Bangalore, India
– sequence: 2
  givenname: Tashmoy
  orcidid: 0009-0001-1662-9378
  surname: Ghosh
  fullname: Ghosh, Tashmoy
  email: tashmoyg@iisc.ac.in
  organization: Human-interactive Robotics (HiRo) Lab, Cyber-Physical Systems, Indian Institute of Science, Bangalore, India
– sequence: 3
  givenname: Anh
  orcidid: 0000-0002-1449-211X
  surname: Nguyen
  fullname: Nguyen, Anh
  email: anh.nguyen@liverpool.ac.uk
  organization: Department of Computer Science, University of Liverpool, Liverpool, U.K
– sequence: 4
  givenname: Ravi
  orcidid: 0000-0002-9058-434X
  surname: Prakash
  fullname: Prakash, Ravi
  email: ravipr@iisc.ac.in
  organization: Human-interactive Robotics (HiRo) Lab, Cyber-Physical Systems, Indian Institute of Science, Bangalore, India
BookMark eNpNkM1rAjEQxUOxUGu999CD0PPaSSYfbntaxLaCIBTrNSRxBMXubpP10P--EYX2NA_mvRne75b16qYmxu45jDmH8mnxUY0FCDVGDRqhvGJ9gcYUaLTu_dM3bJjSHgC4EgZL1Wcvy_V8VT2Pli3VxboJzh8PLv6M5nVHsY3UOX-g0Sq6PYWuyYtq49rOdbumTnfseusOiYaXOWCfr7PV9L1YLN_m02pRBCFNV2BJxjitTBB8IkuQSgIhug3xAEZKteWlDOAmHkRQ3mu_UY6MzxmpndY4YI_nu21svo-UOrtvjrHOLy0KlasgR8wuOLtCbFKKtLVt3H3lLpaDPVGymZI9UbIXSjnycI7siOjPzrkCg4C_eYRi8Q
CODEN IRALC6
Cites_doi 10.1146/annurev-control-100819-063206
10.1109/IROS58592.2024.10803060
10.1109/ICCV51070.2023.00280
10.1109/ICRA48891.2023.10160591
10.1177/0278364911406761
10.1109/ICRA48891.2023.10161068
10.1561/2300000072
10.1109/IROS47612.2022.9981810
10.3389/frobt.2024.1345693
10.1007/s11370-015-0187-9
10.1109/LRA.2024.3410155
10.1109/LRA.2024.3357432
10.1177/0278364919846363
10.1109/ICRA48891.2023.10161317
10.1145/3568162.3578623
10.48550/arXiv.1810.04805
10.1007/s11704-024-40231-1
10.15607/rss.2022.xviii.065
10.1145/3292500.3330701
10.1561/9781680834116
10.1146/annurev-control-101119-071628
10.1177/0278364920917755
10.1162/NECO_a_00393
ContentType Journal Article
Copyright Copyright The Institute of Electrical and Electronics Engineers, Inc. (IEEE) 2025
Copyright_xml – notice: Copyright The Institute of Electrical and Electronics Engineers, Inc. (IEEE) 2025
DBID 97E
RIA
RIE
AAYXX
CITATION
7SC
7SP
8FD
JQ2
L7M
L~C
L~D
DOI 10.1109/LRA.2025.3606309
DatabaseName IEEE Xplore (IEEE)
IEEE All-Society Periodicals Package (ASPP) 1998–Present
IEEE Electronic Library (IEL)
CrossRef
Computer and Information Systems Abstracts
Electronics & Communications Abstracts
Technology Research Database
ProQuest Computer Science Collection
Advanced Technologies Database with Aerospace
Computer and Information Systems Abstracts – Academic
Computer and Information Systems Abstracts Professional
DatabaseTitle CrossRef
Technology Research Database
Computer and Information Systems Abstracts – Academic
Electronics & Communications Abstracts
ProQuest Computer Science Collection
Computer and Information Systems Abstracts
Advanced Technologies Database with Aerospace
Computer and Information Systems Abstracts Professional
DatabaseTitleList
Technology Research Database
Database_xml – sequence: 1
  dbid: RIE
  name: IEEE Electronic Library (IEL)
  url: https://ieeexplore.ieee.org/
  sourceTypes: Publisher
DeliveryMethod fulltext_linktorsrc
Discipline Engineering
EISSN 2377-3766
EndPage 11061
ExternalDocumentID 10_1109_LRA_2025_3606309
11150730
Genre orig-research
GrantInformation_xml – fundername: Kotak IISc AI/ML centre and in part by ARTPARK, IISc Bangalore
GroupedDBID 0R~
97E
AAJGR
AASAJ
AAWTH
ABAZT
ABQJQ
ABVLG
ACGFS
AGQYO
AGSQL
AHBIQ
AKJIK
AKQYR
ALMA_UNASSIGNED_HOLDINGS
ATWAV
BEFXN
BFFAM
BGNUA
BKEBE
BPEOZ
EBS
EJD
IFIPE
IPLJI
JAVBF
KQ8
M43
M~E
O9-
OCL
RIA
RIE
AAYXX
CITATION
7SC
7SP
8FD
JQ2
L7M
L~C
L~D
ID FETCH-LOGICAL-c247t-39e77a657c2184904540e33ade1c07445f194c0a8b02c5bb6bd5ae7b7a646a663
IEDL.DBID RIE
ISICitedReferencesCount 0
ISICitedReferencesURI http://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=Summon&SrcAuth=ProQuest&DestLinkType=CitingArticles&DestApp=WOS_CPL&KeyUT=001574207700004&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D
ISSN 2377-3766
IngestDate Sat Nov 22 13:41:00 EST 2025
Sat Nov 29 07:28:31 EST 2025
Wed Oct 01 07:05:10 EDT 2025
IsPeerReviewed true
IsScholarly true
Issue 11
Language English
License https://ieeexplore.ieee.org/Xplorehelp/downloads/license-information/IEEE.html
https://doi.org/10.15223/policy-029
https://doi.org/10.15223/policy-037
LinkModel DirectLink
MergedId FETCHMERGED-LOGICAL-c247t-39e77a657c2184904540e33ade1c07445f194c0a8b02c5bb6bd5ae7b7a646a663
Notes ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
ORCID 0009-0003-2651-1757
0000-0002-1449-211X
0009-0001-1662-9378
0000-0002-9058-434X
PQID 3251523133
PQPubID 4437225
PageCount 8
ParticipantIDs ieee_primary_11150730
crossref_primary_10_1109_LRA_2025_3606309
proquest_journals_3251523133
PublicationCentury 2000
PublicationDate 2025-11-01
PublicationDateYYYYMMDD 2025-11-01
PublicationDate_xml – month: 11
  year: 2025
  text: 2025-11-01
  day: 01
PublicationDecade 2020
PublicationPlace Piscataway
PublicationPlace_xml – name: Piscataway
PublicationTitle IEEE robotics and automation letters
PublicationTitleAbbrev LRA
PublicationYear 2025
Publisher IEEE
The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Publisher_xml – name: IEEE
– name: The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
References ref13
ref12
Szot (ref19) 2023
ref14
ref31
ref30
Medeiros (ref37) 2023
ref10
ref32
Lowry (ref36) 2014
Yu (ref5) 2023
ref2
ref1
ref17
Team (ref35) 2023
ref16
Hurst (ref33) 2024
Ahn (ref21) 2022; 205
Anjomshoae (ref11) 2019
Rawlings (ref8) 2017
ref24
Huang (ref23) 2023
ref26
ref25
ref20
Huang (ref4) 2023
ref22
(ref34) 2024
ref28
Paraschos (ref15) 2013; 26
ref27
ref29
ref7
ref9
ref3
ref6
Driess (ref18) 2023
References_xml – ident: ref9
  doi: 10.1146/annurev-control-100819-063206
– ident: ref17
  doi: 10.1109/IROS58592.2024.10803060
– ident: ref2
  doi: 10.1109/ICCV51070.2023.00280
– ident: ref22
  doi: 10.1109/ICRA48891.2023.10160591
– year: 2024
  ident: ref33
  article-title: GPT-4o system card
– ident: ref7
  doi: 10.1177/0278364911406761
– start-page: 540
  volume-title: Proc. Conf. Robot Learn.
  year: 2023
  ident: ref4
  article-title: VoxPoser: Composable 3D value maps for robotic manipulation with language models
– start-page: 374
  volume-title: Proc. Conf. Robot Learn.
  year: 2023
  ident: ref5
  article-title: Language to rewards for robotic skill synthesis
– volume: 26
  start-page: 2616
  volume-title: Proc. Adv. Neural Inf. Process. Syst.
  year: 2013
  ident: ref15
  article-title: Probabilistic movement primitives
– start-page: 8469
  volume-title: Proc. Int. Conf. Mach. Learn.
  year: 2023
  ident: ref18
  article-title: PaLM-E: An embodied multimodal language model
– volume-title: Proc. 12th Int. Conf. Learn. Representations
  year: 2023
  ident: ref19
  article-title: Large language models as generalizable policies for embodied tasks
– ident: ref30
  doi: 10.1109/ICRA48891.2023.10161068
– start-page: 1769
  volume-title: Proc. Conf. Robot Learn.
  year: 2023
  ident: ref23
  article-title: Inner monologue: Embodied reasoning through planning with language models
– volume: 205
  start-page: 287
  volume-title: Proc. 6th Conf. Robot Learn.
  year: 2022
  ident: ref21
  article-title: Do as i can, not as i say: Grounding language in robotic affordances
– start-page: 1078
  volume-title: Proc. 18th Int. Conf. Auton. Agents Multiagent Syst.
  year: 2019
  ident: ref11
  article-title: Explainable agents and robots: Results from a systematic literature review
– ident: ref10
  doi: 10.1561/2300000072
– year: 2014
  ident: ref36
  article-title: Concepts and applications of inferential statistics
– ident: ref29
  doi: 10.1109/IROS47612.2022.9981810
– ident: ref31
  doi: 10.3389/frobt.2024.1345693
– year: 2023
  ident: ref35
  article-title: Gemini: A family of highly capable multimodal models
– ident: ref13
  doi: 10.1007/s11370-015-0187-9
– ident: ref6
  doi: 10.1109/LRA.2024.3410155
– ident: ref20
  doi: 10.1109/LRA.2024.3357432
– ident: ref16
  doi: 10.1177/0278364919846363
– year: 2023
  ident: ref37
  article-title: LangSAM: Language segment-anything
– ident: ref3
  doi: 10.1109/ICRA48891.2023.10161317
– ident: ref26
  doi: 10.1145/3568162.3578623
– ident: ref27
  doi: 10.48550/arXiv.1810.04805
– ident: ref1
  doi: 10.1007/s11704-024-40231-1
– volume-title: Model Predictive Control: Theory, Computation, and Design
  year: 2017
  ident: ref8
– year: 2024
  ident: ref34
  article-title: Claude haiku [large language model]
– ident: ref28
  doi: 10.15607/rss.2022.xviii.065
– ident: ref32
  doi: 10.1145/3292500.3330701
– ident: ref12
  doi: 10.1561/9781680834116
– ident: ref24
  doi: 10.1146/annurev-control-101119-071628
– ident: ref25
  doi: 10.1177/0278364920917755
– ident: ref14
  doi: 10.1162/NECO_a_00393
SSID ssj0001527395
Score 2.307959
Snippet Adapting trajectories to dynamic situations and user preferences is crucial for robot operation in unstructured environments with non-expert users. Natural...
SourceID proquest
crossref
ieee
SourceType Aggregation Database
Index Database
Publisher
StartPage 11054
SubjectTerms big data in robotics and automation
Codes
Dynamics
Grounding
human-robot collaboration
Large language models
Motion and path planning
Motion planning
Planning
Robot arms
Robot sensing systems
Robots
Service robots
Training
Trajectories
Trajectory
Translation
Title OVITA: Open-Vocabulary Interpretable Trajectory Adaptations
URI https://ieeexplore.ieee.org/document/11150730
https://www.proquest.com/docview/3251523133
Volume 10
WOSCitedRecordID wos001574207700004&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D
hasFullText 1
inHoldings 1
isFullTextHit
isPrint
journalDatabaseRights – providerCode: PRVIEE
  databaseName: IEEE Electronic Library (IEL)
  customDbUrl:
  eissn: 2377-3766
  dateEnd: 99991231
  omitProxy: false
  ssIdentifier: ssj0001527395
  issn: 2377-3766
  databaseCode: RIE
  dateStart: 20160101
  isFulltext: true
  titleUrlDefault: https://ieeexplore.ieee.org/
  providerName: IEEE
– providerCode: PRVHPJ
  databaseName: ROAD: Directory of Open Access Scholarly Resources
  customDbUrl:
  eissn: 2377-3766
  dateEnd: 99991231
  omitProxy: false
  ssIdentifier: ssj0001527395
  issn: 2377-3766
  databaseCode: M~E
  dateStart: 20160101
  isFulltext: true
  titleUrlDefault: https://road.issn.org
  providerName: ISSN International Centre
link http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV1NS8MwGA5ueNCDnxOnc_TgxUO3tEmWRU9FNhTmFJljt5KPt6DINvYh7OJvN0k7pogHbz00JTxp8r7P-_EEoUu79yCKwGX9OQ9pFulQaWMXhDkpGY1j8KGBYY_3--3RSDwVzeq-FwYAfPEZNNyjz-WbiV66UFkz8u4LsQy9xDnPm7U2ARUnJSbYOhWJRbP3nFgCGLMGaTllKfHD9Pi7VH4dwN6qdPf_OZ8DtFe4j0GSr_ch2oLxEdr9Jip4jG4eh_eD5DpwpSLh0Joq5SpNV8GmvFC9Q2Bt1JsP2K-CxMhpnpCfV9BLtzO4vQuLKxJCHVO-CIkAzmWLce2omvB6ekCINBBp6xxQlkWCaizbCseaKdVShkngyo6hLWm9jRNUHk_GcIoCJklGMJbUyIxmljjI2AgKbU4NzhhEVXS1Ri-d5koYqWcQWKQW6dQhnRZIV1HFobV5rwCqimprvNNir8xTYl0sS4ctWT77Y9g52nFfz1sAa6i8mC3hAm3rj8XrfFZHpYfPTt3_DF-ts7Cy
linkProvider IEEE
linkToHtml http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV1NS8MwGA4yBfXgtzid2oMXD92aNmkWPRVxbFinyBy7hSR9C4psYx_C_r1J2jFFPHjroaHhSZP3fd6PJwhdmb0HGIPN-jPmkxxrX-nMLAi1UjI6CMGFBvop63abgwF_LpvVXS8MALjiM6jbR5fLz0Z6bkNlDezcl8gw9HVKSIiLdq1VSMWKiXG6TEYGvJG-JIYChrQexVZbiv8wPu42lV9HsLMrrd1_zmgP7ZQOpJcUK76P1mB4gLa_yQoeotunfqeX3Hi2WMTvG2OlbK3pwlsVGKoP8IyVench-4WXZHJcpOSnR-i1dd-7a_vlJQm-Dgmb-REHxmRMmbZkjTtFPYgimQHWxj0gNMec6EA2VRBqqlSsMiqBKTOGxNL4G8eoMhwN4QR5VEZ5FASSZDInuaEOMsw4gSYjWZBTwFV0vURPjAstDOE4RMCFQVpYpEWJdBUdWbRW75VAVVFtibcod8tURMbJMoTY0OXTP4Zdos127zEVaaf7cIa27JeKhsAaqswmczhHG_pz9jadXLhf4gvcybLI
openUrl ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=OVITA%3A+Open-Vocabulary+Interpretable+Trajectory+Adaptations&rft.jtitle=IEEE+robotics+and+automation+letters&rft.au=Maurya%2C+Anurag&rft.au=Ghosh%2C+Tashmoy&rft.au=Nguyen%2C+Anh&rft.au=Prakash%2C+Ravi&rft.date=2025-11-01&rft.pub=IEEE&rft.eissn=2377-3766&rft.volume=10&rft.issue=11&rft.spage=11054&rft.epage=11061&rft_id=info:doi/10.1109%2FLRA.2025.3606309&rft.externalDocID=11150730
thumbnail_l http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/lc.gif&issn=2377-3766&client=summon
thumbnail_m http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/mc.gif&issn=2377-3766&client=summon
thumbnail_s http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/sc.gif&issn=2377-3766&client=summon