Efficient Model Learning from Joint-Action Demonstrations for Human-Robot Collaborative Tasks
We present a framework for automatically learning human user models from joint-action demonstrations that enables a robot to compute a robust policy for a collaborative task with a human. First, the demonstrated action sequences are clustered into different human types using an unsupervised learning...
Uloženo v:
| Vydáno v: | Hri '15: ACM/IEEE International Conference on Human-Robot Interaction USB Stick s. 189 - 196 |
|---|---|
| Hlavní autoři: | , , , |
| Médium: | Konferenční příspěvek |
| Jazyk: | angličtina |
| Vydáno: |
ACM
02.03.2015
|
| Témata: | |
| On-line přístup: | Získat plný text |
| Tagy: |
Přidat tag
Žádné tagy, Buďte první, kdo vytvoří štítek k tomuto záznamu!
|
| Abstract | We present a framework for automatically learning human user models from joint-action demonstrations that enables a robot to compute a robust policy for a collaborative task with a human. First, the demonstrated action sequences are clustered into different human types using an unsupervised learning algorithm. A reward function is then learned for each type through the employment of an inverse reinforcement learning algorithm. The learned model is then incorporated into a mixed-observability Markov decision process (MOMDP) formulation, wherein the human type is a partially observable variable. With this framework, we can infer online the human type of a new user that was not included in the training set, and can compute a policy for the robot that will be aligned to the preference of this user. In a human subject experiment (n=30), participants agreed more strongly that the robot anticipated their actions when working with a robot incorporating the proposed framework (p<;0.01), compared to manually annotating robot actions. In trials where participants faced difficulty annotating the robot actions to complete the task, the proposed framework significantly improved team efficiency (p <;0.01). The robot incorporating the framework was also found to be more responsive to human actions compared to policies computed using a hand-coded reward function by a domain expert (p<;0.01). These results indicate that learning human user models from joint-action demonstrations and encoding them in a MOMDP formalism can support effective teaming in human-robot collaborative tasks. |
|---|---|
| AbstractList | We present a framework for automatically learning human user models from joint-action demonstrations that enables a robot to compute a robust policy for a collaborative task with a human. First, the demonstrated action sequences are clustered into different human types using an unsupervised learning algorithm. A reward function is then learned for each type through the employment of an inverse reinforcement learning algorithm. The learned model is then incorporated into a mixed-observability Markov decision process (MOMDP) formulation, wherein the human type is a partially observable variable. With this framework, we can infer online the human type of a new user that was not included in the training set, and can compute a policy for the robot that will be aligned to the preference of this user. In a human subject experiment (n=30), participants agreed more strongly that the robot anticipated their actions when working with a robot incorporating the proposed framework (p<;0.01), compared to manually annotating robot actions. In trials where participants faced difficulty annotating the robot actions to complete the task, the proposed framework significantly improved team efficiency (p <;0.01). The robot incorporating the framework was also found to be more responsive to human actions compared to policies computed using a hand-coded reward function by a domain expert (p<;0.01). These results indicate that learning human user models from joint-action demonstrations and encoding them in a MOMDP formalism can support effective teaming in human-robot collaborative tasks. |
| Author | Ramakrishnan, Ramya Keren Gu Nikolaidis, Stefanos Shah, Julie |
| Author_xml | – sequence: 1 givenname: Stefanos surname: Nikolaidis fullname: Nikolaidis, Stefanos email: snikol@alum.mit.edu organization: Comput. Sci. & Artificial Intell. Lab., Massachusetts Inst. of Technol., Cambridge, MA, USA – sequence: 2 givenname: Ramya surname: Ramakrishnan fullname: Ramakrishnan, Ramya email: ramyaram@mit.edu organization: Comput. Sci. & Artificial Intell. Lab., Massachusetts Inst. of Technol., Cambridge, MA, USA – sequence: 3 surname: Keren Gu fullname: Keren Gu email: kgu@mit.edu organization: Comput. Sci. & Artificial Intell. Lab., Massachusetts Inst. of Technol., Cambridge, MA, USA – sequence: 4 givenname: Julie surname: Shah fullname: Shah, Julie email: julie_a_shah@csail.mit.edu organization: Comput. Sci. & Artificial Intell. Lab., Massachusetts Inst. of Technol., Cambridge, MA, USA |
| BookMark | eNotjE9LwzAcQCMoqLNnD17yBTrzv8lxzLkpFUHmUUaa_irRNpEkCn57N-bp8XjwLtFpiAEQuqZkTqmQt0wZJaSYHylPUGUavQ-EM60ZPUdVzh-EEKa0lMxcoLfVMHjnIRT8FHsYcQs2BR_e8ZDihB-jD6VeuOJjwHcwxZBLsgfLeIgJb74nG-qX2MWCl3EcbRcP-Qfw1ubPfIXOBjtmqP45Q6_3q-1yU7fP64floq0tV7TUWjJGnG4EgGDO6I6S3hpJOwdag3HS9ES7BjrCB-WscZQ6K43hjVKCG8tn6Ob49QCw-0p-sul3t98SxRr-BwGRU_M |
| ContentType | Conference Proceeding |
| DBID | 6IE 6IL CBEJK RIE RIL |
| DOI | 10.1145/2696454.2696455 |
| DatabaseName | IEEE Electronic Library (IEL) Conference Proceedings IEEE Proceedings Order Plan All Online (POP All Online) 1998-present by volume IEEE Xplore All Conference Proceedings IEEE Electronic Library (IEL) IEEE Proceedings Order Plans (POP All) 1998-Present |
| DatabaseTitleList | |
| Database_xml | – sequence: 1 dbid: RIE name: IEEE Electronic Library (IEL) url: https://ieeexplore.ieee.org/ sourceTypes: Publisher |
| DeliveryMethod | fulltext_linktorsrc |
| EISBN | 9781450328821 1450328822 |
| EndPage | 196 |
| ExternalDocumentID | 8520627 |
| Genre | orig-research |
| GroupedDBID | 6IE 6IF 6IL 6IN AAJGR AAWTH ADFMO ALMA_UNASSIGNED_HOLDINGS BEFXN BFFAM BGNUA BKEBE BPEOZ CBEJK IEGSK OCL RIE RIL |
| ID | FETCH-LOGICAL-a361t-85220c874ee42c98b10da951bce88e9c59d08c7eb03f6ca9c11ca5993766439a3 |
| IEDL.DBID | RIE |
| ISICitedReferencesCount | 149 |
| ISICitedReferencesURI | http://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=Summon&SrcAuth=ProQuest&DestLinkType=CitingArticles&DestApp=WOS_CPL&KeyUT=000371986600026&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| IngestDate | Wed Aug 27 03:02:40 EDT 2025 |
| IsDoiOpenAccess | false |
| IsOpenAccess | true |
| IsPeerReviewed | false |
| IsScholarly | false |
| Language | English |
| LinkModel | DirectLink |
| MergedId | FETCHMERGED-LOGICAL-a361t-85220c874ee42c98b10da951bce88e9c59d08c7eb03f6ca9c11ca5993766439a3 |
| PageCount | 8 |
| ParticipantIDs | ieee_primary_8520627 |
| PublicationCentury | 2000 |
| PublicationDate | 2015-03-02 |
| PublicationDateYYYYMMDD | 2015-03-02 |
| PublicationDate_xml | – month: 03 year: 2015 text: 2015-03-02 day: 02 |
| PublicationDecade | 2010 |
| PublicationTitle | Hri '15: ACM/IEEE International Conference on Human-Robot Interaction USB Stick |
| PublicationTitleAbbrev | HRI |
| PublicationYear | 2015 |
| Publisher | ACM |
| Publisher_xml | – name: ACM |
| SSID | ssj0002685529 |
| Score | 2.0669918 |
| Snippet | We present a framework for automatically learning human user models from joint-action demonstrations that enables a robot to compute a robust policy for a... |
| SourceID | ieee |
| SourceType | Publisher |
| StartPage | 189 |
| SubjectTerms | Clustering algorithms Collaboration Computational modeling Data models Markov processes Robots Task analysis |
| Title | Efficient Model Learning from Joint-Action Demonstrations for Human-Robot Collaborative Tasks |
| URI | https://ieeexplore.ieee.org/document/8520627 |
| WOSCitedRecordID | wos000371986600026&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| hasFullText | 1 |
| inHoldings | 1 |
| isFullTextHit | |
| isPrint | |
| link | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV3PS8MwFA6bePCksom_ycGj2do0aZOjzA3xMIZM2EVGfrzKUFtZu_39JmnZELx4SgiEhLyQ9-Ul3_cQuhORzt09QRIVWUmYVsKdg4kmzhckXv5KU8hDsolsOhWLhZx10P2OCwMA4fMZDHw1vOXb0mx8qGwoOPWqul3UzbKs4Wrt4ik0FZxT2ar3xIwPaSq9XNWgKfmv9CnBe0yO_zfuCervaXh4tnMwp6gDRQ-9jYPog-uAfR6zT9wqpL5jzxTBz-WqqMlDYCvgR_jy8K8xcoUdPsUhaE9eSl3WeLTfA1vAc1V9VH30OhnPR0-kzZFAVJLGNXFTo5ERGQNg1Eih48gqh5q0ASFAGi5tJEwGOkry1Chp4tgo7kFJ6rGISs7QQVEWcI5wImPLdc4lSxUDJrwwlbUOA0iTg-biAvX80iy_GxmMZbsql383X6Ejhy14-K5Fr9FBvd7ADTo023pVrW-D7X4AaqCblQ |
| linkProvider | IEEE |
| linkToHtml | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV3PS8MwFA5zCnpS2cTf5uDRbG2adMlR5mTqHEMm7CIjSV9lqKus3f5-k7RsCF48JQRCQl7I-_KS73sIXYtAp_aeIIkKEkmYVsKeg5Em1hdETv5KU0h9sonOcCgmEzmqoZs1FwYA_OczaLmqf8tPMrN0obK24NSp6m6hbc4YDUu21jqiQmPBOZWVfk_IeJvG0glWtcqS_0qg4v3H_f7_Rj5AzQ0RD4_WLuYQ1WDeQG89L_tgO2CXyewTVxqp79hxRfBjNpsX5NbzFfAdfDkAWJo5xxahYh-2Jy-Zzgrc3eyCFeCxyj_yJnq97427fVJlSSAqisOC2KnRwIgOA2DUSKHDIFEWN2kDQoA0XCaBMB3QQZTGRkkThkZxB0tih0ZUdITq82wOxwhHMky4TrlksWLAhJOmShKLAqRJQXNxghpuaabfpRDGtFqV07-br9Buf_w8mA4ehk9naM8iDe4_b9FzVC8WS7hAO2ZVzPLFpbfjD7DUntw |
| openUrl | ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Abook&rft.genre=proceeding&rft.title=Hri+%2715%3A+ACM%2FIEEE+International+Conference+on+Human-Robot+Interaction+USB+Stick&rft.atitle=Efficient+Model+Learning+from+Joint-Action+Demonstrations+for+Human-Robot+Collaborative+Tasks&rft.au=Nikolaidis%2C+Stefanos&rft.au=Ramakrishnan%2C+Ramya&rft.au=Keren+Gu&rft.au=Shah%2C+Julie&rft.date=2015-03-02&rft.pub=ACM&rft.spage=189&rft.epage=196&rft_id=info:doi/10.1145%2F2696454.2696455&rft.externalDocID=8520627 |