Efficient Model Learning from Joint-Action Demonstrations for Human-Robot Collaborative Tasks
We present a framework for automatically learning human user models from joint-action demonstrations that enables a robot to compute a robust policy for a collaborative task with a human. First, the demonstrated action sequences are clustered into different human types using an unsupervised learning...
Saved in:
| Published in: | Hri '15: ACM/IEEE International Conference on Human-Robot Interaction USB Stick pp. 189 - 196 |
|---|---|
| Main Authors: | , , , |
| Format: | Conference Proceeding |
| Language: | English |
| Published: |
ACM
02.03.2015
|
| Subjects: | |
| Online Access: | Get full text |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| Abstract | We present a framework for automatically learning human user models from joint-action demonstrations that enables a robot to compute a robust policy for a collaborative task with a human. First, the demonstrated action sequences are clustered into different human types using an unsupervised learning algorithm. A reward function is then learned for each type through the employment of an inverse reinforcement learning algorithm. The learned model is then incorporated into a mixed-observability Markov decision process (MOMDP) formulation, wherein the human type is a partially observable variable. With this framework, we can infer online the human type of a new user that was not included in the training set, and can compute a policy for the robot that will be aligned to the preference of this user. In a human subject experiment (n=30), participants agreed more strongly that the robot anticipated their actions when working with a robot incorporating the proposed framework (p<;0.01), compared to manually annotating robot actions. In trials where participants faced difficulty annotating the robot actions to complete the task, the proposed framework significantly improved team efficiency (p <;0.01). The robot incorporating the framework was also found to be more responsive to human actions compared to policies computed using a hand-coded reward function by a domain expert (p<;0.01). These results indicate that learning human user models from joint-action demonstrations and encoding them in a MOMDP formalism can support effective teaming in human-robot collaborative tasks. |
|---|---|
| AbstractList | We present a framework for automatically learning human user models from joint-action demonstrations that enables a robot to compute a robust policy for a collaborative task with a human. First, the demonstrated action sequences are clustered into different human types using an unsupervised learning algorithm. A reward function is then learned for each type through the employment of an inverse reinforcement learning algorithm. The learned model is then incorporated into a mixed-observability Markov decision process (MOMDP) formulation, wherein the human type is a partially observable variable. With this framework, we can infer online the human type of a new user that was not included in the training set, and can compute a policy for the robot that will be aligned to the preference of this user. In a human subject experiment (n=30), participants agreed more strongly that the robot anticipated their actions when working with a robot incorporating the proposed framework (p<;0.01), compared to manually annotating robot actions. In trials where participants faced difficulty annotating the robot actions to complete the task, the proposed framework significantly improved team efficiency (p <;0.01). The robot incorporating the framework was also found to be more responsive to human actions compared to policies computed using a hand-coded reward function by a domain expert (p<;0.01). These results indicate that learning human user models from joint-action demonstrations and encoding them in a MOMDP formalism can support effective teaming in human-robot collaborative tasks. |
| Author | Ramakrishnan, Ramya Keren Gu Nikolaidis, Stefanos Shah, Julie |
| Author_xml | – sequence: 1 givenname: Stefanos surname: Nikolaidis fullname: Nikolaidis, Stefanos email: snikol@alum.mit.edu organization: Comput. Sci. & Artificial Intell. Lab., Massachusetts Inst. of Technol., Cambridge, MA, USA – sequence: 2 givenname: Ramya surname: Ramakrishnan fullname: Ramakrishnan, Ramya email: ramyaram@mit.edu organization: Comput. Sci. & Artificial Intell. Lab., Massachusetts Inst. of Technol., Cambridge, MA, USA – sequence: 3 surname: Keren Gu fullname: Keren Gu email: kgu@mit.edu organization: Comput. Sci. & Artificial Intell. Lab., Massachusetts Inst. of Technol., Cambridge, MA, USA – sequence: 4 givenname: Julie surname: Shah fullname: Shah, Julie email: julie_a_shah@csail.mit.edu organization: Comput. Sci. & Artificial Intell. Lab., Massachusetts Inst. of Technol., Cambridge, MA, USA |
| BookMark | eNotjE9LwzAcQCMoqLNnD17yBTrzv8lxzLkpFUHmUUaa_irRNpEkCn57N-bp8XjwLtFpiAEQuqZkTqmQt0wZJaSYHylPUGUavQ-EM60ZPUdVzh-EEKa0lMxcoLfVMHjnIRT8FHsYcQs2BR_e8ZDihB-jD6VeuOJjwHcwxZBLsgfLeIgJb74nG-qX2MWCl3EcbRcP-Qfw1ubPfIXOBjtmqP45Q6_3q-1yU7fP64floq0tV7TUWjJGnG4EgGDO6I6S3hpJOwdag3HS9ES7BjrCB-WscZQ6K43hjVKCG8tn6Ob49QCw-0p-sul3t98SxRr-BwGRU_M |
| ContentType | Conference Proceeding |
| DBID | 6IE 6IL CBEJK RIE RIL |
| DOI | 10.1145/2696454.2696455 |
| DatabaseName | IEEE Electronic Library (IEL) Conference Proceedings IEEE Xplore POP ALL IEEE Xplore All Conference Proceedings IEEE/IET Electronic Library (IEL) (UW System Shared) IEEE Proceedings Order Plans (POP All) 1998-Present |
| DatabaseTitleList | |
| Database_xml | – sequence: 1 dbid: RIE name: IEEE/IET Electronic Library (IEL) (UW System Shared) url: https://ieeexplore.ieee.org/ sourceTypes: Publisher |
| DeliveryMethod | fulltext_linktorsrc |
| EISBN | 9781450328821 1450328822 |
| EndPage | 196 |
| ExternalDocumentID | 8520627 |
| Genre | orig-research |
| GroupedDBID | 6IE 6IF 6IL 6IN AAJGR AAWTH ADFMO ALMA_UNASSIGNED_HOLDINGS BEFXN BFFAM BGNUA BKEBE BPEOZ CBEJK IEGSK OCL RIE RIL |
| ID | FETCH-LOGICAL-a361t-85220c874ee42c98b10da951bce88e9c59d08c7eb03f6ca9c11ca5993766439a3 |
| IEDL.DBID | RIE |
| ISICitedReferencesCount | 149 |
| ISICitedReferencesURI | http://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=Summon&SrcAuth=ProQuest&DestLinkType=CitingArticles&DestApp=WOS_CPL&KeyUT=000371986600026&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| IngestDate | Wed Aug 27 03:02:40 EDT 2025 |
| IsDoiOpenAccess | false |
| IsOpenAccess | true |
| IsPeerReviewed | false |
| IsScholarly | false |
| Language | English |
| LinkModel | DirectLink |
| MergedId | FETCHMERGED-LOGICAL-a361t-85220c874ee42c98b10da951bce88e9c59d08c7eb03f6ca9c11ca5993766439a3 |
| PageCount | 8 |
| ParticipantIDs | ieee_primary_8520627 |
| PublicationCentury | 2000 |
| PublicationDate | 2015-03-02 |
| PublicationDateYYYYMMDD | 2015-03-02 |
| PublicationDate_xml | – month: 03 year: 2015 text: 2015-03-02 day: 02 |
| PublicationDecade | 2010 |
| PublicationTitle | Hri '15: ACM/IEEE International Conference on Human-Robot Interaction USB Stick |
| PublicationTitleAbbrev | HRI |
| PublicationYear | 2015 |
| Publisher | ACM |
| Publisher_xml | – name: ACM |
| SSID | ssj0002685529 |
| Score | 2.0668857 |
| Snippet | We present a framework for automatically learning human user models from joint-action demonstrations that enables a robot to compute a robust policy for a... |
| SourceID | ieee |
| SourceType | Publisher |
| StartPage | 189 |
| SubjectTerms | Clustering algorithms Collaboration Computational modeling Data models Markov processes Robots Task analysis |
| Title | Efficient Model Learning from Joint-Action Demonstrations for Human-Robot Collaborative Tasks |
| URI | https://ieeexplore.ieee.org/document/8520627 |
| WOSCitedRecordID | wos000371986600026&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| hasFullText | 1 |
| inHoldings | 1 |
| isFullTextHit | |
| isPrint | |
| link | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV3PS8MwFA6bePCksom_ycGj3dq0SZOjzA3xMIZM2EVG8voqQ21l7fb3m6RlQ_DiqaEQEvKSvC8v-b5HyJ1RSaaZTgPrG02Q5LGx-yCAXe4I9oQRQyqMTzaRTqdysVCzDrnfcWEQ0T8-w4Er-rv8rISNC5UNJWdOVbdLumkqGq7WLp7ChOScqVa9J0r4kAnl5KoGzZf_Sp_ivcfk-H_tnpD-noZHZzsHc0o6WPTI29iLPtgK1OUx-6StQuo7dUwR-lyuijp48GwF-ohfDv41Rq6oxafUB-2Dl9KUNR3t58AW6VxXH1WfvE7G89FT0OZICHQsojqwXWMhyDRBTBgoaaIw0xY1GUApUQFXWSghRRPGuQCtIIpAcwdKhMMiOj4jB0VZ4DmhLGeYCIvYcmvAMGMK4hwjmYHCUGY5uyA9NzTL70YGY9mOyuXfv6_IkcUW3D_XYtfkoF5v8IYcwrZeVetbb7sfjxScRQ |
| linkProvider | IEEE |
| linkToHtml | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV3PS8MwFA5zCnpS2cTf5uDRbm2atMlR5mTqHEMm7CIjSV9lqK2s3f5-k7RsCF48NRRCQl6S9-Ul3_cQulaCJpLI2DO-UXk0DZXZB7U2yx20OWGEOo6USzYRj0Z8OhXjBrpZc2EAwD0-g44turv8JNdLGyrrckasqu4W2maUEr9ia60jKiTijBFR6_cElHVJJKxgVaf6sl8JVJz_uN__X8sHqL0h4uHx2sUcogZkLfTWd7IPpgK2mcw-ca2R-o4tVwQ_5vOs9G4dXwHfwZcFgJWZC2wQKnZhe-8lV3mJe5tZsAI8kcVH0Uav9_1Jb-DVWRI8GUZB6ZmuEV_zmAJQogVXgZ9Ig5uUBs5BaCYSn-sYlB-mkZZCB4GWzMKSyKIRGR6hZpZncIwwSQnQyGC21JjQT4jQYQoBT7QAnycpOUEtOzSz70oIY1aPyunfv6_Q7mDyPJwNH0ZPZ2jPIA3mHm-Rc9QsF0u4QDt6Vc6LxaWz4w_ekJ-M |
| openUrl | ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Abook&rft.genre=proceeding&rft.title=Hri+%2715%3A+ACM%2FIEEE+International+Conference+on+Human-Robot+Interaction+USB+Stick&rft.atitle=Efficient+Model+Learning+from+Joint-Action+Demonstrations+for+Human-Robot+Collaborative+Tasks&rft.au=Nikolaidis%2C+Stefanos&rft.au=Ramakrishnan%2C+Ramya&rft.au=Keren+Gu&rft.au=Shah%2C+Julie&rft.date=2015-03-02&rft.pub=ACM&rft.spage=189&rft.epage=196&rft_id=info:doi/10.1145%2F2696454.2696455&rft.externalDocID=8520627 |