Efficient Model Learning from Joint-Action Demonstrations for Human-Robot Collaborative Tasks
We present a framework for automatically learning human user models from joint-action demonstrations that enables a robot to compute a robust policy for a collaborative task with a human. First, the demonstrated action sequences are clustered into different human types using an unsupervised learning...
Saved in:
| Published in: | Hri '15: ACM/IEEE International Conference on Human-Robot Interaction USB Stick pp. 189 - 196 |
|---|---|
| Main Authors: | , , , |
| Format: | Conference Proceeding |
| Language: | English |
| Published: |
ACM
02.03.2015
|
| Subjects: | |
| Online Access: | Get full text |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| Abstract | We present a framework for automatically learning human user models from joint-action demonstrations that enables a robot to compute a robust policy for a collaborative task with a human. First, the demonstrated action sequences are clustered into different human types using an unsupervised learning algorithm. A reward function is then learned for each type through the employment of an inverse reinforcement learning algorithm. The learned model is then incorporated into a mixed-observability Markov decision process (MOMDP) formulation, wherein the human type is a partially observable variable. With this framework, we can infer online the human type of a new user that was not included in the training set, and can compute a policy for the robot that will be aligned to the preference of this user. In a human subject experiment (n=30), participants agreed more strongly that the robot anticipated their actions when working with a robot incorporating the proposed framework (p<;0.01), compared to manually annotating robot actions. In trials where participants faced difficulty annotating the robot actions to complete the task, the proposed framework significantly improved team efficiency (p <;0.01). The robot incorporating the framework was also found to be more responsive to human actions compared to policies computed using a hand-coded reward function by a domain expert (p<;0.01). These results indicate that learning human user models from joint-action demonstrations and encoding them in a MOMDP formalism can support effective teaming in human-robot collaborative tasks. |
|---|---|
| AbstractList | We present a framework for automatically learning human user models from joint-action demonstrations that enables a robot to compute a robust policy for a collaborative task with a human. First, the demonstrated action sequences are clustered into different human types using an unsupervised learning algorithm. A reward function is then learned for each type through the employment of an inverse reinforcement learning algorithm. The learned model is then incorporated into a mixed-observability Markov decision process (MOMDP) formulation, wherein the human type is a partially observable variable. With this framework, we can infer online the human type of a new user that was not included in the training set, and can compute a policy for the robot that will be aligned to the preference of this user. In a human subject experiment (n=30), participants agreed more strongly that the robot anticipated their actions when working with a robot incorporating the proposed framework (p<;0.01), compared to manually annotating robot actions. In trials where participants faced difficulty annotating the robot actions to complete the task, the proposed framework significantly improved team efficiency (p <;0.01). The robot incorporating the framework was also found to be more responsive to human actions compared to policies computed using a hand-coded reward function by a domain expert (p<;0.01). These results indicate that learning human user models from joint-action demonstrations and encoding them in a MOMDP formalism can support effective teaming in human-robot collaborative tasks. |
| Author | Ramakrishnan, Ramya Keren Gu Nikolaidis, Stefanos Shah, Julie |
| Author_xml | – sequence: 1 givenname: Stefanos surname: Nikolaidis fullname: Nikolaidis, Stefanos email: snikol@alum.mit.edu organization: Comput. Sci. & Artificial Intell. Lab., Massachusetts Inst. of Technol., Cambridge, MA, USA – sequence: 2 givenname: Ramya surname: Ramakrishnan fullname: Ramakrishnan, Ramya email: ramyaram@mit.edu organization: Comput. Sci. & Artificial Intell. Lab., Massachusetts Inst. of Technol., Cambridge, MA, USA – sequence: 3 surname: Keren Gu fullname: Keren Gu email: kgu@mit.edu organization: Comput. Sci. & Artificial Intell. Lab., Massachusetts Inst. of Technol., Cambridge, MA, USA – sequence: 4 givenname: Julie surname: Shah fullname: Shah, Julie email: julie_a_shah@csail.mit.edu organization: Comput. Sci. & Artificial Intell. Lab., Massachusetts Inst. of Technol., Cambridge, MA, USA |
| BookMark | eNotjE9LwzAcQCMoqLNnD17yBTrzv8lxzLkpFUHmUUaa_irRNpEkCn57N-bp8XjwLtFpiAEQuqZkTqmQt0wZJaSYHylPUGUavQ-EM60ZPUdVzh-EEKa0lMxcoLfVMHjnIRT8FHsYcQs2BR_e8ZDihB-jD6VeuOJjwHcwxZBLsgfLeIgJb74nG-qX2MWCl3EcbRcP-Qfw1ubPfIXOBjtmqP45Q6_3q-1yU7fP64floq0tV7TUWjJGnG4EgGDO6I6S3hpJOwdag3HS9ES7BjrCB-WscZQ6K43hjVKCG8tn6Ob49QCw-0p-sul3t98SxRr-BwGRU_M |
| ContentType | Conference Proceeding |
| DBID | 6IE 6IL CBEJK RIE RIL |
| DOI | 10.1145/2696454.2696455 |
| DatabaseName | IEEE Electronic Library (IEL) Conference Proceedings IEEE Xplore POP ALL IEEE Xplore All Conference Proceedings IEEE Electronic Library (IEL) IEEE Proceedings Order Plans (POP All) 1998-Present |
| DatabaseTitleList | |
| Database_xml | – sequence: 1 dbid: RIE name: IEEE/IET Electronic Library (IEL) (UW System Shared) url: https://ieeexplore.ieee.org/ sourceTypes: Publisher |
| DeliveryMethod | fulltext_linktorsrc |
| EISBN | 9781450328821 1450328822 |
| EndPage | 196 |
| ExternalDocumentID | 8520627 |
| Genre | orig-research |
| GroupedDBID | 6IE 6IF 6IL 6IN AAJGR AAWTH ADFMO ALMA_UNASSIGNED_HOLDINGS BEFXN BFFAM BGNUA BKEBE BPEOZ CBEJK IEGSK OCL RIE RIL |
| ID | FETCH-LOGICAL-a361t-85220c874ee42c98b10da951bce88e9c59d08c7eb03f6ca9c11ca5993766439a3 |
| IEDL.DBID | RIE |
| ISICitedReferencesCount | 149 |
| ISICitedReferencesURI | http://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=Summon&SrcAuth=ProQuest&DestLinkType=CitingArticles&DestApp=WOS_CPL&KeyUT=000371986600026&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| IngestDate | Wed Aug 27 03:02:40 EDT 2025 |
| IsDoiOpenAccess | false |
| IsOpenAccess | true |
| IsPeerReviewed | false |
| IsScholarly | false |
| Language | English |
| LinkModel | DirectLink |
| MergedId | FETCHMERGED-LOGICAL-a361t-85220c874ee42c98b10da951bce88e9c59d08c7eb03f6ca9c11ca5993766439a3 |
| PageCount | 8 |
| ParticipantIDs | ieee_primary_8520627 |
| PublicationCentury | 2000 |
| PublicationDate | 2015-03-02 |
| PublicationDateYYYYMMDD | 2015-03-02 |
| PublicationDate_xml | – month: 03 year: 2015 text: 2015-03-02 day: 02 |
| PublicationDecade | 2010 |
| PublicationTitle | Hri '15: ACM/IEEE International Conference on Human-Robot Interaction USB Stick |
| PublicationTitleAbbrev | HRI |
| PublicationYear | 2015 |
| Publisher | ACM |
| Publisher_xml | – name: ACM |
| SSID | ssj0002685529 |
| Score | 2.0668857 |
| Snippet | We present a framework for automatically learning human user models from joint-action demonstrations that enables a robot to compute a robust policy for a... |
| SourceID | ieee |
| SourceType | Publisher |
| StartPage | 189 |
| SubjectTerms | Clustering algorithms Collaboration Computational modeling Data models Markov processes Robots Task analysis |
| Title | Efficient Model Learning from Joint-Action Demonstrations for Human-Robot Collaborative Tasks |
| URI | https://ieeexplore.ieee.org/document/8520627 |
| WOSCitedRecordID | wos000371986600026&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| hasFullText | 1 |
| inHoldings | 1 |
| isFullTextHit | |
| isPrint | |
| link | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV3PS8MwFA6bePCksom_ycGj2ZK0adKjzA3xMIZM2EVGkr7KUFtZu_39JmnZELx4SgjkB-8F8uUl3_cQutNMKyW4JTnkMYlzd2c1kQbCkwyMjnTEoUk2IadTtViksw6633FhACB8PoOBr4a3_Ky0Gx8qG7pBvapuF3WllA1XaxdP4YkSgqeteg-LxZAnqZerGjSl-JU-JZwek-P_zXuC-nsaHp7tDphT1IGih97GQfTBdcA-j9knbhVS37FniuDnclXU5CGwFfAjfHn41zi5wg6f4hC0Jy-lKWs82u-BLeC5rj6qPnqdjOejJ9LmSCA6SlhN3NI4tUrGADG3qTKMZtqhJmNBKUitSDOqrARDozyxOrWMWS08KEk8FtHRGTooygLOEc4sTUwuuYkNjbWUhkntvCjdFUpSCuIC9bxplt-NDMaytcrl381X6MhhCxG-a_FrdFCvN3CDDu22XlXr2-C7H5iOnGI |
| linkProvider | IEEE |
| linkToHtml | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV3PS8MwFA5zCnpS2cTf5uDRbG2aNOlR5mTqHEMm7CIjSV9lqK2s3f5-k7ZsCF48JQTyg_cC-fKS73sIXStfScmpIQkkjLDE3ll1oIDQMAatAhVQqJJNiNFITqfRuIFu1lwYACg_n0HHVcu3_DgzSxcq69pBnaruFtrmjFG_YmutIyo0lJzTqNbv8Rnv0jByglWdquS_EqiU58f9_v9mPkDtDREPj9dHzCFqQNpCb_1S9sF2wC6T2SeuNVLfseOK4MdsnhbktuQr4Dv4cgCwcnOOLULFZdievGQ6K3BvswtWgCcq_8jb6PW-P-kNSJ0lgagg9Atil0Y9IwUDYNREUvterCxu0gakhMjwKPakEaC9IAmNiozvG8UdLAkdGlHBEWqmWQrHCMfGC3UiqGbaY0oI7Qtl_SjsJUp4HvAT1HKmmX1XQhiz2iqnfzdfod3B5Hk4Gz6Mns7QnkUavPy8Rc9Rs1gs4QLtmFUxzxeXpR9_AOqEn6k |
| openUrl | ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Abook&rft.genre=proceeding&rft.title=Hri+%2715%3A+ACM%2FIEEE+International+Conference+on+Human-Robot+Interaction+USB+Stick&rft.atitle=Efficient+Model+Learning+from+Joint-Action+Demonstrations+for+Human-Robot+Collaborative+Tasks&rft.au=Nikolaidis%2C+Stefanos&rft.au=Ramakrishnan%2C+Ramya&rft.au=Keren+Gu&rft.au=Shah%2C+Julie&rft.date=2015-03-02&rft.pub=ACM&rft.spage=189&rft.epage=196&rft_id=info:doi/10.1145%2F2696454.2696455&rft.externalDocID=8520627 |