HHI-Assist: A Dataset and Benchmark of Human-Human Interaction in Physical Assistance Scenario

The increasing labor shortage and aging population underline the need for assistive robots to support human care recipients. To enable safe and responsive assistance, robots require accurate human motion prediction in physical interaction scenarios. However, this remains a challenging task due to th...

Ausführliche Beschreibung

Gespeichert in:

Bibliographische Detailangaben
Veröffentlicht in:	IEEE robotics and automation letters Jg. 10; H. 9; S. 8746 - 8753
Hauptverfasser:	Saadatnejad, Saeed, Hosseininejad, Reyhaneh, Barreiros, Jose, Tsui, Katherine M., Alahi, Alexandre
Format:	Journal Article
Sprache:	Englisch
Veröffentlicht:	Piscataway IEEE 01.09.2025 The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Schlagworte:	Data sets for robot learning Datasets Diffusion models Dynamics Human motion Human-robot interaction intention recognition Motion capture Noise reduction physical human-robot interaction Predictive models Receivers Robot dynamics Robots Service robots Skeleton Transformers
ISSN:	2377-3766, 2377-3766
Online-Zugang:	Volltext
Tags:	Tag hinzufügen Keine Tags, Fügen Sie den ersten Tag hinzu!

Abstract	The increasing labor shortage and aging population underline the need for assistive robots to support human care recipients. To enable safe and responsive assistance, robots require accurate human motion prediction in physical interaction scenarios. However, this remains a challenging task due to the variability of assistive settings and the complexity of coupled dynamics in physical interactions. In this work, we address these challenges through two key contributions: (1) HHI-Assist , a dataset comprising motion capture clips of human-human interactions in assistive tasks; and (2) a conditional Transformer-based denoising diffusion model for predicting the poses of interacting agents. Our model effectively captures the coupled dynamics between caregivers and care receivers, demonstrating improvements over baselines and strong generalization to unseen scenarios. By advancing interaction-aware motion prediction and introducing a new dataset, our work has the potential to significantly enhance robotic assistance policies.
AbstractList	The increasing labor shortage and aging population underline the need for assistive robots to support human care recipients. To enable safe and responsive assistance, robots require accurate human motion prediction in physical interaction scenarios. However, this remains a challenging task due to the variability of assistive settings and the complexity of coupled dynamics in physical interactions. In this work, we address these challenges through two key contributions: (1) HHI-Assist, a dataset comprising motion capture clips of human-human interactions in assistive tasks; and (2) a conditional Transformer-based denoising diffusion model for predicting the poses of interacting agents. Our model effectively captures the coupled dynamics between caregivers and care receivers, demonstrating improvements over baselines and strong generalization to unseen scenarios. By advancing interaction-aware motion prediction and introducing a new dataset, our work has the potential to significantly enhance robotic assistance policies.
Author	Barreiros, Jose Saadatnejad, Saeed Tsui, Katherine M. Hosseininejad, Reyhaneh Alahi, Alexandre
Author_xml	– sequence: 1 givenname: Saeed orcidid: 0000-0001-7310-8686 surname: Saadatnejad fullname: Saadatnejad, Saeed email: saeed.saadatnejad@epfl.ch organization: VITA Laboratory, EPFL, Lausanne, Switzerland – sequence: 2 givenname: Reyhaneh orcidid: 0009-0003-3638-0260 surname: Hosseininejad fullname: Hosseininejad, Reyhaneh email: reyhaneh.hosseininejad@epfl.ch organization: VITA Laboratory, EPFL, Lausanne, Switzerland – sequence: 3 givenname: Jose orcidid: 0000-0002-6558-8656 surname: Barreiros fullname: Barreiros, Jose email: jose.barreiros@tri.global organization: Toyota Research Institute, Cambridge, MA, USA – sequence: 4 givenname: Katherine M. orcidid: 0000-0003-1262-4083 surname: Tsui fullname: Tsui, Katherine M. email: kate.tsui@tri.global organization: Toyota Research Institute, Cambridge, MA, USA – sequence: 5 givenname: Alexandre orcidid: 0000-0002-5004-1498 surname: Alahi fullname: Alahi, Alexandre email: alexandre.alahi@epfl.ch organization: VITA Laboratory, EPFL, Lausanne, Switzerland
BookMark	eNpNkEtPwzAQhC1UJErpnQMHS5xT_IjtmFsoj1SqBOJxxXKdjZrSOsVOD_33pKQSXHb3MDOr-c7RwDceELqkZEIp0Tfz13zCCBMTLjJJKD1BQ8aVSriScvDvPkPjGFeEECqY4loM0WdRzJI8xjq2tzjH97a1EVpsfYnvwLvlxoYv3FS42G2sT34nnvkWgnVt3Xhce_yy3Mfa2TXuY6x3gN8ceBvq5gKdVnYdYXzcI_Tx-PA-LZL589Nsms8Tx1LVJooySQjPwEIqATImhHAy4ymhJWdEW75gLi116TTJSgsLLqmuJLM2peWiFHyErvvcbWi-dxBbs2p2wXcvDWeplCqVOutUpFe50MQYoDLbUHcN94YScwBpOpDmANIcQXaWq95SA8CfnBJFM634D26gbwQ
CODEN	IRALC6
Cites_doi	10.1109/ICCV51070.2023.00880 10.1109/ICCV.2019.00554 10.23919/ECC.2018.8550170 10.1109/LRA.2020.3010742 10.1609/aaai.v33i01.33018553 10.1109/LRA.2024.3374188 10.1016/j.trc.2022.103705 10.1145/3450626.3459670 10.1109/cvpr42600.2020.00724 10.1109/ICCV48922.2021.01144 10.1007/978-3-030-01249-6_37 10.1109/ICRA.2019.8794134 10.1109/WACV56688.2023.00479 10.1109/ROBOT.2009.5152855 10.1007/978-3-030-58545-7_20 10.1109/CVPR.2019.00589 10.1093/restud/rdab031 10.1109/ICCV51070.2023.00875 10.1109/CVPR52688.2022.01271 10.1109/CVPR52733.2024.00163 10.1109/LRA.2018.2861569 10.1109/ICCV.2019.00958 10.1109/CVPR.2018.00548 10.1109/CVPR52688.2022.01042 10.1109/TPAMI.2013.248 10.1109/ACCESS.2024.3400604 10.1109/ICCV51070.2023.01385 10.1109/ICCV48922.2021.01305 10.1109/3DV53792.2021.00066 10.1109/ICCV.2015.494 10.1109/ICCV48922.2021.01102 10.1177/0278364921990671 10.7759/cureus.42905 10.1109/ICCV.2017.361 10.1109/IROS45743.2020.9340705 10.1109/LRA.2021.3086666 10.1038/d41586-024-01184-4 10.1109/ICRA48891.2023.10160399 10.1109/CVPR52688.2022.00633 10.1109/CVPR.2017.497 10.1109/CVPR52729.2023.01632 10.1109/IROS.2015.7353843 10.15607/RSS.2023.XIX.026 10.1109/LRA.2019.2895266 10.1109/CYBER.2018.8688348 10.1007/978-3-031-72655-2_20 10.1109/ICCV51070.2023.00355 10.1109/TPAMI.2021.3070543 10.1109/ICCV51070.2023.00220
ContentType	Journal Article
Copyright	Copyright The Institute of Electrical and Electronics Engineers, Inc. (IEEE) 2025
Copyright_xml	– notice: Copyright The Institute of Electrical and Electronics Engineers, Inc. (IEEE) 2025
DBID	97E RIA RIE AAYXX CITATION 7SC 7SP 8FD JQ2 L7M L~C L~D
DOI	10.1109/LRA.2025.3586011
DatabaseName	IEEE All-Society Periodicals Package (ASPP) 2005–Present IEEE All-Society Periodicals Package (ASPP) 1998–Present IEEE Electronic Library (IEL) CrossRef Computer and Information Systems Abstracts Electronics & Communications Abstracts Technology Research Database ProQuest Computer Science Collection Advanced Technologies Database with Aerospace Computer and Information Systems Abstracts Academic Computer and Information Systems Abstracts Professional
DatabaseTitle	CrossRef Technology Research Database Computer and Information Systems Abstracts – Academic Electronics & Communications Abstracts ProQuest Computer Science Collection Computer and Information Systems Abstracts Advanced Technologies Database with Aerospace Computer and Information Systems Abstracts Professional
DatabaseTitleList	Technology Research Database
Database_xml	– sequence: 1 dbid: RIE name: IEEE Xplore Digital Library (LUT) url: https://ieeexplore.ieee.org/ sourceTypes: Publisher
DeliveryMethod	fulltext_linktorsrc
Discipline	Engineering
EISSN	2377-3766
EndPage	8753
ExternalDocumentID	10_1109_LRA_2025_3586011 11071897
Genre	orig-research
GroupedDBID	0R~ 97E AAJGR AARMG AASAJ AAWTH ABAZT ABQJQ ABVLG ACGFS AGQYO AGSQL AHBIQ AKJIK AKQYR ALMA_UNASSIGNED_HOLDINGS ATWAV BEFXN BFFAM BGNUA BKEBE BPEOZ EBS EJD IFIPE IPLJI JAVBF KQ8 M43 M~E O9- OCL RIA RIE AAYXX CITATION 7SC 7SP 8FD JQ2 L7M L~C L~D
ID	FETCH-LOGICAL-c247t-71260038eae46ee82555c683401d3209a3b2c4d9dc908daeb3619f62aa41dbd53
IEDL.DBID	RIE
ISICitedReferencesCount	0
ISICitedReferencesURI	http://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=Summon&SrcAuth=ProQuest&DestLinkType=CitingArticles&DestApp=WOS_CPL&KeyUT=001534489800008&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D
ISSN	2377-3766
IngestDate	Sat Nov 22 13:41:14 EST 2025 Sat Nov 29 07:43:07 EST 2025 Wed Aug 27 02:13:16 EDT 2025
IsPeerReviewed	true
IsScholarly	true
Issue	9
Language	English
License	https://ieeexplore.ieee.org/Xplorehelp/downloads/license-information/IEEE.html https://doi.org/10.15223/policy-029 https://doi.org/10.15223/policy-037
LinkModel	DirectLink
MergedId	FETCHMERGED-LOGICAL-c247t-71260038eae46ee82555c683401d3209a3b2c4d9dc908daeb3619f62aa41dbd53
Notes	ObjectType-Article-1 SourceType-Scholarly Journals-1 ObjectType-Feature-2 content type line 14
ORCID	0000-0001-7310-8686 0009-0003-3638-0260 0000-0002-6558-8656 0000-0002-5004-1498 0000-0003-1262-4083
PQID	3246674698
PQPubID	4437225
PageCount	8
ParticipantIDs	proquest_journals_3246674698 crossref_primary_10_1109_LRA_2025_3586011 ieee_primary_11071897
PublicationCentury	2000
PublicationDate	2025-09-01
PublicationDateYYYYMMDD	2025-09-01
PublicationDate_xml	– month: 09 year: 2025 text: 2025-09-01 day: 01
PublicationDecade	2020
PublicationPlace	Piscataway
PublicationPlace_xml	– name: Piscataway
PublicationTitle	IEEE robotics and automation letters
PublicationTitleAbbrev	LRA
PublicationYear	2025
Publisher	IEEE The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Publisher_xml	– name: IEEE – name: The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
References	ref13 ref57 ref12 ref56 ref15 ref59 ref14 ref58 Ho (ref17) 2020 ref53 ref52 ref11 ref55 ref10 ref54 ref16 ref19 Tashiro (ref48) 2021 ref51 Maaten (ref50) 2008; 9 ref46 ref45 ref47 ref44 ref43 Meredith (ref33) 2001; 211 Hosseininejad (ref18) 2025 ref8 (ref36) 2023 Saadatnejad (ref42) 2022 ref7 ref9 ref3 ref6 ref5 ref40 ref34 ref37 ref31 ref30 ref32 ref2 (ref35) 2024 ref1 ref39 ref38 Saadatnejad (ref41) 2024 ref24 ref23 Bahari (ref4) 2025 ref26 ref25 ref20 ref22 ref21 Tedrake (ref49) 2019 ref28 ref27 ref29 ref60
References_xml	– ident: ref47 doi: 10.1109/ICCV51070.2023.00880 – ident: ref29 doi: 10.1109/ICCV.2019.00554 – ident: ref54 doi: 10.23919/ECC.2018.8550170 – ident: ref2 doi: 10.1109/LRA.2020.3010742 – ident: ref25 doi: 10.1609/aaai.v33i01.33018553 – ident: ref43 doi: 10.1109/LRA.2024.3374188 – ident: ref40 doi: 10.1016/j.trc.2022.103705 – ident: ref38 doi: 10.1145/3450626.3459670 – ident: ref13 doi: 10.1109/cvpr42600.2020.00724 – ident: ref6 doi: 10.1109/ICCV48922.2021.01144 – start-page: 22680 volume-title: Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit. year: 2025 ident: ref18 article-title: MotionMap: Representing multimodality in human pose forecasting – ident: ref52 doi: 10.1007/978-3-030-01249-6_37 – ident: ref8 doi: 10.1109/ICRA.2019.8794134 – ident: ref16 doi: 10.1109/WACV56688.2023.00479 – ident: ref45 doi: 10.1109/ROBOT.2009.5152855 – ident: ref57 doi: 10.1007/978-3-030-58545-7_20 – ident: ref59 doi: 10.1109/CVPR.2019.00589 – ident: ref1 doi: 10.1093/restud/rdab031 – ident: ref9 doi: 10.1109/ICCV51070.2023.00875 – ident: ref15 doi: 10.1109/CVPR52688.2022.01271 – volume: 9 year: 2008 ident: ref50 article-title: Visualizing data using t-SNE publication-title: J. Mach. Learn. Res. – ident: ref22 doi: 10.1109/CVPR52733.2024.00163 – ident: ref12 doi: 10.1109/LRA.2018.2861569 – year: 2019 ident: ref49 article-title: Drake: Model-based design and verification for robotics – start-page: 12301 volume-title: Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit. year: 2025 ident: ref4 article-title: Certified human trajectory prediction – ident: ref30 doi: 10.1109/ICCV.2019.00958 – ident: ref26 doi: 10.1109/CVPR.2018.00548 – ident: ref39 doi: 10.1109/CVPR52688.2022.01042 – ident: ref19 doi: 10.1109/TPAMI.2013.248 – ident: ref34 doi: 10.1109/ACCESS.2024.3400604 – ident: ref60 doi: 10.1109/ICCV51070.2023.01385 – ident: ref27 doi: 10.1109/ICCV48922.2021.01305 – volume-title: Proc. Symp. Eur. Assoc. Res. Transp. year: 2022 ident: ref42 article-title: Pedestrian 3D bounding box prediction – ident: ref3 doi: 10.1109/3DV53792.2021.00066 – ident: ref14 doi: 10.1109/ICCV.2015.494 – ident: ref46 doi: 10.1109/ICCV48922.2021.01102 – year: 2023 ident: ref36 article-title: Skeleton marker set: Core (50) – volume-title: Proc. Int. Conf. Learn. Representations year: 2024 ident: ref41 article-title: Social-transmotion: Promptable human trajectory prediction – ident: ref23 doi: 10.1177/0278364921990671 – ident: ref37 doi: 10.7759/cureus.42905 – ident: ref53 doi: 10.1109/ICCV.2017.361 – ident: ref7 doi: 10.1109/IROS45743.2020.9340705 – ident: ref51 doi: 10.1109/LRA.2021.3086666 – ident: ref55 doi: 10.1038/d41586-024-01184-4 – ident: ref44 doi: 10.1109/ICRA48891.2023.10160399 – ident: ref28 doi: 10.1109/CVPR52688.2022.00633 – ident: ref32 doi: 10.1109/CVPR.2017.497 – ident: ref56 doi: 10.1109/CVPR52729.2023.01632 – ident: ref24 doi: 10.1109/IROS.2015.7353843 – start-page: 6840 volume-title: Proc. Adv. Neural Inf. Process. Syst. year: 2020 ident: ref17 article-title: Denoising diffusion probabilistic models – ident: ref10 doi: 10.15607/RSS.2023.XIX.026 – ident: ref11 doi: 10.1109/LRA.2019.2895266 – ident: ref21 doi: 10.1109/CYBER.2018.8688348 – ident: ref20 doi: 10.1007/978-3-031-72655-2_20 – ident: ref58 doi: 10.1109/ICCV51070.2023.00355 – ident: ref31 doi: 10.1109/TPAMI.2021.3070543 – ident: ref5 doi: 10.1109/ICCV51070.2023.00220 – volume: 211 start-page: 241 issue: 241244 year: 2001 ident: ref33 article-title: Motion capture file formats explained publication-title: Dept. Comput. Sci., Univ. Sheffield – start-page: 24804 volume-title: Proc. Adv. Neural Inf. Process. Syst. year: 2021 ident: ref48 article-title: CSDI: Conditional score-based diffusion models for probabilistic time series imputation – year: 2024 ident: ref35 article-title: Prime 17w
SSID	ssj0001527395
Score	2.3019602
Snippet	The increasing labor shortage and aging population underline the need for assistive robots to support human care recipients. To enable safe and responsive...
SourceID	proquest crossref ieee
SourceType	Aggregation Database Index Database Publisher
StartPage	8746
SubjectTerms	Data sets for robot learning Datasets Diffusion models Dynamics Human motion Human-robot interaction intention recognition Motion capture Noise reduction physical human-robot interaction Predictive models Receivers Robot dynamics Robots Service robots Skeleton Transformers
Title	HHI-Assist: A Dataset and Benchmark of Human-Human Interaction in Physical Assistance Scenario
URI	https://ieeexplore.ieee.org/document/11071897 https://www.proquest.com/docview/3246674698
Volume	10
WOSCitedRecordID	wos001534489800008&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D
hasFullText	1
inHoldings	1
isFullTextHit
isPrint
journalDatabaseRights	– providerCode: PRVIEE databaseName: IEEE Xplore Digital Library (LUT) customDbUrl: eissn: 2377-3766 dateEnd: 99991231 omitProxy: false ssIdentifier: ssj0001527395 issn: 2377-3766 databaseCode: RIE dateStart: 20160101 isFulltext: true titleUrlDefault: https://ieeexplore.ieee.org/ providerName: IEEE – providerCode: PRVHPJ databaseName: ROAD: Directory of Open Access Scholarly Resources customDbUrl: eissn: 2377-3766 dateEnd: 99991231 omitProxy: false ssIdentifier: ssj0001527395 issn: 2377-3766 databaseCode: M~E dateStart: 20160101 isFulltext: true titleUrlDefault: https://road.issn.org providerName: ISSN International Centre
link	http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV1LS8QwEB7cxYMefIvrY8nBi4do27RN4m3VlRVUxAfsyZImKS5iV3arR3-7k7SLinjwUkJpQplv2plvkpkB2EeTUahYKVoUMaexzjXNBWNUS9SPiOdaMOWbTfDrazEcypsmWd3nwlhr_eEze-iGfi_fjPWbC5UdOa4SCslb0OKc18laXwEVV0pMJrOtyEAeXd72kABGySFLBPKO8Ifp8b1Ufv2AvVU5X_7n-6zAUuM-kl6N9yrM2XINFr8VFVyHx8HggqLUEb9j0iNnqkJDVRFVGnKCKvn0oibPZFwQH76n_kp8XLBOcSCjktw04JF6GacY5E7bEnn1eAMezvv3pwPadFGgOop5RXnoatAzYZWNU2uRESaJTgVDYmVYFEjF8kjHRhotA2EUkmvkVEUaKRWHJjcJ24R2OS7tFhAuAmNydLGkRkeEByrNUxGmlifuDrcdOJgJOHuti2VknmQEMkMwMgdG1oDRgQ0n0K_nGll2YHcGSdZ8TtMMvb405a7Z5fYf03Zgwa1en_7ahXY1ebN7MK_fq9F00oXW1Ue_6_XlE4p8vVg
linkProvider	IEEE
linkToHtml	http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV3NS-wwEB_8AvXgt7h-vRy8eIi2Tdok3tYvVt66iPrAkyVNUhSxK2v173eSdvGJePBSQmnaMr9pZ36TzAzAHpqMUnOtaVlyQbkpDC0kY9Qo1I9EFEYyHZpNiMFA3t2pqzZZPeTCOOfC5jN34IdhLd8OzZsPlR16rhJLJSZhOuU8iZt0rc-Qii8mptLxYmSkDvvXXaSASXrAUonMI_5ifEI3lW-_4GBXzhd_-UZLsNA6kKTbIL4ME65agfn_ygquwn2vd0FR7ojgEemSU12jqaqJriw5RqV8eNajJzIsSQjg03AkITLYJDmQx4pctfCR5jZeNciNcRUy6-Ea_Ds_uz3p0baPAjUJFzUVsa9Cz6TTjmfOISdMU5NJhtTKsiRSmhWJ4VZZoyJpNdJrZFVllmjNY1vYlK3DVDWs3AYQISNrC3SylEFXREQ6KzIZZ06k_oxwHdgfCzh_acpl5IFmRCpHMHIPRt6C0YE1L9DP61pZdmB7DEneflCvOfp9WSZ8u8vNH6b9gdne7WU_718M_m7BnH9SsxdsG6bq0ZvbgRnzXj--jnaD1nwAFAy_bg
openUrl	ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=HHI-Assist%3A+A+Dataset+and+Benchmark+of+Human-Human+Interaction+in+Physical+Assistance+Scenario&rft.jtitle=IEEE+robotics+and+automation+letters&rft.au=Saadatnejad%2C+Saeed&rft.au=Hosseininejad%2C+Reyhaneh&rft.au=Barreiros%2C+Jose&rft.au=Tsui%2C+Katherine+M&rft.date=2025-09-01&rft.pub=The+Institute+of+Electrical+and+Electronics+Engineers%2C+Inc.+%28IEEE%29&rft.eissn=2377-3766&rft.volume=10&rft.issue=12&rft.spage=8746&rft.epage=8753&rft_id=info:doi/10.1109%2FLRA.2025.3586011&rft.externalDBID=NO_FULL_TEXT
thumbnail_l	http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/lc.gif&issn=2377-3766&client=summon
thumbnail_m	http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/mc.gif&issn=2377-3766&client=summon
thumbnail_s	http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/sc.gif&issn=2377-3766&client=summon