Adaptive Fingers Coordination for Robust Grasp and In-Hand Manipulation Under Disturbances and Unknown Dynamics
We present a control framework for achieving a robust object grasp and manipulation in hand. In-hand manipulation remains a demanding task as the object is never stable and task success relies on carefully synchronizing the fingers' dynamics. Indeed, fingers must simultaneously generate motion...
Saved in:
| Published in: | IEEE transactions on robotics Vol. 39; no. 5; pp. 1 - 18 |
|---|---|
| Main Authors: | , |
| Format: | Journal Article |
| Language: | English |
| Published: |
New York
IEEE
01.10.2023
The Institute of Electrical and Electronics Engineers, Inc. (IEEE) |
| Subjects: | |
| ISSN: | 1552-3098, 1941-0468 |
| Online Access: | Get full text |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| Abstract | We present a control framework for achieving a robust object grasp and manipulation in hand. In-hand manipulation remains a demanding task as the object is never stable and task success relies on carefully synchronizing the fingers' dynamics. Indeed, fingers must simultaneously generate motion while maintaining contact with the object and, by staying within the hand's frame, ensuring that the object remains manipulable. These challenges are exacerbated once the hand gets disturbed or when the internal dynamics of the manipulated object are unknown, such as when it is filled with liquid moving during manipulation. We present a control strategy based on coupled dynamical systems (DSs), whereby the fingers move in synchronization using an intermediate dynamics responsible for coordinating fingers. To adapt to changes in forces due to model uncertainties and unexpected disturbances, we employ an adaptive torque-controller combined with a joint impedance regulator that guarantees high tracking accuracy while adapting to dynamic changes. We validate the approach in multiple experiments on 16-degrees-of-freedom robotic hand grasping and manipulating objects with different mass properties, e.g., uneven or varying mass distribution in a glass half-filled with water. We show that the robot can compensate for disturbances generated by internal dynamics and external perturbations. Additionally, we showcase how our controller, in conjunction with learning from human demonstration, provides a robust solution for more complicated manipulations such as finger gaiting. |
|---|---|
| AbstractList | We present a control framework for achieving a robust object grasp and manipulation in hand. In-hand manipulation remains a demanding task as the object is never stable and task success relies on carefully synchronizing the fingers' dynamics. Indeed, fingers must simultaneously generate motion while maintaining contact with the object and, by staying within the hand's frame, ensuring that the object remains manipulable. These challenges are exacerbated once the hand gets disturbed or when the internal dynamics of the manipulated object are unknown, such as when it is filled with liquid moving during manipulation. We present a control strategy based on coupled dynamical systems (DSs), whereby the fingers move in synchronization using an intermediate dynamics responsible for coordinating fingers. To adapt to changes in forces due to model uncertainties and unexpected disturbances, we employ an adaptive torque-controller combined with a joint impedance regulator that guarantees high tracking accuracy while adapting to dynamic changes. We validate the approach in multiple experiments on 16-degrees-of-freedom robotic hand grasping and manipulating objects with different mass properties, e.g., uneven or varying mass distribution in a glass half-filled with water. We show that the robot can compensate for disturbances generated by internal dynamics and external perturbations. Additionally, we showcase how our controller, in conjunction with learning from human demonstration, provides a robust solution for more complicated manipulations such as finger gaiting. |
| Author | Khadivar, Farshad Billard, Aude |
| Author_xml | – sequence: 1 givenname: Farshad orcidid: 0000-0003-3519-3052 surname: Khadivar fullname: Khadivar, Farshad organization: Learning Algorithms and Systems Laboratory (LASA), École Polytechnique Fédérale de Lausanne (EPFL), Lausanne, Switzerland – sequence: 2 givenname: Aude orcidid: 0000-0002-7076-8010 surname: Billard fullname: Billard, Aude organization: Learning Algorithms and Systems Laboratory (LASA), École Polytechnique Fédérale de Lausanne (EPFL), Lausanne, Switzerland |
| BookMark | eNp9kDtvwjAUha2KSgXavUMHS51D_UrijAjKQ6JCQjBHxrmpTMFO7aQV_77hMVQdOp07fOce6euhjnUWEHqkZEApyV7Wq-WAEcYHnElCmLxBXZoJGhGRyE57xzGLOMnkHeqFsGsJkRHeRW5YqKo2X4Anxr6DD3jknC-MVbVxFpfO45XbNqHGU69ChZUt8NxGs1O-KWuqZn8hN7YAj8cm1I3fKqshnNmN_bDu2-Lx0aqD0eEe3ZZqH-Dhmn20mbyuR7NosZzOR8NFpAUhdVQkXAFXUlLKdAygdJEJkNu4pBpIKoRgQmsqs1LIlNOSxloSVnJgIi0LnfI-er78rbz7bCDU-c413raTOZMpE0nMk6ylyIXS3oXgocwrbw7KH3NK8pPWvNWan7TmV61tJflT0aY-K6i9Mvv_ik-XogGAXztUJERw_gPVyIe8 |
| CODEN | ITREAE |
| CitedBy_id | crossref_primary_10_1016_j_rcim_2025_103021 crossref_primary_10_1017_S0263574724001929 crossref_primary_10_1541_ieejjia_24004855 crossref_primary_10_1109_LRA_2025_3589168 crossref_primary_10_1109_TASE_2025_3528051 crossref_primary_10_1109_TSMC_2024_3500776 crossref_primary_10_1109_TIM_2024_3472894 |
| Cites_doi | 10.1109/ICRA.2015.7139709 10.1007/s10514-014-9402-3 10.1109/TRO.2019.2890990 10.1109/70.897777 10.1109/ROBOT.1998.677060 10.1109/ICRA48506.2021.9561011 10.1007/s10514-018-9772-z 10.1109/IROS.2015.7354264 10.1016/j.robot.2020.103700 10.1109/ICRA.2014.6907059 10.1109/TRO.2009.2014125 10.1109/ICRA.2013.6630637 10.1109/HUMANOIDS.2014.7041347 10.1109/ROBOT.2004.1307247 10.1109/70.880798 10.1109/TRO.2011.2159412 10.1177/0278364919887447 10.1109/ROBIO.2014.7090666 10.1109/ICRA.2011.5980395 10.1007/s10514-014-9397-9 10.1109/ICRA.2018.8461263 10.1109/ROBOT.2006.1641181 10.1109/LRA.2018.2856536 10.1109/LRA.2015.2509025 10.1109/TRO.2020.3003230 10.1109/TRO.2017.2765668 10.1177/0278364919883343 10.1109/ICRA.2019.8794102 10.1016/0005-1098(89)90054-X 10.1177/02783640122067273 10.1109/IROS.2014.6943027 10.1007/978-3-319-32552-1_38 10.1007/s10514-009-9164-5 10.1109/ICRA.2014.6907062 10.1177/0278364918765952 10.1109/ICRA.2012.6224681 10.1145/1390156.1390196 10.1109/LRA.2018.2852779 10.1109/IROS.2012.6386006 10.1093/oso/9780198538493.001.0001 10.1109/ICRA.2018.8460496 10.1109/THMS.2015.2470657 10.1109/TRO.2016.2588879 10.1016/j.robot.2015.09.008 10.1007/978-3-319-32552-1_39 10.1109/ROBOT.2010.5509377 10.1080/01691864.2017.1365011 10.1109/LRA.2020.2974702 10.1109/TRO.2017.2693391 10.1109/ROBOT.2000.844067 |
| ContentType | Journal Article |
| Copyright | Copyright The Institute of Electrical and Electronics Engineers, Inc. (IEEE) 2023 |
| Copyright_xml | – notice: Copyright The Institute of Electrical and Electronics Engineers, Inc. (IEEE) 2023 |
| DBID | 97E RIA RIE AAYXX CITATION 7SC 7SP 7TB 8FD FR3 JQ2 L7M L~C L~D |
| DOI | 10.1109/TRO.2023.3280028 |
| DatabaseName | IEEE All-Society Periodicals Package (ASPP) 2005-present IEEE All-Society Periodicals Package (ASPP) 1998-Present IEEE Electronic Library (IEL) CrossRef Computer and Information Systems Abstracts Electronics & Communications Abstracts Mechanical & Transportation Engineering Abstracts Technology Research Database Engineering Research Database ProQuest Computer Science Collection Advanced Technologies Database with Aerospace Computer and Information Systems Abstracts Academic Computer and Information Systems Abstracts Professional |
| DatabaseTitle | CrossRef Technology Research Database Computer and Information Systems Abstracts – Academic Mechanical & Transportation Engineering Abstracts Electronics & Communications Abstracts ProQuest Computer Science Collection Computer and Information Systems Abstracts Engineering Research Database Advanced Technologies Database with Aerospace Computer and Information Systems Abstracts Professional |
| DatabaseTitleList | Technology Research Database |
| Database_xml | – sequence: 1 dbid: RIE name: IEEE Xplore url: https://ieeexplore.ieee.org/ sourceTypes: Publisher |
| DeliveryMethod | fulltext_linktorsrc |
| Discipline | Engineering |
| EISSN | 1941-0468 |
| EndPage | 18 |
| ExternalDocumentID | 10_1109_TRO_2023_3280028 10146043 |
| Genre | orig-research |
| GrantInformation_xml | – fundername: European Research Council grantid: 741945 funderid: 10.13039/501100000781 |
| GroupedDBID | .DC 0R~ 29I 4.4 5GY 6IK 97E AAJGR AARMG AASAJ AAWTH ABAZT ABQJQ ABVLG ACGFO ACIWK AENEX AGQYO AHBIQ AKJIK AKQYR ALMA_UNASSIGNED_HOLDINGS ATWAV BEFXN BFFAM BGNUA BKEBE BPEOZ CS3 DU5 EBS F5P HZ~ IFIPE IPLJI JAVBF LAI M43 MS~ O9- OCL P2P PQQKQ RIA RIE RNS 5VS AAYXX AETIX AGSQL AIBXA CITATION EJD H~9 VJK 7SC 7SP 7TB 8FD FR3 JQ2 L7M L~C L~D |
| ID | FETCH-LOGICAL-c400t-d63ae3a88112c5eeacd94e8b5f1ce0744424cc189f48731f15c802f3e247fdc73 |
| IEDL.DBID | RIE |
| ISICitedReferencesCount | 17 |
| ISICitedReferencesURI | http://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=Summon&SrcAuth=ProQuest&DestLinkType=CitingArticles&DestApp=WOS_CPL&KeyUT=001005858700001&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| ISSN | 1552-3098 |
| IngestDate | Mon Jun 30 02:13:03 EDT 2025 Tue Nov 18 22:37:43 EST 2025 Sat Nov 29 01:47:31 EST 2025 Wed Aug 27 02:57:13 EDT 2025 |
| IsDoiOpenAccess | false |
| IsOpenAccess | true |
| IsPeerReviewed | true |
| IsScholarly | true |
| Issue | 5 |
| Language | English |
| License | https://ieeexplore.ieee.org/Xplorehelp/downloads/license-information/IEEE.html https://doi.org/10.15223/policy-029 https://doi.org/10.15223/policy-037 |
| LinkModel | DirectLink |
| MergedId | FETCHMERGED-LOGICAL-c400t-d63ae3a88112c5eeacd94e8b5f1ce0744424cc189f48731f15c802f3e247fdc73 |
| Notes | ObjectType-Article-1 SourceType-Scholarly Journals-1 ObjectType-Feature-2 content type line 14 |
| ORCID | 0000-0003-3519-3052 0000-0002-7076-8010 |
| OpenAccessLink | http://infoscience.epfl.ch/record/303245 |
| PQID | 2872465369 |
| PQPubID | 27625 |
| PageCount | 18 |
| ParticipantIDs | ieee_primary_10146043 crossref_citationtrail_10_1109_TRO_2023_3280028 proquest_journals_2872465369 crossref_primary_10_1109_TRO_2023_3280028 |
| PublicationCentury | 2000 |
| PublicationDate | 2023-10-01 |
| PublicationDateYYYYMMDD | 2023-10-01 |
| PublicationDate_xml | – month: 10 year: 2023 text: 2023-10-01 day: 01 |
| PublicationDecade | 2020 |
| PublicationPlace | New York |
| PublicationPlace_xml | – name: New York |
| PublicationTitle | IEEE transactions on robotics |
| PublicationTitleAbbrev | TRO |
| PublicationYear | 2023 |
| Publisher | IEEE The Institute of Electrical and Electronics Engineers, Inc. (IEEE) |
| Publisher_xml | – name: IEEE – name: The Institute of Electrical and Electronics Engineers, Inc. (IEEE) |
| References | ref12 ref15 ref14 ref58 ref53 ref52 ref11 ref10 ref54 dafle (ref40) 0 ref17 ref16 ref19 ref18 farshad (ref45) 0 ref51 ref50 ref46 goodwin (ref9) 2014 ref48 ref47 ref42 karayiannidis (ref20) 0 ref41 ref44 ref43 bishop (ref56) 1995 ref49 ref8 ref7 ref4 ref3 ref6 ref5 ref35 ref34 ref36 ref31 ref30 ref33 ref32 ref2 ref1 ref39 ref38 kumar (ref37) 2016 paszke (ref57) 0 ref24 ref23 ref26 ref25 ref22 ref21 ref28 ref27 ref29 rajeswaran (ref13) 2017 khalil (ref55) 2002; 3 |
| References_xml | – ident: ref19 doi: 10.1109/ICRA.2015.7139709 – ident: ref50 doi: 10.1007/s10514-014-9402-3 – ident: ref21 doi: 10.1109/TRO.2019.2890990 – ident: ref25 doi: 10.1109/70.897777 – start-page: 399 year: 0 ident: ref20 article-title: Adaptive control for pivoting with visual and tactile feedback publication-title: Proc IEEE Int Conf Robot Automat – ident: ref11 doi: 10.1109/ROBOT.1998.677060 – ident: ref35 doi: 10.1109/ICRA48506.2021.9561011 – ident: ref30 doi: 10.1007/s10514-018-9772-z – ident: ref39 doi: 10.1109/IROS.2015.7354264 – year: 2017 ident: ref13 article-title: Learning complex dexterous manipulation with deep reinforcement learning and demonstrations – ident: ref44 doi: 10.1016/j.robot.2020.103700 – ident: ref41 doi: 10.1109/ICRA.2014.6907059 – ident: ref5 doi: 10.1109/TRO.2009.2014125 – ident: ref32 doi: 10.1109/ICRA.2013.6630637 – start-page: 1934 year: 0 ident: ref45 article-title: Efficient configuration exploration in inverse dynamics acquisition of robotic manipulators publication-title: Proc IEEE Int Conf Robot Automat – ident: ref54 doi: 10.1109/HUMANOIDS.2014.7041347 – ident: ref22 doi: 10.1109/ROBOT.2004.1307247 – start-page: 254 year: 0 ident: ref40 article-title: Stable prehensile pushing: In-hand manipulation with alternating sticking contacts publication-title: Proc IEEE Int Conf Robot Automat – ident: ref17 doi: 10.1109/70.880798 – ident: ref43 doi: 10.1109/TRO.2011.2159412 – volume: 3 year: 2002 ident: ref55 publication-title: Nonlinear Systems – ident: ref14 doi: 10.1177/0278364919887447 – ident: ref23 doi: 10.1109/ROBIO.2014.7090666 – ident: ref48 doi: 10.1109/ICRA.2011.5980395 – ident: ref58 doi: 10.1007/s10514-014-9397-9 – ident: ref46 doi: 10.1109/ICRA.2018.8461263 – ident: ref42 doi: 10.1109/ROBOT.2006.1641181 – ident: ref8 doi: 10.1109/LRA.2018.2856536 – ident: ref51 doi: 10.1109/LRA.2015.2509025 – ident: ref34 doi: 10.1109/TRO.2020.3003230 – year: 2016 ident: ref37 article-title: Learning dexterous manipulation policies from experience and imitation – ident: ref24 doi: 10.1109/TRO.2017.2765668 – year: 0 ident: ref57 article-title: Automatic differentiation in PyTorch publication-title: Proc NIPS Workshop – ident: ref49 doi: 10.1177/0278364919883343 – ident: ref38 doi: 10.1109/ICRA.2019.8794102 – ident: ref10 doi: 10.1016/0005-1098(89)90054-X – ident: ref18 doi: 10.1177/02783640122067273 – ident: ref33 doi: 10.1109/IROS.2014.6943027 – ident: ref3 doi: 10.1007/978-3-319-32552-1_38 – ident: ref12 doi: 10.1007/s10514-009-9164-5 – ident: ref1 doi: 10.1109/ICRA.2014.6907062 – ident: ref7 doi: 10.1177/0278364918765952 – ident: ref15 doi: 10.1109/ICRA.2012.6224681 – ident: ref53 doi: 10.1145/1390156.1390196 – ident: ref36 doi: 10.1109/LRA.2018.2852779 – ident: ref52 doi: 10.1109/IROS.2012.6386006 – year: 1995 ident: ref56 publication-title: Neural Networks for Pattern Recognition doi: 10.1093/oso/9780198538493.001.0001 – ident: ref28 doi: 10.1109/ICRA.2018.8460496 – ident: ref2 doi: 10.1109/THMS.2015.2470657 – ident: ref16 doi: 10.1109/TRO.2016.2588879 – ident: ref47 doi: 10.1016/j.robot.2015.09.008 – ident: ref6 doi: 10.1007/978-3-319-32552-1_39 – ident: ref29 doi: 10.1109/ROBOT.2010.5509377 – year: 2014 ident: ref9 publication-title: Adaptive Filtering Prediction and Control – ident: ref26 doi: 10.1080/01691864.2017.1365011 – ident: ref31 doi: 10.1109/LRA.2020.2974702 – ident: ref27 doi: 10.1109/TRO.2017.2693391 – ident: ref4 doi: 10.1109/ROBOT.2000.844067 |
| SSID | ssj0024903 |
| Score | 2.5281208 |
| Snippet | We present a control framework for achieving a robust object grasp and manipulation in hand. In-hand manipulation remains a demanding task as the object is... |
| SourceID | proquest crossref ieee |
| SourceType | Aggregation Database Enrichment Source Index Database Publisher |
| StartPage | 1 |
| SubjectTerms | Controllers Coupled dynamical systems dexterous manipulation Disturbances Dynamical systems End effectors finger gating Fingers grasping Grasping (robotics) Mass distribution object with uneven mass distribution Perturbation robust/adaptive control of robotic systems Robustness (mathematics) Synchronism |
| Title | Adaptive Fingers Coordination for Robust Grasp and In-Hand Manipulation Under Disturbances and Unknown Dynamics |
| URI | https://ieeexplore.ieee.org/document/10146043 https://www.proquest.com/docview/2872465369 |
| Volume | 39 |
| WOSCitedRecordID | wos001005858700001&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| hasFullText | 1 |
| inHoldings | 1 |
| isFullTextHit | |
| isPrint | |
| journalDatabaseRights | – providerCode: PRVIEE databaseName: IEEE Xplore customDbUrl: eissn: 1941-0468 dateEnd: 99991231 omitProxy: false ssIdentifier: ssj0024903 issn: 1552-3098 databaseCode: RIE dateStart: 20040101 isFulltext: true titleUrlDefault: https://ieeexplore.ieee.org/ providerName: IEEE |
| link | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV07T8MwELZoxQADzyIKBXlgYUibxE5sj1VLKQMFVa3ULXJtR6qEkioPfj-2k0ARAokpGWwrynd3vjv77gPgTjCuwxspHcqJcnDItc75gXS0789ZrHAcBNySTZDZjK5W7LUuVre1MEope_lM9c2rPcuXqShNqmxgeGVDF6MWaBFCqmKtr8Z6zNIgm5ZiDnIZbc4kXTZYzF_6hia8j3zjH9Fve5AlVflhie32Mjn-54edgKPaj4TDCvhTsKeSM3C4013wHKRDybfGmsGJzd3lcJTqUHNT5f-g9lbhPF2XeQEfM55vIU8kfEqcqXk-82TTMHtBy40Ex1ogymxtpCS3Y5eJScglcFyR2ucdsJw8LEZTp-ZXcITW3MKRIeIKcUq1zyUCpU2wZFjRdRB7QmnXAmMfC-FRFuuoBnmxFwjq-jFSPiaxFARdgHaSJuoSQBFyKoSLPS4JpiHjMpAkCKWHOSIC4S4YNH88EnXzccOB8RbZIMRlkcYoMhhFNUZdcP85Y1s13vhjbMdgsjOugqMLeg2qUa2aeaRDRN80lQvZ1S_TrsGBWb26stcD7SIr1Q3YF-_FJs9urdR9AOYi1So |
| linkProvider | IEEE |
| linkToHtml | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV1LT8MwDLZ4ScCBN2I8c-DCoVvbJG1yRBtjE9tA05C4VVmSSpNQO60bv58k7XgIgcSpPThqlc92bCfxB3AtuTDpjVIeE7H2SCSMzYVUeSb2FzzVJKVUOLKJeDBgLy_8qbqs7u7CaK3d4TNdt69uL1_lcmFLZQ3LKxv5BK_COiUkDMrrWp-t9bgjQrZNxTzsc7bclfR5YzR8rFui8DoObYTEvq1Cjlblhy92C0x795-_tgc7VSSJbkvo92FFZwew_aW_4CHkt0pMrT9DbVe9K1AzN8nmpKwAIhOvomE-XhRzdD8TxRSJTKFu5nXssy-yyZLbCzl2JNQyKrGYja2eFE72ObMluQy1Slr74gie23ejZserGBY8aWx37qkIC40FYybqklQbJ6w40WxM00BqE1yYSSZSBoynJq_BQRpQyfwwxTokcapkjI9hLcszfQJIRoJJ6ZNAqJiwiAtFVUwjFRCBY4lJDRrLGU9k1X7csmC8Ji4N8XliMEosRkmFUQ1uPkZMy9Ybf8geWUy-yJVw1OB8iWpSGWeRmCQxtG3lIn76y7Ar2OyM-r2k1x08nMGW_VJ5gO8c1uazhb6ADfk2nxSzS6eB7yTM2HE |
| openUrl | ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Adaptive+Fingers+Coordination+for+Robust+Grasp+and+In-Hand+Manipulation+Under+Disturbances+and+Unknown+Dynamics&rft.jtitle=IEEE+transactions+on+robotics&rft.au=Khadivar%2C+Farshad&rft.au=Billard%2C+Aude&rft.date=2023-10-01&rft.pub=The+Institute+of+Electrical+and+Electronics+Engineers%2C+Inc.+%28IEEE%29&rft.issn=1552-3098&rft.eissn=1941-0468&rft.volume=39&rft.issue=5&rft.spage=3350&rft_id=info:doi/10.1109%2FTRO.2023.3280028&rft.externalDBID=NO_FULL_TEXT |
| thumbnail_l | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/lc.gif&issn=1552-3098&client=summon |
| thumbnail_m | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/mc.gif&issn=1552-3098&client=summon |
| thumbnail_s | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/sc.gif&issn=1552-3098&client=summon |