Benchmarking the Sim-to-Real Gap in Cloth Manipulation

Realistic physics engines play a crucial role for learning to manipulate deformable objects such as garments in simulation. By doing so, researchers can circumvent challenges such as sensing the deformation of the object in the real-world. In spite of the extensive use of simulations for this task,...

Celý popis

Uložené v:

Podrobná bibliografia
Vydané v:	IEEE robotics and automation letters Ročník 9; číslo 3; s. 2981 - 2988
Hlavní autori:	Blanco-Mulero, David, Barbany, Oriol, Alcan, Gokhan, Colome, Adria, Torras, Carme, Kyrki, Ville
Médium:	Journal Article
Jazyk:	English
Vydavateľské údaje:	Piscataway IEEE 01.03.2024 The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Predmet:	Benchmark testing Benchmarks bimanual manipulation Cloth Computing time Data sets for robot learning Datasets deformable object manipulation Deformation Dynamics Engines Fabrics Formability Manipulator dynamics Simulation Simulators Source code Stability analysis Task analysis Trajectory
ISSN:	2377-3766, 2377-3766
On-line prístup:	Získať plný text
Tagy:	Pridať tag Žiadne tagy, Buďte prvý, kto otaguje tento záznam!

Abstract	Realistic physics engines play a crucial role for learning to manipulate deformable objects such as garments in simulation. By doing so, researchers can circumvent challenges such as sensing the deformation of the object in the real-world. In spite of the extensive use of simulations for this task, few works have evaluated the reality gap between deformable object simulators and real-world data. We present a benchmark dataset to evaluate the sim-to-real gap in cloth manipulation. The dataset is collected by performing a dynamic as well as a quasi-static cloth manipulation task involving contact with a rigid table. We use the dataset to evaluate the reality gap, computational time, and simulation stability of four popular deformable object simulators: MuJoCo, Bullet, Flex, and SOFA. Additionally, we discuss the benefits and drawbacks of each simulator. The benchmark dataset is open-source. Supplementary material, videos, and code, can be found at https://sites.google.com/view/cloth-sim2real-benchmark .
AbstractList	Realistic physics engines play a crucial role for learning to manipulate deformable objects such as garments in simulation. By doing so, researchers can circumvent challenges such as sensing the deformation of the object in the real-world. In spite of the extensive use of simulations for this task, few works have evaluated the reality gap between deformable object simulators and real-world data. We present a benchmark dataset to evaluate the sim-to-real gap in cloth manipulation. The dataset is collected by performing a dynamic as well as a quasi-static cloth manipulation task involving contact with a rigid table. We use the dataset to evaluate the reality gap, computational time, and simulation stability of four popular deformable object simulators: MuJoCo, Bullet, Flex, and SOFA. Additionally, we discuss the benefits and drawbacks of each simulator. The benchmark dataset is open-source. Supplementary material, videos, and code, can be found at https://sites.google.com/view/cloth-sim2real-benchmark .
Author	Blanco-Mulero, David Torras, Carme Barbany, Oriol Colome, Adria Kyrki, Ville Alcan, Gokhan
Author_xml	– sequence: 1 givenname: David orcidid: 0000-0003-1866-2144 surname: Blanco-Mulero fullname: Blanco-Mulero, David email: david.blancomulero@aalto.fi organization: Department of Electrical Engineering and Automation (EEA), Aalto University, Espoo, Finland – sequence: 2 givenname: Oriol orcidid: 0000-0002-1379-9152 surname: Barbany fullname: Barbany, Oriol email: obarbany@iri.upc.edu organization: Institut de Robótica i Informática Industrial, CSIC-UPC, Barcelona, Spain – sequence: 3 givenname: Gokhan orcidid: 0000-0003-3025-5883 surname: Alcan fullname: Alcan, Gokhan email: gokhan.alcan@aalto.fi organization: Department of Electrical Engineering and Automation (EEA), Aalto University, Espoo, Finland – sequence: 4 givenname: Adria orcidid: 0000-0001-9715-4062 surname: Colome fullname: Colome, Adria email: acolome@iri.upc.edu organization: Institut de Robótica i Informática Industrial, CSIC-UPC, Barcelona, Spain – sequence: 5 givenname: Carme orcidid: 0000-0002-2933-398X surname: Torras fullname: Torras, Carme email: torras@iri.upc.edu organization: Institut de Robótica i Informática Industrial, CSIC-UPC, Barcelona, Spain – sequence: 6 givenname: Ville orcidid: 0000-0002-5230-5549 surname: Kyrki fullname: Kyrki, Ville email: ville.kyrki@aalto.fi organization: Department of Electrical Engineering and Automation (EEA), Aalto University, Espoo, Finland
BookMark	eNp9kDtPwzAURi1UJErpzsAQiTnFj9iOx1JBQSpCKjBbxrmmLqkTHHfg35PSDhUD073Dd-7jnKNBaAIgdEnwhBCsbhbL6YRiWkwYE7gkxQkaUiZlzqQQg6P-DI27bo0xJpxKpvgQiVsIdrUx8dOHjyytIHvxmzw1-RJMnc1Nm_mQzeomrbInE3y7rU3yTbhAp87UHYwPdYTe7u9eZw_54nn-OJsucltgnHJpGCWOgCSSg1IOAFgprSyq0nEjKuHeLbdcVFQ4bBRlpSqLijtDeQUcMBuh6_3cNjZfW-iSXjfbGPqVmioqJWGC0D4l9ikbm66L4LT16ffOFI2vNcF6p0n3mvROkz5o6kH8B2yj72V8_4dc7RHfP3MUL8juGPYDQmxyww
CODEN	IRALC6
CitedBy_id	crossref_primary_10_20965_jrm_2024_p0950 crossref_primary_10_1109_LRA_2024_3436329 crossref_primary_10_4018_IJSSCI_374216 crossref_primary_10_1109_LRA_2024_3405335 crossref_primary_10_1016_j_mechmachtheory_2025_105993 crossref_primary_10_1109_ACCESS_2025_3582523 crossref_primary_10_1177_02783649251317617 crossref_primary_10_1109_TRO_2025_3588448 crossref_primary_10_1109_LRA_2025_3588048 crossref_primary_10_1146_annurev_control_022723_033252 crossref_primary_10_1109_TCYB_2024_3518975 crossref_primary_10_17979_ja_cea_2025_46_12171
Cites_doi	10.1109/ICRA48506.2021.9561391 10.15607/RSS.2020.XVI.065 10.1109/LRA.2022.3158428 10.15607/RSS.2022.XVIII.016 10.1109/IROS55552.2023.10342002 10.1109/CVPR42600.2020.00356 10.1109/CVPR.2019.00459 10.1007/8415_2012_125 10.1109/LRA.2020.2972852 10.1017/9781108348973 10.1109/LRA.2019.2953663 10.1145/2010324.1964966 10.1109/CVPR46437.2021.00551 10.3389/frobt.2020.00082 10.1145/2601097.2601152 10.1109/LRA.2023.3284382 10.1109/IROS47612.2022.9981376 10.1109/TRO.2020.2986921 10.1109/LRA.2022.3174367 10.1109/ICRA46639.2022.9811651 10.1109/IROS.2018.8593512 10.1145/2001858.2002123 10.1109/ICRA48891.2023.10160653 10.1109/LRA.2020.2965891 10.1109/TASE.2021.3064065 10.1109/LRA.2022.3187843 10.1109/ICRA40945.2020.9196659 10.1109/IROS.2012.6386109 10.1109/IROS47612.2022.9981101 10.1109/cvpr.2018.00030
ContentType	Journal Article
Copyright	Copyright The Institute of Electrical and Electronics Engineers, Inc. (IEEE) 2024
Copyright_xml	– notice: Copyright The Institute of Electrical and Electronics Engineers, Inc. (IEEE) 2024
DBID	97E ESBDL RIA RIE AAYXX CITATION 7SC 7SP 8FD JQ2 L7M L~C L~D
DOI	10.1109/LRA.2024.3360814
DatabaseName	IEEE All-Society Periodicals Package (ASPP) 2005–Present IEEE Xplore Open Access Journals IEEE All-Society Periodicals Package (ASPP) 1998–Present IEEE Electronic Library (IEL) CrossRef Computer and Information Systems Abstracts Electronics & Communications Abstracts Technology Research Database ProQuest Computer Science Collection Advanced Technologies Database with Aerospace Computer and Information Systems Abstracts Academic Computer and Information Systems Abstracts Professional
DatabaseTitle	CrossRef Technology Research Database Computer and Information Systems Abstracts – Academic Electronics & Communications Abstracts ProQuest Computer Science Collection Computer and Information Systems Abstracts Advanced Technologies Database with Aerospace Computer and Information Systems Abstracts Professional
DatabaseTitleList	Technology Research Database
Database_xml	– sequence: 1 dbid: RIE name: IEEE Electronic Library (IEL) url: https://ieeexplore.ieee.org/ sourceTypes: Publisher
DeliveryMethod	fulltext_linktorsrc
Discipline	Engineering
EISSN	2377-3766
EndPage	2988
ExternalDocumentID	10_1109_LRA_2024_3360814 10417136
Genre	orig-research
GrantInformation_xml	– fundername: Academy of Finland grantid: 317020 funderid: 10.13039/501100002341 – fundername: CLOTHILDE grantid: ERC-2016-ADG-741930 – fundername: EU's Horizon Europe Programme – fundername: SoftEnable Project grantid: 101070600 – fundername: EU's H2020 Programme – fundername: MCIN/AEI/10.13039/501100011033 – fundername: Business Finland funderid: 10.13039/501100014438 – fundername: CHLOE-GRAPH Project grantid: PID2020-118649RB-I00
GroupedDBID	0R~ 97E AAJGR AARMG AASAJ AAWTH ABAZT ABQJQ ABVLG ACGFS AGQYO AGSQL AHBIQ AKJIK AKQYR ALMA_UNASSIGNED_HOLDINGS ATWAV BEFXN BFFAM BGNUA BKEBE BPEOZ EBS EJD ESBDL IFIPE IPLJI JAVBF KQ8 M43 M~E O9- OCL RIA RIE AAYXX CITATION 7SC 7SP 8FD JQ2 L7M L~C L~D
ID	FETCH-LOGICAL-c400t-7a321f1e7175e99feee387c74d8f5a6d6fbc5c56d26f0a9238984d5fa25de5e03
IEDL.DBID	RIE
ISICitedReferencesCount	14
ISICitedReferencesURI	http://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=Summon&SrcAuth=ProQuest&DestLinkType=CitingArticles&DestApp=WOS_CPL&KeyUT=001165171700005&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D
ISSN	2377-3766
IngestDate	Sun Nov 30 05:13:06 EST 2025 Tue Nov 18 20:15:39 EST 2025 Sat Nov 29 06:03:29 EST 2025 Wed Aug 27 02:17:05 EDT 2025
IsDoiOpenAccess	true
IsOpenAccess	true
IsPeerReviewed	true
IsScholarly	true
Issue	3
Language	English
License	https://creativecommons.org/licenses/by/4.0/legalcode
LinkModel	DirectLink
MergedId	FETCHMERGED-LOGICAL-c400t-7a321f1e7175e99feee387c74d8f5a6d6fbc5c56d26f0a9238984d5fa25de5e03
Notes	ObjectType-Article-1 SourceType-Scholarly Journals-1 ObjectType-Feature-2 content type line 14
ORCID	0000-0002-2933-398X 0000-0002-5230-5549 0000-0001-9715-4062 0000-0003-1866-2144 0000-0003-3025-5883 0000-0002-1379-9152
OpenAccessLink	https://ieeexplore.ieee.org/document/10417136
PQID	2927713612
PQPubID	4437225
PageCount	8
ParticipantIDs	proquest_journals_2927713612 ieee_primary_10417136 crossref_citationtrail_10_1109_LRA_2024_3360814 crossref_primary_10_1109_LRA_2024_3360814
PublicationCentury	2000
PublicationDate	2024-03-01
PublicationDateYYYYMMDD	2024-03-01
PublicationDate_xml	– month: 03 year: 2024 text: 2024-03-01 day: 01
PublicationDecade	2020
PublicationPlace	Piscataway
PublicationPlace_xml	– name: Piscataway
PublicationTitle	IEEE robotics and automation letters
PublicationTitleAbbrev	LRA
PublicationYear	2024
Publisher	IEEE The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Publisher_xml	– name: IEEE – name: The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
References	ref13 ref35 ref12 ref34 ref37 ref14 Lin (ref20) 2021 ref31 ref11 ref33 ref10 ref32 Zhu (ref36) 2020 ref2 ref1 Coumans (ref15) 20162021 ref17 ref39 ref16 ref38 ref19 ref18 Matas (ref25) 2018 Ha (ref4) Chen (ref21) 2023 ref24 Spong (ref30) 2020 ref23 Wu (ref6) 2020 Yan (ref7) 2021 ref22 ref28 ref27 ref29 ref8 ref9 ref3 Antonova (ref26) 2021 Chi (ref5) 2022
References_xml	– ident: ref8 doi: 10.1109/ICRA48506.2021.9561391 – volume-title: Proc. 16th Robot.: Sci. Syst. year: 2020 ident: ref6 article-title: Learning to manipulate deformable objects without demonstrations doi: 10.15607/RSS.2020.XVI.065 – ident: ref2 doi: 10.1109/LRA.2022.3158428 – volume-title: Proc. Robot.: Sci. Syst. year: 2022 ident: ref5 article-title: Iterative residual policy for goal-conditioned dynamic manipulation of deformable objects doi: 10.15607/RSS.2022.XVIII.016 – volume-title: Robot Modeling and Control year: 2020 ident: ref30 – ident: ref12 doi: 10.1109/IROS55552.2023.10342002 – ident: ref34 doi: 10.1109/CVPR42600.2020.00356 – volume-title: Proc. 11th Int. Conf. Learn. Representations year: 2023 ident: ref21 article-title: DaxBench: Benchmarking deformable object manipulation with differentiable physics – ident: ref33 doi: 10.1109/CVPR.2019.00459 – year: 20162021 ident: ref15 article-title: Pybullet, a Python module for physics simulation for games, robotics and machine learning – ident: ref16 doi: 10.1007/8415_2012_125 – ident: ref1 doi: 10.1109/LRA.2020.2972852 – ident: ref37 doi: 10.1017/9781108348973 – ident: ref9 doi: 10.1109/LRA.2019.2953663 – ident: ref17 doi: 10.1145/2010324.1964966 – ident: ref31 doi: 10.1109/CVPR46437.2021.00551 – ident: ref3 doi: 10.3389/frobt.2020.00082 – ident: ref14 doi: 10.1145/2601097.2601152 – ident: ref29 doi: 10.1109/LRA.2023.3284382 – ident: ref11 doi: 10.1109/IROS47612.2022.9981376 – start-page: 432 volume-title: Proc. Conf. Robot Learn. year: 2021 ident: ref20 article-title: SoftGym: Benchmarking deep reinforcement learning for deformable object manipulation – ident: ref28 doi: 10.1109/TRO.2020.2986921 – ident: ref18 doi: 10.1109/LRA.2022.3174367 – start-page: 564 volume-title: Proc. Conf. Robot Learn. year: 2021 ident: ref7 article-title: Learning predictive representations for deformable objects using contrastive estimation – volume-title: Proc. Neural Inf. Process. Syst. Datasets Benchmarks Track year: 2021 ident: ref26 article-title: Dynamic environments with deformable objects – ident: ref23 doi: 10.1109/ICRA46639.2022.9811651 – ident: ref27 doi: 10.1109/IROS.2018.8593512 – ident: ref38 doi: 10.1145/2001858.2002123 – year: 2020 ident: ref36 article-title: Robosuite: A modular simulation framework and benchmark for robot learning – ident: ref32 doi: 10.1109/ICRA48891.2023.10160653 – ident: ref19 doi: 10.1109/LRA.2020.2965891 – ident: ref39 doi: 10.1109/TASE.2021.3064065 – start-page: 2021 volume-title: Proc. 5th Annu. Conf. Robot Learn. ident: ref4 article-title: FlingBot: The unreasonable effectiveness of dynamic manipulation for cloth unfolding – ident: ref22 doi: 10.1109/LRA.2022.3187843 – ident: ref10 doi: 10.1109/ICRA40945.2020.9196659 – ident: ref13 doi: 10.1109/IROS.2012.6386109 – ident: ref24 doi: 10.1109/IROS47612.2022.9981101 – ident: ref35 doi: 10.1109/cvpr.2018.00030 – start-page: 734 volume-title: Proc. Conf. Robot Learn. year: 2018 ident: ref25 article-title: Sim-to-real reinforcement learning for deformable object manipulation
SSID	ssj0001527395
Score	2.4124846
Snippet	Realistic physics engines play a crucial role for learning to manipulate deformable objects such as garments in simulation. By doing so, researchers can...
SourceID	proquest crossref ieee
SourceType	Aggregation Database Enrichment Source Index Database Publisher
StartPage	2981
SubjectTerms	Benchmark testing Benchmarks bimanual manipulation Cloth Computing time Data sets for robot learning Datasets deformable object manipulation Deformation Dynamics Engines Fabrics Formability Manipulator dynamics Simulation Simulators Source code Stability analysis Task analysis Trajectory
Title	Benchmarking the Sim-to-Real Gap in Cloth Manipulation
URI	https://ieeexplore.ieee.org/document/10417136 https://www.proquest.com/docview/2927713612
Volume	9
WOSCitedRecordID	wos001165171700005&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D
hasFullText	1
inHoldings	1
isFullTextHit
isPrint
journalDatabaseRights	– providerCode: PRVIEE databaseName: IEEE Electronic Library (IEL) customDbUrl: eissn: 2377-3766 dateEnd: 99991231 omitProxy: false ssIdentifier: ssj0001527395 issn: 2377-3766 databaseCode: RIE dateStart: 20160101 isFulltext: true titleUrlDefault: https://ieeexplore.ieee.org/ providerName: IEEE – providerCode: PRVHPJ databaseName: ROAD: Directory of Open Access Scholarly Resources customDbUrl: eissn: 2377-3766 dateEnd: 99991231 omitProxy: false ssIdentifier: ssj0001527395 issn: 2377-3766 databaseCode: M~E dateStart: 20160101 isFulltext: true titleUrlDefault: https://road.issn.org providerName: ISSN International Centre
link	http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV07T8MwED5RxAADzyIKpcrAwhCaOI4dj1DxGNoKlYfYIscPUalNEW0Z-e1cnBSKEEhslmLH1nc-38O-O4ATnbGAhlr5WUiYT7UyvtQCCZIFnCRayshVa3js8n4_eXoSt1WwuouFMca4x2fmrGi6u3w9UfPCVYYcTkM0qlgNapzzMljry6FSpBIT8eIqMhDt7uAcDUBCz6KIoeSj30SPq6Xy4wB2UuVq65_r2YbNSn30zkt678CKyXdhYymp4B6wC9x4z2PpnOAe6nfe3XDszyb-AHVC71q-eMPc64yQQl5P5sNF_a46PFxd3ndu_Ko6gq-Q72Y-lxEJbWjQHouNEBZXFyVccaoTG0ummc1UrGKmCbOBRD0uEQnVsZUk1iY2QbQPq_kkNwfgEUltRjNE0FBqhMwIMXj4BAJnsiywDWgvgEtVlTq8qGAxSp0JEYgUoU4LqNMK6gacfo54KdNm_NG3XkC71K9EtQHNBXHSirGmKRGEFx9DcvjLsCNYL_5evhNrwursdW6OYU29zYbT1xbUeu-XLbdzPgA3Fb-c
linkProvider	IEEE
linkToHtml	http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV1LSwMxEB60CurBZ8X63IMXD6u72SS7OdaiVqxFtIq3JZsHFtqtaPX3O5vdakUUvAWSkPBNJvNIZgbgUGc8oKFWfhYS7lOtjC-1QIJkQUwSLWXkqjU8dOJuN3l8FDdVsLqLhTHGuM9n5rhourd8PVJvhasMOZyGaFTxWZhjlJKwDNf6cqkUycQEmzxGBuKkc9tEE5DQ4yjiKPvoN-Hjqqn8uIKdXDlf-eeOVmG5UiC9ZknxNZgx-TosTaUV3AB-ikfvaSidG9xDDc-76w_98ci_Ra3Qu5DPXj_3WgOkkXct8_6kglcd7s_Peq22X9VH8BVy3tiPZURCGxq0yJgRwuLuoiRWMdWJZZJrbjPFFOOacBtI1OQSkVDNrCRMG2aCaBNq-Sg3W-ARSW1GM0TQUGqEzAgxeP0EAleyPLANOJkAl6oqeXhRw2KQOiMiEClCnRZQpxXUDTj6nPFcJs74Y2y9gHZqXIlqA3YnxEkr1npNiSBx0RmS7V-mHcBCu3fdSTuX3asdWCxWKn-N7UJt_PJm9mBevY_7ry_77vx8ANH8wbI
openUrl	ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Benchmarking+the+Sim-to-Real+Gap+in+Cloth+Manipulation&rft.jtitle=IEEE+robotics+and+automation+letters&rft.au=Blanco-Mulero%2C+David&rft.au=Barbany%2C+Oriol&rft.au=Alcan%2C+Gokhan&rft.au=Colome%2C+Adria&rft.date=2024-03-01&rft.pub=IEEE&rft.eissn=2377-3766&rft.volume=9&rft.issue=3&rft.spage=2981&rft.epage=2988&rft_id=info:doi/10.1109%2FLRA.2024.3360814&rft.externalDocID=10417136
thumbnail_l	http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/lc.gif&issn=2377-3766&client=summon
thumbnail_m	http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/mc.gif&issn=2377-3766&client=summon
thumbnail_s	http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/sc.gif&issn=2377-3766&client=summon