Tidiness Score-Guided Monte Carlo Tree Search for Visual Tabletop Rearrangement
In this letter, we present the tidiness score-guided Monte Carlo tree search (TSMCTS), a novel framework designed to address the tabletop tidying up problem using only an RGB-D camera. We address two major problems for tabletop tidying up problem: (1) the lack of public datasets and benchmarks, and...
Gespeichert in:
| Veröffentlicht in: | IEEE robotics and automation letters Jg. 10; H. 10; S. 10090 - 10097 |
|---|---|
| Hauptverfasser: | , , , , |
| Format: | Journal Article |
| Sprache: | Englisch |
| Veröffentlicht: |
Piscataway
IEEE
01.10.2025
The Institute of Electrical and Electronics Engineers, Inc. (IEEE) |
| Schlagworte: | |
| ISSN: | 2377-3766, 2377-3766 |
| Online-Zugang: | Volltext |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| Abstract | In this letter, we present the tidiness score-guided Monte Carlo tree search (TSMCTS), a novel framework designed to address the tabletop tidying up problem using only an RGB-D camera. We address two major problems for tabletop tidying up problem: (1) the lack of public datasets and benchmarks, and (2) the difficulty of specifying the goal configuration of unseen objects. We address the former by presenting the tabletop tidying up (TTU) dataset, a structured dataset collected in simulation. Using this dataset, we train a vision-based discriminator capable of predicting the tidiness score. This discriminator can consistently evaluate the degree of tidiness across unseen configurations, including real-world scenes. Addressing the second problem, we employ Monte Carlo tree search (MCTS) to find tidying trajectories without specifying explicit goals. Instead of providing specific goals, we demonstrate that our MCTS-based planner can find diverse tidied configurations using the tidiness score as a guidance. Consequently, we propose TSMCTS, which integrates a tidiness discriminator with an MCTS-based tidying planner to find optimal tidied arrangements. TSMCTS has successfully demonstrated its capability across various environments, including coffee tables, dining tables, office desks, and bathrooms. |
|---|---|
| AbstractList | In this letter, we present the tidiness score-guided Monte Carlo tree search (TSMCTS), a novel framework designed to address the tabletop tidying up problem using only an RGB-D camera. We address two major problems for tabletop tidying up problem: (1) the lack of public datasets and benchmarks, and (2) the difficulty of specifying the goal configuration of unseen objects. We address the former by presenting the tabletop tidying up (TTU) dataset, a structured dataset collected in simulation. Using this dataset, we train a vision-based discriminator capable of predicting the tidiness score. This discriminator can consistently evaluate the degree of tidiness across unseen configurations, including real-world scenes. Addressing the second problem, we employ Monte Carlo tree search (MCTS) to find tidying trajectories without specifying explicit goals. Instead of providing specific goals, we demonstrate that our MCTS-based planner can find diverse tidied configurations using the tidiness score as a guidance. Consequently, we propose TSMCTS, which integrates a tidiness discriminator with an MCTS-based tidying planner to find optimal tidied arrangements. TSMCTS has successfully demonstrated its capability across various environments, including coffee tables, dining tables, office desks, and bathrooms. |
| Author | Kee, Hogun Kang, Minjae Oh, Wooseok Ahn, Hyemin Oh, Songhwai |
| Author_xml | – sequence: 1 givenname: Hogun orcidid: 0009-0008-3320-5658 surname: Kee fullname: Kee, Hogun email: hogun.kee@rllab.snu.ac.kr organization: Department of Electrical and Computer Engineering and ASRI, Seoul National University, Seoul, South Korea – sequence: 2 givenname: Wooseok surname: Oh fullname: Oh, Wooseok email: wooseok.oh@rllab.snu.ac.kr organization: Department of Electrical and Computer Engineering and ASRI, Seoul National University, Seoul, South Korea – sequence: 3 givenname: Minjae orcidid: 0000-0002-4971-3920 surname: Kang fullname: Kang, Minjae email: minjae.kang@rllab.snu.ac.kr organization: Department of Electrical and Computer Engineering and ASRI, Seoul National University, Seoul, South Korea – sequence: 4 givenname: Hyemin orcidid: 0000-0001-8081-6023 surname: Ahn fullname: Ahn, Hyemin email: hyemin.ahn@unist.ac.kr organization: Artificial Intelligence Graduate School (AIGS), Ulsan National Institute of Science and Technology (UNIST), Ulsan, South Korea – sequence: 5 givenname: Songhwai orcidid: 0000-0002-9781-2018 surname: Oh fullname: Oh, Songhwai email: songhwai@snu.ac.kr organization: Department of Electrical and Computer Engineering and ASRI, Seoul National University, Seoul, South Korea |
| BookMark | eNpNkMFLwzAUxoNMcM7dPXgIeO5MkyZpjmPoFCaDrXoNr-2rdnTNTNqD__06NtDT--B933sfv1syal2LhNzHbBbHzDytNvMZZ1zOhDQ65fyKjLnQOhJaqdE_fUOmIewYY7HkWhg5JuusLusWQ6DbwnmMln1dYknfXdshXYBvHM08It0i-OKbVs7Tzzr00NAM8gY7d6CbYeWh_cI9tt0dua6gCTi9zAn5eHnOFq_Rar18W8xXUcET3UUm0TmWKhfGQMFRVQhpCokBhsIIyDGVMgFenDTP0TBVMhiKlXlaQSGlmJDH892Ddz89hs7uXO_b4aUVPFGCa5XqwcXOrsK7EDxW9uDrPfhfGzN7ImcHcvZEzl7IDZGHc6RGxD97HHOupBZHB-xsFg |
| CODEN | IRALC6 |
| Cites_doi | 10.1109/MRA.2015.2448951 10.1109/CVPR52688.2022.01437 10.1109/ICRA46639.2022.9811931 10.15607/rss.2021.xvii.072 10.1109/ICCV51070.2023.00371 10.15607/rss.2023.xix.029 10.1109/ICRA.2019.8793946 10.1109/ICRA.2015.7139396 10.1109/CVPR52733.2024.02123 10.1109/ICRA48506.2021.9561877 10.1016/j.ijmedinf.2015.03.003 10.15607/rss.2023.xix.031 10.1109/IROS58592.2024.10802562 10.1007/11871842_29 |
| ContentType | Journal Article |
| Copyright | Copyright The Institute of Electrical and Electronics Engineers, Inc. (IEEE) 2025 |
| Copyright_xml | – notice: Copyright The Institute of Electrical and Electronics Engineers, Inc. (IEEE) 2025 |
| DBID | 97E RIA RIE AAYXX CITATION 7SC 7SP 8FD JQ2 L7M L~C L~D |
| DOI | 10.1109/LRA.2025.3597822 |
| DatabaseName | IEEE All-Society Periodicals Package (ASPP) 2005–Present IEEE All-Society Periodicals Package (ASPP) 1998–Present IEEE Electronic Library (IEL) CrossRef Computer and Information Systems Abstracts Electronics & Communications Abstracts Technology Research Database ProQuest Computer Science Collection Advanced Technologies Database with Aerospace Computer and Information Systems Abstracts Academic Computer and Information Systems Abstracts Professional |
| DatabaseTitle | CrossRef Technology Research Database Computer and Information Systems Abstracts – Academic Electronics & Communications Abstracts ProQuest Computer Science Collection Computer and Information Systems Abstracts Advanced Technologies Database with Aerospace Computer and Information Systems Abstracts Professional |
| DatabaseTitleList | Technology Research Database |
| Database_xml | – sequence: 1 dbid: RIE name: IEEE Electronic Library (IEL) url: https://ieeexplore.ieee.org/ sourceTypes: Publisher |
| DeliveryMethod | fulltext_linktorsrc |
| Discipline | Engineering Government |
| EISSN | 2377-3766 |
| EndPage | 10097 |
| ExternalDocumentID | 10_1109_LRA_2025_3597822 11122657 |
| Genre | orig-research |
| GrantInformation_xml | – fundername: Development of Complex Task Planning Technologies for Autonomous Agents – fundername: Korea Government grantid: RS-2019-II191190 – fundername: Robot Learning: Efficient, Safe, and Socially-Acceptable Machine Learning – fundername: Institute of Information & Communications Technology Planning & Evaluation – fundername: Innovative Human Resource Development for Local Intellectualization program – fundername: Korea Government grantid: RS-2024-00336738 – fundername: Korea Government grantid: IITP-2025-RS-2022-00156361 |
| GroupedDBID | 0R~ 97E AAJGR AARMG AASAJ AAWTH ABAZT ABQJQ ABVLG ACGFS AGQYO AGSQL AHBIQ AKJIK AKQYR ALMA_UNASSIGNED_HOLDINGS ATWAV BEFXN BFFAM BGNUA BKEBE BPEOZ EBS EJD IFIPE IPLJI JAVBF KQ8 M43 M~E O9- OCL RIA RIE AAYXX CITATION 7SC 7SP 8FD JQ2 L7M L~C L~D |
| ID | FETCH-LOGICAL-c247t-947bed6b399ac2e6fea88a49a0e393abe8554a2c93ab2be906d0adeddb8fac553 |
| IEDL.DBID | RIE |
| ISICitedReferencesCount | 0 |
| ISICitedReferencesURI | http://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=Summon&SrcAuth=ProQuest&DestLinkType=CitingArticles&DestApp=WOS_CPL&KeyUT=001563969800018&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| ISSN | 2377-3766 |
| IngestDate | Sat Nov 22 13:40:56 EST 2025 Sat Nov 29 07:38:05 EST 2025 Wed Aug 27 07:37:17 EDT 2025 |
| IsPeerReviewed | true |
| IsScholarly | true |
| Issue | 10 |
| Language | English |
| License | https://ieeexplore.ieee.org/Xplorehelp/downloads/license-information/IEEE.html https://doi.org/10.15223/policy-029 https://doi.org/10.15223/policy-037 |
| LinkModel | DirectLink |
| MergedId | FETCHMERGED-LOGICAL-c247t-947bed6b399ac2e6fea88a49a0e393abe8554a2c93ab2be906d0adeddb8fac553 |
| Notes | ObjectType-Article-1 SourceType-Scholarly Journals-1 ObjectType-Feature-2 content type line 14 |
| ORCID | 0000-0002-9781-2018 0009-0008-3320-5658 0000-0002-4971-3920 0000-0001-8081-6023 |
| PQID | 3246327687 |
| PQPubID | 4437225 |
| PageCount | 8 |
| ParticipantIDs | ieee_primary_11122657 crossref_primary_10_1109_LRA_2025_3597822 proquest_journals_3246327687 |
| PublicationCentury | 2000 |
| PublicationDate | 2025-10-01 |
| PublicationDateYYYYMMDD | 2025-10-01 |
| PublicationDate_xml | – month: 10 year: 2025 text: 2025-10-01 day: 01 |
| PublicationDecade | 2020 |
| PublicationPlace | Piscataway |
| PublicationPlace_xml | – name: Piscataway |
| PublicationTitle | IEEE robotics and automation letters |
| PublicationTitleAbbrev | LRA |
| PublicationYear | 2025 |
| Publisher | IEEE The Institute of Electrical and Electronics Engineers, Inc. (IEEE) |
| Publisher_xml | – name: IEEE – name: The Institute of Electrical and Electronics Engineers, Inc. (IEEE) |
| References | Shridhar (ref9) 2022 ref14 ref20 ref11 Wu (ref16) 2022 ref10 ref21 ref2 ref1 ref17 Kapelyukh (ref15) 2023 ref19 ref18 Kapelyukh (ref13) 2022 Kostrikov (ref12) 2022 ref4 ref3 ref6 ref5 Radford (ref7) 2021 Li (ref8) 2022 |
| References_xml | – ident: ref17 doi: 10.1109/MRA.2015.2448951 – ident: ref3 doi: 10.1109/CVPR52688.2022.01437 – ident: ref4 doi: 10.1109/ICRA46639.2022.9811931 – start-page: 894 volume-title: Proc. Conf. Robot Learn. year: 2022 ident: ref9 article-title: Cliport: What and where pathways for robotic manipulation – volume-title: Int. Conf. Learn. year: 2022 ident: ref12 article-title: Offline reinforcement learning with implicit Q-learning – start-page: 31986 volume-title: Proc. Adv. Neural Inf. Process. Syst. year: 2022 ident: ref16 article-title: TARGF: Learning target gradient field to rearrange objects without explicit goal specification – ident: ref2 doi: 10.15607/rss.2021.xvii.072 – ident: ref19 doi: 10.1109/ICCV51070.2023.00371 – year: 2023 ident: ref15 article-title: Workshop on Learning Effective Abstractions for Planning (LEAP) publication-title: Conf. Robot. Learn. – start-page: 740 volume-title: Proc. Conf. Robot Learn. year: 2022 ident: ref13 article-title: My house, my rules: Learning tidying preferences with graph neural networks – ident: ref10 doi: 10.15607/rss.2023.xix.029 – ident: ref1 doi: 10.1109/ICRA.2019.8793946 – ident: ref14 doi: 10.1109/ICRA.2015.7139396 – ident: ref18 doi: 10.1109/CVPR52733.2024.02123 – ident: ref20 doi: 10.1109/ICRA48506.2021.9561877 – start-page: 8748 volume-title: Proc. Int. Conf. Mach. Learn. year: 2021 ident: ref7 article-title: Learning transferable visual models from natural language supervision – start-page: 12888 volume-title: Proc. Int. Conf. Mach. Learn. year: 2022 ident: ref8 article-title: Blip: Bootstrapping language-image pre-training for unified vision-language understanding and generation – ident: ref21 doi: 10.1016/j.ijmedinf.2015.03.003 – ident: ref5 doi: 10.15607/rss.2023.xix.031 – ident: ref6 doi: 10.1109/IROS58592.2024.10802562 – ident: ref11 doi: 10.1007/11871842_29 |
| SSID | ssj0001527395 |
| Score | 2.3118227 |
| Snippet | In this letter, we present the tidiness score-guided Monte Carlo tree search (TSMCTS), a novel framework designed to address the tabletop tidying up problem... |
| SourceID | proquest crossref ieee |
| SourceType | Aggregation Database Index Database Publisher |
| StartPage | 10090 |
| SubjectTerms | Bathrooms Configurations data sets for robot learning Datasets deep learning methods Discriminators Feature extraction Government Manipulation planning Monte Carlo methods Monte Carlo simulation Planning Robot learning Search problems Searching Semantics Training Trajectory Visualization |
| Title | Tidiness Score-Guided Monte Carlo Tree Search for Visual Tabletop Rearrangement |
| URI | https://ieeexplore.ieee.org/document/11122657 https://www.proquest.com/docview/3246327687 |
| Volume | 10 |
| WOSCitedRecordID | wos001563969800018&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| hasFullText | 1 |
| inHoldings | 1 |
| isFullTextHit | |
| isPrint | |
| journalDatabaseRights | – providerCode: PRVIEE databaseName: IEEE Electronic Library (IEL) customDbUrl: eissn: 2377-3766 dateEnd: 99991231 omitProxy: false ssIdentifier: ssj0001527395 issn: 2377-3766 databaseCode: RIE dateStart: 20160101 isFulltext: true titleUrlDefault: https://ieeexplore.ieee.org/ providerName: IEEE – providerCode: PRVHPJ databaseName: ROAD: Directory of Open Access Scholarly Resources customDbUrl: eissn: 2377-3766 dateEnd: 99991231 omitProxy: false ssIdentifier: ssj0001527395 issn: 2377-3766 databaseCode: M~E dateStart: 20160101 isFulltext: true titleUrlDefault: https://road.issn.org providerName: ISSN International Centre |
| link | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV07T8MwED7RiqEMPAqIQqk8sDCkDXnZHquqhaEtqJSqW2Q7FykSaqo-GPnt2E6qghADm4fYse7i-Dv7u-8A7kJUMmSKOej6SgcozHWEYMIJBMrggQqKVrdgNqTjMZvP-UuZrG5zYRDRks-wbZr2Lj_J1dYclXX0utRoIaQVqFBKi2St_YGKkRLj4e4q0uWd4aSrA0AvbPsaNTPP-7H12Foqv37AdlcZnPxzPqdwXMJH0i38fQYHuKjD0TdRwTrU9uVzz-F5miWW2E5ejV6l87jNEkzIyGhSkZ5YvedkukIkBeuYaARLZtl6q18xNTlVm3xJJobNa1IQzIgX8DboT3tPTllDwVFeQDcOD6jEJJIahwjlYZSiYEwEXLjoc19INDQ14SnT9iRyN0pcoSeSSJYKFYb-JVQX-QKvgHDJdV9XBkYzj_kpi5IAkfqSBSwNBW3A_c688bKQyohtiOHyWLsiNq6IS1c04MKYc_9cackGNHcOicvFtI415ot8T8dF9PqPbjdQM6MXJLsmVDerLd7CofrYZOtVCyqjz37Lfi1fCSy-xQ |
| linkProvider | IEEE |
| linkToHtml | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV07T8MwED7xkigDzyLK0wMLQyB17NgeEQKKaAuCgNgi27lKlVCL-uD3YzupCkIMbB7i2LqL4-_s774DOOVoDZdWRhgn1gUoMo60ljpiGg1rCi0w6Ba8tkW3K9_e1GOVrB5yYRAxkM_w3DfDXX4xtFN_VHbh1qVDC1wswjJnjDbLdK35kYoXE1N8dhkZq4v206ULASk_TxxulpT-2HxCNZVfv-Cwr9xs_HNGm7BeAUhyWXp8CxZwsA1r32QFt6E2L6C7Aw9ZvwjUdvLsFSuj22m_wIJ0vCoVudKj9yHJRoik5B0Th2HJa388dUNkPqtqMvwgT57P65MQ_Bvr8HJznV21oqqKQmQpE5NIMWGwSI1DItpSTHuopdRM6RgTlWiDnqimqfVtalDFaRFrN5HCyJ62nCe7sDQYDnAPiDLK9Y0N86p5MunJtGCIIjGSyR7XogFnM_PmH6VYRh6CjFjlzhW5d0VeuaIBdW_O-XOVJRtwOHNIXi2nce5QX5pQFxmJ_T-6ncBqK-u08_Zd9_4Aan6kknJ3CEuT0RSPYMV-Tvrj0XH4Zr4AmsnA2w |
| openUrl | ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Tidiness+Score-Guided+Monte+Carlo+Tree+Search+for+Visual+Tabletop+Rearrangement&rft.jtitle=IEEE+robotics+and+automation+letters&rft.au=Kee%2C+Hogun&rft.au=Oh%2C+Wooseok&rft.au=Kang%2C+Minjae&rft.au=Ahn%2C+Hyemin&rft.date=2025-10-01&rft.issn=2377-3766&rft.eissn=2377-3766&rft.volume=10&rft.issue=10&rft.spage=10090&rft.epage=10097&rft_id=info:doi/10.1109%2FLRA.2025.3597822&rft.externalDBID=n%2Fa&rft.externalDocID=10_1109_LRA_2025_3597822 |
| thumbnail_l | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/lc.gif&issn=2377-3766&client=summon |
| thumbnail_m | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/mc.gif&issn=2377-3766&client=summon |
| thumbnail_s | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/sc.gif&issn=2377-3766&client=summon |