Arena-Bench: A Benchmarking Suite for Obstacle Avoidance Approaches in Highly Dynamic Environments
The ability to autonomously navigate safely, especially within dynamic environments, is paramount for mobile robotics. In recent years, DRL approaches have shown superior performance in dynamic obstacle avoidance. However, these learning-based approaches are often developed in specially designed sim...
Uloženo v:
| Vydáno v: | IEEE robotics and automation letters Ročník 7; číslo 4; s. 9477 - 9484 |
|---|---|
| Hlavní autoři: | , , , , , , , , , , , |
| Médium: | Journal Article |
| Jazyk: | angličtina |
| Vydáno: |
Piscataway
IEEE
01.10.2022
The Institute of Electrical and Electronics Engineers, Inc. (IEEE) |
| Témata: | |
| ISSN: | 2377-3766, 2377-3766 |
| On-line přístup: | Získat plný text |
| Tagy: |
Přidat tag
Žádné tagy, Buďte první, kdo vytvoří štítek k tomuto záznamu!
|
| Abstract | The ability to autonomously navigate safely, especially within dynamic environments, is paramount for mobile robotics. In recent years, DRL approaches have shown superior performance in dynamic obstacle avoidance. However, these learning-based approaches are often developed in specially designed simulation environments and are hard to test against conventional planning approaches. Furthermore, the integration and deployment of these approaches into real robotic platforms are not yet completely solved. In this letter, we present Arena-bench, a benchmark suite to train, test, and evaluate navigation planners on different robotic platforms within 3D environments. It provides tools to design and generate highly dynamic evaluation worlds, scenarios, and tasks for autonomous navigation and is fully integrated into the robot operating system. To demonstrate the functionalities of our suite, we trained a DRL agent on our platform and compared it against a variety of existing different model-based and learning-based navigation approaches on a variety of relevant metrics. Finally, we deployed the approaches towards real robots and demonstrated the reproducibility of the results. The code is publicly available at github.com/ignc-research/arena-bench |
|---|---|
| AbstractList | The ability to autonomously navigate safely, especially within dynamic environments, is paramount for mobile robotics. In recent years, DRL approaches have shown superior performance in dynamic obstacle avoidance. However, these learning-based approaches are often developed in specially designed simulation environments and are hard to test against conventional planning approaches. Furthermore, the integration and deployment of these approaches into real robotic platforms are not yet completely solved. In this letter, we present Arena-bench, a benchmark suite to train, test, and evaluate navigation planners on different robotic platforms within 3D environments. It provides tools to design and generate highly dynamic evaluation worlds, scenarios, and tasks for autonomous navigation and is fully integrated into the robot operating system. To demonstrate the functionalities of our suite, we trained a DRL agent on our platform and compared it against a variety of existing different model-based and learning-based navigation approaches on a variety of relevant metrics. Finally, we deployed the approaches towards real robots and demonstrated the reproducibility of the results. The code is publicly available at github.com/ignc-research/arena-bench |
| Author | Khorsandi, Niloufar Le, Tuan Anh Cox, Johannes Carstens, Reyk Kastner, Linh Kmiecik, Jacek Meinardus, Boris Bhuiyan, Teham Pichel, Duc Fatloun, Bassel Lambrecht, Jens Treis, Elias |
| Author_xml | – sequence: 1 givenname: Linh orcidid: 0000-0001-5263-4687 surname: Kastner fullname: Kastner, Linh email: linhdoan@win.tu-berlin.de organization: Chair Industry Grade Networks and Clouds, Faculty of Electrical Engineering, and Computer Science, Berlin Institute of Technology, Berlin, Germany – sequence: 2 givenname: Teham surname: Bhuiyan fullname: Bhuiyan, Teham email: t.buiyyan@gmail.com organization: Chair Industry Grade Networks and Clouds, Faculty of Electrical Engineering, and Computer Science, Berlin Institute of Technology, Berlin, Germany – sequence: 3 givenname: Tuan Anh surname: Le fullname: Le, Tuan Anh email: anh.maxiking@hotmail.de organization: Chair Industry Grade Networks and Clouds, Faculty of Electrical Engineering, and Computer Science, Berlin Institute of Technology, Berlin, Germany – sequence: 4 givenname: Elias surname: Treis fullname: Treis, Elias email: e.treis@tu-berlin.de organization: Chair Industry Grade Networks and Clouds, Faculty of Electrical Engineering, and Computer Science, Berlin Institute of Technology, Berlin, Germany – sequence: 5 givenname: Johannes surname: Cox fullname: Cox, Johannes email: j.cox@campus.tu-berlin.de organization: Chair Industry Grade Networks and Clouds, Faculty of Electrical Engineering, and Computer Science, Berlin Institute of Technology, Berlin, Germany – sequence: 6 givenname: Boris orcidid: 0000-0002-4327-3055 surname: Meinardus fullname: Meinardus, Boris email: b.meinardus@campus.tu-berlin.de organization: Chair Industry Grade Networks and Clouds, Faculty of Electrical Engineering, and Computer Science, Berlin Institute of Technology, Berlin, Germany – sequence: 7 givenname: Jacek surname: Kmiecik fullname: Kmiecik, Jacek email: j.kmiecik@tu-berlin.de organization: Chair Industry Grade Networks and Clouds, Faculty of Electrical Engineering, and Computer Science, Berlin Institute of Technology, Berlin, Germany – sequence: 8 givenname: Reyk surname: Carstens fullname: Carstens, Reyk email: reyk.carstens@tu-berlin.de organization: Chair Industry Grade Networks and Clouds, Faculty of Electrical Engineering, and Computer Science, Berlin Institute of Technology, Berlin, Germany – sequence: 9 givenname: Duc surname: Pichel fullname: Pichel, Duc email: duc.pichel@tu-berlin.de organization: Chair Industry Grade Networks and Clouds, Faculty of Electrical Engineering, and Computer Science, Berlin Institute of Technology, Berlin, Germany – sequence: 10 givenname: Bassel surname: Fatloun fullname: Fatloun, Bassel email: ba.f.prosoft@gmail.com organization: Chair Industry Grade Networks and Clouds, Faculty of Electrical Engineering, and Computer Science, Berlin Institute of Technology, Berlin, Germany – sequence: 11 givenname: Niloufar surname: Khorsandi fullname: Khorsandi, Niloufar email: niloufar.k@tu-berlin.de organization: Chair Industry Grade Networks and Clouds, Faculty of Electrical Engineering, and Computer Science, Berlin Institute of Technology, Berlin, Germany – sequence: 12 givenname: Jens orcidid: 0000-0002-1017-9548 surname: Lambrecht fullname: Lambrecht, Jens email: lambrecht@tu-berlin.de organization: Chair Industry Grade Networks and Clouds, Faculty of Electrical Engineering, and Computer Science, Berlin Institute of Technology, Berlin, Germany |
| BookMark | eNp9kEtPAjEUhRuDiYjsTdw0cT3Y10xbd6OimJCQ-FhPOp0OFIcW24GEf-8gxBgXru5ZnO-ee8856DnvDACXGI0wRvJm-pKPCCJkRLFESGQnoE8o5wnlWdb7pc_AMMYlQginhFOZ9kGZB-NUcmecXtzCHH6LlQof1s3h68a2BtY-wFkZW6UbA_Ott5VyulPrdfBKL0yE1sGJnS-aHXzYObWyGo7d1gbvVsa18QKc1qqJZnicA_D-OH67nyTT2dPzfT5NNJG4TZTmuiqpZEbUtCK1YrzksiRSaMFKShThQpQkxZLSmvHKVFqyOq2EoKyiKKMDcH3Y2931uTGxLZZ-E1wXWZBMppixjLPOhQ4uHXyMwdTFOtju4V2BUbEvs-jKLPZlFscyOyT7g2jbqtZ61wZlm__AqwNojTE_OVIQnqacfgF2GIKe |
| CODEN | IRALC6 |
| CitedBy_id | crossref_primary_10_1007_s10462_025_11129_6 crossref_primary_10_1145_3700599 crossref_primary_10_3390_app122312377 crossref_primary_10_3390_s23136073 crossref_primary_10_1002_rob_22506 crossref_primary_10_1109_ACCESS_2024_3404881 crossref_primary_10_1109_LRA_2023_3316072 crossref_primary_10_1109_LRA_2024_3355641 crossref_primary_10_3390_s24092794 crossref_primary_10_1109_ACCESS_2023_3299178 crossref_primary_10_1109_LRA_2025_3597898 crossref_primary_10_1016_j_robot_2025_105020 crossref_primary_10_3390_technologies12060082 crossref_primary_10_3390_app13158925 |
| Cites_doi | 10.1109/LRA.2019.2899918 10.1109/LRA.2021.3068913 10.1109/ICRA.2016.7487514 10.1109/ICRA.2018.8461096 10.1145/3406499.3418760 10.1109/IVS.2017.7995802 10.1109/IROS47612.2022.9982021 10.1109/IROS.2018.8593871 10.1109/ICRA48506.2021.9560951 10.1109/ICRA48506.2021.9561901 10.1109/CCTA.2017.8062434 10.1109/IROS.2017.8202312 10.1109/TSSC.1968.300136 10.1109/100.580977 10.1109/ICRA46639.2022.9811797 10.1109/IROS51168.2021.9636226 10.1109/MRA.2015.2448276 10.1109/TITS.2021.3134702 10.1109/IROS51168.2021.9636039 10.1007/978-3-319-91590-6_1 10.1109/TITS.2021.3054625 10.1007/978-3-030-58526-6_42 10.1103/PhysRevE.51.4282 10.1109/ACCESS.2020.2975643 10.1109/IROS45743.2020.9341783 10.1109/ECC.2015.7331052 |
| ContentType | Journal Article |
| Copyright | Copyright The Institute of Electrical and Electronics Engineers, Inc. (IEEE) 2022 |
| Copyright_xml | – notice: Copyright The Institute of Electrical and Electronics Engineers, Inc. (IEEE) 2022 |
| DBID | 97E RIA RIE AAYXX CITATION 7SC 7SP 8FD JQ2 L7M L~C L~D |
| DOI | 10.1109/LRA.2022.3190086 |
| DatabaseName | IEEE All-Society Periodicals Package (ASPP) 2005–Present IEEE All-Society Periodicals Package (ASPP) 1998–Present IEEE Electronic Library (IEL) CrossRef Computer and Information Systems Abstracts Electronics & Communications Abstracts Technology Research Database ProQuest Computer Science Collection Advanced Technologies Database with Aerospace Computer and Information Systems Abstracts Academic Computer and Information Systems Abstracts Professional |
| DatabaseTitle | CrossRef Technology Research Database Computer and Information Systems Abstracts – Academic Electronics & Communications Abstracts ProQuest Computer Science Collection Computer and Information Systems Abstracts Advanced Technologies Database with Aerospace Computer and Information Systems Abstracts Professional |
| DatabaseTitleList | Technology Research Database |
| Database_xml | – sequence: 1 dbid: RIE name: IEEE Electronic Library (IEL) url: https://ieeexplore.ieee.org/ sourceTypes: Publisher |
| DeliveryMethod | fulltext_linktorsrc |
| Discipline | Engineering |
| EISSN | 2377-3766 |
| EndPage | 9484 |
| ExternalDocumentID | 10_1109_LRA_2022_3190086 9827557 |
| Genre | orig-research |
| GroupedDBID | 0R~ 97E AAJGR AARMG AASAJ AAWTH ABAZT ABQJQ ABVLG ACGFS AGQYO AGSQL AHBIQ AKJIK AKQYR ALMA_UNASSIGNED_HOLDINGS ATWAV BEFXN BFFAM BGNUA BKEBE BPEOZ EBS EJD IFIPE IPLJI JAVBF KQ8 M43 M~E O9- OCL RIA RIE AAYXX CITATION 7SC 7SP 8FD JQ2 L7M L~C L~D |
| ID | FETCH-LOGICAL-c291t-ac7cdb394e8f3d2fa47b79b298c84b32a2788b251933f47dedc94f5d8834d3063 |
| IEDL.DBID | RIE |
| ISICitedReferencesCount | 26 |
| ISICitedReferencesURI | http://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=Summon&SrcAuth=ProQuest&DestLinkType=CitingArticles&DestApp=WOS_CPL&KeyUT=000831182500024&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| ISSN | 2377-3766 |
| IngestDate | Mon Jun 30 04:50:24 EDT 2025 Tue Nov 18 21:39:27 EST 2025 Sat Nov 29 06:03:19 EST 2025 Wed Aug 27 02:25:28 EDT 2025 |
| IsPeerReviewed | true |
| IsScholarly | true |
| Issue | 4 |
| Language | English |
| License | https://ieeexplore.ieee.org/Xplorehelp/downloads/license-information/IEEE.html https://doi.org/10.15223/policy-029 https://doi.org/10.15223/policy-037 |
| LinkModel | DirectLink |
| MergedId | FETCHMERGED-LOGICAL-c291t-ac7cdb394e8f3d2fa47b79b298c84b32a2788b251933f47dedc94f5d8834d3063 |
| Notes | ObjectType-Article-1 SourceType-Scholarly Journals-1 ObjectType-Feature-2 content type line 14 |
| ORCID | 0000-0001-5263-4687 0000-0002-4327-3055 0000-0002-1017-9548 |
| PQID | 2695144674 |
| PQPubID | 4437225 |
| PageCount | 8 |
| ParticipantIDs | ieee_primary_9827557 crossref_citationtrail_10_1109_LRA_2022_3190086 proquest_journals_2695144674 crossref_primary_10_1109_LRA_2022_3190086 |
| PublicationCentury | 2000 |
| PublicationDate | 2022-10-01 |
| PublicationDateYYYYMMDD | 2022-10-01 |
| PublicationDate_xml | – month: 10 year: 2022 text: 2022-10-01 day: 01 |
| PublicationDecade | 2020 |
| PublicationPlace | Piscataway |
| PublicationPlace_xml | – name: Piscataway |
| PublicationTitle | IEEE robotics and automation letters |
| PublicationTitleAbbrev | LRA |
| PublicationYear | 2022 |
| Publisher | IEEE The Institute of Electrical and Electronics Engineers, Inc. (IEEE) |
| Publisher_xml | – name: IEEE – name: The Institute of Electrical and Electronics Engineers, Inc. (IEEE) |
| References | ref13 ref12 ref15 ref14 ref30 ref11 ref2 ref1 ref17 ref16 ref19 ref18 ref24 ref25 ref20 ref22 Gerkey (ref23) 2003 ref28 ref27 ref29 ref8 ref7 ref9 ref4 ref3 ref6 ref5 Ra (ref10) 2006; 88 Brockman (ref21) 2016 |
| References_xml | – ident: ref2 doi: 10.1109/LRA.2019.2899918 – ident: ref17 doi: 10.1109/LRA.2021.3068913 – ident: ref9 doi: 10.1109/ICRA.2016.7487514 – ident: ref4 doi: 10.1109/ICRA.2018.8461096 – ident: ref24 doi: 10.1145/3406499.3418760 – ident: ref18 doi: 10.1109/IVS.2017.7995802 – ident: ref20 doi: 10.1109/IROS47612.2022.9982021 – ident: ref16 doi: 10.1109/IROS.2018.8593871 – ident: ref5 doi: 10.1109/ICRA48506.2021.9560951 – ident: ref8 doi: 10.1109/ICRA48506.2021.9561901 – year: 2016 ident: ref21 article-title: Openai Gym – ident: ref15 doi: 10.1109/CCTA.2017.8062434 – ident: ref3 doi: 10.1109/IROS.2017.8202312 – ident: ref27 doi: 10.1109/TSSC.1968.300136 – ident: ref14 doi: 10.1109/100.580977 – ident: ref30 doi: 10.1109/ICRA46639.2022.9811797 – ident: ref12 doi: 10.1109/IROS51168.2021.9636226 – ident: ref19 doi: 10.1109/MRA.2015.2448276 – ident: ref7 doi: 10.1109/TITS.2021.3134702 – ident: ref28 doi: 10.1109/IROS51168.2021.9636039 – start-page: 317 volume-title: Proc. 11th Int. Conf. Adv. Robot. year: 2003 ident: ref23 article-title: The player/stage project: Tools for multi-robot and distributed sensor systems – ident: ref22 doi: 10.1007/978-3-319-91590-6_1 – ident: ref6 doi: 10.1109/TITS.2021.3054625 – ident: ref25 doi: 10.1007/978-3-030-58526-6_42 – ident: ref11 doi: 10.1103/PhysRevE.51.4282 – ident: ref1 doi: 10.1109/ACCESS.2020.2975643 – volume: 88 start-page: 90 volume-title: Proc. Workshop Benchmarking Robot., IEEE/RSJ Int. Conf. Intell. Robots Syst. year: 2006 ident: ref10 article-title: Steps towards the automatic evaluation of robot obstacle avoidance algorithms – ident: ref29 doi: 10.1109/IROS45743.2020.9341783 – ident: ref13 doi: 10.1109/ECC.2015.7331052 |
| SSID | ssj0001527395 |
| Score | 2.4218528 |
| Snippet | The ability to autonomously navigate safely, especially within dynamic environments, is paramount for mobile robotics. In recent years, DRL approaches have... |
| SourceID | proquest crossref ieee |
| SourceType | Aggregation Database Enrichment Source Index Database Publisher |
| StartPage | 9477 |
| SubjectTerms | Autonomous navigation Benchmark testing Collision avoidance Learning Measurement motion and path planning Navigation Obstacle avoidance Platforms reinforcement learning Reproducibility Robotics Robots Software tools for benchmarking and reproducibility Task analysis Three-dimensional displays |
| Title | Arena-Bench: A Benchmarking Suite for Obstacle Avoidance Approaches in Highly Dynamic Environments |
| URI | https://ieeexplore.ieee.org/document/9827557 https://www.proquest.com/docview/2695144674 |
| Volume | 7 |
| WOSCitedRecordID | wos000831182500024&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| hasFullText | 1 |
| inHoldings | 1 |
| isFullTextHit | |
| isPrint | |
| journalDatabaseRights | – providerCode: PRVIEE databaseName: IEEE Electronic Library (IEL) customDbUrl: eissn: 2377-3766 dateEnd: 99991231 omitProxy: false ssIdentifier: ssj0001527395 issn: 2377-3766 databaseCode: RIE dateStart: 20160101 isFulltext: true titleUrlDefault: https://ieeexplore.ieee.org/ providerName: IEEE – providerCode: PRVHPJ databaseName: ROAD: Directory of Open Access Scholarly Resources customDbUrl: eissn: 2377-3766 dateEnd: 99991231 omitProxy: false ssIdentifier: ssj0001527395 issn: 2377-3766 databaseCode: M~E dateStart: 20160101 isFulltext: true titleUrlDefault: https://road.issn.org providerName: ISSN International Centre |
| link | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV07T8MwED61FQMMvAqiUCoPLEiEFseuY7YArRhKQbzULbIdR1QqKepLYuG3YztpKQIhsXmwrSifk_vufPcdwBFXluVT6TWpsCU5knhSY-5ZhR96JhLdcAp8zx3W7Qa9Hr8rwMmiFkZr7ZLP9Kkdurv8eKimNlRW5wFmlLIiFBlrZrVaX_EUqyTG6fwmssHrnfvQ-H8YG7eUW-b-zfK4Vio__r_OqLQ3_vc4m7Cek0cUZmhvQUGn27C2JClYBhmOdCq8C3P4Xs5RiNzgVbiIOHqYGoKJDE1Ft9KwQrMHCmfDfmyRR2GuLq7HqJ8im_8xeEdXWcN61Fqqh9uBp3br8fLay_soeArzs4knFFOx9DnRQeLHOBGEScYl5oEKiPSxwMYPlraE1fcTwmIdK04SGgeBT2LjUvi7UEqHqd4DxHlizFniBwQrQq1-odKYCeOUUMYaglagPn_HkcpFxm2vi0HknI0GjwwqkUUlylGpwPFixVsmsPHH3LJFYTEvB6AC1TmMUf4FjiPcNNyR2F4q-7-vOoBVu3eWmFeF0mQ01YewomaT_nhUg-LNR6vmjtgnr5jNUA |
| linkProvider | IEEE |
| linkToHtml | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV3JTsMwEB1BQQIObAVRVh-4IBGaOjaOuYWlAlEKgoK4RbbjiEqQom4Sf4_tpKUIhMTNBzuJ8pzMm_HMG4B9rizLp9I7psKW5EjiSY25ZxV-aE2k2ncKfE8N1myGz8_8bgoOx7UwWmuXfKaP7NCd5ScdNbChsioPMaOUTcMMJQT7ebXWV0TFaolxOjqL9Hm1cR8ZDxBj45hyy92_2R7XTOXHH9iZlfrS_x5oGRYL-oiiHO8VmNLZKixMiAqWQUZdnQnv1Gy_lxMUITd4Ey4mjh4GhmIiQ1TRrTS80FwDRcNOO7HYo6jQF9c91M6QzQB5_UDnect6dDFREbcGj_WL1tmlV3RS8BTmtb4nFFOJDDjRYRokOBWEScYl5qEKiQywwMYTlraINQhSwhKdKE5SmoRhQBLjVATrUMo6md4AxHlqDFoahAQrQq2CodKYCeOWUMZ8QStQHb3jWBUy47bbxWvs3A2fxwaV2KISF6hU4GC84j2X2PhjbtmiMJ5XAFCB7RGMcfEN9mJ8bNgjsd1UNn9ftQdzl62bRty4al5vwby9T56mtw2lfnegd2BWDfvtXnfXbbRPqa_PZg |
| openUrl | ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Arena-Bench%3A+A+Benchmarking+Suite+for+Obstacle+Avoidance+Approaches+in+Highly+Dynamic+Environments&rft.jtitle=IEEE+robotics+and+automation+letters&rft.au=Kastner%2C+Linh&rft.au=Bhuiyan%2C+Teham&rft.au=Le%2C+Tuan+Anh&rft.au=Treis%2C+Elias&rft.date=2022-10-01&rft.pub=The+Institute+of+Electrical+and+Electronics+Engineers%2C+Inc.+%28IEEE%29&rft.eissn=2377-3766&rft.volume=7&rft.issue=4&rft.spage=9477&rft_id=info:doi/10.1109%2FLRA.2022.3190086&rft.externalDBID=NO_FULL_TEXT |
| thumbnail_l | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/lc.gif&issn=2377-3766&client=summon |
| thumbnail_m | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/mc.gif&issn=2377-3766&client=summon |
| thumbnail_s | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/sc.gif&issn=2377-3766&client=summon |