Deep Reinforcement Learning With Dynamic Graphs for Adaptive Informative Path Planning
Autonomousrobots are often employed for data collection due to their efficiency and low labour costs. A key task in robotic data acquisition is planning paths through an initially unknown environment to collect observations given platform-specific resource constraints, such as limited battery life....
Uloženo v:
| Vydáno v: | IEEE robotics and automation letters Ročník 9; číslo 9; s. 7747 - 7754 |
|---|---|
| Hlavní autoři: | , , , , |
| Médium: | Journal Article |
| Jazyk: | angličtina |
| Vydáno: |
IEEE
01.09.2024
|
| Témata: | |
| ISSN: | 2377-3766, 2377-3766 |
| On-line přístup: | Získat plný text |
| Tagy: |
Přidat tag
Žádné tagy, Buďte první, kdo vytvoří štítek k tomuto záznamu!
|
| Abstract | Autonomousrobots are often employed for data collection due to their efficiency and low labour costs. A key task in robotic data acquisition is planning paths through an initially unknown environment to collect observations given platform-specific resource constraints, such as limited battery life. Adaptive online path planning in 3D environments is challenging due to the large set of valid actions and the presence of unknown occlusions. To address these issues, we propose a novel deep reinforcement learning approach for adaptively replanning robot paths to map targets of interest in unknown 3D environments. A key aspect of our approach is a dynamically constructed graph that restricts planning actions local to the robot, allowing us to react to newly discovered static obstacles and targets of interest. For replanning, we propose a new reward function that balances between exploring the unknown environment and exploiting online-discovered targets of interest. Our experiments show that our method enables more efficient target discovery compared to state-of-the-art learning and non-learning baselines. We also showcase our approach for orchard monitoring using an unmanned aerial vehicle in a photorealistic simulator. |
|---|---|
| AbstractList | Autonomousrobots are often employed for data collection due to their efficiency and low labour costs. A key task in robotic data acquisition is planning paths through an initially unknown environment to collect observations given platform-specific resource constraints, such as limited battery life. Adaptive online path planning in 3D environments is challenging due to the large set of valid actions and the presence of unknown occlusions. To address these issues, we propose a novel deep reinforcement learning approach for adaptively replanning robot paths to map targets of interest in unknown 3D environments. A key aspect of our approach is a dynamically constructed graph that restricts planning actions local to the robot, allowing us to react to newly discovered static obstacles and targets of interest. For replanning, we propose a new reward function that balances between exploring the unknown environment and exploiting online-discovered targets of interest. Our experiments show that our method enables more efficient target discovery compared to state-of-the-art learning and non-learning baselines. We also showcase our approach for orchard monitoring using an unmanned aerial vehicle in a photorealistic simulator. |
| Author | Ruckin, Julius Magistri, Federico Popovic, Marija Vashisth, Apoorva Stachniss, Cyrill |
| Author_xml | – sequence: 1 givenname: Apoorva orcidid: 0009-0005-3741-9053 surname: Vashisth fullname: Vashisth, Apoorva organization: Department of Mechanical Engineering, Indian Institute of Technology, Kharagpur, India – sequence: 2 givenname: Julius orcidid: 0000-0002-6308-0533 surname: Ruckin fullname: Ruckin, Julius organization: Institute of Geodesy and Geoinformation, Cluster of Excellence PhenoRob, University of Bonn, Bonn, Germany – sequence: 3 givenname: Federico orcidid: 0000-0003-2815-5760 surname: Magistri fullname: Magistri, Federico organization: Institute of Geodesy and Geoinformation, Cluster of Excellence PhenoRob, University of Bonn, Bonn, Germany – sequence: 4 givenname: Cyrill orcidid: 0000-0003-1173-6972 surname: Stachniss fullname: Stachniss, Cyrill organization: Institute of Geodesy and Geoinformation, Cluster of Excellence PhenoRob, University of Bonn, Bonn, Germany – sequence: 5 givenname: Marija orcidid: 0000-0002-8830-6033 surname: Popovic fullname: Popovic, Marija email: m.popovic@tudelft.nl organization: Institute of Geodesy and Geoinformation, Cluster of Excellence PhenoRob, University of Bonn, Bonn, Germany |
| BookMark | eNp9kD1PwzAQhi0EEqV0Z2DwH0jxR2InY9VCqRSJquJjjK7OmRo1TuRESP33JLRDxcB07_A-d7rnhlz62iMhd5xNOWfZQ76ZTQUT8VTGgvM0vSAjIbWOpFbq8ixfk0nbfjHGeCK0zJIReV8gNnSDzts6GKzQdzRHCN75T_rhuh1dHDxUztBlgGbX0r5GZyU0nftGuhqoCn7zGvryeg9-QG_JlYV9i5PTHJO3p8fX-XOUvyxX81keGaHiLpLcKibtVpVqGydK8KzkDJU1CNKksAUpwQgOTNvMskyVFkCrUpuMcV1mRo4JO-41oW7bgLZogqsgHArOikFN0aspBjXFSU2PqD-IcV3_Qu27AG7_H3h_BB0int1JdNoblT-j33Om |
| CODEN | IRALC6 |
| CitedBy_id | crossref_primary_10_3390_jmse13050934 crossref_primary_10_1109_LRA_2025_3557233 crossref_primary_10_1002_rob_22479 crossref_primary_10_1016_j_patcog_2025_112400 crossref_primary_10_1016_j_robot_2024_104814 crossref_primary_10_1109_LRA_2025_3555940 crossref_primary_10_3390_drones8110682 crossref_primary_10_3390_drones9090650 crossref_primary_10_1109_ACCESS_2025_3551088 crossref_primary_10_3390_drones8100521 crossref_primary_10_1109_TASE_2025_3604174 crossref_primary_10_3390_agriculture15181917 crossref_primary_10_1016_j_ijepes_2025_110468 |
| Cites_doi | 10.1109/IROS55552.2023.10341674 10.1109/IROS47612.2022.9982287 10.1109/ROBOT.2010.5509714 10.1109/IROS.2017.8202317 10.1109/LRA.2016.2520560 10.1007/978-3-030-92790-5_24 10.1109/ICRA.2012.6224902 10.1109/OCEANSAP.2016.7485429 10.1109/ROBOT.2004.1308026 10.1109/TRO.2023.3313811 10.1109/ICRA.2019.8794345 10.1109/ICRA46639.2022.9812025 10.1109/ICRA.2019.8794402 10.1109/ICRA.2014.6907763 10.1177/0278364915596378 10.1016/j.comcom.2012.08.021 10.1109/TSMC.2019.2946646 10.7551/mitpress/3206.001.0001 10.1007/s10514-020-09903-2 10.3390/drones2010009 10.1177/0278364914533443 10.1109/IROS45743.2020.9341781 10.1109/ICRA48891.2023.10160859 10.1109/LRA.2022.3154026 10.1109/IROS.2012.6385653 10.1016/j.robot.2022.104288 10.1109/INFOCOM41043.2020.9155528 10.1109/IROS47612.2022.9981992 10.1109/ICRA48506.2021.9561963 10.1109/IROS45743.2020.9340918 10.1109/LRA.2020.2969191 10.1109/ICRA46639.2022.9811800 10.1109/IROS40897.2019.8967969 10.1109/ROBOT.2005.1570193 10.1109/ICRA.2017.7989582 10.1002/rob.21722 10.1109/ICRA48891.2023.10160565 10.1109/ECMR50962.2021.9568796 10.1016/j.neucom.2022.11.020 |
| ContentType | Journal Article |
| DBID | 97E RIA RIE AAYXX CITATION |
| DOI | 10.1109/LRA.2024.3421188 |
| DatabaseName | IEEE All-Society Periodicals Package (ASPP) 2005–Present IEEE All-Society Periodicals Package (ASPP) 1998–Present IEEE Electronic Library (IEL) CrossRef |
| DatabaseTitle | CrossRef |
| DatabaseTitleList | |
| Database_xml | – sequence: 1 dbid: RIE name: IEEE Electronic Library (IEL) url: https://ieeexplore.ieee.org/ sourceTypes: Publisher |
| DeliveryMethod | fulltext_linktorsrc |
| Discipline | Engineering Agriculture Forestry |
| EISSN | 2377-3766 |
| EndPage | 7754 |
| ExternalDocumentID | 10_1109_LRA_2024_3421188 10578000 |
| Genre | orig-research |
| GrantInformation_xml | – fundername: Deutsche Forschungsgemeinschaft grantid: EXC-2070 – 390732324 funderid: 10.13039/501100001659 |
| GroupedDBID | 0R~ 97E AAJGR AARMG AASAJ AAWTH ABAZT ABQJQ ABVLG ACGFS AGQYO AGSQL AHBIQ AKJIK AKQYR ALMA_UNASSIGNED_HOLDINGS ATWAV BEFXN BFFAM BGNUA BKEBE BPEOZ EBS EJD IFIPE IPLJI JAVBF KQ8 M43 M~E O9- OCL RIA RIE AAYXX CITATION |
| ID | FETCH-LOGICAL-c264t-31f603fb6d6b456219d10e6fcea3c8aba33ac21a07f9f096dfaa76d7c9017d9c3 |
| IEDL.DBID | RIE |
| ISICitedReferencesCount | 14 |
| ISICitedReferencesURI | http://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=Summon&SrcAuth=ProQuest&DestLinkType=CitingArticles&DestApp=WOS_CPL&KeyUT=001283670800007&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| ISSN | 2377-3766 |
| IngestDate | Sat Nov 29 06:03:31 EST 2025 Tue Nov 18 22:18:56 EST 2025 Wed Aug 27 02:29:06 EDT 2025 |
| IsPeerReviewed | true |
| IsScholarly | true |
| Issue | 9 |
| Language | English |
| License | https://ieeexplore.ieee.org/Xplorehelp/downloads/license-information/IEEE.html https://doi.org/10.15223/policy-029 https://doi.org/10.15223/policy-037 |
| LinkModel | DirectLink |
| MergedId | FETCHMERGED-LOGICAL-c264t-31f603fb6d6b456219d10e6fcea3c8aba33ac21a07f9f096dfaa76d7c9017d9c3 |
| ORCID | 0000-0003-1173-6972 0000-0002-8830-6033 0009-0005-3741-9053 0000-0002-6308-0533 0000-0003-2815-5760 |
| PageCount | 8 |
| ParticipantIDs | crossref_citationtrail_10_1109_LRA_2024_3421188 crossref_primary_10_1109_LRA_2024_3421188 ieee_primary_10578000 |
| PublicationCentury | 2000 |
| PublicationDate | 2024-09-01 |
| PublicationDateYYYYMMDD | 2024-09-01 |
| PublicationDate_xml | – month: 09 year: 2024 text: 2024-09-01 day: 01 |
| PublicationDecade | 2020 |
| PublicationTitle | IEEE robotics and automation letters |
| PublicationTitleAbbrev | LRA |
| PublicationYear | 2024 |
| Publisher | IEEE |
| Publisher_xml | – name: IEEE |
| References | ref13 ref35 Schulman (ref40) 2017 ref12 ref34 ref15 ref37 ref14 ref36 ref31 ref11 ref33 ref10 ref32 ref2 ref1 ref17 ref39 ref16 ref38 ref19 ref18 ref24 Magistri (ref8) 2019 ref23 ref26 ref25 ref20 ref42 ref41 ref22 ref21 Cao (ref30) 2023 ref28 ref27 ref29 ref7 ref9 ref4 ref3 ref6 ref5 |
| References_xml | – ident: ref5 doi: 10.1109/IROS55552.2023.10341674 – ident: ref26 doi: 10.1109/IROS47612.2022.9982287 – ident: ref9 doi: 10.1109/ROBOT.2010.5509714 – ident: ref42 doi: 10.1109/IROS.2017.8202317 – ident: ref28 doi: 10.1109/LRA.2016.2520560 – ident: ref7 doi: 10.1007/978-3-030-92790-5_24 – ident: ref14 doi: 10.1109/ICRA.2012.6224902 – ident: ref2 doi: 10.1109/OCEANSAP.2016.7485429 – ident: ref1 doi: 10.1109/ROBOT.2004.1308026 – ident: ref4 doi: 10.1109/TRO.2023.3313811 – ident: ref23 doi: 10.1109/ICRA.2019.8794345 – ident: ref32 doi: 10.1109/ICRA46639.2022.9812025 – ident: ref21 doi: 10.1109/ICRA.2019.8794402 – ident: ref16 doi: 10.1109/ICRA.2014.6907763 – ident: ref19 doi: 10.1177/0278364915596378 – ident: ref13 doi: 10.1016/j.comcom.2012.08.021 – ident: ref35 doi: 10.1109/TSMC.2019.2946646 – ident: ref38 doi: 10.7551/mitpress/3206.001.0001 – ident: ref18 doi: 10.1007/s10514-020-09903-2 – ident: ref27 doi: 10.3390/drones2010009 – ident: ref34 doi: 10.1177/0278364914533443 – start-page: 1928 volume-title: Proc. Conf. Robot Learn. (CoRL) year: 2023 ident: ref30 article-title: CAtNIPP: Context-aware attention-based network for informative path planning – ident: ref24 doi: 10.1109/IROS45743.2020.9341781 – ident: ref41 doi: 10.1109/ICRA48891.2023.10160859 – ident: ref17 doi: 10.1109/LRA.2022.3154026 – ident: ref3 doi: 10.1109/IROS.2012.6385653 – ident: ref25 doi: 10.1016/j.robot.2022.104288 – ident: ref29 doi: 10.1109/INFOCOM41043.2020.9155528 – ident: ref36 doi: 10.1109/IROS47612.2022.9981992 – ident: ref22 doi: 10.1109/ICRA48506.2021.9561963 – year: 2017 ident: ref40 article-title: Proximal policy optimization algorithms – ident: ref6 doi: 10.1109/IROS45743.2020.9340918 – ident: ref20 doi: 10.1109/LRA.2020.2969191 – ident: ref37 doi: 10.1109/ICRA46639.2022.9811800 – ident: ref33 doi: 10.1109/IROS40897.2019.8967969 – ident: ref39 doi: 10.1109/ROBOT.2005.1570193 – ident: ref15 doi: 10.1109/ICRA.2017.7989582 – ident: ref10 doi: 10.1002/rob.21722 – ident: ref11 doi: 10.1109/ICRA48891.2023.10160565 – start-page: 67 volume-title: Proc. Int. Micro Air Veh. Competition Conf. year: 2019 ident: ref8 article-title: Using prior information to improve crop/weed classification by MAV swarms – ident: ref31 doi: 10.1109/ECMR50962.2021.9568796 – ident: ref12 doi: 10.1016/j.neucom.2022.11.020 |
| SSID | ssj0001527395 |
| Score | 2.37124 |
| Snippet | Autonomousrobots are often employed for data collection due to their efficiency and low labour costs. A key task in robotic data acquisition is planning paths... |
| SourceID | crossref ieee |
| SourceType | Enrichment Source Index Database Publisher |
| StartPage | 7747 |
| SubjectTerms | Agricultural robots Agriculture Autonomous aerial vehicles Forestry Motion and path planning Motion planning Path planning Reinforcement learning Robot sensing systems robotics and automation in agriculture and forestry Three-dimensional displays Vehicle dynamics |
| Title | Deep Reinforcement Learning With Dynamic Graphs for Adaptive Informative Path Planning |
| URI | https://ieeexplore.ieee.org/document/10578000 |
| Volume | 9 |
| WOSCitedRecordID | wos001283670800007&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| hasFullText | 1 |
| inHoldings | 1 |
| isFullTextHit | |
| isPrint | |
| journalDatabaseRights | – providerCode: PRVIEE databaseName: IEEE Electronic Library (IEL) customDbUrl: eissn: 2377-3766 dateEnd: 99991231 omitProxy: false ssIdentifier: ssj0001527395 issn: 2377-3766 databaseCode: RIE dateStart: 20160101 isFulltext: true titleUrlDefault: https://ieeexplore.ieee.org/ providerName: IEEE – providerCode: PRVHPJ databaseName: ROAD: Directory of Open Access Scholarly Resources customDbUrl: eissn: 2377-3766 dateEnd: 99991231 omitProxy: false ssIdentifier: ssj0001527395 issn: 2377-3766 databaseCode: M~E dateStart: 20160101 isFulltext: true titleUrlDefault: https://road.issn.org providerName: ISSN International Centre |
| link | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV1NT8IwGG6EGKMHPxAjfpAevHgYbOvW0iNR0AMSQvzgtvQTSQwQGB797bZdES6aeOuWt8myp2mft-37PADcJIynLNIo4CkXQUI1C3hIdBDFPKU6RYkq1PV7pN9vjUZ04IvVXS2MUspdPlMN23Rn-XImVnarrGk9aQ3BMRl6iRBSFGttNlSslBhN10eRIW32hm2TAMZJAyUmzXHeKpulZ8tLxS0l3aN_fsQxOPScEbYLkE_AjppWwEF7vPC6Gco8bekKVsCeNdy0Lm6n4PVeqTkcKieQKtxeIPSaqmP4Nsnf4X3hSQ8frHT1Epow2JZsbqdB6IuVXHtguCJcexxVwUu383z3GHgvhUAYypObqVbjEGmOJeY26YmojEKFtVAMiRbjDCEm4ogZnKg2aY3UjBEsiTB8gUgq0BkoT2dTdQ6gCnXcwhoLw20SLglDmlk3PBwyQxYQqoHm-jdnwguNW7-Lj8wlHCHNDDCZBSbzwNTA7U-PeSGy8Uds1WKyFVfAcfHL-0uwb7sX18KuQDlfrNQ12BWf-WS5qIPS01en7gbSNxKtxnQ |
| linkProvider | IEEE |
| linkToHtml | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV1bT8IwGG2MGi8PXhAjXvvgiw-Dbd06-riIiHESQlB5W3pFEgOEi7_ftivCiya-dcu3ZelZ2vO1_c4B4DaiLKaBQh6LGfcioqjH_ER5QchiomIUyUJdP0va7Xq_TzquWN3Wwkgp7eEzWTVNu5cvxnxhlspqxpNWExydoW_FURQGRbnWaknFiImReLkZ6ZNa1k11ChhGVaSjA-uuspp81txU7GTSPPznZxyBA8caYVrAfAw25KgE9tPB1ClnSH21pixYAjvGctP4uJ2At4aUE9iVViKV29VA6FRVB_B9OP-AjcKVHj4a8eoZ1GEwFXRiBkLoypVsu6PZIly6HJXBa_Ohd9_ynJuCxzXpmevBVmEfKYYFZibtCYgIfIkVlxTxOmUUIcrDgGqkiNKJjVCUJlgkXDOGRBCOTsHmaDySZwBKX4V1rDDX7CZiIqFIUeOHh32q6QJCFVBbdnPOndS4cbz4zG3K4ZNcA5MbYHIHTAXc_TwxKWQ2_ogtG0zW4go4zn-5fwN2W72XLM-e2s8XYM-8qjgkdgk259OFvALb_Gs-nE2v7e_0DdvjyIo |
| openUrl | ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Deep+Reinforcement+Learning+With+Dynamic+Graphs+for+Adaptive+Informative+Path+Planning&rft.jtitle=IEEE+robotics+and+automation+letters&rft.au=Vashisth%2C+Apoorva&rft.au=Ruckin%2C+Julius&rft.au=Magistri%2C+Federico&rft.au=Stachniss%2C+Cyrill&rft.date=2024-09-01&rft.pub=IEEE&rft.eissn=2377-3766&rft.volume=9&rft.issue=9&rft.spage=7747&rft.epage=7754&rft_id=info:doi/10.1109%2FLRA.2024.3421188&rft.externalDocID=10578000 |
| thumbnail_l | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/lc.gif&issn=2377-3766&client=summon |
| thumbnail_m | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/mc.gif&issn=2377-3766&client=summon |
| thumbnail_s | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/sc.gif&issn=2377-3766&client=summon |