MAMBPO: Sample-efficient multi-robot reinforcement learning using learned world models
Multi-robot systems can benefit from reinforcement learning (RL) algorithms that learn behaviours in a small number of trials, a property known as sample efficiency. This research thus investigates the use of learned world models to improve sample efficiency. We present a novel multi-agent model-bas...
Uloženo v:
| Vydáno v: | Proceedings of the ... IEEE/RSJ International Conference on Intelligent Robots and Systems s. 5635 - 5640 |
|---|---|
| Hlavní autoři: | , , |
| Médium: | Konferenční příspěvek |
| Jazyk: | angličtina |
| Vydáno: |
IEEE
27.09.2021
|
| Témata: | |
| ISSN: | 2153-0866 |
| On-line přístup: | Získat plný text |
| Tagy: |
Přidat tag
Žádné tagy, Buďte první, kdo vytvoří štítek k tomuto záznamu!
|
| Abstract | Multi-robot systems can benefit from reinforcement learning (RL) algorithms that learn behaviours in a small number of trials, a property known as sample efficiency. This research thus investigates the use of learned world models to improve sample efficiency. We present a novel multi-agent model-based RL algorithm: Multi-Agent Model-Based Policy Optimization (MAMBPO), utilizing the Centralized Learning for Decentralized Execution (CLDE) framework. CLDE algorithms allow a group of agents to act in a fully decentralized manner after training. This is a desirable property for many systems comprising of multiple robots. MAMBPO uses a learned world model to improve sample efficiency compared to model-free Multi-Agent Soft Actor-Critic (MASAC). We demonstrate this on two simulated multi-robot tasks, where MAMBPO achieves a similar performance to MASAC, but requires far fewer samples to do so. Through this, we take an important step towards making real-life learning for multi-robot systems possible. |
|---|---|
| AbstractList | Multi-robot systems can benefit from reinforcement learning (RL) algorithms that learn behaviours in a small number of trials, a property known as sample efficiency. This research thus investigates the use of learned world models to improve sample efficiency. We present a novel multi-agent model-based RL algorithm: Multi-Agent Model-Based Policy Optimization (MAMBPO), utilizing the Centralized Learning for Decentralized Execution (CLDE) framework. CLDE algorithms allow a group of agents to act in a fully decentralized manner after training. This is a desirable property for many systems comprising of multiple robots. MAMBPO uses a learned world model to improve sample efficiency compared to model-free Multi-Agent Soft Actor-Critic (MASAC). We demonstrate this on two simulated multi-robot tasks, where MAMBPO achieves a similar performance to MASAC, but requires far fewer samples to do so. Through this, we take an important step towards making real-life learning for multi-robot systems possible. |
| Author | de Croon, Guido C.H.E. Willemsen, Daniel Coppola, Mario |
| Author_xml | – sequence: 1 givenname: Daniel surname: Willemsen fullname: Willemsen, Daniel email: j.daniel.willemsen@gmail.com organization: Delft University of Technology,MAVLab, Control & Operations Department, Faculty of Aerospace Engineering,Delft,The Netherlands,2628 HS – sequence: 2 givenname: Mario surname: Coppola fullname: Coppola, Mario email: m.coppola@tudelft.nl organization: Delft University of Technology,MAVLab, Control & Operations Department, Faculty of Aerospace Engineering,Delft,The Netherlands,2628 HS – sequence: 3 givenname: Guido C.H.E. surname: de Croon fullname: de Croon, Guido C.H.E. email: g.c.h.e.decroon@tudelft.nl organization: Delft University of Technology,MAVLab, Control & Operations Department, Faculty of Aerospace Engineering,Delft,The Netherlands,2628 HS |
| BookMark | eNotkNtKw0AURUdRsK39AkHyA6nnZJK5-FZL1UJLxKqvZTJzRkZyKZMU8e-l2pe9WWxYD3vMLtquJcZuEWaIoO9Wr-W2QBRqlkGGMy14obg4Y2MUoshRYl6cs1GGBU9BCXHFpn3_BQAIUistRuxjM988vJT3ydY0-5pS8j7YQO2QNId6CGnsqm5IIoXWd9FSc1xqMrEN7Wdy6I_5h-SS7y7WLmk6R3V_zS69qXuannrC3h-Xb4vndF0-rRbzdRoy4ENKYL3zTisvrefSeMy8IYvSYC50lRe596qSkBF3mrgGowmdJZKikhw4n7Cbf28got0-hsbEn93pBv4LPqVVtQ |
| ContentType | Conference Proceeding |
| DBID | 6IE 6IH CBEJK RIE RIO |
| DOI | 10.1109/IROS51168.2021.9635836 |
| DatabaseName | IEEE Electronic Library (IEL) Conference Proceedings IEEE Proceedings Order Plan (POP) 1998-present by volume IEEE Xplore All Conference Proceedings IEEE Electronic Library (IEL) IEEE Proceedings Order Plans (POP) 1998-present |
| DatabaseTitleList | |
| Database_xml | – sequence: 1 dbid: RIE name: IEEE Electronic Library (IEL) url: https://ieeexplore.ieee.org/ sourceTypes: Publisher |
| DeliveryMethod | fulltext_linktorsrc |
| Discipline | Engineering |
| EISBN | 1665417145 9781665417143 |
| EISSN | 2153-0866 |
| EndPage | 5640 |
| ExternalDocumentID | 9635836 |
| Genre | orig-research |
| GroupedDBID | 6IE 6IF 6IH 6IL 6IN AAJGR AAWTH ABLEC ACGFS ADZIZ ALMA_UNASSIGNED_HOLDINGS BEFXN BFFAM BGNUA BKEBE BPEOZ CBEJK CHZPO IEGSK IJVOP M43 OCL RIE RIL RIO RNS |
| ID | FETCH-LOGICAL-i203t-e0cfdfd98f7cf37af12faec17a1469b454ff8b702e3d9e390a9e1dcee76b73033 |
| IEDL.DBID | RIE |
| ISICitedReferencesCount | 14 |
| ISICitedReferencesURI | http://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=Summon&SrcAuth=ProQuest&DestLinkType=CitingArticles&DestApp=WOS_CPL&KeyUT=000755125504071&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| IngestDate | Tue May 06 03:33:40 EDT 2025 |
| IsPeerReviewed | false |
| IsScholarly | true |
| Language | English |
| LinkModel | DirectLink |
| MergedId | FETCHMERGED-LOGICAL-i203t-e0cfdfd98f7cf37af12faec17a1469b454ff8b702e3d9e390a9e1dcee76b73033 |
| PageCount | 6 |
| ParticipantIDs | ieee_primary_9635836 |
| PublicationCentury | 2000 |
| PublicationDate | 2021-Sept.-27 |
| PublicationDateYYYYMMDD | 2021-09-27 |
| PublicationDate_xml | – month: 09 year: 2021 text: 2021-Sept.-27 day: 27 |
| PublicationDecade | 2020 |
| PublicationTitle | Proceedings of the ... IEEE/RSJ International Conference on Intelligent Robots and Systems |
| PublicationTitleAbbrev | IROS |
| PublicationYear | 2021 |
| Publisher | IEEE |
| Publisher_xml | – name: IEEE |
| SSID | ssj0001079896 |
| Score | 2.3757644 |
| Snippet | Multi-robot systems can benefit from reinforcement learning (RL) algorithms that learn behaviours in a small number of trials, a property known as sample... |
| SourceID | ieee |
| SourceType | Publisher |
| StartPage | 5635 |
| SubjectTerms | Intelligent robots Multi-robot systems Optimization Reinforcement learning Task analysis Training |
| Title | MAMBPO: Sample-efficient multi-robot reinforcement learning using learned world models |
| URI | https://ieeexplore.ieee.org/document/9635836 |
| WOSCitedRecordID | wos000755125504071&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| hasFullText | 1 |
| inHoldings | 1 |
| isFullTextHit | |
| isPrint | |
| link | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV07T8MwELbaigEWHi3iLQ-MuHXiJI7ZAFGBRB-igLpVfpyrLi1KU34_thu1ILGwJZHsWHdOfD5_930IXUcmM7mRlFghU5KklBEFaUzAGhBGKSZlUC154f1-Ph6LYQ3dbGphACCAz6DtL8NZvlnolU-VddxkSXOW1VGd82xdq7XNp1AucpFVRcARFZ3n18HIhROZB3DFUbtq_EtFJSwi3f3_vf4AtbbVeHi4WWcOUQ3mR2jvB5FgE3307nr3w8EtHknP9uts4IkhXG84AAZJsVCLEhcQaFJ1yAjiSi9iij30fbq-BYMDhSoOAjnLFnrvPr49PJFKMYHMYspKAlRbY43ILdeWcWmj2ErQEZfuhyhUkibW5orTGJgRwASVAiLjxs8z5T51xo5RY76YwwnCnCVMu760diEXOMcJcKEOJG6HRoEadYqa3kKTzzUpxqQyztnfj8_RrneCB1rE_AI1ymIFl2hHf5WzZXEVPPkNX8CigA |
| linkProvider | IEEE |
| linkToHtml | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV07T8MwELZKQQIWHgXxxgMjaZ04iWM2QFSt6Eu0oG6VY5-rLg1KU34_thu1ILGwOZZsWXdOfDl_930I3fkqVokSxNNcRF4YEeqlEAUeaAVcpSkVwqmWdFivl4zHfFBB9-taGABw4DOo26a7y1eZXNpUWcNsliih8RbajsIwIKtqrU1GhTCe8LgsA_YJb7Tf-kMTUMQWwhX49XL4Lx0Vd4w0D_63gEN0sqnHw4P1SXOEKjA_Rvs_qARr6KP72H0a9B_wUFi-X2MFSw1hZsMOMujlWZoVOAdHlCpdThCXihFTbMHv09UjKOxIVLGTyFmcoPfmy-i55ZWaCd4sILTwgEittOKJZlJTJrQfaAHSZ8J8EnkaRqHWScpIAFRxoJwIDr4y62dxal52Sk9RdZ7N4QxhRkMqzVxSmqALjOs4mGAHQvOPRoCo9BzVrIUmnytajElpnIu_u2_RbmvU7Uw67d7rJdqzDrGwi4BdoWqRL-Ea7civYrbIb5xXvwEAU6XH |
| openUrl | ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=proceeding&rft.title=Proceedings+of+the+...+IEEE%2FRSJ+International+Conference+on+Intelligent+Robots+and+Systems&rft.atitle=MAMBPO%3A+Sample-efficient+multi-robot+reinforcement+learning+using+learned+world+models&rft.au=Willemsen%2C+Daniel&rft.au=Coppola%2C+Mario&rft.au=de+Croon%2C+Guido+C.H.E.&rft.date=2021-09-27&rft.pub=IEEE&rft.eissn=2153-0866&rft.spage=5635&rft.epage=5640&rft_id=info:doi/10.1109%2FIROS51168.2021.9635836&rft.externalDocID=9635836 |