MAMBPO: Sample-efficient multi-robot reinforcement learning using learned world models
Multi-robot systems can benefit from reinforcement learning (RL) algorithms that learn behaviours in a small number of trials, a property known as sample efficiency. This research thus investigates the use of learned world models to improve sample efficiency. We present a novel multi-agent model-bas...
Uloženo v:
| Vydáno v: | Proceedings of the ... IEEE/RSJ International Conference on Intelligent Robots and Systems s. 5635 - 5640 |
|---|---|
| Hlavní autoři: | , , |
| Médium: | Konferenční příspěvek |
| Jazyk: | angličtina |
| Vydáno: |
IEEE
27.09.2021
|
| Témata: | |
| ISSN: | 2153-0866 |
| On-line přístup: | Získat plný text |
| Tagy: |
Přidat tag
Žádné tagy, Buďte první, kdo vytvoří štítek k tomuto záznamu!
|
| Abstract | Multi-robot systems can benefit from reinforcement learning (RL) algorithms that learn behaviours in a small number of trials, a property known as sample efficiency. This research thus investigates the use of learned world models to improve sample efficiency. We present a novel multi-agent model-based RL algorithm: Multi-Agent Model-Based Policy Optimization (MAMBPO), utilizing the Centralized Learning for Decentralized Execution (CLDE) framework. CLDE algorithms allow a group of agents to act in a fully decentralized manner after training. This is a desirable property for many systems comprising of multiple robots. MAMBPO uses a learned world model to improve sample efficiency compared to model-free Multi-Agent Soft Actor-Critic (MASAC). We demonstrate this on two simulated multi-robot tasks, where MAMBPO achieves a similar performance to MASAC, but requires far fewer samples to do so. Through this, we take an important step towards making real-life learning for multi-robot systems possible. |
|---|---|
| AbstractList | Multi-robot systems can benefit from reinforcement learning (RL) algorithms that learn behaviours in a small number of trials, a property known as sample efficiency. This research thus investigates the use of learned world models to improve sample efficiency. We present a novel multi-agent model-based RL algorithm: Multi-Agent Model-Based Policy Optimization (MAMBPO), utilizing the Centralized Learning for Decentralized Execution (CLDE) framework. CLDE algorithms allow a group of agents to act in a fully decentralized manner after training. This is a desirable property for many systems comprising of multiple robots. MAMBPO uses a learned world model to improve sample efficiency compared to model-free Multi-Agent Soft Actor-Critic (MASAC). We demonstrate this on two simulated multi-robot tasks, where MAMBPO achieves a similar performance to MASAC, but requires far fewer samples to do so. Through this, we take an important step towards making real-life learning for multi-robot systems possible. |
| Author | de Croon, Guido C.H.E. Willemsen, Daniel Coppola, Mario |
| Author_xml | – sequence: 1 givenname: Daniel surname: Willemsen fullname: Willemsen, Daniel email: j.daniel.willemsen@gmail.com organization: Delft University of Technology,MAVLab, Control & Operations Department, Faculty of Aerospace Engineering,Delft,The Netherlands,2628 HS – sequence: 2 givenname: Mario surname: Coppola fullname: Coppola, Mario email: m.coppola@tudelft.nl organization: Delft University of Technology,MAVLab, Control & Operations Department, Faculty of Aerospace Engineering,Delft,The Netherlands,2628 HS – sequence: 3 givenname: Guido C.H.E. surname: de Croon fullname: de Croon, Guido C.H.E. email: g.c.h.e.decroon@tudelft.nl organization: Delft University of Technology,MAVLab, Control & Operations Department, Faculty of Aerospace Engineering,Delft,The Netherlands,2628 HS |
| BookMark | eNotkNtKw0AURUdRsK39AkHyA6nnZJK5-FZL1UJLxKqvZTJzRkZyKZMU8e-l2pe9WWxYD3vMLtquJcZuEWaIoO9Wr-W2QBRqlkGGMy14obg4Y2MUoshRYl6cs1GGBU9BCXHFpn3_BQAIUistRuxjM988vJT3ydY0-5pS8j7YQO2QNId6CGnsqm5IIoXWd9FSc1xqMrEN7Wdy6I_5h-SS7y7WLmk6R3V_zS69qXuannrC3h-Xb4vndF0-rRbzdRoy4ENKYL3zTisvrefSeMy8IYvSYC50lRe596qSkBF3mrgGowmdJZKikhw4n7Cbf28got0-hsbEn93pBv4LPqVVtQ |
| ContentType | Conference Proceeding |
| DBID | 6IE 6IH CBEJK RIE RIO |
| DOI | 10.1109/IROS51168.2021.9635836 |
| DatabaseName | IEEE Electronic Library (IEL) Conference Proceedings IEEE Proceedings Order Plan (POP) 1998-present by volume IEEE Xplore All Conference Proceedings IEEE Electronic Library (IEL) IEEE Proceedings Order Plans (POP) 1998-present |
| DatabaseTitleList | |
| Database_xml | – sequence: 1 dbid: RIE name: IEEE Electronic Library (IEL) url: https://ieeexplore.ieee.org/ sourceTypes: Publisher |
| DeliveryMethod | fulltext_linktorsrc |
| Discipline | Engineering |
| EISBN | 1665417145 9781665417143 |
| EISSN | 2153-0866 |
| EndPage | 5640 |
| ExternalDocumentID | 9635836 |
| Genre | orig-research |
| GroupedDBID | 6IE 6IF 6IH 6IL 6IN AAJGR AAWTH ABLEC ACGFS ADZIZ ALMA_UNASSIGNED_HOLDINGS BEFXN BFFAM BGNUA BKEBE BPEOZ CBEJK CHZPO IEGSK IJVOP M43 OCL RIE RIL RIO RNS |
| ID | FETCH-LOGICAL-i203t-e0cfdfd98f7cf37af12faec17a1469b454ff8b702e3d9e390a9e1dcee76b73033 |
| IEDL.DBID | RIE |
| ISICitedReferencesCount | 14 |
| ISICitedReferencesURI | http://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=Summon&SrcAuth=ProQuest&DestLinkType=CitingArticles&DestApp=WOS_CPL&KeyUT=000755125504071&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| IngestDate | Tue May 06 03:33:40 EDT 2025 |
| IsPeerReviewed | false |
| IsScholarly | true |
| Language | English |
| LinkModel | DirectLink |
| MergedId | FETCHMERGED-LOGICAL-i203t-e0cfdfd98f7cf37af12faec17a1469b454ff8b702e3d9e390a9e1dcee76b73033 |
| PageCount | 6 |
| ParticipantIDs | ieee_primary_9635836 |
| PublicationCentury | 2000 |
| PublicationDate | 2021-Sept.-27 |
| PublicationDateYYYYMMDD | 2021-09-27 |
| PublicationDate_xml | – month: 09 year: 2021 text: 2021-Sept.-27 day: 27 |
| PublicationDecade | 2020 |
| PublicationTitle | Proceedings of the ... IEEE/RSJ International Conference on Intelligent Robots and Systems |
| PublicationTitleAbbrev | IROS |
| PublicationYear | 2021 |
| Publisher | IEEE |
| Publisher_xml | – name: IEEE |
| SSID | ssj0001079896 |
| Score | 2.3757644 |
| Snippet | Multi-robot systems can benefit from reinforcement learning (RL) algorithms that learn behaviours in a small number of trials, a property known as sample... |
| SourceID | ieee |
| SourceType | Publisher |
| StartPage | 5635 |
| SubjectTerms | Intelligent robots Multi-robot systems Optimization Reinforcement learning Task analysis Training |
| Title | MAMBPO: Sample-efficient multi-robot reinforcement learning using learned world models |
| URI | https://ieeexplore.ieee.org/document/9635836 |
| WOSCitedRecordID | wos000755125504071&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| hasFullText | 1 |
| inHoldings | 1 |
| isFullTextHit | |
| isPrint | |
| link | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV07T8MwELZKxQALjxbxlgdG3CZxEttsgKhg6EMUULfKj3PVpUFpyu_HdqMWJBY221Ji65zY5_N334fQTebcBCtjRYSy7oACVhBlFSOG5yaioHJBZRCbYIMBn0zEqIFuN7kwABDAZ9DxxXCXbwq98qGyrvtYMk7zHbTDWL7O1drGUyImuMjrJOA4Et2X1-HYuRO5B3Alcad--JeKSthEegf_6_4QtbfZeHi02WeOUAMWx2j_B5FgC3307_sPo-EdHkvP9ksgEEO4t-EAGCRloYoKlxBoUnWICOJaL2KGPfR9tq6CwYFCFQeBnGUbvfee3h6fSa2YQOZJRCsCkbbGGsEt05YyaePEStAxk25BFCrNUmu5YlEC1AigIpICYuPGz3LlfnVKT1BzUSzgFOFM8lQBVc69iFPJuXK-mnQro0mVpTbTZ6jlLTT9XJNiTGvjnP_dfIH2_CR4oEXCLlGzKldwhXb1VzVfltdhJr8Bf8Chuw |
| linkProvider | IEEE |
| linkToHtml | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV3LTgMhFCW1mqgbH63xLQuXTjszzAzgTo1NG_uKraa7BoZL003HTKd-v4CTVhM37oAEQnheLueeg9BtbMwELQLpcanNAwU096SW1FMsUT4BmXAinNgE7ffZZMKHFXS3joUBAAc-g4ZNur98laUr6yprmsUSM5JsoW2rnFVGa208Kj7ljCdlGHDg82bndTAyBkViIVxh0Cir_9JRcddI6-B_HThE9U08Hh6ub5ojVIHFMdr_QSVYQ--9h97jcHCPR8Ly_XrgqCFMa9hBBr08k1mBc3BEqanzCeJSMWKGLfh99p0FhR2JKnYSOcs6ems9j5_aXqmZ4M1DnxQe-KlWWnGmaaoJFToItYA0oMIciVxGcaQ1k9QPgSgOhPuCQ6BM_2kizWYn5ARVF9kCThGOBYskEGkMjCASjEljrQlzNqpIaqLj9AzV7AhNP75pMabl4Jz_XXyDdtvjXnfa7fRfLtCenRALuwjpJaoW-Qqu0E76WcyX-bWb1S8aKKUE |
| openUrl | ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=proceeding&rft.title=Proceedings+of+the+...+IEEE%2FRSJ+International+Conference+on+Intelligent+Robots+and+Systems&rft.atitle=MAMBPO%3A+Sample-efficient+multi-robot+reinforcement+learning+using+learned+world+models&rft.au=Willemsen%2C+Daniel&rft.au=Coppola%2C+Mario&rft.au=de+Croon%2C+Guido+C.H.E.&rft.date=2021-09-27&rft.pub=IEEE&rft.eissn=2153-0866&rft.spage=5635&rft.epage=5640&rft_id=info:doi/10.1109%2FIROS51168.2021.9635836&rft.externalDocID=9635836 |