Coordination in Large Multiagent Reinforcement Learning Problems

Large distributed systems often require intelligent behavior. Although multiagent reinforcement learning can be applied to such systems, several yet unsolved challenges arise due to the large number of simultaneous learners. Among others, these include exponential growth of state-action spaces and c...

Celý popis

Uloženo v:
Podrobná bibliografie
Vydáno v:2011 IEEE/WIC/ACM International Conferences on Web Intelligence and Intelligent Agent Technology Ročník 2; s. 236 - 239
Hlavní autoři: Kemmerich, T., Buning, H. K.
Médium: Konferenční příspěvek
Jazyk:angličtina
Vydáno: IEEE 01.08.2011
Témata:
Convergence
Cooperative Stochastic Games
Coordination
DSL
Educational institutions
Electronic mail
Games
Joints
Learning
Multiagent Reinforcement Learning
ISBN:9781457713736, 145771373X
On-line přístup:Získat plný text
Tagy: Přidat tag
Žádné tagy, Buďte první, kdo vytvoří štítek k tomuto záznamu!
Abstract Large distributed systems often require intelligent behavior. Although multiagent reinforcement learning can be applied to such systems, several yet unsolved challenges arise due to the large number of simultaneous learners. Among others, these include exponential growth of state-action spaces and coordination. In this work, we deal with these two issues. Therefore, we consider a subclass of stochastic games called cooperative sequential stage games. With the help of a stateless distributed learning algorithm we solve the problem of growing state-action spaces. Then, we present six different techniques to coordinate action selection during the learning process. We prove a property of the learning algorithm that helps to reduce computational costs of one technique. An experimental analysis in a distributed agent partitioning problem with hundreds of agents reveals that the proposed techniques can lead to higher quality solutions and increase convergence speed compared to the basic approach. Some techniques even outperform a state-of-the-art special purpose approach.
AbstractList Large distributed systems often require intelligent behavior. Although multiagent reinforcement learning can be applied to such systems, several yet unsolved challenges arise due to the large number of simultaneous learners. Among others, these include exponential growth of state-action spaces and coordination. In this work, we deal with these two issues. Therefore, we consider a subclass of stochastic games called cooperative sequential stage games. With the help of a stateless distributed learning algorithm we solve the problem of growing state-action spaces. Then, we present six different techniques to coordinate action selection during the learning process. We prove a property of the learning algorithm that helps to reduce computational costs of one technique. An experimental analysis in a distributed agent partitioning problem with hundreds of agents reveals that the proposed techniques can lead to higher quality solutions and increase convergence speed compared to the basic approach. Some techniques even outperform a state-of-the-art special purpose approach.
Author Buning, H. K.
Kemmerich, T.
Author_xml – sequence: 1
  givenname: T.
  surname: Kemmerich
  fullname: Kemmerich, T.
  email: kemmerich@upb.de
  organization: Int. Grad. Sch. Dynamic Intell. Syst., Univ. of Paderborn, Paderborn, Germany
– sequence: 2
  givenname: H. K.
  surname: Buning
  fullname: Buning, H. K.
  email: kbcsl@upb.de
  organization: Dept. of Comput. Sci., Univ. of Paderborn, Paderborn, Germany
BookMark eNotjM1KxDAURiMqqGO3btz0BVpzkzTp3TkMjhYqioy4HNL2tkTaRNK68O0df87m48DHuWAnPnhi7Ap4DsDx5q3KqvUuFxwgV-qIJWhKbjQWqgCpjn8dVGEMSCP1GUvm-Z0f0BoR4ZzdbkKInfN2ccGnzqe1jQOlj5_j4uxAfklfyPk-xJamH6vJRu_8kD7H0Iw0zZfstLfjTMn_rtjr9m63ecjqp_tqs64zC5ovmYSO9500KJumKUm1aIQVfWGlpEYo7K2wCB0VCg1XBpG4AJIoqe11d7it2PVf1xHR_iO6ycavveaKm1LKb_47TK0
ContentType Conference Proceeding
DBID 6IE
6IL
CBEJK
RIE
RIL
DOI 10.1109/WI-IAT.2011.44
DatabaseName IEEE Electronic Library (IEL) Conference Proceedings
IEEE Proceedings Order Plan All Online (POP All Online) 1998-present by volume
IEEE Xplore All Conference Proceedings
IEEE Electronic Library (IEL)
IEEE Proceedings Order Plans (POP All) 1998-Present
DatabaseTitleList
Database_xml – sequence: 1
  dbid: RIE
  name: IEEE Electronic Library (IEL)
  url: https://ieeexplore.ieee.org/
  sourceTypes: Publisher
DeliveryMethod fulltext_linktorsrc
Discipline Computer Science
EISBN 9780769545134
0769545130
EndPage 239
ExternalDocumentID 6040783
Genre orig-research
GroupedDBID 6IE
6IL
ACM
ALMA_UNASSIGNED_HOLDINGS
APO
CBEJK
GUFHI
LHSKQ
RIB
RIC
RIE
RIL
ID FETCH-LOGICAL-a160t-31d0fd3793bbb8e4c972a2f5a33eb249fa2a91de549704799e021e393ecf6d5a3
IEDL.DBID RIE
ISBN 9781457713736
145771373X
IngestDate Wed Aug 27 02:48:18 EDT 2025
IsPeerReviewed false
IsScholarly false
Language English
LinkModel DirectLink
MergedId FETCHMERGED-LOGICAL-a160t-31d0fd3793bbb8e4c972a2f5a33eb249fa2a91de549704799e021e393ecf6d5a3
PageCount 4
ParticipantIDs ieee_primary_6040783
PublicationCentury 2000
PublicationDate 2011-Aug.
PublicationDateYYYYMMDD 2011-08-01
PublicationDate_xml – month: 08
  year: 2011
  text: 2011-Aug.
PublicationDecade 2010
PublicationTitle 2011 IEEE/WIC/ACM International Conferences on Web Intelligence and Intelligent Agent Technology
PublicationTitleAbbrev wi-iat
PublicationYear 2011
Publisher IEEE
Publisher_xml – name: IEEE
SSID ssj0000669991
ssj0001120470
Score 1.4713038
Snippet Large distributed systems often require intelligent behavior. Although multiagent reinforcement learning can be applied to such systems, several yet unsolved...
SourceID ieee
SourceType Publisher
StartPage 236
SubjectTerms Convergence
Cooperative Stochastic Games
Coordination
DSL
Educational institutions
Electronic mail
Games
Joints
Learning
Multiagent Reinforcement Learning
Title Coordination in Large Multiagent Reinforcement Learning Problems
URI https://ieeexplore.ieee.org/document/6040783
Volume 2
hasFullText 1
inHoldings 1
isFullTextHit
isPrint
link http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlZ3PS8MwFMcf2_Dgaeom_iYHj9albZofN2U4HIwxZOJuI01eZZdO9sO_3yTtNgQv3tpSSkhJXt43-X4ewL3gmSrcMiFyreTeksMjnUkaoVA5MoYqzQNdfyTGYzmbqUkDHvZeGEQMh8_w0V-GvXy7NFsvlfU4DbtOTWgKwSuv1l5PcaHTr3UO-kqcUCZo8HJlwqViIp3tEE_1Pa8hjjFVvY9hNHyeVkhPxn6VWgmRZtD-XxtPoHuw7JHJPhidQgPLM2jvajaQegh34Km_dOnmotIAyaIkI38UnAQfrvY2K_KGAaZqgm5Iav7qp_-4rzyz7sL74GXaf43qKgqRjjnduEnW0sKmbhzmeS6RGSUSnRSZTlOXVTNV6ESr2KJLFIXnzSt0YR9TlaIpuHWvnUOrXJZ4AYRzNxmiNZnUnisotJTKYqJNwiwzkl5Cx3fI_KsCZczrvrj6-_E1HO8EWhrfQGuz2uItHJnvzWK9ugt_9we9EaCe
linkProvider IEEE
linkToHtml http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlZ3PS8MwFMcfcwp6mrqJv83Bo3VpmybNTRHHhnUMmbjbSJNX2WWT_fDvN0m7DcGLt7aUElKSl_dNvp8HcCt4Igu7TAhsK7mz5PBAJSkNUMgcGUMZ556un4l-Px2N5KAGdxsvDCL6w2d47y79Xr6Z6ZWTytqc-l2nHdhNGIto6dbaKCo2eLrVzlZhCSPKBPVurkTYZEzEozXkqbrnFcYxpLL90Qt6j8MS6snYr2IrPtZ0Gv9r5SG0tqY9MtiEoyOo4fQYGuuqDaQaxE14eJrZhHNSqoBkMiWZOwxOvBNXOaMVeUOPU9VeOSQVgfXTfdzVnlm04L3zPHzqBlUdhUCFnC7tNGtoYWI7EvM8T5FpKSIVFYmKY5tXM1moSMnQoE0VhSPOS7SBH2MZoy64sa-dQH06m-IpEM7tdIhGJ6lyZEGh0lQajJSOmGE6pWfQdB0y_ipRGeOqL87_fnwD-93hazbOev2XCzhYy7U0vIT6cr7CK9jT38vJYn7t__QPTFyj5Q
openUrl ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Abook&rft.genre=proceeding&rft.title=2011+IEEE%2FWIC%2FACM+International+Conferences+on+Web+Intelligence+and+Intelligent+Agent+Technology&rft.atitle=Coordination+in+Large+Multiagent+Reinforcement+Learning+Problems&rft.au=Kemmerich%2C+T.&rft.au=Buning%2C+H.+K.&rft.date=2011-08-01&rft.pub=IEEE&rft.isbn=9781457713736&rft.volume=2&rft.spage=236&rft.epage=239&rft_id=info:doi/10.1109%2FWI-IAT.2011.44&rft.externalDocID=6040783
thumbnail_l http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=9781457713736/lc.gif&client=summon&freeimage=true
thumbnail_m http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=9781457713736/mc.gif&client=summon&freeimage=true
thumbnail_s http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=9781457713736/sc.gif&client=summon&freeimage=true