CVAE-based Far-sighted Intention Inference for Opponent Modeling in Multi-agent Reinforcement Learning

Most interactive environments are non-stationary for agents, as the behaviors of their opponents continually change, which can impair the performance of reinforcement learning algorithms. This impairment can be alleviated by modeling opponents to predict their future movements. To predict more preci...

Full description

Saved in:
Bibliographic Details
Published in:Chinese Control Conference pp. 5847 - 5851
Main Authors: Pei, Yu, Xu, XiaoPeng, Liu, Zhong, Wang, Kuo, Zhu, Li, Wang, Dong
Format: Conference Proceeding
Language:English
Published: Technical Committee on Control Theory, Chinese Association of Automation 28.07.2024
Subjects:
Analytical models
Attention mechanisms
conditional variational autoencoder (CVAE)
Decision making
Games
intention inference
Multi-agent reinforcement learning
opponent modeling
Prediction algorithms
Predictive models
Reinforcement learning
ISSN:1934-1768
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Most interactive environments are non-stationary for agents, as the behaviors of their opponents continually change, which can impair the performance of reinforcement learning algorithms. This impairment can be alleviated by modeling opponents to predict their future movements. To predict more precisely and further into the future compared to current opponent modeling approaches, we developed a CVAE-based Far-sighted Intention Inference method (CFI2), including a Trajectory Prediction Module (TPM) and a Trajectory Analysis Module (TAM). TPM synthesizes complex interactions between agents using an attention mechanism and achieves robust far-sighted prediction with a conditional variational autoencoder (CVAE). TAM enables agents to analyze trajectories by assigning attention to the predicted movements of their opponents, corresponding to their impacts on the future. We conducted experiments in Drone Game where CFI2 achieves significantly higher rewards more rapidly than baseline methods. It is proven that agents can make better decisions by incorporating long-term predictions, just like the decision-making process of humans.
ISSN:1934-1768
DOI:10.23919/CCC63176.2024.10662404