Multi-Agent Reinforcement Learning-Based Digital Twin Migration Over Wireless Networks

To reduce the synchronization latency in digital twin (DT)-enabled wireless edge networks, the DT migration provides an efficient roaming solution among edge servers by following users' trajectories. In this work, we formulate a joint DT migration, communication and computation resource managem...

Ausführliche Beschreibung

Gespeichert in:
Bibliographische Detailangaben
Veröffentlicht in:IEEE International Conference on Communications (2003) S. 2779 - 2784
Hauptverfasser: Chen, Zhixiong, Yi, Wenqiang, Nallanathan, Arumugam
Format: Tagungsbericht
Sprache:Englisch
Veröffentlicht: IEEE 09.06.2024
Schlagworte:
Convex functions
Digital twin migration
Digital twins
multi-agent reinforcement learning
Reinforcement learning
Simulation
Trajectory
Transforms
Wireless networks
ISSN:1938-1883
Online-Zugang:Volltext
Tags: Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
Beschreibung
Zusammenfassung:To reduce the synchronization latency in digital twin (DT)-enabled wireless edge networks, the DT migration provides an efficient roaming solution among edge servers by following users' trajectories. In this work, we formulate a joint DT migration, communication and computation resource management problem to minimize the data synchronization latency, where the time-varying network states and user mobility are considered. By decoupling edge servers under a deterministic migration strategy, we first derive the optimal communication and computation resource management policies at each server using convex optimization methods. For the DT migration problem between different servers, we transform it as a decentralized partially observable Markov decision process (Dec-POMDP). Then, we propose a novel agent-contribution-enabled multiagent reinforcement learning (AC-MARL) algorithm to enable distributed DT migration for users, in which the counterfactual baseline method is adopted to characterize the contribution of each agent and facilitate cooperation among agents. Simulation results show that the proposed DT migration scheme is able to reduce 30% data synchronization latency for users compared to the benchmark schemes.
ISSN:1938-1883
DOI:10.1109/ICC51166.2024.10622790