Multi-Agent Reinforcement Learning-Based Digital Twin Migration Over Wireless Networks

To reduce the synchronization latency in digital twin (DT)-enabled wireless edge networks, the DT migration provides an efficient roaming solution among edge servers by following users' trajectories. In this work, we formulate a joint DT migration, communication and computation resource managem...

Celý popis

Uložené v:
Podrobná bibliografia
Vydané v:IEEE International Conference on Communications (2003) s. 2779 - 2784
Hlavní autori: Chen, Zhixiong, Yi, Wenqiang, Nallanathan, Arumugam
Médium: Konferenčný príspevok..
Jazyk:English
Vydavateľské údaje: IEEE 09.06.2024
Predmet:
Convex functions
Digital twin migration
Digital twins
multi-agent reinforcement learning
Reinforcement learning
Simulation
Trajectory
Transforms
Wireless networks
ISSN:1938-1883
On-line prístup:Získať plný text
Tagy: Pridať tag
Žiadne tagy, Buďte prvý, kto otaguje tento záznam!
Popis
Shrnutí:To reduce the synchronization latency in digital twin (DT)-enabled wireless edge networks, the DT migration provides an efficient roaming solution among edge servers by following users' trajectories. In this work, we formulate a joint DT migration, communication and computation resource management problem to minimize the data synchronization latency, where the time-varying network states and user mobility are considered. By decoupling edge servers under a deterministic migration strategy, we first derive the optimal communication and computation resource management policies at each server using convex optimization methods. For the DT migration problem between different servers, we transform it as a decentralized partially observable Markov decision process (Dec-POMDP). Then, we propose a novel agent-contribution-enabled multiagent reinforcement learning (AC-MARL) algorithm to enable distributed DT migration for users, in which the counterfactual baseline method is adopted to characterize the contribution of each agent and facilitate cooperation among agents. Simulation results show that the proposed DT migration scheme is able to reduce 30% data synchronization latency for users compared to the benchmark schemes.
ISSN:1938-1883
DOI:10.1109/ICC51166.2024.10622790