Multi-Agent Reinforcement Learning-Based Digital Twin Migration Over Wireless Networks - Chen, Zhixiong; Yi, Wenqiang; Nallanathan, Arumugam (IEEE International Conference on Communications (2003), 2024)
To reduce the synchronization latency in digital twin (DT)-enabled wireless edge networks, the DT migration provides an efficient roaming solution among edge servers by following users' trajectories. In this work, we formulate a joint DT migration, communication and computation resource management problem to minimize the data synchronization latency, where the time-varying network states and user mobility are considered. By decoupling edge servers under a deterministic migration strategy, we first derive the optimal communication and computation resource management policies at each server using convex optimization methods. For the DT migration problem between different servers, we transform it as a decentralized partially observable Markov decision process (Dec-POMDP). Then, we propose a novel agent-contribution-enabled multiagent reinforcement learning (AC-MARL) algorithm to enable distributed DT migration for users, in which the counterfactual baseline method is adopted to characterize the contribution of each agent and facilitate cooperation among agents. Simulation results show that the proposed DT migration scheme is able to reduce 30% data synchronization latency for users compared to the benchmark schemes.