Multi-Agent Reinforcement Learning-Based Digital Twin Migration Over Wireless Networks

To reduce the synchronization latency in digital twin (DT)-enabled wireless edge networks, the DT migration provides an efficient roaming solution among edge servers by following users' trajectories. In this work, we formulate a joint DT migration, communication and computation resource managem...

Full description

Saved in:
Bibliographic Details
Published in:IEEE International Conference on Communications (2003) pp. 2779 - 2784
Main Authors: Chen, Zhixiong, Yi, Wenqiang, Nallanathan, Arumugam
Format: Conference Proceeding
Language:English
Published: IEEE 09.06.2024
Subjects:
Convex functions
Digital twin migration
Digital twins
multi-agent reinforcement learning
Reinforcement learning
Simulation
Trajectory
Transforms
Wireless networks
ISSN:1938-1883
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:To reduce the synchronization latency in digital twin (DT)-enabled wireless edge networks, the DT migration provides an efficient roaming solution among edge servers by following users' trajectories. In this work, we formulate a joint DT migration, communication and computation resource management problem to minimize the data synchronization latency, where the time-varying network states and user mobility are considered. By decoupling edge servers under a deterministic migration strategy, we first derive the optimal communication and computation resource management policies at each server using convex optimization methods. For the DT migration problem between different servers, we transform it as a decentralized partially observable Markov decision process (Dec-POMDP). Then, we propose a novel agent-contribution-enabled multiagent reinforcement learning (AC-MARL) algorithm to enable distributed DT migration for users, in which the counterfactual baseline method is adopted to characterize the contribution of each agent and facilitate cooperation among agents. Simulation results show that the proposed DT migration scheme is able to reduce 30% data synchronization latency for users compared to the benchmark schemes.
ISSN:1938-1883
DOI:10.1109/ICC51166.2024.10622790