The self supervised multimodal semantic transmission mechanism for complex network environments

With the rapid development of intelligent transportation systems, the challenge of achieving efficient and accurate multimodal traffic data transmission and collaborative processing in complex network environments with bandwidth limitations, signal interference, and high concurrency has become a key...

Ausführliche Beschreibung

Gespeichert in:
Bibliographische Detailangaben
Veröffentlicht in:Scientific reports Jg. 15; H. 1; S. 29899 - 21
Hauptverfasser: Zou, Jiajun, Wan, Zhiping, Wang, Feng, Ye, Shitong, Liu, Shaojiang
Format: Journal Article
Sprache:Englisch
Veröffentlicht: London Nature Publishing Group UK 14.08.2025
Nature Publishing Group
Nature Portfolio
Schlagworte:
631/477
639/166
639/705
Algorithms
Bandwidths
Collaboration
Communication
Data transmission
Efficiency
Graph neural network
Humanities and Social Sciences
Intelligent transportation
Latency
Learning
Lidar
multidisciplinary
Multimodal semantic communication
Neural networks
Optimization
Reinforcement
Reinforcement learning
Resource management
Science
Science (multidisciplinary)
Self-supervised learning
Semantics
Spectrum allocation
Traffic accidents & safety
Traffic control
Traffic flow
Vehicles
Intelligent transportation
Multimodal semantic communication
Graph neural network
Self-supervised learning
Reinforcement learning
ISSN:2045-2322, 2045-2322
Online-Zugang:Volltext
Tags: Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
Beschreibung
Zusammenfassung:With the rapid development of intelligent transportation systems, the challenge of achieving efficient and accurate multimodal traffic data transmission and collaborative processing in complex network environments with bandwidth limitations, signal interference, and high concurrency has become a key issue that needs to be addressed. This paper proposes a Self-supervised Multi-modal and Reinforcement learning-based Traffic data semantic collaboration Transmission mechanism (SMART), aiming to optimize the transmission efficiency and robustness of multimodal data through a combination of self-supervised learning and reinforcement learning. The sending end employs a self-supervised conditional variational autoencoder and Transformer-DRL-based dynamic semantic compression strategy to intelligently filter and transmit the most core semantic information from video, radar, and LiDAR data. The receiving end combines Transformer and graph neural networks for deep decoding and feature fusion of multimodal data, while also using reinforcement learning self-supervised multi-task optimization engine to collaboratively enhance multiple task scenarios (such as traffic accident detection and vehicle behavior recognition). Experimental results show that SMART significantly outperforms traditional methods in low signal-to-noise ratio, high packet loss rate, and large-scale concurrency environments, excelling in key indicators such as semantic similarity, transmission efficiency, robustness, and end-to-end latency, demonstrating its effectiveness and innovation in smart transportation scenarios.
Bibliographie:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
content type line 23
ISSN:2045-2322
2045-2322
DOI:10.1038/s41598-025-15162-x