Wireless Deep Video Semantic Transmission

In this paper, we design a new class of high-efficiency deep joint source-channel coding methods to achieve end-to-end video transmission over wireless channels. The proposed methods exploit nonlinear transform and conditional coding architecture to adaptively extract semantic features across video...

Celý popis

Uložené v:
Podrobná bibliografia
Vydané v:IEEE journal on selected areas in communications Ročník 41; číslo 1; s. 214 - 229
Hlavní autori: Wang, Sixian, Dai, Jincheng, Liang, Zijian, Niu, Kai, Si, Zhongwei, Dong, Chao, Qin, Xiaoqi, Zhang, Ping
Médium: Journal Article
Jazyk:English
Vydavateľské údaje: New York IEEE 01.01.2023
The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Predmet:
Channels
Coding
Computer architecture
Design optimization
Domains
Encoding
Entropy
Feature extraction
Frames (data processing)
joint source-channel coding
Machine vision
nonlinear transform
Performance measurement
rate-distortion
Semantic communications
Semantics
Task analysis
Transforms
Transmission rate (communications)
Video communication
Video transmission
Vision systems
Wireless communication
Wireless sensor networks
ISSN:0733-8716, 1558-0008
On-line prístup:Získať plný text
Tagy: Pridať tag
Žiadne tagy, Buďte prvý, kto otaguje tento záznam!
Popis
Shrnutí:In this paper, we design a new class of high-efficiency deep joint source-channel coding methods to achieve end-to-end video transmission over wireless channels. The proposed methods exploit nonlinear transform and conditional coding architecture to adaptively extract semantic features across video frames, and transmit semantic feature domain representations over wireless channels via deep joint source-channel coding. Our framework is collected under the name deep video semantic transmission (DVST). In particular, benefiting from the strong temporal prior provided by the feature domain context, the learned nonlinear transform function becomes temporally adaptive, resulting in a richer and more accurate entropy model guiding the transmission of current frame. Accordingly, a novel rate adaptive transmission mechanism is developed to customize deep joint source-channel coding for video sources. It learns to allocate the limited channel bandwidth within and among video frames to maximize the overall transmission performance. The whole DVST design is formulated as an optimization problem whose goal is to minimize the end-to-end transmission rate-distortion performance under perceptual quality metrics or machine vision task performance metrics. Across standard video source test sequences and various communication scenarios, experiments show that our DVST can generally surpass traditional wireless video coded transmission schemes. The proposed DVST framework can well support future semantic communications due to its video content-aware and machine vision task integration abilities.
Bibliografia:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
ISSN:0733-8716
1558-0008
DOI:10.1109/JSAC.2022.3221977