Video Variational Deep Atmospheric Turbulence Correction

This paper presents a novel variational deep-learning approach for video atmospheric turbulence correction. We modify and tailor the Nonlinear Activation Free Network (NAFNet) architecture for video restoration, introducing a new transformer-based channel attention mechanism to exploit long-range hi...

Celý popis

Uložené v:
Podrobná bibliografia
Vydané v:IEEE access Ročník 12; s. 127368 - 127379
Hlavní autori: Lopez-Tapia, Santiago, Wang, Xijun, Katsaggelos, Aggelos K.
Médium: Journal Article
Jazyk:English
Vydavateľské údaje: Piscataway IEEE 2024
The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Predmet:
Atmospheric correction
Atmospheric modeling
Atmospheric turbulence
Deep learning
Degradation
Feature extraction
Image enhancement
Image restoration
Noise prediction
Nonlinear distortion
Training
video restoration
ISSN:2169-3536, 2169-3536
On-line prístup:Získať plný text
Tagy: Pridať tag
Žiadne tagy, Buďte prvý, kto otaguje tento záznam!
Popis
Shrnutí:This paper presents a novel variational deep-learning approach for video atmospheric turbulence correction. We modify and tailor the Nonlinear Activation Free Network (NAFNet) architecture for video restoration, introducing a new transformer-based channel attention mechanism to exploit long-range high-level relations among frames. Close-range low-level relations are handled using 3D convolutions. We further boost the model's performance by including it in a variational inference framework. We achieve this objective by conditioning the model on features extracted by a variational autoencoder (VAE). Furthermore, we enhance these features by incorporating information relevant to the image formation process into the VAE features. This is done via a new loss function based on the prediction of the parameters of the geometrical distortion, the spatially variant blur, and the noise, all responsible for the video degradation. Experiments on synthetic and physically simulated video datasets demonstrate the effectiveness and reliability of the proposed method and validate its superiority over existing state-of-the-art approaches.
Bibliografia:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
ISSN:2169-3536
2169-3536
DOI:10.1109/ACCESS.2024.3455289