Neural Schrödinger bridge for unpaired real-world image deraining

Given the significant differences between domains, current unpaired learning methods struggle to accurately map the relationship between rainy and clear images. To this end, we introduce a neural Schrödinger bridge (NSB) for unpaired real-world image deraining, which utilizes the stochastic differen...

Ausführliche Beschreibung

Gespeichert in:
Bibliographische Detailangaben
Veröffentlicht in:Information sciences Jg. 682; S. 121199
Hauptverfasser: Wen, Yuanbo, Gao, Tao, Chen, Ting
Format: Journal Article
Sprache:Englisch
Veröffentlicht: Elsevier Inc 01.11.2024
Schlagworte:
Computer vision
Contrastive learning
Image deraining
Schrödinger bridge
Unpaired learning
Computer vision
Image deraining
Schrödinger bridge
Unpaired learning
Contrastive learning
ISSN:0020-0255
Online-Zugang:Volltext
Tags: Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
Beschreibung
Zusammenfassung:Given the significant differences between domains, current unpaired learning methods struggle to accurately map the relationship between rainy and clear images. To this end, we introduce a neural Schrödinger bridge (NSB) for unpaired real-world image deraining, which utilizes the stochastic differential equations to capture the mapping relationships between rainy and clear domains. Meanwhile, we frame the deraining process as a Lagrangian problem using the Kullback-Leibler divergence between the data distribution and the model distribution. Additionally, by leveraging the capabilities of the contrastive language-image pre-training model (CLIP), our research shows that the CLIP prior helps differentiate between rainy and clear images. Building on this, we reformulate the Schrödinger bridge problem as a series of adversarial learning tasks using both image and prompt representations. To our knowledge, our approach is the first to use the Schrödinger bridge in unpaired image deraining. Extensive experiments show that our proposed NSB model outperforms existing unpaired deraining methods in both quantitative and qualitative evaluations.
ISSN:0020-0255
DOI:10.1016/j.ins.2024.121199