Zobrazit v EDS

Rain removal method for single image of dual-branch joint network based on sparse transformer

Uloženo v:
Podrobná bibliografie
Název: Rain removal method for single image of dual-branch joint network based on sparse transformer
Autoři: Fangfang Qin, Zongpu Jia, Xiaoyan Pang, Shan Zhao
Zdroj: Complex & Intelligent Systems, Vol 11, Iss 1, Pp 1-19 (2024)
Informace o vydavateli: Springer, 2024.
Rok vydání: 2024
Sbírka: LCC:Electronic computers. Computer science
LCC:Information technology
Témata: Image deraining, Self-attention, Dual-branch, Sparse transformer, Deep learning, Electronic computers. Computer science, QA75.5-76.95, Information technology, T58.5-58.64
Popis: Abstract In response to image degradation caused by rain during image acquisition, this paper proposes a rain removal method for single image of dual-branch joint network based on a sparse Transformer (DBSTNet). The developed model comprises a rain removal subnet and a background recovery subnet. The former extracts rain trace information utilizing a rain removal strategy, while the latter employs this information to restore background details. Furthermore, a U-shaped encoder-decoder branch (UEDB) focuses on local features to mitigate the impact of rainwater on background detail textures. UEDB incorporates a feature refinement unit to maximize the contribution of the channel attention mechanism in recovering local detail features. Additionally, since tokens with low relevance in the Transformer may influence image recovery, this study introduces a residual sparse Transformer branch (RSTB) to overcome the limitations of the Convolutional Neural Network’s (CNN’s) receptive field. Indeed, RSTB preserves the most valuable self-attention values for the aggregation of features, facilitating high-quality image reconstruction from a global perspective. Finally, the parallel dual-branch joint module, composed of RSTB and UEDB branches, effectively captures the local context and global structure, culminating in a clear background image. Experimental validation on synthetic and real datasets demonstrates that rain removal images exhibit richer detail information, significantly improving the overall visual effect.
Druh dokumentu: article
Popis souboru: electronic resource
Jazyk: English
ISSN: 2199-4536
2198-6053
Relation: https://doaj.org/toc/2199-4536; https://doaj.org/toc/2198-6053
DOI: 10.1007/s40747-024-01711-w
Přístupová URL adresa: https://doaj.org/article/b143d60a3b9d48b4b59b558c4114e8f5
Přístupové číslo: edsdoj.b143d60a3b9d48b4b59b558c4114e8f5
Databáze: Directory of Open Access Journals
Popis
Abstrakt:Abstract In response to image degradation caused by rain during image acquisition, this paper proposes a rain removal method for single image of dual-branch joint network based on a sparse Transformer (DBSTNet). The developed model comprises a rain removal subnet and a background recovery subnet. The former extracts rain trace information utilizing a rain removal strategy, while the latter employs this information to restore background details. Furthermore, a U-shaped encoder-decoder branch (UEDB) focuses on local features to mitigate the impact of rainwater on background detail textures. UEDB incorporates a feature refinement unit to maximize the contribution of the channel attention mechanism in recovering local detail features. Additionally, since tokens with low relevance in the Transformer may influence image recovery, this study introduces a residual sparse Transformer branch (RSTB) to overcome the limitations of the Convolutional Neural Network’s (CNN’s) receptive field. Indeed, RSTB preserves the most valuable self-attention values for the aggregation of features, facilitating high-quality image reconstruction from a global perspective. Finally, the parallel dual-branch joint module, composed of RSTB and UEDB branches, effectively captures the local context and global structure, culminating in a clear background image. Experimental validation on synthetic and real datasets demonstrates that rain removal images exhibit richer detail information, significantly improving the overall visual effect.
ISSN:21994536
21986053
DOI:10.1007/s40747-024-01711-w