VSS-SpatioNet: a multi-scale feature fusion network for multimodal image integrations

Infrared and visible image fusion (vis-ir) enhances diagnostic accuracy in medical imaging and biological analysis. Existing CNN-based and Transformer-based methods face computational inefficiencies in modeling global dependencies. The author proposes VSS-SpatioNet, a lightweight architecture that r...

Ausführliche Beschreibung

Gespeichert in:
Bibliographische Detailangaben
Veröffentlicht in:Scientific reports Jg. 15; H. 1; S. 9306 - 20
1. Verfasser: Xiang, Zeyu
Format: Journal Article
Sprache:Englisch
Veröffentlicht: London Nature Publishing Group UK 18.03.2025
Nature Publishing Group
Nature Portfolio
Schlagworte:
631/114/1305
631/114/1564
631/114/2401
639/624/1075
639/705/117
639/705/258
Biological analysis
Deep learning
Efficiency
Humanities and Social Sciences
Infrared and visible image fusion
Lightweight asymmetric encoder-decoder
Medical imaging
Medical imaging applications
Multi-scale feature integration
multidisciplinary
Science
Science (multidisciplinary)
Sensors
Visual state space (VSS) module
Wavelet transforms
Visual state space (VSS) module
Multi-scale feature integration
Medical imaging applications
Lightweight asymmetric encoder-decoder
Infrared and visible image fusion
ISSN:2045-2322, 2045-2322
Online-Zugang:Volltext
Tags: Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
Beschreibung
Zusammenfassung:Infrared and visible image fusion (vis-ir) enhances diagnostic accuracy in medical imaging and biological analysis. Existing CNN-based and Transformer-based methods face computational inefficiencies in modeling global dependencies. The author proposes VSS-SpatioNet, a lightweight architecture that replaces self-attention in Transformers with a Visual State Space (VSS) module for efficient dependency modeling. The framework employs an asymmetric encoder-decoder with a multi-scale autoencoder and a novel VSS-Spatial (VS) fusion block for local-global feature integration. Evaluations on TNO, Harvard Medical, and RoadScene datasets demonstrate superior performance. On TNO, VSS-SpatioNet achieves state-of-the-art Entropy (En = 7.0058) and Mutual Information (MI = 14.0116), outperforming 12 benchmark methods. For RoadScene, it attains gradient-based fusion performance ( =0.5712), Piella’s metric ( =0.7926), and average gradient (AG = 5.2994), surpassing prior works. On Harvard Medical, the VS strategy improves Mean Gradient by 18.7% (0.0224 vs. 0.0198) against FusionGAN, validating enhanced feature preservation. Results confirm the framework’s efficacy in medical applications, particularly precise tissue characterization.
Bibliographie:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
content type line 23
ISSN:2045-2322
2045-2322
DOI:10.1038/s41598-025-93143-w