DMFusion: A dual-branch multi-scale feature fusion network for medical multi-modal image fusion

In the field of medical imaging, high-quality multi-modal image fusion is crucial for improving diagnostic accuracy. By integrating information from different imaging modalities, medical multi-modal image fusion provides more comprehensive and accurate images. However, many existing fusion methods e...

Ausführliche Beschreibung

Gespeichert in:

Bibliographische Detailangaben
Veröffentlicht in:	Biomedical signal processing and control Jg. 105; S. 107572
Hauptverfasser:	Ma, Gengchen, Qiu, Xihe, Tan, Xiaoyu
Format:	Journal Article
Sprache:	Englisch
Veröffentlicht:	Elsevier Ltd 01.07.2025
Schlagworte:	CNN-transformer Dual-branch Medical image fusion Multi-modality images Multi-scale Medical image fusion CNN-transformer Multi-scale Multi-modality images Dual-branch
ISSN:	1746-8094
Online-Zugang:	Volltext
Tags:	Tag hinzufügen Keine Tags, Fügen Sie den ersten Tag hinzu!

Beschreibung
Zusammenfassung:	In the field of medical imaging, high-quality multi-modal image fusion is crucial for improving diagnostic accuracy. By integrating information from different imaging modalities, medical multi-modal image fusion provides more comprehensive and accurate images. However, many existing fusion methods either overlook the unique information of each modality or fail to capture commonalities, resulting in incomplete fused images. To address this challenge, we propose an advanced medical multi-modal image fusion framework called Dual-Branch Multi-Scale Feature Fusion (DMFusion), aiming to optimize the fusion performance of multi-modal medical images. The DMFusion framework is based on a dual-branch autoencoder (AE) structure, where one branch is dedicated to extracting modality-specific distinctive features, and the other branch focuses on capturing shared features between modalities. This design allows DMFusion to not only preserve key features of each modality but also to effectively integrate their common information. Furthermore, our encoder employs multi-scale feature extraction techniques, enhancing the model’s perception of image details and allowing effective capture and fusion of image features at various scales. During the fusion process, both the encoder and decoder employ lightweight self-attention mechanisms. The encoder uses designed selection rules to precisely select salient features from the two branches, which are then fed into the decoder to achieve deep fusion. This decoder employs advanced image reconstruction techniques to generate fused images with richer texture details and better visual quality. Through qualitative and quantitative experiments on the publicly available Harvard Medical dataset and a dataset of abdominal multi-modal medical images from China, our method has demonstrated superior performance in medical image fusion tasks. The results indicate that the DMFusion framework can effectively enhance the accuracy of medical image fusion, providing new insights for future research on multi-modal image fusion. •Dual-branch autoencoder for multi-modal medical image fusion.•Modality-specific and shared features extracted by separate branches.•Multi-scale feature extraction enhances perception of image details.•Lightweight self-attention and selection rules enable precise feature fusion.
ISSN:	1746-8094
DOI:	10.1016/j.bspc.2025.107572