DMFusion: A dual-branch multi-scale feature fusion network for medical multi-modal image fusion - Ma, Gengchen; Qiu, Xihe; Tan, Xiaoyu (Biomedical signal processing and control, 2025)
In the field of medical imaging, high-quality multi-modal image fusion is crucial for improving diagnostic accuracy. By integrating information from different imaging modalities, medical multi-modal image fusion provides more comprehensive and accurate images. However, many existing fusion methods either overlook the unique information of each modality or fail to capture commonalities, resulting in incomplete fused images. To address this challenge, we propose an advanced medical multi-modal image fusion framework called Dual-Branch Multi-Scale Feature Fusion (DMFusion), aiming to optimize the fusion performance of multi-modal medical images. The DMFusion framework is based on a dual-branch autoencoder (AE) structure, where one branch is dedicated to extracting modality-specific distinctive features, and the other branch focuses on capturing shared features between modalities. This design allows DMFusion to not only preserve key features of each modality but also to effectively integrate their common information. Furthermore, our encoder employs multi-scale feature extraction techniques, enhancing the model’s perception of image details and allowing effective capture and fusion of image features at various scales. During the fusion process, both the encoder and decoder employ lightweight self-attention mechanisms. The encoder uses designed selection rules to precisely select salient features from the two branches, which are then fed into the decoder to achieve deep fusion. This decoder employs advanced image reconstruction techniques to generate fused images with richer texture details and better visual quality. Through qualitative and quantitative experiments on the publicly available Harvard Medical dataset and a dataset of abdominal multi-modal medical images from China, our method has demonstrated superior performance in medical image fusion tasks. The results indicate that the DMFusion framework can effectively enhance the accuracy of medical image fusion, providing new insights for future research on multi-modal image fusion. •Dual-branch autoencoder for multi-modal medical image fusion.•Modality-specific and shared features extracted by separate branches.•Multi-scale feature extraction enhances perception of image details.•Lightweight self-attention and selection rules enable precise feature fusion.