View in EDS

A multispectral pansharpening method based on CNN-DI network with mixture of experts.

Saved in:
Bibliographic Details
Title: A multispectral pansharpening method based on CNN-DI network with mixture of experts.
Authors: Guo, Zhongyuan1 (AUTHOR) mojo1023@163.com, Lei, Jia1,2 (AUTHOR) leijia03@stu.xjtu.edu.cn, Zhou, Shihua1 (AUTHOR) zhoushihua@dlu.edu.cn, Wang, Bin1 (AUTHOR) wangbin@dlu.edu.cn, Kasabov, Nikola K.3 (AUTHOR) nkasabov@aut.ac.nz
Source: Applied Soft Computing. Oct2025, Vol. 182, pN.PAG-N.PAG. 1p.
Subject Terms: Multispectral imaging, Image fusion, Artificial neural networks, Ensemble learning, Image processing, Deep learning, Multisensor data fusion
Abstract: The process of fusing two complementary data, panchromatic and multispectral images, to create high-resolution multispectral (HRMS) images is known as pansharpening. Combining detail injection (DI) methods with convolutional neural networks (CNN) for improved HRMS image fusion quality is a research hotspot due to their interpretability and large-scale data processing capabilities, respectively. Nevertheless, the current hybrid models typically concatenate CNN and traditional techniques, limiting the ability to utilize the benefits of both approaches. This paper presents a new hybrid network, multispectral pansharpening method based on CNN-DI network with mixture of experts (CDN-MoE), using detail injection theory to design a deep learning framework. Specifically, we first create the mixture of detail inject experts network (MoDIE-Net) that mixes training pairs of full- and reduced-resolution images to enhance model generalization. Next, the adaptive correlation residual network (ACR-Net) is suggested to find the correlation between the spectral and spatial features of the source images. Finally, the global information injection network (GII-Net) is established to strengthen the accuracy of fusion results by integrating the context of input images. Additionally, to reduce the loss of spectral features during the upsampling process, the spectral reconstruction network (SR-Net) is proposed. We perform both qualitative and quantitative experiments on the GaoFen-2, IKONOS, and WorldView-2 datasets at various resolutions. Our approach has advantages over other SOTA pansharpening methods currently available in terms of visual effects and objective metrics. • This paper proposes a novel CNN-DI network with mixture of experts for pansharpening. • CDN-MoE can achieve hybrid training of full- and reduced-resolution datasets with mixture of experts model. • The spatial–spectral adaptive module is built to find the correlation between space and spectrum. [ABSTRACT FROM AUTHOR]
Database: Supplemental Index
Description
Abstract:The process of fusing two complementary data, panchromatic and multispectral images, to create high-resolution multispectral (HRMS) images is known as pansharpening. Combining detail injection (DI) methods with convolutional neural networks (CNN) for improved HRMS image fusion quality is a research hotspot due to their interpretability and large-scale data processing capabilities, respectively. Nevertheless, the current hybrid models typically concatenate CNN and traditional techniques, limiting the ability to utilize the benefits of both approaches. This paper presents a new hybrid network, multispectral pansharpening method based on CNN-DI network with mixture of experts (CDN-MoE), using detail injection theory to design a deep learning framework. Specifically, we first create the mixture of detail inject experts network (MoDIE-Net) that mixes training pairs of full- and reduced-resolution images to enhance model generalization. Next, the adaptive correlation residual network (ACR-Net) is suggested to find the correlation between the spectral and spatial features of the source images. Finally, the global information injection network (GII-Net) is established to strengthen the accuracy of fusion results by integrating the context of input images. Additionally, to reduce the loss of spectral features during the upsampling process, the spectral reconstruction network (SR-Net) is proposed. We perform both qualitative and quantitative experiments on the GaoFen-2, IKONOS, and WorldView-2 datasets at various resolutions. Our approach has advantages over other SOTA pansharpening methods currently available in terms of visual effects and objective metrics. • This paper proposes a novel CNN-DI network with mixture of experts for pansharpening. • CDN-MoE can achieve hybrid training of full- and reduced-resolution datasets with mixture of experts model. • The spatial–spectral adaptive module is built to find the correlation between space and spectrum. [ABSTRACT FROM AUTHOR]
ISSN:15684946
DOI:10.1016/j.asoc.2025.113499