MSCSCformer: Multiscale Convolutional Sparse Coding-Based Transformer for Pansharpening

With the increasing significance of high-quality, high-resolution multispectral images (HRMSs) in various domains, pansharpening, which fuses low-resolution multispectral images (LRMSs) with high-resolution panchromatic (PAN) images, has gained considerable attention. However, current deep-learning...

Full description

Saved in:
Bibliographic Details
Published in:IEEE transactions on geoscience and remote sensing Vol. 62; pp. 1 - 12
Main Authors: Ye, Yongxu, Wang, Tingting, Fang, Faming, Zhang, Guixu
Format: Journal Article
Language:English
Published: New York IEEE 2024
The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Subjects:
Ablation
Coding
Convolutional neural networks
Deep unfolding network
Feature extraction
High resolution
Image quality
Image resolution
Information processing
Iterative methods
multiscale convolution sparse coding (MS-CSC)
Optimization
Pansharpening
pansharpening (PAN)
remote sensing
Spectral bands
Task analysis
transformer
Transformers
ISSN:0196-2892, 1558-0644
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:With the increasing significance of high-quality, high-resolution multispectral images (HRMSs) in various domains, pansharpening, which fuses low-resolution multispectral images (LRMSs) with high-resolution panchromatic (PAN) images, has gained considerable attention. However, current deep-learning (DL) methods have limitations in capturing global long-range dependencies and incorporating spectral characteristics across different spectral bands of multispectral (MS) images. Additionally, model-based approaches do not effectively utilize the multiscale information between LRMS and HRMS data, limiting their further performance enhancement. To address these limitations, we propose a new observation model based on multiscale convolutional sparse coding (MS-CSC) and design a novel multiscale hybrid spatial-spectral transformer (MSHST) for the unfolding networks. The MS-CSC-based observation model aims to fuse multiscale information, while the MSHST incorporates spatial self-attention to capture global long-range dependencies and spectral self-attention to capture the interband correlation. Experimental results demonstrate the superiority of our method over other state-of-the-art approaches in both reduced-resolution and full-resolution evaluations. Ablation experiments further validate the effectiveness of the proposed multiscale model and MSHST. Code is available at https://github.com/Eternityyx/MSCSCformer .
Bibliography:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
ISSN:0196-2892
1558-0644
DOI:10.1109/TGRS.2024.3391355