MSCSCformer: Multiscale Convolutional Sparse Coding-Based Transformer for Pansharpening

With the increasing significance of high-quality, high-resolution multispectral images (HRMSs) in various domains, pansharpening, which fuses low-resolution multispectral images (LRMSs) with high-resolution panchromatic (PAN) images, has gained considerable attention. However, current deep-learning...

Celý popis

Uloženo v:

Podrobná bibliografie
Vydáno v:	IEEE transactions on geoscience and remote sensing Ročník 62; s. 1 - 12
Hlavní autoři:	Ye, Yongxu, Wang, Tingting, Fang, Faming, Zhang, Guixu
Médium:	Journal Article
Jazyk:	angličtina
Vydáno:	New York IEEE 2024 The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Témata:	Ablation Coding Convolutional neural networks Deep unfolding network Feature extraction High resolution Image quality Image resolution Information processing Iterative methods multiscale convolution sparse coding (MS-CSC) Optimization Pansharpening pansharpening (PAN) remote sensing Spectral bands Task analysis transformer Transformers
ISSN:	0196-2892, 1558-0644
On-line přístup:	Získat plný text
Tagy:	Přidat tag Žádné tagy, Buďte první, kdo vytvoří štítek k tomuto záznamu!

Abstract	With the increasing significance of high-quality, high-resolution multispectral images (HRMSs) in various domains, pansharpening, which fuses low-resolution multispectral images (LRMSs) with high-resolution panchromatic (PAN) images, has gained considerable attention. However, current deep-learning (DL) methods have limitations in capturing global long-range dependencies and incorporating spectral characteristics across different spectral bands of multispectral (MS) images. Additionally, model-based approaches do not effectively utilize the multiscale information between LRMS and HRMS data, limiting their further performance enhancement. To address these limitations, we propose a new observation model based on multiscale convolutional sparse coding (MS-CSC) and design a novel multiscale hybrid spatial-spectral transformer (MSHST) for the unfolding networks. The MS-CSC-based observation model aims to fuse multiscale information, while the MSHST incorporates spatial self-attention to capture global long-range dependencies and spectral self-attention to capture the interband correlation. Experimental results demonstrate the superiority of our method over other state-of-the-art approaches in both reduced-resolution and full-resolution evaluations. Ablation experiments further validate the effectiveness of the proposed multiscale model and MSHST. Code is available at https://github.com/Eternityyx/MSCSCformer .
AbstractList	With the increasing significance of high-quality, high-resolution multispectral images (HRMSs) in various domains, pansharpening, which fuses low-resolution multispectral images (LRMSs) with high-resolution panchromatic (PAN) images, has gained considerable attention. However, current deep-learning (DL) methods have limitations in capturing global long-range dependencies and incorporating spectral characteristics across different spectral bands of multispectral (MS) images. Additionally, model-based approaches do not effectively utilize the multiscale information between LRMS and HRMS data, limiting their further performance enhancement. To address these limitations, we propose a new observation model based on multiscale convolutional sparse coding (MS-CSC) and design a novel multiscale hybrid spatial–spectral transformer (MSHST) for the unfolding networks. The MS-CSC-based observation model aims to fuse multiscale information, while the MSHST incorporates spatial self-attention to capture global long-range dependencies and spectral self-attention to capture the interband correlation. Experimental results demonstrate the superiority of our method over other state-of-the-art approaches in both reduced-resolution and full-resolution evaluations. Ablation experiments further validate the effectiveness of the proposed multiscale model and MSHST. Code is available at https://github.com/Eternityyx/MSCSCformer .
Author	Wang, Tingting Fang, Faming Zhang, Guixu Ye, Yongxu
Author_xml	– sequence: 1 givenname: Yongxu orcidid: 0000-0003-1914-9780 surname: Ye fullname: Ye, Yongxu email: 51215901060@stu.ecnu.edu organization: School of Computer Science and Technology, East China Normal University, Shanghai, China – sequence: 2 givenname: Tingting orcidid: 0009-0008-8433-8063 surname: Wang fullname: Wang, Tingting email: tingtingwang@cs.ecnu.edu.cn organization: School of Computer Science and Technology, East China Normal University, Shanghai, China – sequence: 3 givenname: Faming orcidid: 0000-0003-4511-4813 surname: Fang fullname: Fang, Faming email: fmfang@cs.ecnu.edu.cn organization: School of Computer Science and Technology, East China Normal University, Shanghai, China – sequence: 4 givenname: Guixu orcidid: 0000-0003-4720-6607 surname: Zhang fullname: Zhang, Guixu email: gxzhang@cs.ecnu.edu.cn organization: School of Computer Science and Technology, East China Normal University, Shanghai, China
BookMark	eNp9kFFLwzAQx4NMcJt-AMGHgs-duaRJE9-06BQ2FDvxsSRtoh1dO5NW8Nvb0j2IDz4dd_x-x91_hiZ1UxuEzgEvALC82ixf0gXBJFpQKoEydoSmwJgIMY-iCZpikDwkQpITNPN-izFEDOIpelunSZrYxu2Muw7WXdWWPleVCZKm_mqqri2bWlVBulfOD8OirN_DW-VNEWycqv1oBn0Jnvv2Q7m9qXvmFB1bVXlzdqhz9Hp_t0kewtXT8jG5WYU5kVEbFlZwBlFOBbOs0MQqwbUEQgxogy3XlvICLOhYUi4V0FwrAC2tAKG1UHSOLse9e9d8dsa32bbpXH-yzyiOZCQkp7Kn4pHKXeO9MzbLy1YNv7VOlVUGOBtSzIYUsyHF7JBib8Ifc-_KnXLf_zoXo1MaY37xDPMYOP0BtpuAlw
CODEN	IGRSD2
CitedBy_id	crossref_primary_10_3390_e27060567 crossref_primary_10_1109_TGRS_2025_3580088 crossref_primary_10_1109_TGRS_2025_3585606 crossref_primary_10_1109_TGRS_2025_3599130 crossref_primary_10_1109_TGRS_2025_3544755 crossref_primary_10_1109_JSTARS_2025_3585751
Cites_doi	10.1016/j.inffus.2012.05.003 10.1109/TGRS.2021.3139190 10.1109/MGRS.2022.3187652 10.1080/01431161.2012.744882 10.1109/JSTARS.2022.3193182 10.1109/CVPR52688.2022.00183 10.1109/MGRS.2022.3170092 10.1109/TGRS.2020.3048257 10.1109/TGRS.2020.3015878 10.1109/CVPR46437.2021.00142 10.1109/TGRS.2022.3197438 10.1109/TIP.2013.2258355 10.1109/TCYB.2019.2910151 10.1109/TGRS.2022.3168465 10.1109/TPAMI.2023.3279050 10.1109/TGRS.2010.2067219 10.1109/TGRS.2020.2964627 10.1137/080716542 10.1016/j.inffus.2021.09.002 10.1109/ICCV.2015.69 10.1109/TGRS.2007.904923 10.1109/TNNLS.2021.3084745 10.1109/JSTARS.2014.2347072 10.1109/TGRS.2010.2051674 10.1109/IGARSS.2017.8128408 10.1109/TGRS.2019.2906073 10.1049/el:20080522 10.1109/LGRS.2007.909934 10.1109/TGRS.2022.3225563 10.1109/MGRS.2022.3145854 10.1109/LGRS.2013.2281996 10.1109/TGRS.2007.901007 10.1109/TGRS.2022.3152425 10.1109/TPAMI.2015.2439281 10.1109/TGRS.2022.3227405 10.1109/TGRS.2022.3163887 10.1109/tgrs.2021.3115501 10.3390/rs8070594 10.1109/TGRS.2020.3007884 10.1109/ICCV48922.2021.00986 10.1109/TIP.2003.819861 10.1109/CVPR.2018.00695 10.1016/j.inffus.2020.04.006 10.1109/TGRS.2014.2361734 10.1109/TIP.2018.2866954 10.1109/CVPR52688.2022.00564 10.1109/tnnls.2023.3313202 10.1109/CVPR52688.2022.00181
ContentType	Journal Article
Copyright	Copyright The Institute of Electrical and Electronics Engineers, Inc. (IEEE) 2024
Copyright_xml	– notice: Copyright The Institute of Electrical and Electronics Engineers, Inc. (IEEE) 2024
DBID	97E RIA RIE AAYXX CITATION 7UA 8FD C1K F1W FR3 H8D H96 KR7 L.G L7M
DOI	10.1109/TGRS.2024.3391355
DatabaseName	IEEE Xplore (IEEE) IEEE All-Society Periodicals Package (ASPP) 1998–Present IEEE Electronic Library (IEL) CrossRef Water Resources Abstracts Technology Research Database Environmental Sciences and Pollution Management ASFA: Aquatic Sciences and Fisheries Abstracts Engineering Research Database Aerospace Database Aquatic Science & Fisheries Abstracts (ASFA) 2: Ocean Technology, Policy & Non-Living Resources Civil Engineering Abstracts Aquatic Science & Fisheries Abstracts (ASFA) Professional Advanced Technologies Database with Aerospace
DatabaseTitle	CrossRef Aerospace Database Civil Engineering Abstracts Aquatic Science & Fisheries Abstracts (ASFA) Professional Aquatic Science & Fisheries Abstracts (ASFA) 2: Ocean Technology, Policy & Non-Living Resources Technology Research Database ASFA: Aquatic Sciences and Fisheries Abstracts Engineering Research Database Advanced Technologies Database with Aerospace Water Resources Abstracts Environmental Sciences and Pollution Management
DatabaseTitleList	Aerospace Database
Database_xml	– sequence: 1 dbid: RIE name: IEEE Xplore url: https://ieeexplore.ieee.org/ sourceTypes: Publisher
DeliveryMethod	fulltext_linktorsrc
Discipline	Engineering Physics
EISSN	1558-0644
EndPage	12
ExternalDocumentID	10_1109_TGRS_2024_3391355 10506716
Genre	orig-research
GrantInformation_xml	– fundername: National Key Research and Development Program of China grantid: 2022ZD0161800 funderid: 10.13039/501100012166 – fundername: National Natural Science Foundation of China grantid: 62271203; 62202173 funderid: 10.13039/501100001809 – fundername: Shanghai Rising-Star Program grantid: 21QA1402500 funderid: 10.13039/501100013105
GroupedDBID	-~X 0R~ 29I 4.4 5GY 5VS 6IK 97E AAJGR AARMG AASAJ AAWTH ABAZT ABQJQ ABVLG ACGFO ACGFS ACIWK ACNCT AENEX AETIX AFRAH AGQYO AGSQL AHBIQ AI. AIBXA AKJIK AKQYR ALLEH ALMA_UNASSIGNED_HOLDINGS ASUFR ATWAV BEFXN BFFAM BGNUA BKEBE BPEOZ CS3 DU5 EBS EJD F5P HZ~ H~9 IBMZZ ICLAB IFIPE IFJZH IPLJI JAVBF LAI M43 O9- OCL P2P RIA RIE RNS RXW TAE TN5 VH1 Y6R AAYXX CITATION 7UA 8FD C1K F1W FR3 H8D H96 KR7 L.G L7M
ID	FETCH-LOGICAL-c294t-df86514c385f5db2fa86b9122e1be0f6bf36d1f1b79369a13cba11b9f818bb8a3
IEDL.DBID	RIE
ISICitedReferencesCount	11
ISICitedReferencesURI	http://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=Summon&SrcAuth=ProQuest&DestLinkType=CitingArticles&DestApp=WOS_CPL&KeyUT=001214652600018&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D
ISSN	0196-2892
IngestDate	Mon Jun 30 08:22:19 EDT 2025 Sat Nov 29 03:32:35 EST 2025 Tue Nov 18 22:28:56 EST 2025 Wed Aug 27 02:06:29 EDT 2025
IsPeerReviewed	true
IsScholarly	true
Language	English
License	https://ieeexplore.ieee.org/Xplorehelp/downloads/license-information/IEEE.html https://doi.org/10.15223/policy-029 https://doi.org/10.15223/policy-037
LinkModel	DirectLink
MergedId	FETCHMERGED-LOGICAL-c294t-df86514c385f5db2fa86b9122e1be0f6bf36d1f1b79369a13cba11b9f818bb8a3
Notes	ObjectType-Article-1 SourceType-Scholarly Journals-1 ObjectType-Feature-2 content type line 14
ORCID	0000-0003-1914-9780 0000-0003-4511-4813 0009-0008-8433-8063 0000-0003-4720-6607
PQID	3049489639
PQPubID	85465
PageCount	12
ParticipantIDs	ieee_primary_10506716 crossref_citationtrail_10_1109_TGRS_2024_3391355 crossref_primary_10_1109_TGRS_2024_3391355 proquest_journals_3049489639
PublicationCentury	2000
PublicationDate	20240000 2024-00-00 20240101
PublicationDateYYYYMMDD	2024-01-01
PublicationDate_xml	– year: 2024 text: 20240000
PublicationDecade	2020
PublicationPlace	New York
PublicationPlace_xml	– name: New York
PublicationTitle	IEEE transactions on geoscience and remote sensing
PublicationTitleAbbrev	TGRS
PublicationYear	2024
Publisher	IEEE The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Publisher_xml	– name: IEEE – name: The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
References	ref13 ref12 ref15 ref14 ref52 ref11 ref17 ref16 ref19 ref18 Yuhas (ref47); 1 Wald (ref43) 1997; 63 Schowengerdt (ref10) 2006 ref51 ref46 ref45 ref48 ref42 ref41 ref44 ref49 ref8 ref7 ref9 ref4 ref3 ref6 ref5 ref40 ref35 ref34 ref37 ref36 ref31 ref30 ref33 ref32 Dosovitskiy (ref50) ref2 ref1 ref39 ref38 ref24 ref23 ref26 ref25 ref20 ref22 ref21 ref28 ref27 ref29
References_xml	– ident: ref3 doi: 10.1016/j.inffus.2012.05.003 – ident: ref39 doi: 10.1109/TGRS.2021.3139190 – ident: ref44 doi: 10.1109/MGRS.2022.3187652 – ident: ref14 doi: 10.1080/01431161.2012.744882 – ident: ref18 doi: 10.1109/JSTARS.2022.3193182 – ident: ref42 doi: 10.1109/CVPR52688.2022.00183 – ident: ref49 doi: 10.1109/MGRS.2022.3170092 – ident: ref20 doi: 10.1109/TGRS.2020.3048257 – ident: ref23 doi: 10.1109/TGRS.2020.3015878 – ident: ref35 doi: 10.1109/CVPR46437.2021.00142 – ident: ref36 doi: 10.1109/TGRS.2022.3197438 – ident: ref13 doi: 10.1109/TIP.2013.2258355 – ident: ref15 doi: 10.1109/TCYB.2019.2910151 – ident: ref28 doi: 10.1109/TGRS.2022.3168465 – ident: ref34 doi: 10.1109/TPAMI.2023.3279050 – ident: ref16 doi: 10.1109/TGRS.2010.2067219 – ident: ref2 doi: 10.1109/TGRS.2020.2964627 – ident: ref41 doi: 10.1137/080716542 – ident: ref25 doi: 10.1016/j.inffus.2021.09.002 – ident: ref19 doi: 10.1109/ICCV.2015.69 – ident: ref48 doi: 10.1109/TGRS.2007.904923 – ident: ref31 doi: 10.1109/TNNLS.2021.3084745 – ident: ref17 doi: 10.1109/JSTARS.2014.2347072 – ident: ref6 doi: 10.1109/TGRS.2010.2051674 – ident: ref4 doi: 10.1109/IGARSS.2017.8128408 – ident: ref8 doi: 10.1109/TGRS.2019.2906073 – volume-title: Remote Sensing: Models and Methods for Image Processing year: 2006 ident: ref10 – ident: ref45 doi: 10.1049/el:20080522 – ident: ref9 doi: 10.1109/LGRS.2007.909934 – ident: ref38 doi: 10.1109/TGRS.2022.3225563 – ident: ref1 doi: 10.1109/MGRS.2022.3145854 – ident: ref11 doi: 10.1109/LGRS.2013.2281996 – ident: ref7 doi: 10.1109/TGRS.2007.901007 – ident: ref29 doi: 10.1109/TGRS.2022.3152425 – ident: ref24 doi: 10.1109/TPAMI.2015.2439281 – ident: ref30 doi: 10.1109/TGRS.2022.3227405 – ident: ref33 doi: 10.1109/TGRS.2022.3163887 – ident: ref37 doi: 10.1109/tgrs.2021.3115501 – ident: ref21 doi: 10.3390/rs8070594 – ident: ref22 doi: 10.1109/TGRS.2020.3007884 – ident: ref52 doi: 10.1109/ICCV48922.2021.00986 – ident: ref46 doi: 10.1109/TIP.2003.819861 – volume: 1 start-page: 1 volume-title: Proc. JPL, Summaries 3rd Annu. JPL Airborne Geosci. Workshop. AVIRIS Workshop ident: ref47 article-title: Discrimination among semi-arid landscape endmembers using the spectral angle mapper (SAM) algorithm – ident: ref40 doi: 10.1109/CVPR.2018.00695 – ident: ref32 doi: 10.1016/j.inffus.2020.04.006 – start-page: 1 volume-title: Proc. ICLR ident: ref50 article-title: An image is worth 16×16 words: Transformers for image recognition at scale – ident: ref5 doi: 10.1109/TGRS.2014.2361734 – ident: ref12 doi: 10.1109/TIP.2018.2866954 – volume: 63 start-page: 691 issue: 6 year: 1997 ident: ref43 article-title: Fusion of satellite images of different spatial resolutions: Assessing the quality of resulting images publication-title: Photogramm. Eng. Remote Sens. – ident: ref51 doi: 10.1109/CVPR52688.2022.00564 – ident: ref27 doi: 10.1109/tnnls.2023.3313202 – ident: ref26 doi: 10.1109/CVPR52688.2022.00181
SSID	ssj0014517
Score	2.4924226
Snippet	With the increasing significance of high-quality, high-resolution multispectral images (HRMSs) in various domains, pansharpening, which fuses low-resolution...
SourceID	proquest crossref ieee
SourceType	Aggregation Database Enrichment Source Index Database Publisher
StartPage	1
SubjectTerms	Ablation Coding Convolutional neural networks Deep unfolding network Feature extraction High resolution Image quality Image resolution Information processing Iterative methods multiscale convolution sparse coding (MS-CSC) Optimization Pansharpening pansharpening (PAN) remote sensing Spectral bands Task analysis transformer Transformers
Title	MSCSCformer: Multiscale Convolutional Sparse Coding-Based Transformer for Pansharpening
URI	https://ieeexplore.ieee.org/document/10506716 https://www.proquest.com/docview/3049489639
Volume	62
WOSCitedRecordID	wos001214652600018&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D
hasFullText	1
inHoldings	1
isFullTextHit
isPrint
journalDatabaseRights	– providerCode: PRVIEE databaseName: IEEE Xplore customDbUrl: eissn: 1558-0644 dateEnd: 99991231 omitProxy: false ssIdentifier: ssj0014517 issn: 0196-2892 databaseCode: RIE dateStart: 19800101 isFulltext: true titleUrlDefault: https://ieeexplore.ieee.org/ providerName: IEEE
link	http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV1LTxsxEB6VCCQ4AKVUDYVqDz1VMl2vd712b20E9NAi1KQtt5UfY_VQJSiB_H7GXgdFQiBx2odsa7Wf7Xn5mwH4iA132rWSOWMMq0XwzCiLrGl1aY1ufZ2ItH9-tJeX6vpaX2WyeuLCIGI6fIan8TbF8v3M3UVXGa3whjZXLjdgo21lT9Z6CBnUDc_caMnIiqhyCJOX-vPk4teYTMGqPhVCcxFpfWtCKFVVebQVJ_lyvvfCL9uH3axIFl975F_DK5wewM5aesED2ErHO93iDfz9OR6NR1E_xfmXIpFuFwQOFqPZdJknHw02viEzN76MAo19IwHni8lKs8V5QZfiih7_xaBNdKgcwu_zs8noO8slFZirdH3LfFCSVCQnVBMab6tglLSaVxVyi2WQNgjpeeC2jYX-DBfOGs6tDiTXrVVGvIXBdDbFd1B4NBzJ2CGbDWuNXilH6qNqGl_XRrowhHL1jzuX843Hshf_u2R3lLqLsHQRli7DMoRPD11u-mQbzzU-jDisNewhGMLxCskur8dFJ1IaHNps9NET3d7Ddhy9964cw-B2focnsOmWBMj8Q5pq9yIU0YE
linkProvider	IEEE
linkToHtml	http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV1Lb9QwEB5BAQEHHqWIhQI5cEJyG8d2YnODFaWI7apiF-gt8mMsDmi32m37-xk73molBBKnPGQnUT7b8_I3A_AGFffGdy3z1lomRQzMaodMdaZ21nRBZiLt90k3neqzM3NayOqZC4OIefMZHqTTHMsPS3-ZXGU0wxUtrry9CbeUlE090LWugwZS8cKObhnZEU0JYvLaHM4_fZ2RMdjIAyEMF4nYtyWGcl2VPxbjLGGOHv7ntz2CB0WVrN4P2D-GG7jYhftbCQZ34U7e4OnXT-DHyWw8GycNFVfvqky7XRM8WI2Xi6sy_Ohhs3MydNPNJNLYBxJxoZpvdFtcVXSoTunyZwrbJJfKHnw7-jgfH7NSVIH5xsgLFqJuSUnyQquogmui1a0zvGmQO6xj66JoA4_cdanUn-XCO8u5M5Eku3Paiqews1gu8BlUAS1HMnfIakNpMGjtSYHUSgUpbevjCOrNP-59yTieCl_86rPlUZs-wdInWPoCywjeXnc5H9Jt_KvxXsJhq-EAwQj2N0j2ZUaue5ET4dByY57_pdtruHs8P5n0k8_TLy_gXnrT4GvZh52L1SW-hNv-isBZvcrD7jcxgtTI
openUrl	ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=MSCSCformer%3A+Multiscale+Convolutional+Sparse+Coding-Based+Transformer+for+Pansharpening&rft.jtitle=IEEE+transactions+on+geoscience+and+remote+sensing&rft.au=Ye%2C+Yongxu&rft.au=Wang%2C+Tingting&rft.au=Fang%2C+Faming&rft.au=Zhang%2C+Guixu&rft.date=2024&rft.pub=IEEE&rft.issn=0196-2892&rft.volume=62&rft.spage=1&rft.epage=12&rft_id=info:doi/10.1109%2FTGRS.2024.3391355&rft.externalDocID=10506716
thumbnail_l	http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/lc.gif&issn=0196-2892&client=summon
thumbnail_m	http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/mc.gif&issn=0196-2892&client=summon
thumbnail_s	http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/sc.gif&issn=0196-2892&client=summon