MSSL: Hyperspectral and Panchromatic Images Fusion via Multiresolution Spatial-Spectral Feature Learning Networks

The fusion of hyperspectral (HS) and panchromatic (PAN) images aims to generate a fused HS image that combines spectral information of the HS image with spatial information of the PAN image. In this article, we propose a multiresolution spatial-spectral feature learning (MSSL) framework for fusing H...

Ausführliche Beschreibung

Gespeichert in:

Bibliographische Detailangaben
Veröffentlicht in:	IEEE transactions on geoscience and remote sensing Jg. 60; S. 1 - 13
Hauptverfasser:	Qu, Jiahui, Shi, Yanzi, Xie, Weiying, Li, Yunsong, Wu, Xianyun, Du, Qian
Format:	Journal Article
Sprache:	Englisch
Veröffentlicht:	New York IEEE 2022 The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Schlagworte:	Bayes methods Computer applications Computer vision Convolutional autoencoder (CAE) Data mining Feature extraction hyperspectral (HS) pansharpening image fusion Image processing Image reconstruction Image resolution Learning multiresolution Pansharpening Remote sensing Spatial data Spatial discrimination learning Spatial resolution spatial–spectral feature Spectra
ISSN:	0196-2892, 1558-0644
Online-Zugang:	Volltext
Tags:	Tag hinzufügen Keine Tags, Fügen Sie den ersten Tag hinzu!

Abstract	The fusion of hyperspectral (HS) and panchromatic (PAN) images aims to generate a fused HS image that combines spectral information of the HS image with spatial information of the PAN image. In this article, we propose a multiresolution spatial-spectral feature learning (MSSL) framework for fusing HS and PAN images. The proposed MSSL transforms the existing deep and complex network into several simple and shallow subnetworks to simplify the feature learning process. MSSL upsamples the HS image while downsamples the PAN image and designs multiresolution 3-D convolutional autoencoder (CAEs) networks with a spectral constraint to learn complete spatial-spectral features of the HS image. MSSL designs multiresolution 2-D CAEs with spatial constraint to extract spatial features of the PAN image, with a low computational cost. In order to effectively generate the pansharpened HS image with high spatial and spectral fidelity, a multiresolution residual network is presented to reconstruct the HS image from the extracted spatial-spectral features. Extensive experiments are conducted on three widely used remote sensing data sets in comparison with state-of-the-art HS image fusion methods, demonstrating the superiority of the proposed MSSL method. Code is available at https://github.com/Jiahuiqu/MSSL .
AbstractList	The fusion of hyperspectral (HS) and panchromatic (PAN) images aims to generate a fused HS image that combines spectral information of the HS image with spatial information of the PAN image. In this article, we propose a multiresolution spatial–spectral feature learning (MSSL) framework for fusing HS and PAN images. The proposed MSSL transforms the existing deep and complex network into several simple and shallow subnetworks to simplify the feature learning process. MSSL upsamples the HS image while downsamples the PAN image and designs multiresolution 3-D convolutional autoencoder (CAEs) networks with a spectral constraint to learn complete spatial–spectral features of the HS image. MSSL designs multiresolution 2-D CAEs with spatial constraint to extract spatial features of the PAN image, with a low computational cost. In order to effectively generate the pansharpened HS image with high spatial and spectral fidelity, a multiresolution residual network is presented to reconstruct the HS image from the extracted spatial–spectral features. Extensive experiments are conducted on three widely used remote sensing data sets in comparison with state-of-the-art HS image fusion methods, demonstrating the superiority of the proposed MSSL method. Code is available at https://github.com/Jiahuiqu/MSSL .
Author	Qu, Jiahui Shi, Yanzi Xie, Weiying Li, Yunsong Wu, Xianyun Du, Qian
Author_xml	– sequence: 1 givenname: Jiahui orcidid: 0000-0002-3925-2884 surname: Qu fullname: Qu, Jiahui email: jhqu@xidian.edu.cn organization: State Key Laboratory of Integrated Service Network, Xidian University, Xi'an, China – sequence: 2 givenname: Yanzi orcidid: 0000-0002-7717-985X surname: Shi fullname: Shi, Yanzi email: yzshi_xidian@163.com organization: State Key Laboratory of Integrated Service Network, Xidian University, Xi'an, China – sequence: 3 givenname: Weiying orcidid: 0000-0001-8310-024X surname: Xie fullname: Xie, Weiying email: wyxie@xidian.edu.cn organization: State Key Laboratory of Integrated Service Network, Xidian University, Xi'an, China – sequence: 4 givenname: Yunsong orcidid: 0000-0002-0234-6270 surname: Li fullname: Li, Yunsong email: ysli@mail.xidian.edu.cn organization: State Key Laboratory of Integrated Service Network, Xidian University, Xi'an, China – sequence: 5 givenname: Xianyun orcidid: 0000-0002-4450-3801 surname: Wu fullname: Wu, Xianyun email: xywu@mail.xidian.edu.cn organization: State Key Laboratory of Integrated Service Network, Xidian University, Xi'an, China – sequence: 6 givenname: Qian orcidid: 0000-0001-8354-7500 surname: Du fullname: Du, Qian email: du@ece.msstate.edu organization: Department of Electronic and Computer Engineering, Mississippi State University, Starkville, MS, USA
BookMark	eNp9kMFOwzAQRC0EEqXwAYiLJc4pXttxEm4IUUBqARE4R467LS5pEmwHxN-TqMCBA6eVRvNmNXNAduumRkKOgU0AWHb2dP2YTzjjMBFMKZHIHTKCOE4jpqTcJSMGmYp4mvF9cuD9mjGQMSQj8jbP89k5vfls0fkWTXC6orpe0AddmxfXbHSwht5u9Ao9nXbeNjV9t5rOuypYh76pujBoedsbdRXlPxlT1KFzSGeoXW3rFb3D8NG4V39I9pa68nj0fcfkeXr1dHkTze6vby8vZpHhiQyR0CzlpVJZ3yYVmElUUpWgMobIRclKWcYGYGnKRQqx5qbXJI85clzoZanFmJxuc1vXvHXoQ7FuOlf3LwuugEHC0kT0rmTrMq7x3uGyMDbooVLfwlYFsGLYtxj2LYZ9i-99exL-kK2zG-0-_2VOtoxFxF9_JlLFJYgvqCiJoQ
CODEN	IGRSD2
CitedBy_id	crossref_primary_10_1109_LGRS_2021_3094216 crossref_primary_10_1109_JAS_2023_123681 crossref_primary_10_1109_TGRS_2022_3189624 crossref_primary_10_3390_rs13183674 crossref_primary_10_1109_TGRS_2025_3583877 crossref_primary_10_1109_TGRS_2025_3592011 crossref_primary_10_1109_JSTARS_2022_3220974 crossref_primary_10_1109_TGRS_2021_3133670 crossref_primary_10_3390_rs17111947 crossref_primary_10_1109_TGRS_2022_3146296 crossref_primary_10_1109_MGRS_2024_3495516 crossref_primary_10_1109_TGRS_2023_3339337 crossref_primary_10_1109_TGRS_2022_3224480 crossref_primary_10_1109_TGRS_2024_3500036 crossref_primary_10_1109_MGRS_2024_3509139 crossref_primary_10_3390_rs15112853 crossref_primary_10_1109_TGRS_2023_3244992
Cites_doi	10.14358/PERS.72.5.591 10.1109/TII.2020.3024578 10.1109/TPAMI.2015.2439281 10.1109/LGRS.2017.2736020 10.1109/LGRS.2018.2878773 10.1109/TGRS.2020.2966550 10.1007/978-1-4612-2544-7_17 10.1109/TGRS.2007.901007 10.1109/34.192463 10.1109/LGRS.2013.2281996 10.1109/TPAMI.2016.2644615 10.1109/TGRS.2019.2917759 10.1109/LGRS.2017.2751559 10.1109/TIP.2015.2458572 10.1109/TGRS.2014.2361734 10.1109/JSTARS.2015.2420582 10.1109/TGRS.2008.916211 10.1109/TGRS.2014.2375320 10.1109/TGRS.2019.2960391 10.1111/j.1477-9730.2006.00353.x 10.1109/TGRS.2007.904923 10.1109/JSTSP.2015.2407855 10.1109/TGRS.2007.912448 10.1109/LGRS.2017.2668299 10.1109/TGRS.2019.2949082 10.7551/mitpress/7503.003.0024 10.1109/TCOM.1983.1095851 10.1080/014311600750037499 10.1016/j.rse.2002.08.002 10.1109/78.157290 10.1109/TGRS.2011.2161320 10.3390/rs8070594 10.1016/S1566-2535(01)00036-7 10.1109/TGRS.2020.2982940 10.1109/TGRS.2020.2965961
ContentType	Journal Article
Copyright	Copyright The Institute of Electrical and Electronics Engineers, Inc. (IEEE) 2022
Copyright_xml	– notice: Copyright The Institute of Electrical and Electronics Engineers, Inc. (IEEE) 2022
DBID	97E RIA RIE AAYXX CITATION 7UA 8FD C1K F1W FR3 H8D H96 KR7 L.G L7M
DOI	10.1109/TGRS.2021.3066374
DatabaseName	IEEE All-Society Periodicals Package (ASPP) 2005–Present IEEE All-Society Periodicals Package (ASPP) 1998–Present IEEE Electronic Library (IEL) CrossRef Water Resources Abstracts Technology Research Database Environmental Sciences and Pollution Management ASFA: Aquatic Sciences and Fisheries Abstracts Engineering Research Database Aerospace Database Aquatic Science & Fisheries Abstracts (ASFA) 2: Ocean Technology, Policy & Non-Living Resources Civil Engineering Abstracts Aquatic Science & Fisheries Abstracts (ASFA) Professional Advanced Technologies Database with Aerospace
DatabaseTitle	CrossRef Aerospace Database Civil Engineering Abstracts Aquatic Science & Fisheries Abstracts (ASFA) Professional Aquatic Science & Fisheries Abstracts (ASFA) 2: Ocean Technology, Policy & Non-Living Resources Technology Research Database ASFA: Aquatic Sciences and Fisheries Abstracts Engineering Research Database Advanced Technologies Database with Aerospace Water Resources Abstracts Environmental Sciences and Pollution Management
DatabaseTitleList	Aerospace Database
Database_xml	– sequence: 1 dbid: RIE name: IEEE Electronic Library (IEL) url: https://ieeexplore.ieee.org/ sourceTypes: Publisher
DeliveryMethod	fulltext_linktorsrc
Discipline	Engineering Physics
EISSN	1558-0644
EndPage	13
ExternalDocumentID	10_1109_TGRS_2021_3066374 9386241
Genre	orig-research
GrantInformation_xml	– fundername: National Natural Science Foundation of China grantid: 61571345; 91538101; 61501346; 61502367; 61701360 funderid: 10.13039/501100001809 – fundername: Open Fund of Key Laboratory of Geospatial Technology for the Middle and Lower Yellow River Regions (Henan University), Ministry of Education grantid: GTYR201904 – fundername: Ten Thousand Talent Program of the Natural Science Basic Research Plan in Shaanxi Province of China grantid: 2016JQ6023 funderid: 10.13039/501100017596 – fundername: Supported by Yangtze River Scholar Bonus Schemes of China grantid: CJT160102 – fundername: Higher Education Discipline Innovation Project; 111 Project grantid: B08038 funderid: 10.13039/501100013314
GroupedDBID	-~X 0R~ 29I 4.4 5GY 5VS 6IK 97E AAJGR AARMG AASAJ AAWTH ABAZT ABQJQ ABVLG ACGFO ACGFS ACIWK ACNCT AENEX AETIX AFRAH AGQYO AGSQL AHBIQ AI. AIBXA AKJIK AKQYR ALLEH ALMA_UNASSIGNED_HOLDINGS ASUFR ATWAV BEFXN BFFAM BGNUA BKEBE BPEOZ CS3 DU5 EBS EJD F5P HZ~ H~9 IBMZZ ICLAB IFIPE IFJZH IPLJI JAVBF LAI M43 O9- OCL P2P RIA RIE RNS RXW TAE TN5 VH1 Y6R AAYXX CITATION 7UA 8FD C1K F1W FR3 H8D H96 KR7 L.G L7M
ID	FETCH-LOGICAL-c274t-3a082b66966383e94e646b1690ee23b0b4b5c11fcbd815a2c3b04252e2edafba3
IEDL.DBID	RIE
ISSN	0196-2892
IngestDate	Tue Aug 26 15:40:28 EDT 2025 Tue Nov 18 22:24:33 EST 2025 Sat Nov 29 02:50:12 EST 2025 Wed Aug 27 05:07:52 EDT 2025
IsPeerReviewed	true
IsScholarly	true
Language	English
License	https://ieeexplore.ieee.org/Xplorehelp/downloads/license-information/IEEE.html https://doi.org/10.15223/policy-029 https://doi.org/10.15223/policy-037
LinkModel	DirectLink
MergedId	FETCHMERGED-LOGICAL-c274t-3a082b66966383e94e646b1690ee23b0b4b5c11fcbd815a2c3b04252e2edafba3
Notes	ObjectType-Article-1 SourceType-Scholarly Journals-1 ObjectType-Feature-2 content type line 14
ORCID	0000-0002-0234-6270 0000-0002-4450-3801 0000-0001-8354-7500 0000-0002-3925-2884 0000-0002-7717-985X 0000-0001-8310-024X
PQID	2610170873
PQPubID	85465
PageCount	13
ParticipantIDs	crossref_citationtrail_10_1109_TGRS_2021_3066374 crossref_primary_10_1109_TGRS_2021_3066374 proquest_journals_2610170873 ieee_primary_9386241
PublicationCentury	2000
PublicationDate	20220000 2022-00-00 20220101
PublicationDateYYYYMMDD	2022-01-01
PublicationDate_xml	– year: 2022 text: 20220000
PublicationDecade	2020
PublicationPlace	New York
PublicationPlace_xml	– name: New York
PublicationTitle	IEEE transactions on geoscience and remote sensing
PublicationTitleAbbrev	TGRS
PublicationYear	2022
Publisher	IEEE The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Publisher_xml	– name: IEEE – name: The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
References	ref13 ref35 ref34 ref15 ref14 ref36 Shettigara (ref10) 1992; 58 ref31 ref30 ref11 ref33 ref32 ref2 ref1 Chavez (ref9) 1989; 55 ref17 ref16 ref19 ref18 Laben (ref12) 2000 Wald (ref39) ref24 ref23 ref26 ref25 ref20 Yuhas (ref37) ref41 ref22 ref21 ref28 ref27 ref29 ref8 ref7 ref4 ref3 ref6 Wald (ref38) 1997; 63 ref5 ref40
References_xml	– ident: ref21 doi: 10.14358/PERS.72.5.591 – ident: ref29 doi: 10.1109/TII.2020.3024578 – ident: ref32 doi: 10.1109/TPAMI.2015.2439281 – ident: ref34 doi: 10.1109/LGRS.2017.2736020 – ident: ref31 doi: 10.1109/LGRS.2018.2878773 – ident: ref15 doi: 10.1109/TGRS.2020.2966550 – volume: 63 start-page: 691 issue: 6 year: 1997 ident: ref38 article-title: Fusion of satellite images of different spatial resolutions: Assessing the quality of resulting images publication-title: Photogramm. Eng. Remote Sens. – ident: ref17 doi: 10.1007/978-1-4612-2544-7_17 – ident: ref13 doi: 10.1109/TGRS.2007.901007 – ident: ref16 doi: 10.1109/34.192463 – volume: 55 start-page: 339 issue: 3 year: 1989 ident: ref9 article-title: Extracting spectral contrast in Landsat thematic mapper image data using selective principal component analysis publication-title: Photogramm. Eng. Remote Sens. – ident: ref22 doi: 10.1109/LGRS.2013.2281996 – volume-title: Process for enhancing the spatial resolution of multispectral imagery using pan-sharpening year: 2000 ident: ref12 – ident: ref30 doi: 10.1109/TPAMI.2016.2644615 – ident: ref41 doi: 10.1109/TGRS.2019.2917759 – ident: ref5 doi: 10.1109/LGRS.2017.2751559 – ident: ref26 doi: 10.1109/TIP.2015.2458572 – ident: ref8 doi: 10.1109/TGRS.2014.2361734 – ident: ref40 doi: 10.1109/JSTARS.2015.2420582 – ident: ref11 doi: 10.1109/TGRS.2008.916211 – ident: ref25 doi: 10.1109/TGRS.2014.2375320 – ident: ref4 doi: 10.1109/TGRS.2019.2960391 – ident: ref2 doi: 10.1111/j.1477-9730.2006.00353.x – ident: ref23 doi: 10.1109/TGRS.2007.904923 – ident: ref24 doi: 10.1109/JSTSP.2015.2407855 – ident: ref14 doi: 10.1109/TGRS.2007.912448 – ident: ref35 doi: 10.1109/LGRS.2017.2668299 – start-page: 99 volume-title: Proc. Int. Conf. Fusion Earth Data. ident: ref39 article-title: Quality of high resolution synthesised images: Is there a simple criterion? – ident: ref6 doi: 10.1109/TGRS.2019.2949082 – ident: ref36 doi: 10.7551/mitpress/7503.003.0024 – ident: ref19 doi: 10.1109/TCOM.1983.1095851 – ident: ref20 doi: 10.1080/014311600750037499 – start-page: 147 volume-title: Proc. 3rd Annu. JPL Airborne Geosci. Workshop ident: ref37 article-title: Discrimination among semi-arid landscape endmembers using the spectral angle mapper (SAM) algorithm – ident: ref1 doi: 10.1016/j.rse.2002.08.002 – ident: ref18 doi: 10.1109/78.157290 – ident: ref27 doi: 10.1109/TGRS.2011.2161320 – ident: ref33 doi: 10.3390/rs8070594 – ident: ref7 doi: 10.1016/S1566-2535(01)00036-7 – ident: ref28 doi: 10.1109/TGRS.2020.2982940 – ident: ref3 doi: 10.1109/TGRS.2020.2965961 – volume: 58 start-page: 561 issue: 5 year: 1992 ident: ref10 article-title: A generalized component substitution technique for spatial enhancement of multispectral images using a higher resolution data set publication-title: Photogram. Eng. Remote Sens.
SSID	ssj0014517
Score	2.4746225
Snippet	The fusion of hyperspectral (HS) and panchromatic (PAN) images aims to generate a fused HS image that combines spectral information of the HS image with...
SourceID	proquest crossref ieee
SourceType	Aggregation Database Enrichment Source Index Database Publisher
StartPage	1
SubjectTerms	Bayes methods Computer applications Computer vision Convolutional autoencoder (CAE) Data mining Feature extraction hyperspectral (HS) pansharpening image fusion Image processing Image reconstruction Image resolution Learning multiresolution Pansharpening Remote sensing Spatial data Spatial discrimination learning Spatial resolution spatial–spectral feature Spectra
Title	MSSL: Hyperspectral and Panchromatic Images Fusion via Multiresolution Spatial-Spectral Feature Learning Networks
URI	https://ieeexplore.ieee.org/document/9386241 https://www.proquest.com/docview/2610170873
Volume	60
hasFullText	1
inHoldings	1
isFullTextHit
isPrint
journalDatabaseRights	– providerCode: PRVIEE databaseName: IEEE Electronic Library (IEL) customDbUrl: eissn: 1558-0644 dateEnd: 99991231 omitProxy: false ssIdentifier: ssj0014517 issn: 0196-2892 databaseCode: RIE dateStart: 19800101 isFulltext: true titleUrlDefault: https://ieeexplore.ieee.org/ providerName: IEEE
link	http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV1LS8NAEB5UFPTgo1WsL_bgSdyad7LeRKwKWsQq9Bb2FS1oq0nr73d2sy2KIngLIRtCvtnd-Wb2mwE4NB2sQ49H1BOK0aiQjHLFCyqUCfOjh1LEhW02kXa7Wb_P7ubgeKaF0Vrbw2e6bS5tLl-N5MSEyk5YaOQMyHXm0zSptVqzjEEU-04anVAkEYHLYPoeO3m4vO8hEwz8dmg22DT6tgfZpio_VmK7vXTW_vdh67Dq3EhyVuO-AXN62ICVL8UFG7BkD3fKqgnvt73ezSm5QspZKytLHMqHitwh5M_lyFZtJdevuLRUpDMx8TPyMeDEinORjjvrJKZ9MZor7U3fYfzHSamJK9L6RLr1ofJqEx47Fw_nV9S1WqASaemYhhxdAZEkSH6QsmoW6SRKhEmhaR2EwhORiKXvF1KozI95IPEezvZABxqRFTzcgoXhaKi3gWRxkfFUJsrLZMR8JTjyYI6IhDJTXsJa4E1_fi5dHXLTDuMlt3zEY7nBKzd45Q6vFhzNhrzVRTj-erhpAJo96LBpwd4U4dxN0ypH-mjqB6G57vw-aheWA6N3sDGXPVgYlxO9D4vyYzyoygNrgZ9ej9l7
linkProvider	IEEE
linkToHtml	http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV1LT9tAEB4hHiocCuWhhuceekJdsrbXjpdbVTUEESLUBImbtS8DEk1aO-H3M7veRFRUlbhZltey_M3uzjez3wzAF9fBOmGSU6aMoLzUgkojS6qMC_Ojh1KmpW820RkM8rs7cbMEXxdaGGutP3xmz9ylz-WbiZ65UFlbJE7OgFxnJeU8Zo1aa5Ez4GkUxNEZRRoRhxxmxER7dPFziFwwjs4St8V2-F-7kG-r8mYt9htMd_N9n7YFH4MjSb41yH-CJTveho1X5QW3Yc0f79T1Dvy5Hg7756SHpLPRVlY4VI4NuUHQH6qJr9tKLn_h4lKT7sxF0MjzoyRenouEPNgncQ2M0WDpcP4O50HOKktCmdZ7MmiOlde7cNv9Mfreo6HZAtVITKc0kegMqCxD-oOk1QpuM54pl0SzNk4UU1ylOopKrUwepTLWeA_ne2xji9gqmezB8ngytp-B5GmZy47ODMs1F5FREpmwREQSnRuWiRaw-c8vdKhE7hpiPBWekTBROLwKh1cR8GrB6WLI76YMx_8e3nEALR4M2LTgcI5wESZqXSCBdBWE0GD3_z3qBD70Rtf9on85uDqA9dipH3wE5hCWp9XMHsGqfp4-1tWxt8YX1RDcwg
openUrl	ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=MSSL%3A+Hyperspectral+and+Panchromatic+Images+Fusion+via+Multiresolution+Spatial-Spectral+Feature+Learning+Networks&rft.jtitle=IEEE+transactions+on+geoscience+and+remote+sensing&rft.au=Qu%2C+Jiahui&rft.au=Shi%2C+Yanzi&rft.au=Xie%2C+Weiying&rft.au=Li%2C+Yunsong&rft.date=2022&rft.pub=IEEE&rft.issn=0196-2892&rft.volume=60&rft.spage=1&rft.epage=13&rft_id=info:doi/10.1109%2FTGRS.2021.3066374&rft.externalDocID=9386241
thumbnail_l	http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/lc.gif&issn=0196-2892&client=summon
thumbnail_m	http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/mc.gif&issn=0196-2892&client=summon
thumbnail_s	http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/sc.gif&issn=0196-2892&client=summon