Convolutional Analysis Sparse Coding for Multimodal Image Super-Resolution

With multimodal imaging systems in place, recent research focus has been directed toward exploiting the knowledge from different imaging modalities to solve inverse problems, one example being image super-resolution, considered in this letter. The goal of multimodal image super-resolution (MISR) is...

Ausführliche Beschreibung

Gespeichert in:

Bibliographische Detailangaben
Veröffentlicht in:	IEEE sensors letters Jg. 8; H. 6; S. 1 - 4
Hauptverfasser:	Kumar, Kriti, Majumdar, Angshul, Kumar, A. Anil, Chandra, M. Girish
Format:	Journal Article
Sprache:	Englisch
Veröffentlicht:	Piscataway IEEE 01.06.2024 The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Schlagworte:	Artificial neural networks Benchmark testing Convolutional codes convolutional sparse coding convolutional transform learning (CTL) Image coding Image enhancement Image reconstruction Image resolution Inverse problems joint optimization Learning multimodal image super-resolution (MISR) Performance enhancement Sensor applications Superresolution Task analysis Transforms
ISSN:	2475-1472, 2475-1472
Online-Zugang:	Volltext
Tags:	Tag hinzufügen Keine Tags, Fügen Sie den ersten Tag hinzu!

Abstract	With multimodal imaging systems in place, recent research focus has been directed toward exploiting the knowledge from different imaging modalities to solve inverse problems, one example being image super-resolution, considered in this letter. The goal of multimodal image super-resolution (MISR) is to enhance the resolution of low-resolution (LR) images from the target modality by taking guidance from high-resolution (HR) images from a different modality. Conventional methods for MISR employing convolutional neural networks typically adopt an encoder-decoder architecture that often demands massive data for optimal reconstruction and tends to overfit in data-limited scenarios. The proposed work presents a convolutional analysis sparse coding-based method, employing convolutional transforms that eliminate the need for learning the decoder network and learns unique filters. This reduces the number of trainable parameters, making it suitable for learning with limited data. A joint optimization formulation is presented that learns dedicated convolutional transforms for both the LR images of target modality and the HR images of guidance modality, and a fusing (combining) transform that combines the respective transform features to reconstruct the HR images of target modality. Unlike the dictionary-based synthesis sparse coding methods for MISR, the proposed approach offers enhanced performance with reduced complexity due to some of the inherent advantages of transform learning. The effectiveness of the proposed method is validated using RGB-Near Infrared and RGB-Multispectral datasets. Experimental findings demonstrate enhanced reconstruction performance achieved by the proposed method, compared with the state-of-the-art techniques. Moreover, the proposed method demonstrates improved robustness to noise compared with other sparse coding-based methods.
AbstractList	With multimodal imaging systems in place, recent research focus has been directed toward exploiting the knowledge from different imaging modalities to solve inverse problems, one example being image super-resolution, considered in this letter. The goal of multimodal image super-resolution (MISR) is to enhance the resolution of low-resolution (LR) images from the target modality by taking guidance from high-resolution (HR) images from a different modality. Conventional methods for MISR employing convolutional neural networks typically adopt an encoder–decoder architecture that often demands massive data for optimal reconstruction and tends to overfit in data-limited scenarios. The proposed work presents a convolutional analysis sparse coding-based method, employing convolutional transforms that eliminate the need for learning the decoder network and learns unique filters. This reduces the number of trainable parameters, making it suitable for learning with limited data. A joint optimization formulation is presented that learns dedicated convolutional transforms for both the LR images of target modality and the HR images of guidance modality, and a fusing (combining) transform that combines the respective transform features to reconstruct the HR images of target modality. Unlike the dictionary-based synthesis sparse coding methods for MISR, the proposed approach offers enhanced performance with reduced complexity due to some of the inherent advantages of transform learning. The effectiveness of the proposed method is validated using RGB-Near Infrared and RGB-Multispectral datasets. Experimental findings demonstrate enhanced reconstruction performance achieved by the proposed method, compared with the state-of-the-art techniques. Moreover, the proposed method demonstrates improved robustness to noise compared with other sparse coding-based methods.
Author	Kumar, A. Anil Majumdar, Angshul Kumar, Kriti Chandra, M. Girish
Author_xml	– sequence: 1 givenname: Kriti orcidid: 0000-0003-0484-7404 surname: Kumar fullname: Kumar, Kriti email: kriti.kumar@tcs.com organization: TCS Research, Bangalore, India – sequence: 2 givenname: Angshul orcidid: 0000-0002-1065-3000 surname: Majumdar fullname: Majumdar, Angshul organization: Indraprastha Institute of Information Technology Delhi, New Delhi, India – sequence: 3 givenname: A. Anil orcidid: 0000-0002-0661-5976 surname: Kumar fullname: Kumar, A. Anil organization: TCS Research, Bangalore, India – sequence: 4 givenname: M. Girish orcidid: 0000-0002-1479-0504 surname: Chandra fullname: Chandra, M. Girish organization: TCS Research, Bangalore, India
BookMark	eNpNkE1Lw0AQhhdRsNb-AfEQ8Jy6n93NsYSqlapg9LzsJrMlJc3G3Ubovze1PfQyM4f3GV6eG3TZ-hYQuiN4SgjOHlfF4r2YUkz5lHHMiMwu0IhyKVLCJb08u6_RJMYNxpgoKjHDI_Sa-_bXN_2u9q1pkvkw9rGOSdGZECHJfVW368T5kLz1za7e-mpILbdmDUnRdxDST4gn_BZdOdNEmJz2GH0_Lb7yl3T18bzM56u0HHrsUgGZkdJwILJSmXWKCqqMdCCUlY474Yh0lhNLrGIlLikFY7OsAmwZlzPLxujh-LcL_qeHuNMb34eheNQMz7gkQnExpOgxVQYfYwCnu1BvTdhrgvVBm_7Xpg_a9EnbAN0foRoAzgDBBMkU-wOl5Gtb
CODEN	ISLECD
Cites_doi	10.1109/TCI.2018.2840334 10.1287/moor.2022.1256 10.1109/JSTSP.2015.2417131 10.1109/TPAMI.2012.213 10.23919/EUSIPCO55093.2022.9909616 10.1109/TSP.2012.2226449 10.23919/Eusipco47968.2020.9287506 10.1109/TIP.2019.2944270 10.1007/978-3-030-04182-3_15 10.1109/JAS.2023.123681 10.1109/TIP.2022.3141251 10.1109/JPROC.2015.2449668 10.1109/TCI.2019.2916502 10.1109/CVPR.2011.5995660 10.1109/JSEN.2022.3229046 10.1109/TPAMI.2018.2890623 10.1109/JSEN.2022.3195243 10.1109/ICASSP39728.2021.9413490 10.1109/CVPR.2011.5995637 10.1109/TPAMI.2015.2417569 10.1109/LSP.2022.3159149 10.1145/1276377.1276497
ContentType	Journal Article
Copyright	Copyright The Institute of Electrical and Electronics Engineers, Inc. (IEEE) 2024
Copyright_xml	– notice: Copyright The Institute of Electrical and Electronics Engineers, Inc. (IEEE) 2024
DBID	97E RIA RIE AAYXX CITATION 7SP 8FD L7M
DOI	10.1109/LSENS.2024.3403179
DatabaseName	IEEE All-Society Periodicals Package (ASPP) 2005–Present IEEE All-Society Periodicals Package (ASPP) 1998–Present IEEE/IET Electronic Library CrossRef Electronics & Communications Abstracts Technology Research Database Advanced Technologies Database with Aerospace
DatabaseTitle	CrossRef Technology Research Database Advanced Technologies Database with Aerospace Electronics & Communications Abstracts
DatabaseTitleList	Technology Research Database
Database_xml	– sequence: 1 dbid: RIE name: IEEE/IET Electronic Library url: https://ieeexplore.ieee.org/ sourceTypes: Publisher
DeliveryMethod	fulltext_linktorsrc
EISSN	2475-1472
EndPage	4
ExternalDocumentID	10_1109_LSENS_2024_3403179 10535198
Genre	orig-research
GroupedDBID	0R~ 6IK 97E AAJGR AASAJ AAWTH ABAZT ABJNI ABQJQ ABVLG ACGFS AGQYO AHBIQ AKJIK AKQYR ALMA_UNASSIGNED_HOLDINGS ATWAV BEFXN BFFAM BGNUA BKEBE BPEOZ EBS EJD IFIPE IPLJI JAVBF OCL RIA RIE AAYXX CITATION 7SP 8FD L7M
ID	FETCH-LOGICAL-c247t-5e9a77a4e17d89bf82528a7fe58b7f4f5f17fb41b1b83c0c22eab99de0b3476b3
IEDL.DBID	RIE
ISICitedReferencesCount	0
ISICitedReferencesURI	http://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=Summon&SrcAuth=ProQuest&DestLinkType=CitingArticles&DestApp=WOS_CPL&KeyUT=001239997500001&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D
ISSN	2475-1472
IngestDate	Mon Jun 30 04:34:53 EDT 2025 Sat Nov 29 05:53:53 EST 2025 Wed Aug 27 01:58:18 EDT 2025
IsPeerReviewed	true
IsScholarly	true
Issue	6
Language	English
License	https://ieeexplore.ieee.org/Xplorehelp/downloads/license-information/IEEE.html https://doi.org/10.15223/policy-029 https://doi.org/10.15223/policy-037
LinkModel	DirectLink
MergedId	FETCHMERGED-LOGICAL-c247t-5e9a77a4e17d89bf82528a7fe58b7f4f5f17fb41b1b83c0c22eab99de0b3476b3
Notes	ObjectType-Article-1 SourceType-Scholarly Journals-1 ObjectType-Feature-2 content type line 14
ORCID	0000-0002-1065-3000 0000-0002-0661-5976 0000-0002-1479-0504 0000-0003-0484-7404
PQID	3064715845
PQPubID	4437223
PageCount	4
ParticipantIDs	ieee_primary_10535198 proquest_journals_3064715845 crossref_primary_10_1109_LSENS_2024_3403179
PublicationCentury	2000
PublicationDate	2024-06-01
PublicationDateYYYYMMDD	2024-06-01
PublicationDate_xml	– month: 06 year: 2024 text: 2024-06-01 day: 01
PublicationDecade	2020
PublicationPlace	Piscataway
PublicationPlace_xml	– name: Piscataway
PublicationTitle	IEEE sensors letters
PublicationTitleAbbrev	LSENS
PublicationYear	2024
Publisher	IEEE The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Publisher_xml	– name: IEEE – name: The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
References	ref13 Kingma (ref21) 2015 ref12 ref23 ref15 ref14 ref20 ref11 ref22 ref10 ref2 ref1 ref17 ref16 ref19 ref18 ref8 ref7 ref9 ref4 ref3 ref6 ref5
References_xml	– ident: ref15 doi: 10.1109/TCI.2018.2840334 – ident: ref19 doi: 10.1287/moor.2022.1256 – ident: ref16 doi: 10.1109/JSTSP.2015.2417131 – ident: ref3 doi: 10.1109/TPAMI.2012.213 – ident: ref11 doi: 10.23919/EUSIPCO55093.2022.9909616 – ident: ref20 doi: 10.1109/TSP.2012.2226449 – ident: ref17 doi: 10.23919/Eusipco47968.2020.9287506 – ident: ref13 doi: 10.1109/TIP.2019.2944270 – ident: ref18 doi: 10.1007/978-3-030-04182-3_15 – ident: ref6 doi: 10.1109/JAS.2023.123681 – ident: ref12 doi: 10.1109/TIP.2022.3141251 – ident: ref2 doi: 10.1109/JPROC.2015.2449668 – ident: ref9 doi: 10.1109/TCI.2019.2916502 – ident: ref22 doi: 10.1109/CVPR.2011.5995660 – ident: ref1 doi: 10.1109/JSEN.2022.3229046 – ident: ref7 doi: 10.1109/TPAMI.2018.2890623 – ident: ref8 doi: 10.1109/JSEN.2022.3195243 – ident: ref10 doi: 10.1109/ICASSP39728.2021.9413490 – ident: ref23 doi: 10.1109/CVPR.2011.5995637 – ident: ref5 doi: 10.1109/TPAMI.2015.2417569 – ident: ref14 doi: 10.1109/LSP.2022.3159149 – ident: ref4 doi: 10.1145/1276377.1276497 – volume-title: Proc. 3rd Int. Conf. Learn. Representations year: 2015 ident: ref21 article-title: Adam: A method for stochastic optimization
SSID	ssj0001827030
Score	2.2573922
Snippet	With multimodal imaging systems in place, recent research focus has been directed toward exploiting the knowledge from different imaging modalities to solve...
SourceID	proquest crossref ieee
SourceType	Aggregation Database Index Database Publisher
StartPage	1
SubjectTerms	Artificial neural networks Benchmark testing Convolutional codes convolutional sparse coding convolutional transform learning (CTL) Image coding Image enhancement Image reconstruction Image resolution Inverse problems joint optimization Learning multimodal image super-resolution (MISR) Performance enhancement Sensor applications Superresolution Task analysis Transforms
Title	Convolutional Analysis Sparse Coding for Multimodal Image Super-Resolution
URI	https://ieeexplore.ieee.org/document/10535198 https://www.proquest.com/docview/3064715845
Volume	8
WOSCitedRecordID	wos001239997500001&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D
hasFullText	1
inHoldings	1
isFullTextHit
isPrint
journalDatabaseRights	– providerCode: PRVIEE databaseName: IEEE/IET Electronic Library customDbUrl: eissn: 2475-1472 dateEnd: 99991231 omitProxy: false ssIdentifier: ssj0001827030 issn: 2475-1472 databaseCode: RIE dateStart: 20170101 isFulltext: true titleUrlDefault: https://ieeexplore.ieee.org/ providerName: IEEE
link	http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV1LSwMxEB5s8eDFB1asVtmDN9m6j-xOcpTSoiJFWIXeliQ7AQ_tlr5-v0l2i4J48LawybJ8mczMN5nJANxJSxJkRVFIHKuQobF6UEoe8orSTKdZZFLjm03gdMpnM_HWFqv7Whgi8slnNHSP_iy_qvXWhcrsDs9cPznegQ5i3hRrfQdUeOKkd18YE4mH12I8LSwFTNgwZVZ4XbrWD-Pju6n8UsHerkxO_vlHp3DcOpDBY7PiZ3BAi3N4GdWLXStE7mV700hQLC1vpWBUOwsVWP808AW387qyo57nVpcExXZJq9BF8ZvpPfiYjN9HT2HbJSHUCcNNmJGQiJJRjBUXyljKl3CJhjKu0DCTmRiNYrGKFU91pJOEpBLCro9KGeYqvYDuol7QJQSCC9LWQ1EorAK1xJmRljlVPEeFPNJ9uN_DVy6byzBKTyIiUXqwSwd22YLdh54D7MfIBqs-DPaQl-2GWZeOCGFsvaHs6o9p13Dkvt6kaQ2gu1lt6QYO9W7zuV7deln4AnW5tE8
linkProvider	IEEE
linkToHtml	http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV1LSwMxEA5aBb34wIrVqnvwJlv3kTTJUYql1boIW6G3kGQn4KHd0tfvN8lusSAevC1swi5fJjPzTWYyCD1ISxJkAVEIjBYhpsbqQSlZyApIiU5JZFLjm03QLGOTCf-oi9V9LQwA-OQz6LhHf5ZflHrtQmV2hxPXT47towOCcRJV5Vo_IRWWOPndlsZE_GmUv2S5JYEJ7qTYiq9L2NoxP76fyi8l7C1L__Sf_3SGTmoXMniu1vwc7cHsAr32ytmmFiP3sr5rJMjnlrlC0CudjQqshxr4kttpWdhRw6nVJkG-nsMidHH8anoTffZfxr1BWPdJCHWC6SokwCWlEkNMC8aVsaQvYZIaIExRgw0xMTUKxypWLNWRThKQinO7QirFtKvSS9SYlTO4QgFnHLT1URTlVoVa6oxByy4UrEsVZZFuocctfGJeXYchPI2IuPBgCwe2qMFuoaYDbGdkhVULtbeQi3rLLIWjQjS2_hC5_mPaPToajN9HYjTM3m7QsftSlbTVRo3VYg236FBvVl_LxZ2Xi28DvbeW
openUrl	ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Convolutional+Analysis+Sparse+Coding+for+Multimodal+Image+Super-Resolution&rft.jtitle=IEEE+sensors+letters&rft.au=Kumar%2C+Kriti&rft.au=Majumdar%2C+Angshul&rft.au=Kumar%2C+A.+Anil&rft.au=Chandra%2C+M.+Girish&rft.date=2024-06-01&rft.issn=2475-1472&rft.eissn=2475-1472&rft.volume=8&rft.issue=6&rft.spage=1&rft.epage=4&rft_id=info:doi/10.1109%2FLSENS.2024.3403179&rft.externalDBID=n%2Fa&rft.externalDocID=10_1109_LSENS_2024_3403179
thumbnail_l	http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/lc.gif&issn=2475-1472&client=summon
thumbnail_m	http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/mc.gif&issn=2475-1472&client=summon
thumbnail_s	http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/sc.gif&issn=2475-1472&client=summon