Learning Image Formation and Regularization in Unrolling AMP for Lensless Image Reconstruction
This paper proposes an unrolling learnable approximate message passing recurrent neural network (called ULAMP-Net) for lensless image reconstruction. By unrolling the optimization iterations, key modules and parameters are made learnable to achieve high reconstruction quality. Specifically, observat...
Gespeichert in:
| Veröffentlicht in: | IEEE transactions on computational imaging Jg. 8; S. 479 - 489 |
|---|---|
| Hauptverfasser: | , , , , , |
| Format: | Journal Article |
| Sprache: | Englisch |
| Veröffentlicht: |
Piscataway
IEEE
2022
The Institute of Electrical and Electronics Engineers, Inc. (IEEE) |
| Schlagworte: | |
| ISSN: | 2573-0436, 2333-9403 |
| Online-Zugang: | Volltext |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| Abstract | This paper proposes an unrolling learnable approximate message passing recurrent neural network (called ULAMP-Net) for lensless image reconstruction. By unrolling the optimization iterations, key modules and parameters are made learnable to achieve high reconstruction quality. Specifically, observation matrices are rectified on the fly through network learning to suppress systematic errors in the measurement of the point spread function. We devise a domain transformation structure to achieve a more powerful representation and propose a learnable multistage threshold function to accommodate a much richer family of priors with only a small amount of parameters. Finally, we introduce a multi-layer perceptron (MLP) module to enhance the input and an attention mechanism as an output module to refine the final results. Experimental results on display captured dataset and real scene data demonstrate that, compared with the state-of-the-art methods, our method achieves the best reconstruction quality with low computational complexity and the tiny model size on the testing set. Our code will be released in https://github.com/Xiangjun-TJU/ULAMP-NET . |
|---|---|
| AbstractList | This paper proposes an unrolling learnable approximate message passing recurrent neural network (called ULAMP-Net) for lensless image reconstruction. By unrolling the optimization iterations, key modules and parameters are made learnable to achieve high reconstruction quality. Specifically, observation matrices are rectified on the fly through network learning to suppress systematic errors in the measurement of the point spread function. We devise a domain transformation structure to achieve a more powerful representation and propose a learnable multistage threshold function to accommodate a much richer family of priors with only a small amount of parameters. Finally, we introduce a multi-layer perceptron (MLP) module to enhance the input and an attention mechanism as an output module to refine the final results. Experimental results on display captured dataset and real scene data demonstrate that, compared with the state-of-the-art methods, our method achieves the best reconstruction quality with low computational complexity and the tiny model size on the testing set. Our code will be released in https://github.com/Xiangjun-TJU/ULAMP-NET . |
| Author | Yue, Huihui Yin, Xiangjun Cui, Xingyu Yue, Huanjing Yang, Jingyu Zhang, Mengxi |
| Author_xml | – sequence: 1 givenname: Jingyu orcidid: 0000-0002-7521-7920 surname: Yang fullname: Yang, Jingyu email: yjy@tju.edu.cn organization: School of Electrical and Information Engineering, Tianjin University, Tianjin, China – sequence: 2 givenname: Xiangjun orcidid: 0000-0002-4829-9019 surname: Yin fullname: Yin, Xiangjun email: yinxiangjun@tju.edu.cn organization: School of Electrical and Information Engineering, Tianjin University, Tianjin, China – sequence: 3 givenname: Mengxi surname: Zhang fullname: Zhang, Mengxi email: mengxizhang@tju.edu.cn organization: School of Electrical and Information Engineering, Tianjin University, Tianjin, China – sequence: 4 givenname: Huihui surname: Yue fullname: Yue, Huihui email: yuehuihui@tju.edu.cn organization: School of Electrical and Information Engineering, Tianjin University, Tianjin, China – sequence: 5 givenname: Xingyu surname: Cui fullname: Cui, Xingyu email: cuixingyu@tju.edu.cn organization: School of Electrical and Information Engineering, Tianjin University, Tianjin, China – sequence: 6 givenname: Huanjing orcidid: 0000-0003-2517-9783 surname: Yue fullname: Yue, Huanjing email: huanjing.yue@tju.edu.cn organization: School of Electrical and Information Engineering, Tianjin University, Tianjin, China |
| BookMark | eNp9kE1PwkAQhjcGExG5m3hp4rk4s7P9OhIiSoLRELjabMuUlJRd3C0H_fUWIR48eNrJ5nneybzXomesYSFuEUaIkD0sJ7ORBClHhCmqhC5EXxJRmCmgXjdHCYWgKL4SQ--3AIAqk5TGffE-Z-1MbTbBbKc3HEyt2-m2tibQZh0seHNotKu_Tl-1CVbG2aY58uOXt6CyLpiz8Q17fw5YcGmNb92hPCo34rLSjefh-R2I1fRxOXkO569Ps8l4HpYywzZMkbMCqZBxUqZUxajTkrTWqkJJRaJpLZVSqAAYNWOkU0SudMSgVKEqoIG4P-Xunf04sG_zrT04063Mu8yMMhmB7Kj4RJXOeu-4ysu6_TmtdbpucoT8WGfe1Zkf68zPdXYi_BH3rt5p9_mfcndSamb-xbMUMI6AvgG7LoFh |
| CODEN | ITCIAJ |
| CitedBy_id | crossref_primary_10_1109_TETCI_2024_3375022 crossref_primary_10_1145_3701731 crossref_primary_10_1109_TIM_2024_3375987 crossref_primary_10_1109_TCI_2023_3315853 |
| Cites_doi | 10.1364/OL.411228 10.1109/TMI.2021.3054167 10.1109/JSTSP.2020.2977507 10.1109/TSP.2017.2708040 10.1023/A:1007966830741 10.1109/TCI.2020.3010360 10.1109/ICIEA48937.2020.9248182 10.1364/OE.19.004294 10.1007/978-3-319-46475-6_43 10.1109/TCI.2016.2593662 10.1109/MSP.2016.2581921 10.1109/TPAMI.2020.2987489 10.1109/TPAMI.2020.3033882 10.1016/j.imavis.2020.103871 10.1364/OPTICA.6.001185 10.1364/ao.56.00f189 10.1038/s41377-020-00380-x 10.1007/978-3-030-60636-7_32 10.1162/NECO_a_00907 10.1364/AO.28.004996 10.1364/ao.58.000509 10.1109/ICCV.2019.00795 10.1109/TPAMI.2018.2873610 10.1364/OPTICA.397214 10.1109/TIP.2020.3044472 10.1007/978-3-319-24574-4_28 10.1364/OE.27.028075 10.1364/OE.424075 10.1109/CVPR.2018.00196 10.1109/ICASSP39728.2021.9414728 10.1364/OPTICA.4.001117 10.1364/OL.390810 10.1137/18M1169655 10.1126/sciadv.1701548 10.1364/OPTICA.5.000001 10.1109/WACV48630.2021.00045 10.1109/I2MTC50364.2021.9460040 10.1109/CVPR.2019.01257 10.1109/CVPR46437.2021.01595 10.1038/s41377-020-0289-9 10.1007/BF00171998 10.1109/JSTSP.2020.3037516 10.1109/TCI.2021.3114542 |
| ContentType | Journal Article |
| Copyright | Copyright The Institute of Electrical and Electronics Engineers, Inc. (IEEE) 2022 |
| Copyright_xml | – notice: Copyright The Institute of Electrical and Electronics Engineers, Inc. (IEEE) 2022 |
| DBID | 97E RIA RIE AAYXX CITATION 7SC 8FD JQ2 L7M L~C L~D |
| DOI | 10.1109/TCI.2022.3181473 |
| DatabaseName | IEEE Xplore (IEEE) IEEE All-Society Periodicals Package (ASPP) 1998–Present IEEE Electronic Library (IEL) CrossRef Computer and Information Systems Abstracts Technology Research Database ProQuest Computer Science Collection Advanced Technologies Database with Aerospace Computer and Information Systems Abstracts Academic Computer and Information Systems Abstracts Professional |
| DatabaseTitle | CrossRef Computer and Information Systems Abstracts Technology Research Database Computer and Information Systems Abstracts – Academic Advanced Technologies Database with Aerospace ProQuest Computer Science Collection Computer and Information Systems Abstracts Professional |
| DatabaseTitleList | Computer and Information Systems Abstracts |
| Database_xml | – sequence: 1 dbid: RIE name: IEEE Electronic Library (IEL) url: https://ieeexplore.ieee.org/ sourceTypes: Publisher |
| DeliveryMethod | fulltext_linktorsrc |
| Discipline | Applied Sciences |
| EISSN | 2333-9403 |
| EndPage | 489 |
| ExternalDocumentID | 10_1109_TCI_2022_3181473 9801650 |
| Genre | orig-research |
| GrantInformation_xml | – fundername: National Natural Science Foundation of China grantid: 62072331; 61771339 funderid: 10.13039/501100001809 |
| GroupedDBID | 0R~ 6IK 97E AAJGR AARMG AASAJ AAWTH ABAZT ABJNI ABQJQ ABVLG ACGFS AGQYO AGSQL AHBIQ AKJIK AKQYR ALMA_UNASSIGNED_HOLDINGS ATWAV BEFXN BFFAM BGNUA BKEBE BPEOZ EBS EJD IFIPE IPLJI JAVBF O9- OCL PQQKQ RIA RIE AAYXX CITATION 7SC 8FD JQ2 L7M L~C L~D |
| ID | FETCH-LOGICAL-c291t-81e9b13b267c83f61a8c3aaa4f123b7a3d24441400e1ae15a811efa5e044b4f03 |
| IEDL.DBID | RIE |
| ISICitedReferencesCount | 8 |
| ISICitedReferencesURI | http://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=Summon&SrcAuth=ProQuest&DestLinkType=CitingArticles&DestApp=WOS_CPL&KeyUT=000838376400001&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| ISSN | 2573-0436 |
| IngestDate | Mon Jun 30 05:12:10 EDT 2025 Sat Nov 29 05:06:23 EST 2025 Tue Nov 18 21:46:37 EST 2025 Wed Aug 27 02:23:54 EDT 2025 |
| IsPeerReviewed | false |
| IsScholarly | true |
| Language | English |
| License | https://ieeexplore.ieee.org/Xplorehelp/downloads/license-information/IEEE.html https://doi.org/10.15223/policy-029 https://doi.org/10.15223/policy-037 |
| LinkModel | DirectLink |
| MergedId | FETCHMERGED-LOGICAL-c291t-81e9b13b267c83f61a8c3aaa4f123b7a3d24441400e1ae15a811efa5e044b4f03 |
| Notes | ObjectType-Article-1 SourceType-Scholarly Journals-1 ObjectType-Feature-2 content type line 14 |
| ORCID | 0000-0003-2517-9783 0000-0002-7521-7920 0000-0002-4829-9019 |
| PQID | 2679392502 |
| PQPubID | 2040412 |
| PageCount | 11 |
| ParticipantIDs | crossref_citationtrail_10_1109_TCI_2022_3181473 ieee_primary_9801650 crossref_primary_10_1109_TCI_2022_3181473 proquest_journals_2679392502 |
| PublicationCentury | 2000 |
| PublicationDate | 20220000 2022-00-00 20220101 |
| PublicationDateYYYYMMDD | 2022-01-01 |
| PublicationDate_xml | – year: 2022 text: 20220000 |
| PublicationDecade | 2020 |
| PublicationPlace | Piscataway |
| PublicationPlace_xml | – name: Piscataway |
| PublicationTitle | IEEE transactions on computational imaging |
| PublicationTitleAbbrev | TCI |
| PublicationYear | 2022 |
| Publisher | IEEE The Institute of Electrical and Electronics Engineers, Inc. (IEEE) |
| Publisher_xml | – name: IEEE – name: The Institute of Electrical and Electronics Engineers, Inc. (IEEE) |
| References | ref13 ref35 ref12 ref34 ref15 Stork (ref20) 2013 ref14 ref31 ref30 ref11 ref33 ref10 ref32 Diederik (ref43) 2014 ref2 ref1 ref17 ref39 ref16 ref38 ref19 ref18 Busboom (ref36) 1998; 8 ref24 ref46 ref23 ref45 ref26 ref25 ref42 ref41 ref22 ref44 ref21 ref28 ref27 Amir (ref37) 2007 ref29 ref8 ref7 ref9 ref4 ref3 ref6 ref5 ref40 |
| References_xml | – ident: ref12 doi: 10.1364/OL.411228 – start-page: 186 volume-title: Proc. 7th Int. Conf. Sensor Technol. Appl. year: 2013 ident: ref20 article-title: Lensless ultra-miniature CMOS computational imagers and sensors – ident: ref45 doi: 10.1109/TMI.2021.3054167 – ident: ref26 doi: 10.1109/JSTSP.2020.2977507 – ident: ref38 doi: 10.1109/TSP.2017.2708040 – volume: 8 start-page: 97 year: 1998 ident: ref36 article-title: Uniformly redundant arrays publication-title: Exp. Astron. doi: 10.1023/A:1007966830741 – ident: ref18 doi: 10.1109/TCI.2020.3010360 – ident: ref33 doi: 10.1109/ICIEA48937.2020.9248182 – ident: ref21 doi: 10.1364/OE.19.004294 – ident: ref42 doi: 10.1007/978-3-319-46475-6_43 – ident: ref9 doi: 10.1109/TCI.2016.2593662 – ident: ref3 doi: 10.1109/MSP.2016.2581921 – ident: ref10 doi: 10.1109/TPAMI.2020.2987489 – ident: ref14 doi: 10.1109/TPAMI.2020.3033882 – ident: ref46 doi: 10.1016/j.imavis.2020.103871 – ident: ref6 doi: 10.1364/OPTICA.6.001185 – ident: ref4 doi: 10.1364/ao.56.00f189 – ident: ref7 doi: 10.1038/s41377-020-00380-x – ident: ref27 doi: 10.1007/978-3-030-60636-7_32 – ident: ref41 doi: 10.1162/NECO_a_00907 – ident: ref2 doi: 10.1364/AO.28.004996 – ident: ref5 doi: 10.1364/ao.58.000509 – ident: ref1 doi: 10.1109/ICCV.2019.00795 – ident: ref29 doi: 10.1109/TPAMI.2018.2873610 – ident: ref22 doi: 10.1364/OPTICA.397214 – ident: ref39 doi: 10.1109/TIP.2020.3044472 – ident: ref44 doi: 10.1007/978-3-319-24574-4_28 – ident: ref16 doi: 10.1364/OE.27.028075 – start-page: 1 volume-title: Proc. 3rd Int. Conf. Learn. Representations year: 2014 ident: ref43 article-title: Adam: A method for stochastic optimization – ident: ref35 doi: 10.1364/OE.424075 – ident: ref40 doi: 10.1109/CVPR.2018.00196 – ident: ref23 doi: 10.1109/ICASSP39728.2021.9414728 – ident: ref32 doi: 10.1364/OPTICA.4.001117 – ident: ref15 doi: 10.1364/OL.390810 – ident: ref24 doi: 10.1137/18M1169655 – ident: ref19 doi: 10.1126/sciadv.1701548 – start-page: 553 volume-title: Proc. Int. Conf. Neural Inf. Process. Syst. year: 2007 ident: ref37 article-title: Fixing max-product: Convergent message passing algorithms for MAP LP-Relaxations – ident: ref11 doi: 10.1364/OPTICA.5.000001 – ident: ref13 doi: 10.1109/WACV48630.2021.00045 – ident: ref25 doi: 10.1109/I2MTC50364.2021.9460040 – ident: ref28 doi: 10.1109/CVPR.2019.01257 – ident: ref30 doi: 10.1109/CVPR46437.2021.01595 – ident: ref8 doi: 10.1038/s41377-020-0289-9 – ident: ref17 doi: 10.1007/BF00171998 – ident: ref31 doi: 10.1109/JSTSP.2020.3037516 – ident: ref34 doi: 10.1109/TCI.2021.3114542 |
| SSID | ssj0001492386 |
| Score | 2.2297816 |
| Snippet | This paper proposes an unrolling learnable approximate message passing recurrent neural network (called ULAMP-Net) for lensless image reconstruction. By... |
| SourceID | proquest crossref ieee |
| SourceType | Aggregation Database Enrichment Source Index Database Publisher |
| StartPage | 479 |
| SubjectTerms | approximate message passing algorithm Cameras Current measurement deep unfolding method Image reconstruction Imaging Learning Lensless imaging Measurement uncertainty Message passing Modules Multilayer perceptrons Multilayers Optimization Parameters Point spread functions Prototypes Recurrent neural networks Regularization spatial-channel attention Systematic errors Training |
| Title | Learning Image Formation and Regularization in Unrolling AMP for Lensless Image Reconstruction |
| URI | https://ieeexplore.ieee.org/document/9801650 https://www.proquest.com/docview/2679392502 |
| Volume | 8 |
| WOSCitedRecordID | wos000838376400001&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| hasFullText | 1 |
| inHoldings | 1 |
| isFullTextHit | |
| isPrint | |
| journalDatabaseRights | – providerCode: PRVIEE databaseName: IEEE Electronic Library (IEL) customDbUrl: eissn: 2333-9403 dateEnd: 99991231 omitProxy: false ssIdentifier: ssj0001492386 issn: 2573-0436 databaseCode: RIE dateStart: 20150101 isFulltext: true titleUrlDefault: https://ieeexplore.ieee.org/ providerName: IEEE |
| link | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV3PS8MwFA46PHhx_sT5ixy8CNY1TbI0xyEOBR1DJniyvKapCNrJNv37fUnTDVEEbz0koc2XvPe-pO97hJwmEtCNCBtJ4HkkZK4iSCSPCgW5Ait7mhe-2IQaDtPHRz1aIeeLXBhrrf_5zF64R3-XX0zMhzsq6-rUJd8gQV9VStW5WsvzFKc05gs74iLkkVNWb24lY90dX94gF0wSpKgpE4p_80K-rMoPW-wdzKD9v1fbJBshkKT9GvktsmKrbdIOQSUNW3a2Q56CguozvXlD20EHTbYihaqg974U_TQkY9KXij5U01qnm_bvRhRDWnqLTPcVDWIYwBHWpezsLnkYXI0vr6NQVCEyiWbzKGVW54znSU-ZlJc9BqnhACBK9GEIDy_Q4QukXbFlYJmElDFbgrSxELkoY75HWtWksvuEllqD0BAbBUiqlNKGldZVkIHSsCLXHdJtJjkzQXHcFb54zTzziHWGsGQOlizA0iFnix7vtdrGH213HAyLdgGBDjlqcMzCdpxl-LEaA0EZJwe_9zok627s-mzliLRwEu0xWTOf85fZ9MSvtC9wY9AM |
| linkProvider | IEEE |
| linkToHtml | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV1NTxsxEB1FtBJcSsuHSEtbH7ggsWS9tuP1EaFGRE0ihILEidWs14uQYFMlgd_fsdcJqkBIve3B3g8_2zNvvPMG4ChTSGZEukShKBOpSp1gpkRSaSw1OtU3ogrFJvRkkt_cmMsOnKxzYZxz4eczd-ovw1l-NbNPPlTWM7lPviGC_kFJmfE2W-slouK1xkJpR5qGIvHa6qtzydT0pudDYoNZRiQ151KLf-xQKKzyajcOJmaw_X8v9xk-RVeSnbXYf4GOa3ZgO7qVLC7axS7cRg3VOzZ8pN2DDVb5igybil2FYvTzmI7J7ht23cxbpW52Nr5k5NSyEXHdB9oS4w08ZX0Rnt2D68Gv6flFEssqJDYzfJnk3JmSizLra5uLus8xtwIRZU1WjAASFZl8ScQrdRwdV5hz7mpULpWylHUq9mGjmTXuAFhtDEqDqdVItEprY3ntfA0ZrC2vStOF3mqQCxs1x33pi4cicI_UFARL4WEpIixdOF73-NPqbbzTdtfDsG4XEejC4QrHIi7IRUEfa8gVVGn29e1eP2HzYjoeFaPh5Pc32PLPaSMth7BBA-q-w0f7vLxfzH-EWfcXbHzTUw |
| openUrl | ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Learning+Image+Formation+and+Regularization+in+Unrolling+AMP+for+Lensless+Image+Reconstruction&rft.jtitle=IEEE+transactions+on+computational+imaging&rft.au=Yang%2C+Jingyu&rft.au=Yin%2C+Xiangjun&rft.au=Zhang%2C+Mengxi&rft.au=Yue%2C+Huihui&rft.date=2022&rft.issn=2573-0436&rft.eissn=2333-9403&rft.volume=8&rft.spage=479&rft.epage=489&rft_id=info:doi/10.1109%2FTCI.2022.3181473&rft.externalDBID=n%2Fa&rft.externalDocID=10_1109_TCI_2022_3181473 |
| thumbnail_l | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/lc.gif&issn=2573-0436&client=summon |
| thumbnail_m | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/mc.gif&issn=2573-0436&client=summon |
| thumbnail_s | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/sc.gif&issn=2573-0436&client=summon |