Macroscopic Analysis of Vector Approximate Message Passing in a Model-Mismatched Setting
In this study, macroscopic properties of the vector approximate message passing (VAMP) algorithm for inference of generalized linear models are investigated using a non-rigorous heuristic method of statistical mechanics when the true posterior cannot be used and the measurement matrix is a sample fr...
Uložené v:
| Vydané v: | IEEE transactions on information theory Ročník 68; číslo 8; s. 5579 - 5600 |
|---|---|
| Hlavní autori: | , |
| Médium: | Journal Article |
| Jazyk: | English |
| Vydavateľské údaje: |
New York
IEEE
01.08.2022
The Institute of Electrical and Electronics Engineers, Inc. (IEEE) |
| Predmet: | |
| ISSN: | 0018-9448, 1557-9654 |
| On-line prístup: | Získať plný text |
| Tagy: |
Pridať tag
Žiadne tagy, Buďte prvý, kto otaguje tento záznam!
|
| Abstract | In this study, macroscopic properties of the vector approximate message passing (VAMP) algorithm for inference of generalized linear models are investigated using a non-rigorous heuristic method of statistical mechanics when the true posterior cannot be used and the measurement matrix is a sample from rotation-invariant random matrix ensembles. The focus is on the correspondence between the non-rigorous replica analysis of statistical mechanics and the performance assessment of VAMP in the model-mismatched setting. The correspondence of this kind is well-known when the measurement matrix has independent and identically distributed entries. However, when the measurement matrix follows a general rotation-invariant matrix ensemble, the correspondence has been validated only under limited cases, such as the Bayes optimal inference or the convex empirical risk minimization. The result presented in this paper is to extend the scope of such correspondence. Herein, we heuristically derive the explicit formula of state-evolution equations, which macroscopically describe VAMP dynamics for the current model-mismatched case, and show that their fixed point is generally consistent with the replica symmetric solution obtained by the replica method of statistical mechanics. We also show that the fixed point of VAMP can exhibit a microscopic instability, which indicates that message variables continue to move by VAMP while their macroscopically summarized quantities converge to fixed values. The critical condition the for microscopic instability agrees with that for breaking the replica symmetry that is derived within the non-rigorous replica analysis. The results of the numerical experiments cross-check our findings. |
|---|---|
| AbstractList | In this study, macroscopic properties of the vector approximate message passing (VAMP) algorithm for inference of generalized linear models are investigated using a non-rigorous heuristic method of statistical mechanics when the true posterior cannot be used and the measurement matrix is a sample from rotation-invariant random matrix ensembles. The focus is on the correspondence between the non-rigorous replica analysis of statistical mechanics and the performance assessment of VAMP in the model-mismatched setting. The correspondence of this kind is well-known when the measurement matrix has independent and identically distributed entries. However, when the measurement matrix follows a general rotation-invariant matrix ensemble, the correspondence has been validated only under limited cases, such as the Bayes optimal inference or the convex empirical risk minimization. The result presented in this paper is to extend the scope of such correspondence. Herein, we heuristically derive the explicit formula of state-evolution equations, which macroscopically describe VAMP dynamics for the current model-mismatched case, and show that their fixed point is generally consistent with the replica symmetric solution obtained by the replica method of statistical mechanics. We also show that the fixed point of VAMP can exhibit a microscopic instability, which indicates that message variables continue to move by VAMP while their macroscopically summarized quantities converge to fixed values. The critical condition the for microscopic instability agrees with that for breaking the replica symmetry that is derived within the non-rigorous replica analysis. The results of the numerical experiments cross-check our findings. |
| Author | Kabashima, Yoshiyuki Takahashi, Takashi |
| Author_xml | – sequence: 1 givenname: Takashi orcidid: 0000-0002-0173-5370 surname: Takahashi fullname: Takahashi, Takashi email: takashi-takahashi@g.ecc.u-tokyo.ac.jp organization: Department of Physics, Graduate School of Science, Institute for Physics of Intelligence, The University of Tokyo, Bunkyo City, Tokyo, Japan – sequence: 2 givenname: Yoshiyuki orcidid: 0000-0002-2949-7108 surname: Kabashima fullname: Kabashima, Yoshiyuki email: kaba@phys.u-tokyo.ac.jp organization: Department of Physics, Graduate School of Science, Institute for Physics of Intelligence, The University of Tokyo, Bunkyo City, Tokyo, Japan |
| BookMark | eNp9kE1LAzEQhoNUsK3eBS8Bz1vzsckmx1L8KLQoWMXbkmZna8q6WTcp2H9vSsWDB0_DMM87zDwjNGh9CwhdUjKhlOib1Xw1YYSxCaeS85ydoCEVosi0FPkADQmhKtN5rs7QKIRtanNB2RC9LY3tfbC-cxZPW9PsgwvY1_gVbPQ9nnZd77_ch4mAlxCC2QB-MiG4doNdiw1e-gqabOlCQuw7VPgZYkzTc3RamybAxU8do5e729XsIVs83s9n00VmmaYxYxy4VURqwoSivChEbWhRK61FLSno3BSqosQqWSvGuSkoIWu-JhoqUpM142N0fdyb7vzcQYjl1u_69EgomVRKaialTpQ8UodnQw91aV000fk29sY1JSXlwWKZLJYHi-WPxRQkf4Jdn2z0-_8iV8eIA4BfXBe5IILxb3Bcfb4 |
| CODEN | IETTAW |
| CitedBy_id | crossref_primary_10_1109_TIT_2023_3321575 crossref_primary_10_1088_1751_8121_adce65 crossref_primary_10_1088_1742_5468_adbe3f crossref_primary_10_1109_TIT_2022_3222913 crossref_primary_10_1109_TSP_2023_3301137 |
| Cites_doi | 10.1080/00018732.2016.1211393 10.1109/ISIT44484.2020.9173999 10.1109/ISIT.2018.8437792 10.1103/PhysRevLett.75.2847 10.1088/0305-4470/36/43/030 10.1088/0305-4470/13/4/009 10.1109/ISIT.2015.7282968 10.1109/ITW.2016.7606840 10.1103/PhysRevLett.86.3695 10.1093/acprof:oso/9780198570837.001.0001 10.1093/imaiai/iay021 10.1109/TIT.2022.3222913 10.1088/1751-8113/41/32/324013 10.1088/1751-8113/42/1/015005 10.1109/TIT.2019.2916359 10.1088/1742-5468/abb8c9 10.1088/0305-4470/11/5/028 10.1088/0305-4470/21/1/030 10.1103/PhysRevE.99.062140 10.1088/1742-6596/95/1/012001 10.1109/ISIT.2018.8437522 10.1088/0305-4470/32/21/302 10.1109/ISIT.2019.8849790 10.1088/1742-5468/ac1403 10.1109/LSP.2014.2351822 10.1051/jphys:0198900500200305700 10.1088/0022-3719/12/11/008 10.1109/TIT.2016.2556702 10.1017/9781108120494 10.1109/ISIT.2017.8006578 10.1109/TIT.2010.2094817 10.1007/s00220-013-1862-3 10.1088/1751-8113/49/11/114002 10.1109/ACCESS.2017.2653119 10.1109/JSAIT.2020.2986321 10.1142/0271 10.1093/imaiai/iat004 10.1007/3-540-29060-5 10.1109/TIT.2013.2250578 10.1103/PhysRevE.64.056131 10.1109/ITW.2014.6970819 10.1109/ACSSC.2016.7869633 10.1561/2200000092 10.1109/ISIT.2014.6874828 10.1109/PGEC.1965.264137 10.1073/pnas.1802705116 |
| ContentType | Journal Article |
| Copyright | Copyright The Institute of Electrical and Electronics Engineers, Inc. (IEEE) 2022 |
| Copyright_xml | – notice: Copyright The Institute of Electrical and Electronics Engineers, Inc. (IEEE) 2022 |
| DBID | 97E RIA RIE AAYXX CITATION 7SC 7SP 8FD JQ2 L7M L~C L~D |
| DOI | 10.1109/TIT.2022.3163342 |
| DatabaseName | IEEE All-Society Periodicals Package (ASPP) 2005–Present IEEE All-Society Periodicals Package (ASPP) 1998–Present IEEE Electronic Library (IEL) CrossRef Computer and Information Systems Abstracts Electronics & Communications Abstracts Technology Research Database ProQuest Computer Science Collection Advanced Technologies Database with Aerospace Computer and Information Systems Abstracts Academic Computer and Information Systems Abstracts Professional |
| DatabaseTitle | CrossRef Technology Research Database Computer and Information Systems Abstracts – Academic Electronics & Communications Abstracts ProQuest Computer Science Collection Computer and Information Systems Abstracts Advanced Technologies Database with Aerospace Computer and Information Systems Abstracts Professional |
| DatabaseTitleList | Technology Research Database |
| Database_xml | – sequence: 1 dbid: RIE name: IEEE Electronic Library (IEL) url: https://ieeexplore.ieee.org/ sourceTypes: Publisher |
| DeliveryMethod | fulltext_linktorsrc |
| Discipline | Engineering Computer Science |
| EISSN | 1557-9654 |
| EndPage | 5600 |
| ExternalDocumentID | 10_1109_TIT_2022_3163342 9745052 |
| Genre | orig-research |
| GrantInformation_xml | – fundername: Japan Society for the Promotion of Science (JSPS) KAKENHI grantid: 19J0711; 17H00764; 21K21310 funderid: 10.13039/501100001691 – fundername: Japan Science and Technology Agency (JST) Core Research for Evolutional Science and Technology (CREST), Japan grantid: JPMJCR1912 funderid: 10.13039/501100003382 |
| GroupedDBID | -~X .DC 0R~ 29I 3EH 4.4 5GY 5VS 6IK 97E AAJGR AARMG AASAJ AAWTH ABAZT ABFSI ABQJQ ABVLG ACGFO ACGFS ACGOD ACIWK AENEX AETEA AETIX AGQYO AGSQL AHBIQ AI. AIBXA AKJIK AKQYR ALLEH ALMA_UNASSIGNED_HOLDINGS ASUFR ATWAV BEFXN BFFAM BGNUA BKEBE BPEOZ CS3 DU5 E.L EBS EJD F5P HZ~ H~9 IAAWW IBMZZ ICLAB IDIHD IFIPE IFJZH IPLJI JAVBF LAI M43 MS~ O9- OCL P2P PQQKQ RIA RIE RNS RXW TAE TN5 VH1 VJK AAYXX CITATION 7SC 7SP 8FD JQ2 L7M L~C L~D |
| ID | FETCH-LOGICAL-c291t-23e3c8069025813775fa17f8995f61e94a78d10c86f8233a7100b3b09ed0f0b23 |
| IEDL.DBID | RIE |
| ISICitedReferencesCount | 8 |
| ISICitedReferencesURI | http://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=Summon&SrcAuth=ProQuest&DestLinkType=CitingArticles&DestApp=WOS_CPL&KeyUT=000838527100041&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| ISSN | 0018-9448 |
| IngestDate | Sun Nov 09 06:08:59 EST 2025 Sat Nov 29 03:31:48 EST 2025 Tue Nov 18 20:59:37 EST 2025 Wed Aug 27 02:25:44 EDT 2025 |
| IsPeerReviewed | true |
| IsScholarly | true |
| Issue | 8 |
| Language | English |
| License | https://ieeexplore.ieee.org/Xplorehelp/downloads/license-information/IEEE.html https://doi.org/10.15223/policy-029 https://doi.org/10.15223/policy-037 |
| LinkModel | DirectLink |
| MergedId | FETCHMERGED-LOGICAL-c291t-23e3c8069025813775fa17f8995f61e94a78d10c86f8233a7100b3b09ed0f0b23 |
| Notes | ObjectType-Article-1 SourceType-Scholarly Journals-1 ObjectType-Feature-2 content type line 14 |
| ORCID | 0000-0002-0173-5370 0000-0002-2949-7108 |
| PQID | 2688692669 |
| PQPubID | 36024 |
| PageCount | 22 |
| ParticipantIDs | proquest_journals_2688692669 ieee_primary_9745052 crossref_citationtrail_10_1109_TIT_2022_3163342 crossref_primary_10_1109_TIT_2022_3163342 |
| PublicationCentury | 2000 |
| PublicationDate | 2022-08-01 |
| PublicationDateYYYYMMDD | 2022-08-01 |
| PublicationDate_xml | – month: 08 year: 2022 text: 2022-08-01 day: 01 |
| PublicationDecade | 2020 |
| PublicationPlace | New York |
| PublicationPlace_xml | – name: New York |
| PublicationTitle | IEEE transactions on information theory |
| PublicationTitleAbbrev | TIT |
| PublicationYear | 2022 |
| Publisher | IEEE The Institute of Electrical and Electronics Engineers, Inc. (IEEE) |
| Publisher_xml | – name: IEEE – name: The Institute of Electrical and Electronics Engineers, Inc. (IEEE) |
| References | ref13 ref12 ref15 ref14 ref11 ref10 ref17 ref16 ref19 ref18 ref50 ref46 ref45 ref48 ref47 ref42 ref41 ref44 ref43 ref49 ref8 ref7 ref9 ref4 ref3 ref6 ref5 ref40 Maillard (ref32) 2020; 33 ref35 ref34 ref37 ref36 ref30 ref33 ref2 ref1 ref39 ref38 Minka (ref29) Gerbelot (ref31) ref24 ref23 ref26 ref25 ref20 ref22 ref21 ref27 Opper (ref28) 2005; 6 |
| References_xml | – ident: ref13 doi: 10.1080/00018732.2016.1211393 – ident: ref1 doi: 10.1109/ISIT44484.2020.9173999 – ident: ref4 doi: 10.1109/ISIT.2018.8437792 – ident: ref42 doi: 10.1103/PhysRevLett.75.2847 – ident: ref14 doi: 10.1088/0305-4470/36/43/030 – ident: ref41 doi: 10.1088/0305-4470/13/4/009 – ident: ref23 doi: 10.1109/ISIT.2015.7282968 – ident: ref21 doi: 10.1109/ITW.2016.7606840 – ident: ref27 doi: 10.1103/PhysRevLett.86.3695 – ident: ref9 doi: 10.1093/acprof:oso/9780198570837.001.0001 – ident: ref18 doi: 10.1093/imaiai/iay021 – ident: ref30 doi: 10.1109/TIT.2022.3222913 – ident: ref49 doi: 10.1088/1751-8113/41/32/324013 – ident: ref50 doi: 10.1088/1751-8113/42/1/015005 – ident: ref2 doi: 10.1109/TIT.2019.2916359 – ident: ref34 doi: 10.1088/1742-5468/abb8c9 – ident: ref11 doi: 10.1088/0305-4470/11/5/028 – ident: ref47 doi: 10.1088/0305-4470/21/1/030 – ident: ref33 doi: 10.1103/PhysRevE.99.062140 – ident: ref10 doi: 10.1088/1742-6596/95/1/012001 – ident: ref37 doi: 10.1109/ISIT.2018.8437522 – start-page: 362 volume-title: Proc. 17th Conf. Uncertainty Artif. Intell ident: ref29 article-title: Expectation propagation for approximate Bayesian inference – start-page: 1682 volume-title: Proc. Conf. Learn. Theory ident: ref31 article-title: Asymptotic errors for high-dimensional convex penalized linear regression beyond Gaussian matrices – ident: ref43 doi: 10.1088/0305-4470/32/21/302 – ident: ref25 doi: 10.1109/ISIT.2019.8849790 – ident: ref45 doi: 10.1088/1742-5468/ac1403 – ident: ref20 doi: 10.1109/LSP.2014.2351822 – ident: ref48 doi: 10.1051/jphys:0198900500200305700 – ident: ref40 doi: 10.1088/0022-3719/12/11/008 – ident: ref36 doi: 10.1109/TIT.2016.2556702 – ident: ref39 doi: 10.1017/9781108120494 – ident: ref5 doi: 10.1109/ISIT.2017.8006578 – ident: ref16 doi: 10.1109/TIT.2010.2094817 – ident: ref15 doi: 10.1007/s00220-013-1862-3 – ident: ref24 doi: 10.1088/1751-8113/49/11/114002 – volume: 6 start-page: 2177 year: 2005 ident: ref28 article-title: Expectation consistent approximate inference publication-title: J. Mach. Learn. Res. – ident: ref6 doi: 10.1109/ACCESS.2017.2653119 – ident: ref7 doi: 10.1109/JSAIT.2020.2986321 – ident: ref12 doi: 10.1142/0271 – ident: ref17 doi: 10.1093/imaiai/iat004 – ident: ref35 doi: 10.1007/3-540-29060-5 – ident: ref8 doi: 10.1109/TIT.2013.2250578 – ident: ref26 doi: 10.1103/PhysRevE.64.056131 – volume: 33 start-page: 11071 volume-title: Advances in Neural Information Processing Systems year: 2020 ident: ref32 article-title: Phase retrieval in high dimensions: Statistical and computational phase transitions – ident: ref22 doi: 10.1109/ITW.2014.6970819 – ident: ref3 doi: 10.1109/ACSSC.2016.7869633 – ident: ref19 doi: 10.1561/2200000092 – ident: ref44 doi: 10.1109/ISIT.2014.6874828 – ident: ref46 doi: 10.1109/PGEC.1965.264137 – ident: ref38 doi: 10.1073/pnas.1802705116 |
| SSID | ssj0014512 |
| Score | 2.5117588 |
| Snippet | In this study, macroscopic properties of the vector approximate message passing (VAMP) algorithm for inference of generalized linear models are investigated... |
| SourceID | proquest crossref ieee |
| SourceType | Aggregation Database Enrichment Source Index Database Publisher |
| StartPage | 5579 |
| SubjectTerms | Algorithms Analytical models Approximation algorithms compressive sensing Empirical analysis Generalized linear models Heuristic algorithms Heuristic methods Inference Inference algorithms Invariants Mathematical models Matrices (mathematics) Message passing Message passing algorithms Optimization Performance assessment phase transitions random matrices Rotation Rotation measurement Statistical mechanics Statistical models Symmetry |
| Title | Macroscopic Analysis of Vector Approximate Message Passing in a Model-Mismatched Setting |
| URI | https://ieeexplore.ieee.org/document/9745052 https://www.proquest.com/docview/2688692669 |
| Volume | 68 |
| WOSCitedRecordID | wos000838527100041&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| hasFullText | 1 |
| inHoldings | 1 |
| isFullTextHit | |
| isPrint | |
| journalDatabaseRights | – providerCode: PRVIEE databaseName: IEEE Electronic Library (IEL) customDbUrl: eissn: 1557-9654 dateEnd: 99991231 omitProxy: false ssIdentifier: ssj0014512 issn: 0018-9448 databaseCode: RIE dateStart: 19630101 isFulltext: true titleUrlDefault: https://ieeexplore.ieee.org/ providerName: IEEE |
| link | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV1NSwMxEB2qeNCD32K1Sg5eBNdmk_1IjkUsemgpWKW3JZudyIK0xVbx55tkd4uiCN72MLOEfZnNTDJ5D-BCa2GT1NQE2hgZRHHOA2ViFviG94JFSY7Ui02kw6GYTOSoBVeruzCI6JvP8No9-rP8Yqbf3FZZ1-a-TndtDdbSNK3uaq1ODKI4rJjBQxvAtuZojiSp7I7vx7YQZMzWpwnnEfu2BHlNlR8_Yr-69Hf-N65d2K6zSNKrYN-DFk73YadRaCB1wO7D1he6wQOYDJQbkZ7NS00aNhIyM-TJb92TnuMX_yhtDotk4KRRnpGMbHJtnUk5JYo44bSXYFAurIkFuyAP6NumD-Gxfzu-uQtqZYVAMxkuA8aRa-FIilksHOdgbFSYGlt7xSYJUUYqFUVItUiMYJwrRwGU85xKLKihOeNHsD6dTfEYCFOCYqilyp1EeI624uQKmTE0jnPDRRu6zcfOdE077tQvXjJfflCZWXgyB09Ww9OGy5XHvKLc-MP2wMGxsquRaEOnwTOrY3KRsUSIRNqERJ787nUKm-7dVXtfB9aXr294Bhv6fVkuXs_9dPsEYY3RBg |
| linkProvider | IEEE |
| linkToHtml | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV1NT-MwEB1BQWL3wDfaLl8-cEEi1LHj1D6iFQgErZAoqLfIccYoUtUi2iJ-_tpOUrECrcQthxnFyvPEM_b4PYATY6RLUrs2MtaqKBE5j7QVLAoN7wVL0hxpEJvo9vtyOFT3S3C2uAuDiKH5DM_9YzjLLyZm7rfKOi739bpry7AikoTF1W2txZlBIuKKGzx2IeyqjuZQkqrO4GbgSkHGXIWacp6wfxahoKry6Vcc1perje-NbBPW6zySXFTAb8ESjrdho9FoIHXIbsPPD4SDOzDsaT8iM3kpDWn4SMjEkqeweU8uPMP4e-myWCQ9L47yjOTepdfOmZRjoomXThtFvXLqTBzcBXnA0Di9C49Xl4M_11GtrRAZpuJZxDhyIz1NMRPSsw4Kq-OuddWXsGmMKtFdWcTUyNRKxrn2JEA5z6nCglqaM74HrfFkjL-AMC0pxkbp3IuE5-hqTq6RWUuFyC2Xbeg0HzszNfG4178YZaEAoSpz8GQenqyGpw2nC4-XinTjP7Y7Ho6FXY1EGw4aPLM6KqcZS6VMlUtJ1O-vvY5h7XrQu8vubvq3-_DDv6dq9juA1ux1joewat5m5fT1KEy9v2sN1E0 |
| openUrl | ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Macroscopic+Analysis+of+Vector+Approximate+Message+Passing+in+a+Model-Mismatched+Setting&rft.jtitle=IEEE+transactions+on+information+theory&rft.au=Takahashi%2C+Takashi&rft.au=Kabashima%2C+Yoshiyuki&rft.date=2022-08-01&rft.pub=IEEE&rft.issn=0018-9448&rft.volume=68&rft.issue=8&rft.spage=5579&rft.epage=5600&rft_id=info:doi/10.1109%2FTIT.2022.3163342&rft.externalDocID=9745052 |
| thumbnail_l | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/lc.gif&issn=0018-9448&client=summon |
| thumbnail_m | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/mc.gif&issn=0018-9448&client=summon |
| thumbnail_s | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/sc.gif&issn=0018-9448&client=summon |