Macroscopic Analysis of Vector Approximate Message Passing in a Model-Mismatched Setting

In this study, macroscopic properties of the vector approximate message passing (VAMP) algorithm for inference of generalized linear models are investigated using a non-rigorous heuristic method of statistical mechanics when the true posterior cannot be used and the measurement matrix is a sample fr...

Celý popis

Uloženo v:
Podrobná bibliografie
Vydáno v:IEEE transactions on information theory Ročník 68; číslo 8; s. 5579 - 5600
Hlavní autoři: Takahashi, Takashi, Kabashima, Yoshiyuki
Médium: Journal Article
Jazyk:angličtina
Vydáno: New York IEEE 01.08.2022
The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Témata:
Algorithms
Analytical models
Approximation algorithms
compressive sensing
Empirical analysis
Generalized linear models
Heuristic algorithms
Heuristic methods
Inference
Inference algorithms
Invariants
Mathematical models
Matrices (mathematics)
Message passing
Message passing algorithms
Optimization
Performance assessment
phase transitions
random matrices
Rotation
Rotation measurement
Statistical mechanics
Statistical models
Symmetry
ISSN:0018-9448, 1557-9654
On-line přístup:Získat plný text
Tagy: Přidat tag
Žádné tagy, Buďte první, kdo vytvoří štítek k tomuto záznamu!
Abstract In this study, macroscopic properties of the vector approximate message passing (VAMP) algorithm for inference of generalized linear models are investigated using a non-rigorous heuristic method of statistical mechanics when the true posterior cannot be used and the measurement matrix is a sample from rotation-invariant random matrix ensembles. The focus is on the correspondence between the non-rigorous replica analysis of statistical mechanics and the performance assessment of VAMP in the model-mismatched setting. The correspondence of this kind is well-known when the measurement matrix has independent and identically distributed entries. However, when the measurement matrix follows a general rotation-invariant matrix ensemble, the correspondence has been validated only under limited cases, such as the Bayes optimal inference or the convex empirical risk minimization. The result presented in this paper is to extend the scope of such correspondence. Herein, we heuristically derive the explicit formula of state-evolution equations, which macroscopically describe VAMP dynamics for the current model-mismatched case, and show that their fixed point is generally consistent with the replica symmetric solution obtained by the replica method of statistical mechanics. We also show that the fixed point of VAMP can exhibit a microscopic instability, which indicates that message variables continue to move by VAMP while their macroscopically summarized quantities converge to fixed values. The critical condition the for microscopic instability agrees with that for breaking the replica symmetry that is derived within the non-rigorous replica analysis. The results of the numerical experiments cross-check our findings.
AbstractList In this study, macroscopic properties of the vector approximate message passing (VAMP) algorithm for inference of generalized linear models are investigated using a non-rigorous heuristic method of statistical mechanics when the true posterior cannot be used and the measurement matrix is a sample from rotation-invariant random matrix ensembles. The focus is on the correspondence between the non-rigorous replica analysis of statistical mechanics and the performance assessment of VAMP in the model-mismatched setting. The correspondence of this kind is well-known when the measurement matrix has independent and identically distributed entries. However, when the measurement matrix follows a general rotation-invariant matrix ensemble, the correspondence has been validated only under limited cases, such as the Bayes optimal inference or the convex empirical risk minimization. The result presented in this paper is to extend the scope of such correspondence. Herein, we heuristically derive the explicit formula of state-evolution equations, which macroscopically describe VAMP dynamics for the current model-mismatched case, and show that their fixed point is generally consistent with the replica symmetric solution obtained by the replica method of statistical mechanics. We also show that the fixed point of VAMP can exhibit a microscopic instability, which indicates that message variables continue to move by VAMP while their macroscopically summarized quantities converge to fixed values. The critical condition the for microscopic instability agrees with that for breaking the replica symmetry that is derived within the non-rigorous replica analysis. The results of the numerical experiments cross-check our findings.
Author Kabashima, Yoshiyuki
Takahashi, Takashi
Author_xml – sequence: 1
  givenname: Takashi
  orcidid: 0000-0002-0173-5370
  surname: Takahashi
  fullname: Takahashi, Takashi
  email: takashi-takahashi@g.ecc.u-tokyo.ac.jp
  organization: Department of Physics, Graduate School of Science, Institute for Physics of Intelligence, The University of Tokyo, Bunkyo City, Tokyo, Japan
– sequence: 2
  givenname: Yoshiyuki
  orcidid: 0000-0002-2949-7108
  surname: Kabashima
  fullname: Kabashima, Yoshiyuki
  email: kaba@phys.u-tokyo.ac.jp
  organization: Department of Physics, Graduate School of Science, Institute for Physics of Intelligence, The University of Tokyo, Bunkyo City, Tokyo, Japan
BookMark eNp9kE1LAzEQhoNUsK3eBS8Bz1vzsckmx1L8KLQoWMXbkmZna8q6WTcp2H9vSsWDB0_DMM87zDwjNGh9CwhdUjKhlOib1Xw1YYSxCaeS85ydoCEVosi0FPkADQmhKtN5rs7QKIRtanNB2RC9LY3tfbC-cxZPW9PsgwvY1_gVbPQ9nnZd77_ch4mAlxCC2QB-MiG4doNdiw1e-gqabOlCQuw7VPgZYkzTc3RamybAxU8do5e729XsIVs83s9n00VmmaYxYxy4VURqwoSivChEbWhRK61FLSno3BSqosQqWSvGuSkoIWu-JhoqUpM142N0fdyb7vzcQYjl1u_69EgomVRKaialTpQ8UodnQw91aV000fk29sY1JSXlwWKZLJYHi-WPxRQkf4Jdn2z0-_8iV8eIA4BfXBe5IILxb3Bcfb4
CODEN IETTAW
CitedBy_id crossref_primary_10_1109_TIT_2023_3321575
crossref_primary_10_1088_1751_8121_adce65
crossref_primary_10_1088_1742_5468_adbe3f
crossref_primary_10_1109_TIT_2022_3222913
crossref_primary_10_1109_TSP_2023_3301137
Cites_doi 10.1080/00018732.2016.1211393
10.1109/ISIT44484.2020.9173999
10.1109/ISIT.2018.8437792
10.1103/PhysRevLett.75.2847
10.1088/0305-4470/36/43/030
10.1088/0305-4470/13/4/009
10.1109/ISIT.2015.7282968
10.1109/ITW.2016.7606840
10.1103/PhysRevLett.86.3695
10.1093/acprof:oso/9780198570837.001.0001
10.1093/imaiai/iay021
10.1109/TIT.2022.3222913
10.1088/1751-8113/41/32/324013
10.1088/1751-8113/42/1/015005
10.1109/TIT.2019.2916359
10.1088/1742-5468/abb8c9
10.1088/0305-4470/11/5/028
10.1088/0305-4470/21/1/030
10.1103/PhysRevE.99.062140
10.1088/1742-6596/95/1/012001
10.1109/ISIT.2018.8437522
10.1088/0305-4470/32/21/302
10.1109/ISIT.2019.8849790
10.1088/1742-5468/ac1403
10.1109/LSP.2014.2351822
10.1051/jphys:0198900500200305700
10.1088/0022-3719/12/11/008
10.1109/TIT.2016.2556702
10.1017/9781108120494
10.1109/ISIT.2017.8006578
10.1109/TIT.2010.2094817
10.1007/s00220-013-1862-3
10.1088/1751-8113/49/11/114002
10.1109/ACCESS.2017.2653119
10.1109/JSAIT.2020.2986321
10.1142/0271
10.1093/imaiai/iat004
10.1007/3-540-29060-5
10.1109/TIT.2013.2250578
10.1103/PhysRevE.64.056131
10.1109/ITW.2014.6970819
10.1109/ACSSC.2016.7869633
10.1561/2200000092
10.1109/ISIT.2014.6874828
10.1109/PGEC.1965.264137
10.1073/pnas.1802705116
ContentType Journal Article
Copyright Copyright The Institute of Electrical and Electronics Engineers, Inc. (IEEE) 2022
Copyright_xml – notice: Copyright The Institute of Electrical and Electronics Engineers, Inc. (IEEE) 2022
DBID 97E
RIA
RIE
AAYXX
CITATION
7SC
7SP
8FD
JQ2
L7M
L~C
L~D
DOI 10.1109/TIT.2022.3163342
DatabaseName IEEE All-Society Periodicals Package (ASPP) 2005–Present
IEEE All-Society Periodicals Package (ASPP) 1998–Present
IEEE Xplore
CrossRef
Computer and Information Systems Abstracts
Electronics & Communications Abstracts
Technology Research Database
ProQuest Computer Science Collection
Advanced Technologies Database with Aerospace
Computer and Information Systems Abstracts – Academic
Computer and Information Systems Abstracts Professional
DatabaseTitle CrossRef
Technology Research Database
Computer and Information Systems Abstracts – Academic
Electronics & Communications Abstracts
ProQuest Computer Science Collection
Computer and Information Systems Abstracts
Advanced Technologies Database with Aerospace
Computer and Information Systems Abstracts Professional
DatabaseTitleList
Technology Research Database
Database_xml – sequence: 1
  dbid: RIE
  name: IEEE Electronic Library (IEL)
  url: https://ieeexplore.ieee.org/
  sourceTypes: Publisher
DeliveryMethod fulltext_linktorsrc
Discipline Engineering
Computer Science
EISSN 1557-9654
EndPage 5600
ExternalDocumentID 10_1109_TIT_2022_3163342
9745052
Genre orig-research
GrantInformation_xml – fundername: Japan Society for the Promotion of Science (JSPS) KAKENHI
  grantid: 19J0711; 17H00764; 21K21310
  funderid: 10.13039/501100001691
– fundername: Japan Science and Technology Agency (JST) Core Research for Evolutional Science and Technology (CREST), Japan
  grantid: JPMJCR1912
  funderid: 10.13039/501100003382
GroupedDBID -~X
.DC
0R~
29I
3EH
4.4
5GY
5VS
6IK
97E
AAJGR
AARMG
AASAJ
AAWTH
ABAZT
ABFSI
ABQJQ
ABVLG
ACGFO
ACGFS
ACGOD
ACIWK
AENEX
AETEA
AETIX
AGQYO
AGSQL
AHBIQ
AI.
AIBXA
AKJIK
AKQYR
ALLEH
ALMA_UNASSIGNED_HOLDINGS
ASUFR
ATWAV
BEFXN
BFFAM
BGNUA
BKEBE
BPEOZ
CS3
DU5
E.L
EBS
EJD
F5P
HZ~
H~9
IAAWW
IBMZZ
ICLAB
IDIHD
IFIPE
IFJZH
IPLJI
JAVBF
LAI
M43
MS~
O9-
OCL
P2P
PQQKQ
RIA
RIE
RNS
RXW
TAE
TN5
VH1
VJK
AAYXX
CITATION
7SC
7SP
8FD
JQ2
L7M
L~C
L~D
ID FETCH-LOGICAL-c291t-23e3c8069025813775fa17f8995f61e94a78d10c86f8233a7100b3b09ed0f0b23
IEDL.DBID RIE
ISICitedReferencesCount 8
ISICitedReferencesURI http://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=Summon&SrcAuth=ProQuest&DestLinkType=CitingArticles&DestApp=WOS_CPL&KeyUT=000838527100041&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D
ISSN 0018-9448
IngestDate Sun Nov 09 06:08:59 EST 2025
Sat Nov 29 03:31:48 EST 2025
Tue Nov 18 20:59:37 EST 2025
Wed Aug 27 02:25:44 EDT 2025
IsPeerReviewed true
IsScholarly true
Issue 8
Language English
License https://ieeexplore.ieee.org/Xplorehelp/downloads/license-information/IEEE.html
https://doi.org/10.15223/policy-029
https://doi.org/10.15223/policy-037
LinkModel DirectLink
MergedId FETCHMERGED-LOGICAL-c291t-23e3c8069025813775fa17f8995f61e94a78d10c86f8233a7100b3b09ed0f0b23
Notes ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
ORCID 0000-0002-0173-5370
0000-0002-2949-7108
PQID 2688692669
PQPubID 36024
PageCount 22
ParticipantIDs proquest_journals_2688692669
ieee_primary_9745052
crossref_citationtrail_10_1109_TIT_2022_3163342
crossref_primary_10_1109_TIT_2022_3163342
PublicationCentury 2000
PublicationDate 2022-08-01
PublicationDateYYYYMMDD 2022-08-01
PublicationDate_xml – month: 08
  year: 2022
  text: 2022-08-01
  day: 01
PublicationDecade 2020
PublicationPlace New York
PublicationPlace_xml – name: New York
PublicationTitle IEEE transactions on information theory
PublicationTitleAbbrev TIT
PublicationYear 2022
Publisher IEEE
The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Publisher_xml – name: IEEE
– name: The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
References ref13
ref12
ref15
ref14
ref11
ref10
ref17
ref16
ref19
ref18
ref50
ref46
ref45
ref48
ref47
ref42
ref41
ref44
ref43
ref49
ref8
ref7
ref9
ref4
ref3
ref6
ref5
ref40
Maillard (ref32) 2020; 33
ref35
ref34
ref37
ref36
ref30
ref33
ref2
ref1
ref39
ref38
Minka (ref29)
Gerbelot (ref31)
ref24
ref23
ref26
ref25
ref20
ref22
ref21
ref27
Opper (ref28) 2005; 6
References_xml – ident: ref13
  doi: 10.1080/00018732.2016.1211393
– ident: ref1
  doi: 10.1109/ISIT44484.2020.9173999
– ident: ref4
  doi: 10.1109/ISIT.2018.8437792
– ident: ref42
  doi: 10.1103/PhysRevLett.75.2847
– ident: ref14
  doi: 10.1088/0305-4470/36/43/030
– ident: ref41
  doi: 10.1088/0305-4470/13/4/009
– ident: ref23
  doi: 10.1109/ISIT.2015.7282968
– ident: ref21
  doi: 10.1109/ITW.2016.7606840
– ident: ref27
  doi: 10.1103/PhysRevLett.86.3695
– ident: ref9
  doi: 10.1093/acprof:oso/9780198570837.001.0001
– ident: ref18
  doi: 10.1093/imaiai/iay021
– ident: ref30
  doi: 10.1109/TIT.2022.3222913
– ident: ref49
  doi: 10.1088/1751-8113/41/32/324013
– ident: ref50
  doi: 10.1088/1751-8113/42/1/015005
– ident: ref2
  doi: 10.1109/TIT.2019.2916359
– ident: ref34
  doi: 10.1088/1742-5468/abb8c9
– ident: ref11
  doi: 10.1088/0305-4470/11/5/028
– ident: ref47
  doi: 10.1088/0305-4470/21/1/030
– ident: ref33
  doi: 10.1103/PhysRevE.99.062140
– ident: ref10
  doi: 10.1088/1742-6596/95/1/012001
– ident: ref37
  doi: 10.1109/ISIT.2018.8437522
– start-page: 362
  volume-title: Proc. 17th Conf. Uncertainty Artif. Intell
  ident: ref29
  article-title: Expectation propagation for approximate Bayesian inference
– start-page: 1682
  volume-title: Proc. Conf. Learn. Theory
  ident: ref31
  article-title: Asymptotic errors for high-dimensional convex penalized linear regression beyond Gaussian matrices
– ident: ref43
  doi: 10.1088/0305-4470/32/21/302
– ident: ref25
  doi: 10.1109/ISIT.2019.8849790
– ident: ref45
  doi: 10.1088/1742-5468/ac1403
– ident: ref20
  doi: 10.1109/LSP.2014.2351822
– ident: ref48
  doi: 10.1051/jphys:0198900500200305700
– ident: ref40
  doi: 10.1088/0022-3719/12/11/008
– ident: ref36
  doi: 10.1109/TIT.2016.2556702
– ident: ref39
  doi: 10.1017/9781108120494
– ident: ref5
  doi: 10.1109/ISIT.2017.8006578
– ident: ref16
  doi: 10.1109/TIT.2010.2094817
– ident: ref15
  doi: 10.1007/s00220-013-1862-3
– ident: ref24
  doi: 10.1088/1751-8113/49/11/114002
– volume: 6
  start-page: 2177
  year: 2005
  ident: ref28
  article-title: Expectation consistent approximate inference
  publication-title: J. Mach. Learn. Res.
– ident: ref6
  doi: 10.1109/ACCESS.2017.2653119
– ident: ref7
  doi: 10.1109/JSAIT.2020.2986321
– ident: ref12
  doi: 10.1142/0271
– ident: ref17
  doi: 10.1093/imaiai/iat004
– ident: ref35
  doi: 10.1007/3-540-29060-5
– ident: ref8
  doi: 10.1109/TIT.2013.2250578
– ident: ref26
  doi: 10.1103/PhysRevE.64.056131
– volume: 33
  start-page: 11071
  volume-title: Advances in Neural Information Processing Systems
  year: 2020
  ident: ref32
  article-title: Phase retrieval in high dimensions: Statistical and computational phase transitions
– ident: ref22
  doi: 10.1109/ITW.2014.6970819
– ident: ref3
  doi: 10.1109/ACSSC.2016.7869633
– ident: ref19
  doi: 10.1561/2200000092
– ident: ref44
  doi: 10.1109/ISIT.2014.6874828
– ident: ref46
  doi: 10.1109/PGEC.1965.264137
– ident: ref38
  doi: 10.1073/pnas.1802705116
SSID ssj0014512
Score 2.5117588
Snippet In this study, macroscopic properties of the vector approximate message passing (VAMP) algorithm for inference of generalized linear models are investigated...
SourceID proquest
crossref
ieee
SourceType Aggregation Database
Enrichment Source
Index Database
Publisher
StartPage 5579
SubjectTerms Algorithms
Analytical models
Approximation algorithms
compressive sensing
Empirical analysis
Generalized linear models
Heuristic algorithms
Heuristic methods
Inference
Inference algorithms
Invariants
Mathematical models
Matrices (mathematics)
Message passing
Message passing algorithms
Optimization
Performance assessment
phase transitions
random matrices
Rotation
Rotation measurement
Statistical mechanics
Statistical models
Symmetry
Title Macroscopic Analysis of Vector Approximate Message Passing in a Model-Mismatched Setting
URI https://ieeexplore.ieee.org/document/9745052
https://www.proquest.com/docview/2688692669
Volume 68
WOSCitedRecordID wos000838527100041&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D
hasFullText 1
inHoldings 1
isFullTextHit
isPrint
journalDatabaseRights – providerCode: PRVIEE
  databaseName: IEEE Electronic Library (IEL)
  customDbUrl:
  eissn: 1557-9654
  dateEnd: 99991231
  omitProxy: false
  ssIdentifier: ssj0014512
  issn: 0018-9448
  databaseCode: RIE
  dateStart: 19630101
  isFulltext: true
  titleUrlDefault: https://ieeexplore.ieee.org/
  providerName: IEEE
link http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV1JSwMxGP2oxYMerCvWjRy8CI6dSTozybGIooeWglV6GzKZRAZkWrqIP98vmQVFEbzN4csQ8rJ8L8t7AJc6MkIFfurxNNNeP06ZxwVnnlbILwwyaCVTZzYRj0Z8OhXjFlw3b2G01u7ymb6xn-4sP5uptd0q62Hua33XNmAjjuPyrVZzYtAPg1IZPMABjJyjPpL0RW_yOEEiSCny04ixPv22BDlPlR8TsVtd7jv_q9cu7FRZJBmUsO9BSxf70KkdGkg1YPdh-4vc4AFMh9LWSM3muSK1GgmZGfLitu7JwOqLf-SYw2oytNYor5qMMbnGwiQviCTWOO3NG-ZLDEGwM_Kk3bXpQ3i-v5vcPniVs4KnqAhWHmWaKW5FimnIreZgaGQQG-ReoYkCLfoy5lngKx4ZThmTVgIoZakvdOYbP6XsCNrFrNDHQJSITIwsESdw5Gohk9gSIsiklJh60lR1oVc3dqIq2XHrfvGWOPrhiwThSSw8SQVPF66aEvNScuOP2AMLRxNXIdGFsxrPpBqTy4RGnEcCExJx8nupU9iy_y6v951Be7VY63PYVO-rfLm4cN3tEy6D0TY
linkProvider IEEE
linkToHtml http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV1JS8QwGP1wA_XgLo5rDl4E6zRJl-Qooig6g-AocytpmkhhmBFnFH--X9J2UBTBWw9faMjL8r0s7wEcm8RKTcM8EHlhgijNeSCk4IHRyC8sMmitcm82kXa7ot-X9zNwOn0LY4zxl8_Mmfv0Z_nFSL-5rbI25r7Od20W5uMoYrR6rTU9M4hiWmmDUxzCyDqaQ8lQtns3PaSCjCFDTTiP2LdFyLuq_JiK_fpytfq_mq3BSp1HkvMK-HWYMcMNWG08Gkg9ZDdg-Yvg4Cb0O8rVSI9eSk0aPRIysuTJb96Tc6cw_lFiFmtIx5mjPBtyj-k1FiblkCjirNMGQaccYwjCXZAH4y9Ob8Hj1WXv4jqovRUCzSSdBIwbroWTKWaxcKqDsVU0tci-YptQIyOVioKGWiRWMM6VEwHKeR5KU4Q2zBnfhrnhaGh2gGiZ2BR5Ik7hyNZirrAlJC2UUph8sly3oN00dqZr4XHnfzHIPAEJZYbwZA6erIanBSfTEi-V6MYfsZsOjmlcjUQL9hs8s3pUjjOWCJFITEnk7u-ljmDxute5y-5uurd7sOT-U13224e5yeubOYAF_T4px6-Hvut9Ajrz1H0
openUrl ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Macroscopic+Analysis+of+Vector+Approximate+Message+Passing+in+a+Model-Mismatched+Setting&rft.jtitle=IEEE+transactions+on+information+theory&rft.au=Takahashi%2C+Takashi&rft.au=Kabashima%2C+Yoshiyuki&rft.date=2022-08-01&rft.pub=IEEE&rft.issn=0018-9448&rft.volume=68&rft.issue=8&rft.spage=5579&rft.epage=5600&rft_id=info:doi/10.1109%2FTIT.2022.3163342&rft.externalDocID=9745052
thumbnail_l http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/lc.gif&issn=0018-9448&client=summon
thumbnail_m http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/mc.gif&issn=0018-9448&client=summon
thumbnail_s http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/sc.gif&issn=0018-9448&client=summon