Bethe Free Energy and Extrinsics in Approximate Message Passing
The Bethe Free Energy (BFE) has been found to be closely connected to various message passing algorithms. Studies have indicated that the BFE shares stationary points with message passing algorithms like Belief Propagation (BP) and Expectation Propagation (EP). Generalized Approximate Message Passin...
Uložené v:
| Vydané v: | Conference record - Asilomar Conference on Signals, Systems, & Computers s. 897 - 901 |
|---|---|
| Hlavní autori: | , |
| Médium: | Konferenčný príspevok.. |
| Jazyk: | English |
| Vydavateľské údaje: |
IEEE
29.10.2023
|
| Predmet: | |
| ISSN: | 2576-2303 |
| On-line prístup: | Získať plný text |
| Tagy: |
Pridať tag
Žiadne tagy, Buďte prvý, kto otaguje tento záznam!
|
| Abstract | The Bethe Free Energy (BFE) has been found to be closely connected to various message passing algorithms. Studies have indicated that the BFE shares stationary points with message passing algorithms like Belief Propagation (BP) and Expectation Propagation (EP). Generalized Approximate Message Passing (GAMP) algorithms have demonstrated significant efficacy in signal recovery. Nevertheless, they may encounter convergence issues. To address these convergence issues, algorithms based on the minimization of the large system limit (LSL) BFE have been introduced. In this paper, we explore the BFE within the context of Generalized Linear Models (GLMs). Applying a BFE based EP approach leads to the re(G)VAMP algorithm which provides asymptotically exact marginal posteriors based on asymptotically Gaussian extrinsics. It also provides equivalent Gaussian priors and hence an equivalent overall Gaussian linear model, which allows the application of large random matrix theory. We show how this leads to the LSL BFE on which GAMP is based. We also reveal the intimate relation of extrinsics to Component-Wise Conditionally Unbiased Minimum Mean Squared Error (CWCU MMSE) estimation for which we provide a novel shortcut derivation in the GLM. |
|---|---|
| AbstractList | The Bethe Free Energy (BFE) has been found to be closely connected to various message passing algorithms. Studies have indicated that the BFE shares stationary points with message passing algorithms like Belief Propagation (BP) and Expectation Propagation (EP). Generalized Approximate Message Passing (GAMP) algorithms have demonstrated significant efficacy in signal recovery. Nevertheless, they may encounter convergence issues. To address these convergence issues, algorithms based on the minimization of the large system limit (LSL) BFE have been introduced. In this paper, we explore the BFE within the context of Generalized Linear Models (GLMs). Applying a BFE based EP approach leads to the re(G)VAMP algorithm which provides asymptotically exact marginal posteriors based on asymptotically Gaussian extrinsics. It also provides equivalent Gaussian priors and hence an equivalent overall Gaussian linear model, which allows the application of large random matrix theory. We show how this leads to the LSL BFE on which GAMP is based. We also reveal the intimate relation of extrinsics to Component-Wise Conditionally Unbiased Minimum Mean Squared Error (CWCU MMSE) estimation for which we provide a novel shortcut derivation in the GLM. |
| Author | Slock, Dirk Zhao, Zilu |
| Author_xml | – sequence: 1 givenname: Zilu surname: Zhao fullname: Zhao, Zilu email: zilu.zhao@eurecom.fr organization: EURECOM,Communication Systems Department,France – sequence: 2 givenname: Dirk surname: Slock fullname: Slock, Dirk email: dirk.slock@eurecom.fr organization: EURECOM,Communication Systems Department,France |
| BookMark | eNo1j7FOwzAURQ0Cibb0DxgsMSfYfjaOJ1QqByoVygBz5SQvqRG4kZ2h_fsGAdNZ7r06d0ouwj4gIbec5Zwzc7ey1i43r6UySshcMAE5Z1JrxuCMzI02BSgGQmkpz8lk5H0mgMEVmab0ydhYKMSEPDzisENaRkRqA8buSF1oqD0M0Yfk60R9oIu-j_uD_3YD0hdMyXVI31xKPnTX5LJ1Xwnnf5yRj9K-L5-z9eZptVyss50wfMiKFkY17hSvHNS1LrCoa2MUSOlqUWnXQqUZb6FxIBwCUyh-Uqh0JVWDMCM3v7seEbd9HGXicft_GE7pK01Y |
| ContentType | Conference Proceeding |
| DBID | 6IE 6IH CBEJK RIE RIO |
| DOI | 10.1109/IEEECONF59524.2023.10477003 |
| DatabaseName | IEEE Electronic Library (IEL) Conference Proceedings IEEE Proceedings Order Plan (POP) 1998-present by volume IEEE Xplore All Conference Proceedings IEEE Electronic Library (IEL) IEEE Proceedings Order Plans (POP) 1998-present |
| DatabaseTitleList | |
| Database_xml | – sequence: 1 dbid: RIE name: IEEE Electronic Library (IEL) url: https://ieeexplore.ieee.org/ sourceTypes: Publisher |
| DeliveryMethod | fulltext_linktorsrc |
| Discipline | Engineering |
| EISBN | 9798350325744 |
| EISSN | 2576-2303 |
| EndPage | 901 |
| ExternalDocumentID | 10477003 |
| Genre | orig-research |
| GrantInformation_xml | – fundername: ORANGE funderid: 10.13039/501100003951 |
| GroupedDBID | 6IE 6IF 6IH 6IK 6IL 6IM 6IN AAJGR AAWTH ABLEC ACGFS ADZIZ ALMA_UNASSIGNED_HOLDINGS BEFXN BFFAM BGNUA BKEBE BPEOZ CBEJK CHZPO IEGSK IJVOP IPLJI M43 OCL RIE RIL RIO |
| ID | FETCH-LOGICAL-h291t-8f34771a51ba3cc78e8cc995344ac2b7af3b701f3da32ae305e2c78ee57b45de3 |
| IEDL.DBID | RIE |
| ISICitedReferencesCount | 0 |
| ISICitedReferencesURI | http://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=Summon&SrcAuth=ProQuest&DestLinkType=CitingArticles&DestApp=WOS_CPL&KeyUT=001207755100162&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| IngestDate | Wed Aug 27 02:09:41 EDT 2025 |
| IsDoiOpenAccess | false |
| IsOpenAccess | true |
| IsPeerReviewed | false |
| IsScholarly | true |
| Language | English |
| LinkModel | DirectLink |
| MergedId | FETCHMERGED-LOGICAL-h291t-8f34771a51ba3cc78e8cc995344ac2b7af3b701f3da32ae305e2c78ee57b45de3 |
| OpenAccessLink | https://hal.science/hal-04501734 |
| PageCount | 5 |
| ParticipantIDs | ieee_primary_10477003 |
| PublicationCentury | 2000 |
| PublicationDate | 2023-Oct.-29 |
| PublicationDateYYYYMMDD | 2023-10-29 |
| PublicationDate_xml | – month: 10 year: 2023 text: 2023-Oct.-29 day: 29 |
| PublicationDecade | 2020 |
| PublicationTitle | Conference record - Asilomar Conference on Signals, Systems, & Computers |
| PublicationTitleAbbrev | IEEECONF |
| PublicationYear | 2023 |
| Publisher | IEEE |
| Publisher_xml | – name: IEEE |
| SSID | ssj0020282 |
| Score | 2.2572658 |
| Snippet | The Bethe Free Energy (BFE) has been found to be closely connected to various message passing algorithms. Studies have indicated that the BFE shares stationary... |
| SourceID | ieee |
| SourceType | Publisher |
| StartPage | 897 |
| SubjectTerms | Approximation algorithms Computers Context modeling Convergence Estimation Message passing Minimization |
| Title | Bethe Free Energy and Extrinsics in Approximate Message Passing |
| URI | https://ieeexplore.ieee.org/document/10477003 |
| WOSCitedRecordID | wos001207755100162&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| hasFullText | 1 |
| inHoldings | 1 |
| isFullTextHit | |
| isPrint | |
| link | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV1JT0MhECbaGKMXtxr3kOiVtg8eAiejTV88aO1BTW8Ny6C9vJr21fTnC3RRDx68EbaQmcDMAN83CF21vNUy05wwcIbkxjGimFOEWxMMltTgEpXS64PodmW_r3oLsHrCwgBA-nwGjVhMb_luZKfxqqwZaQVE4vZcF-J6DtZaRVcxeNhElwsSzWaMldpP3YIrTuPdCWWN5fBfiVSSHSl2_rmCXVT_RuTh3srW7KE1KPfR9g8ywQN0cxexu7gYA-BOQvRhXTrcmVWhPehigoclvo0U4rNhcFMBP8bsJ29h3uA_hxnq6KXoPLfvySI9AnmnKquI9CwsJsg5M5pZKyRIa5XiLM-1pUZoz4xoZZ45zaiGsLGBxl7Ahcm5A3aIauWohCOEM-9d2M25suBzGuHlEOIMGbwNypgDfozqURaDjzkDxmAphpM_6k_RVpR4POOpOkO1ajyFc7RhP6vhZHyR9PYFOeWYeg |
| linkProvider | IEEE |
| linkToHtml | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV3JTsMwEB2hglgubEXsWIJrSuNFiU8IqkZFtKGHgnqrHHsCvaSoTVE_H9tdgAMHbpFjjaxn2TNj-70BuKnnWsWhEgFDkwU8MyyQzMhA6Mw6rFih8VJKr-0oTeN-X3YXZHXPhUFE__gMa-7T3-WbkZ66o7JbJysQeW3PdcE5rc_pWqv8yqUPm3C9kNG8ddlS4zlNhBTUnZ5QVlsa-FVKxXuSZPefY9iD6jcnj3RX3mYf1rA4gJ0fcoKHcPfg2LskGSOSpuf0EVUY0pyV9r-djQkZFuTeiYjPhjZQRdJx9U_erF0bQVsLVXhJmr1GK1gUSAjeqQzLIM6ZHYxFOswU0zqKMdZaSsE4V5pmkcpZFtXDnBnFqEK7tJG6XiiijAuD7AgqxajAYyBhnhu7nrnUmFtkbZiANtOIbbxBGTMoTqDqsBh8zDUwBksYTv9ov4KtVq_THrQf06cz2Hboux2fynOolOMpXsCG_iyHk_Gln8MvW0ebwQ |
| openUrl | ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Abook&rft.genre=proceeding&rft.title=Conference+record+-+Asilomar+Conference+on+Signals%2C+Systems%2C+%26+Computers&rft.atitle=Bethe+Free+Energy+and+Extrinsics+in+Approximate+Message+Passing&rft.au=Zhao%2C+Zilu&rft.au=Slock%2C+Dirk&rft.date=2023-10-29&rft.pub=IEEE&rft.eissn=2576-2303&rft.spage=897&rft.epage=901&rft_id=info:doi/10.1109%2FIEEECONF59524.2023.10477003&rft.externalDocID=10477003 |