Bethe Free Energy and Extrinsics in Approximate Message Passing

The Bethe Free Energy (BFE) has been found to be closely connected to various message passing algorithms. Studies have indicated that the BFE shares stationary points with message passing algorithms like Belief Propagation (BP) and Expectation Propagation (EP). Generalized Approximate Message Passin...

Celý popis

Uloženo v:
Podrobná bibliografie
Vydáno v:Conference record - Asilomar Conference on Signals, Systems, & Computers s. 897 - 901
Hlavní autoři: Zhao, Zilu, Slock, Dirk
Médium: Konferenční příspěvek
Jazyk:angličtina
Vydáno: IEEE 29.10.2023
Témata:
Approximation algorithms
Computers
Context modeling
Convergence
Estimation
Message passing
Minimization
ISSN:2576-2303
On-line přístup:Získat plný text
Tagy: Přidat tag
Žádné tagy, Buďte první, kdo vytvoří štítek k tomuto záznamu!
Abstract The Bethe Free Energy (BFE) has been found to be closely connected to various message passing algorithms. Studies have indicated that the BFE shares stationary points with message passing algorithms like Belief Propagation (BP) and Expectation Propagation (EP). Generalized Approximate Message Passing (GAMP) algorithms have demonstrated significant efficacy in signal recovery. Nevertheless, they may encounter convergence issues. To address these convergence issues, algorithms based on the minimization of the large system limit (LSL) BFE have been introduced. In this paper, we explore the BFE within the context of Generalized Linear Models (GLMs). Applying a BFE based EP approach leads to the re(G)VAMP algorithm which provides asymptotically exact marginal posteriors based on asymptotically Gaussian extrinsics. It also provides equivalent Gaussian priors and hence an equivalent overall Gaussian linear model, which allows the application of large random matrix theory. We show how this leads to the LSL BFE on which GAMP is based. We also reveal the intimate relation of extrinsics to Component-Wise Conditionally Unbiased Minimum Mean Squared Error (CWCU MMSE) estimation for which we provide a novel shortcut derivation in the GLM.
AbstractList The Bethe Free Energy (BFE) has been found to be closely connected to various message passing algorithms. Studies have indicated that the BFE shares stationary points with message passing algorithms like Belief Propagation (BP) and Expectation Propagation (EP). Generalized Approximate Message Passing (GAMP) algorithms have demonstrated significant efficacy in signal recovery. Nevertheless, they may encounter convergence issues. To address these convergence issues, algorithms based on the minimization of the large system limit (LSL) BFE have been introduced. In this paper, we explore the BFE within the context of Generalized Linear Models (GLMs). Applying a BFE based EP approach leads to the re(G)VAMP algorithm which provides asymptotically exact marginal posteriors based on asymptotically Gaussian extrinsics. It also provides equivalent Gaussian priors and hence an equivalent overall Gaussian linear model, which allows the application of large random matrix theory. We show how this leads to the LSL BFE on which GAMP is based. We also reveal the intimate relation of extrinsics to Component-Wise Conditionally Unbiased Minimum Mean Squared Error (CWCU MMSE) estimation for which we provide a novel shortcut derivation in the GLM.
Author Slock, Dirk
Zhao, Zilu
Author_xml – sequence: 1
  givenname: Zilu
  surname: Zhao
  fullname: Zhao, Zilu
  email: zilu.zhao@eurecom.fr
  organization: EURECOM,Communication Systems Department,France
– sequence: 2
  givenname: Dirk
  surname: Slock
  fullname: Slock, Dirk
  email: dirk.slock@eurecom.fr
  organization: EURECOM,Communication Systems Department,France
BookMark eNo1j7FOwzAURQ0Cibb0DxgsMSfYfjaOJ1QqByoVygBz5SQvqRG4kZ2h_fsGAdNZ7r06d0ouwj4gIbec5Zwzc7ey1i43r6UySshcMAE5Z1JrxuCMzI02BSgGQmkpz8lk5H0mgMEVmab0ydhYKMSEPDzisENaRkRqA8buSF1oqD0M0Yfk60R9oIu-j_uD_3YD0hdMyXVI31xKPnTX5LJ1Xwnnf5yRj9K-L5-z9eZptVyss50wfMiKFkY17hSvHNS1LrCoa2MUSOlqUWnXQqUZb6FxIBwCUyh-Uqh0JVWDMCM3v7seEbd9HGXicft_GE7pK01Y
ContentType Conference Proceeding
DBID 6IE
6IH
CBEJK
RIE
RIO
DOI 10.1109/IEEECONF59524.2023.10477003
DatabaseName IEEE Electronic Library (IEL) Conference Proceedings
IEEE Proceedings Order Plan (POP) 1998-present by volume
IEEE Xplore All Conference Proceedings
IEEE Electronic Library (IEL)
IEEE Proceedings Order Plans (POP) 1998-present
DatabaseTitleList
Database_xml – sequence: 1
  dbid: RIE
  name: IEEE Electronic Library (IEL)
  url: https://ieeexplore.ieee.org/
  sourceTypes: Publisher
DeliveryMethod fulltext_linktorsrc
Discipline Engineering
EISBN 9798350325744
EISSN 2576-2303
EndPage 901
ExternalDocumentID 10477003
Genre orig-research
GrantInformation_xml – fundername: ORANGE
  funderid: 10.13039/501100003951
GroupedDBID 6IE
6IF
6IH
6IK
6IL
6IM
6IN
AAJGR
AAWTH
ABLEC
ACGFS
ADZIZ
ALMA_UNASSIGNED_HOLDINGS
BEFXN
BFFAM
BGNUA
BKEBE
BPEOZ
CBEJK
CHZPO
IEGSK
IJVOP
IPLJI
M43
OCL
RIE
RIL
RIO
ID FETCH-LOGICAL-h291t-8f34771a51ba3cc78e8cc995344ac2b7af3b701f3da32ae305e2c78ee57b45de3
IEDL.DBID RIE
ISICitedReferencesCount 0
ISICitedReferencesURI http://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=Summon&SrcAuth=ProQuest&DestLinkType=CitingArticles&DestApp=WOS_CPL&KeyUT=001207755100162&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D
IngestDate Wed Aug 27 02:09:41 EDT 2025
IsDoiOpenAccess false
IsOpenAccess true
IsPeerReviewed false
IsScholarly true
Language English
LinkModel DirectLink
MergedId FETCHMERGED-LOGICAL-h291t-8f34771a51ba3cc78e8cc995344ac2b7af3b701f3da32ae305e2c78ee57b45de3
OpenAccessLink https://hal.science/hal-04501734
PageCount 5
ParticipantIDs ieee_primary_10477003
PublicationCentury 2000
PublicationDate 2023-Oct.-29
PublicationDateYYYYMMDD 2023-10-29
PublicationDate_xml – month: 10
  year: 2023
  text: 2023-Oct.-29
  day: 29
PublicationDecade 2020
PublicationTitle Conference record - Asilomar Conference on Signals, Systems, & Computers
PublicationTitleAbbrev IEEECONF
PublicationYear 2023
Publisher IEEE
Publisher_xml – name: IEEE
SSID ssj0020282
Score 2.2572658
Snippet The Bethe Free Energy (BFE) has been found to be closely connected to various message passing algorithms. Studies have indicated that the BFE shares stationary...
SourceID ieee
SourceType Publisher
StartPage 897
SubjectTerms Approximation algorithms
Computers
Context modeling
Convergence
Estimation
Message passing
Minimization
Title Bethe Free Energy and Extrinsics in Approximate Message Passing
URI https://ieeexplore.ieee.org/document/10477003
WOSCitedRecordID wos001207755100162&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D
hasFullText 1
inHoldings 1
isFullTextHit
isPrint
link http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV25TgMxELUgQggariBuWYJ2k6yPtV0hQFnREFKAlC7yMQtpNijZoHw-ns0BFBR0vjUay_aM7feGkBvhwHiICynw0EmEtJA4jp9pjAjMF9x4vQg2oXo9PRiY_hKsXmNhAKD-fAYtTNZv-WHsZ3hV1kZaAVVze24qlS3AWmvvCp2HbXK9JNFso6_08NzLpZEM704Yb626_wqkUp8j-d4_JdgnzW9EHu2vz5oDsgHlIdn9QSZ4RG7vEbtL8wkA7daIPmrLQLvzKtbHuZjSUUnvkEJ8PopmKtAnjH7yFseN9nMcoUle8-7Lw2OyDI-QvDOTVokueBQmtTJ1lnuvNGjvjZFcCOuZU7bgTnXSggfLmYW4sIFhK5DKCRmAH5NGOS7hhFAjtNU88yGgxxRz0W4UwINwktssZKekiboYfiwYMIYrNZz9UX5OdlDjuMczc0Ea1WQGl2TLf1aj6eSqnrcvmbeYPw
linkProvider IEEE
linkToHtml http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV3JTsMwELVQQSwXtiJ2LME1beOliU8IqkZFtKGHIvVWOfYEeklRmqJ-Pp50AQ4cuHnTyBrL9ow97w0hdyIBZcBtJMttwxNSg5dwDKZRwjKTcmXCRbKJII7D4VD1l2D1EgsDAGXwGdSwWP7l24mZ4VNZHWkFgpLbc1MKwRoLuNbav0L3YZvcLmk06-gttV7iSCrJ8PWE8dpKwK9UKuVNEu3_cw4HpPqNyaP99W1zSDYgOyJ7P-gEj8n9I6J3aZQD0HaJ6aM6s7Q9L1y_W40pHWf0AUnE52NnqALtYf6TNyfXWdBOQpW8Ru1Bq-MtEyR470z5hRem3E3G19JPNDcmCCE0RinJhdCGJYFOeRI0_JRbzZkGt7WB4SiQQSKkBX5CKtkkg1NClQh1yJvGWvSZXM1ZjgK4FYnkummbZ6SKuhh9LDgwRis1nP_RfkN2OoNed9R9ip8vyC5qH098pi5JpchncEW2zGcxnubX5Rp-Abe5m4Y
openUrl ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Abook&rft.genre=proceeding&rft.title=Conference+record+-+Asilomar+Conference+on+Signals%2C+Systems%2C+%26+Computers&rft.atitle=Bethe+Free+Energy+and+Extrinsics+in+Approximate+Message+Passing&rft.au=Zhao%2C+Zilu&rft.au=Slock%2C+Dirk&rft.date=2023-10-29&rft.pub=IEEE&rft.eissn=2576-2303&rft.spage=897&rft.epage=901&rft_id=info:doi/10.1109%2FIEEECONF59524.2023.10477003&rft.externalDocID=10477003