Bethe Free Energy and Extrinsics in Approximate Message Passing

The Bethe Free Energy (BFE) has been found to be closely connected to various message passing algorithms. Studies have indicated that the BFE shares stationary points with message passing algorithms like Belief Propagation (BP) and Expectation Propagation (EP). Generalized Approximate Message Passin...

Full description

Saved in:
Bibliographic Details
Published in:Conference record - Asilomar Conference on Signals, Systems, & Computers pp. 897 - 901
Main Authors: Zhao, Zilu, Slock, Dirk
Format: Conference Proceeding
Language:English
Published: IEEE 29.10.2023
Subjects:
Approximation algorithms
Computers
Context modeling
Convergence
Estimation
Message passing
Minimization
ISSN:2576-2303
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:The Bethe Free Energy (BFE) has been found to be closely connected to various message passing algorithms. Studies have indicated that the BFE shares stationary points with message passing algorithms like Belief Propagation (BP) and Expectation Propagation (EP). Generalized Approximate Message Passing (GAMP) algorithms have demonstrated significant efficacy in signal recovery. Nevertheless, they may encounter convergence issues. To address these convergence issues, algorithms based on the minimization of the large system limit (LSL) BFE have been introduced. In this paper, we explore the BFE within the context of Generalized Linear Models (GLMs). Applying a BFE based EP approach leads to the re(G)VAMP algorithm which provides asymptotically exact marginal posteriors based on asymptotically Gaussian extrinsics. It also provides equivalent Gaussian priors and hence an equivalent overall Gaussian linear model, which allows the application of large random matrix theory. We show how this leads to the LSL BFE on which GAMP is based. We also reveal the intimate relation of extrinsics to Component-Wise Conditionally Unbiased Minimum Mean Squared Error (CWCU MMSE) estimation for which we provide a novel shortcut derivation in the GLM.
ISSN:2576-2303
DOI:10.1109/IEEECONF59524.2023.10477003