Counting in Graph Covers: A Combinatorial Characterization of the Bethe Entropy Function

We present a combinatorial characterization of the Bethe entropy function of a factor graph, such a characterization being in contrast to the original, analytical, definition of this function. We achieve this combinatorial characterization by counting valid configurations in finite graph covers of t...

Ausführliche Beschreibung

Gespeichert in:
Bibliographische Detailangaben
Veröffentlicht in:IEEE transactions on information theory Jg. 59; H. 9; S. 6018 - 6048
1. Verfasser: Vontobel, Pascal O.
Format: Journal Article
Sprache:Englisch
Veröffentlicht: New York, NY IEEE 01.09.2013
Institute of Electrical and Electronics Engineers
The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Schlagworte:
Applied sciences
Approximation methods
Bethe approximation
Bethe entropy
Bethe partition function
Coding theory
Coding, codes
Decoding
Entropy
Exact sciences and technology
graph cover
graph-cover decoding
Graphical models
Hamming codes
Information theory
Information, signal and communications theory
Iterative decoding
Linear programming
linear programming decoding
Low density parity check codes
message-passing algorithm
method of types
pseudomarginal vector
Signal and communications theory
Sum product algorithm
sum-product algorithm (SPA)
Telecommunications and information theory
Vectors
Similarity
Energy function
Entropy
message-passing algorithm
Iterative decoding
Bethe approximation
graph cover
Combinatorial method
sum-product algorithm (SPA)
Parity check codes
Minimal distance
Hamming distance
Bethe entropy
pseudomarginal vector
Maximin problem
Linear programming
linear programming decoding
Algorithm
Free energy
graph-cover decoding
Partition function
Message passing
Bethe partition function
Function minimization
Error correcting code
method of types
ISSN:0018-9448, 1557-9654
Online-Zugang:Volltext
Tags: Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
Beschreibung
Zusammenfassung:We present a combinatorial characterization of the Bethe entropy function of a factor graph, such a characterization being in contrast to the original, analytical, definition of this function. We achieve this combinatorial characterization by counting valid configurations in finite graph covers of the factor graph. Analogously, we give a combinatorial characterization of the Bethe partition function, whose original definition was also of an analytical nature. As we point out, our approach has similarities to the replica method, but also stark differences. The above findings are a natural backdrop for introducing a decoder for graph-based codes that we will call symbolwise graph-cover decoding, a decoder that extends our earlier work on blockwise graph-cover decoding. Both graph-cover decoders are theoretical tools that help toward a better understanding of message-passing iterative decoding, namely blockwise graph-cover decoding links max-product (min-sum) algorithm decoding with linear programming decoding, and symbolwise graph-cover decoding links sum-product algorithm decoding with Bethe free energy function minimization at temperature one. In contrast to the Gibbs entropy function, which is a concave function, the Bethe entropy function is in general not concave everywhere. In particular, we show that every code picked from an ensemble of regular low-density parity-check codes with minimum Hamming distance growing (with high probability) linearly with the block length has a Bethe entropy function that is convex in certain regions of its domain.
Bibliographie:SourceType-Scholarly Journals-1
ObjectType-Feature-1
content type line 14
ISSN:0018-9448
1557-9654
DOI:10.1109/TIT.2013.2264715