On the Rényi Divergence, Joint Range of Relative Entropies, and a Channel Coding Theorem

This paper starts by considering the minimization of the Rényi divergence subject to a constraint on the total variation distance. Based on the solution of this optimization problem, the exact locus of the points (D(QIIP 1 ), D(QIIP 2 )) is determined when P 1 , P 2 , and Q are arbitrary probabilit...

Celý popis

Uložené v:
Podrobná bibliografia
Vydané v:IEEE transactions on information theory Ročník 62; číslo 1; s. 23 - 34
Hlavný autor: Sason, Igal
Médium: Journal Article
Jazyk:English
Vydavateľské údaje: New York IEEE 01.01.2016
The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Predmet:
Alphabets
Binary codes
Binomial distribution
Block codes
Channels
Chernoff information
distance spectrum
Divergence
Entropy
error exponent
Information theory
Maximum likelihood decoding
Maximum likelihood method
Minimization
Optimization
Q measurement
relative entropy
Renyi divergence
total variation distance
Upper bound
maximum-likelihood decoding
Rényi divergence
error exponent
relative entropy
distance spectrum
Chernoff information
total variation distance
ISSN:0018-9448, 1557-9654
On-line prístup:Získať plný text
Tagy: Pridať tag
Žiadne tagy, Buďte prvý, kto otaguje tento záznam!
Popis
Shrnutí:This paper starts by considering the minimization of the Rényi divergence subject to a constraint on the total variation distance. Based on the solution of this optimization problem, the exact locus of the points (D(QIIP 1 ), D(QIIP 2 )) is determined when P 1 , P 2 , and Q are arbitrary probability measures which are mutually absolutely continuous, and the total variation distance between P 1 and P 2 is not below a given value. It is further shown that all the points of this convex region are attained by probability measures which are defined on a binary alphabet. This characterization yields a geometric interpretation of the minimal Chernoff information subject to a constraint on the variational distance. This paper also derives an exponential upper bound on the performance of binary linear block codes (or code ensembles) under maximum-likelihood decoding. Its derivation relies on the Gallager bounding technique, and it reproduces the Shulman-Feder bound as a special case. The bound is expressed in terms of the Rényi divergence from the normalized distance spectrum of the code (or the average distance spectrum of the ensemble) to the binomially distributed distance spectrum of the capacity-achieving ensemble of random block codes. This exponential bound provides a quantitative measure of the degradation in performance of binary linear block codes (or code ensembles) as a function of the deviation of their distance spectra from the binomial distribution. An efficient use of this bound is considered.
Bibliografia:SourceType-Scholarly Journals-1
ObjectType-Feature-1
content type line 14
ObjectType-Article-1
ObjectType-Feature-2
content type line 23
ISSN:0018-9448
1557-9654
DOI:10.1109/TIT.2015.2504100