The Lossy Common Information of Correlated Sources

The two most prevalent notions of common information (CI) are due to Wyner and Gács-Körner and both the notions can be stated as two different characteristic points in the lossless Gray-Wyner region. Although the information theoretic characterizations for these two CI quantities can be easily eva...

Full description

Saved in:
Bibliographic Details
Published in:IEEE transactions on information theory Vol. 60; no. 6; pp. 3238 - 3253
Main Authors: Viswanatha, Kumar B., Akyol, Emrah, Rose, Kenneth
Format: Journal Article
Language:English
Published: New York, NY IEEE 01.06.2014
Institute of Electrical and Electronics Engineers
The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Subjects:
Applied sciences
Coding, codes
Decoding
Distortion
Distortion measurement
Entropy
Exact sciences and technology
Foundations
Gaussian
Information theory
Information, signal and communications theory
Joints
Lossless
Markov processes
Networks
Normal distribution
Random variables
Rate-distortion
Shape
Signal and communications theory
Telecommunications and information theory
Gray¿Wyner network
Common information
multiterminal source coding
Lossless circuit
Source coding
Lossy medium
Entropy
Autoregressive model
Decoding
Chi square
Gaussian process
Gray-Wyner network
Information source
Random variable
Information theory
ISSN:0018-9448, 1557-9654
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:The two most prevalent notions of common information (CI) are due to Wyner and Gács-Körner and both the notions can be stated as two different characteristic points in the lossless Gray-Wyner region. Although the information theoretic characterizations for these two CI quantities can be easily evaluated for random variables with infinite entropy (e.g., continuous random variables), their operational significance is applicable only to the lossless framework. The primary objective of this paper is to generalize these two CI notions to the lossy Gray-Wyner network, which hence extends the theoretical foundation to general sources and distortion measures. We begin by deriving a single letter characterization for the lossy generalization of Wyner's CI, defined as the minimum rate on the shared branch of the Gray-Wyner network, maintaining minimum sum transmit rate when the two decoders reconstruct the sources subject to individual distortion constraints. To demonstrate its use, we compute the CI of bivariate Gaussian random variables for the entire regime of distortions. We then similarly generalize Gács and Körner's definition to the lossy framework. The latter half of this paper focuses on studying the tradeoff between the total transmit rate and receive rate in the Gray-Wyner network. We show that this tradeoff yields a contour of points on the surface of the Gray-Wyner region, which passes through both the Wyner and Gács-Körner operating points, and thereby provides a unified framework to understand the different notions of CI. We further show that this tradeoff generalizes the two notions of CI to the excess sum transmit rate and receive rate regimes, respectively.
Bibliography:SourceType-Scholarly Journals-1
ObjectType-Feature-1
content type line 14
ObjectType-Article-2
content type line 23
ISSN:0018-9448
1557-9654
DOI:10.1109/TIT.2014.2315805