Collaborative Information Bottleneck

This paper investigates a multi-terminal source coding problem under a logarithmic loss fidelity which does not necessarily lead to an additive distortion measure. The problem is motivated by an extension of the information bottleneck method to a multi-source scenario where several encoders have to...

Celý popis

Uloženo v:
Podrobná bibliografie
Vydáno v:IEEE transactions on information theory Ročník 65; číslo 2; s. 787 - 815
Hlavní autoři: Vera, Matias, Rey Vega, Leonardo, Piantanida, Pablo
Médium: Journal Article
Jazyk:angličtina
Vydáno: New York IEEE 01.02.2019
The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Institute of Electrical and Electronics Engineers
Témata:
Coders
Collaboration
Complexity
Complexity theory
Cooperation
Decoders
Decoding
Descriptions
Distortion
Distortion measurement
distributed source coding
information bottleneck
Information Theory
interactive lossy source coding
logarithmic loss
Mathematics
Multi-terminal source coding
Mutual information
noisy rate-distortion
Shannon theory
side information
Source coding
Statistical models
Statistics
Tradeoffs
Interactive lossy source coding
Information Bottleneck
Logarithmic loss
Noisy rate-distortion
Multi-terminal source coding
Side information
Shannon theory
Distributed source coding
ISSN:0018-9448, 1557-9654
On-line přístup:Získat plný text
Tagy: Přidat tag
Žádné tagy, Buďte první, kdo vytvoří štítek k tomuto záznamu!
Popis
Shrnutí:This paper investigates a multi-terminal source coding problem under a logarithmic loss fidelity which does not necessarily lead to an additive distortion measure. The problem is motivated by an extension of the information bottleneck method to a multi-source scenario where several encoders have to build cooperatively rate-limited descriptions of their sources in order to maximize information with respect to other unobserved (hidden) sources. More precisely, we study fundamental information-theoretic limits of the so-called: 1) two-way collaborative information bottleneck (TW-CIB) and 2) the collaborative distributed information bottleneck (CDIB) problems. The TW-CIB problem consists of two distant encoders that separately observe marginal (dependent) components X 1 and X 2 and can cooperate through multiple exchanges of limited information with the aim of extracting information about hidden variables (Y 1 , Y 2 ), which can be arbitrarily dependent on (X 1 , X 2 ). On the other hand, in CDIB, there are two cooperating encoders which separately observe X 1 and X 2 and a third node which can listen to the exchanges between the two encoders in order to obtain information about a hidden variable Y. The relevance (figure-of-merit) is measured in terms of a normalized (per-sample) multi-letter mutual information metric (log-loss fidelity), and an interesting tradeoff arises by constraining the complexity of descriptions, measured in terms of the rates needed for the exchanges between the encoders and decoders involved. Inner and outer bounds to the complexity-relevance region of these problems are derived from which optimality is characterized for several cases of interest. Our resulting theoretical complexity-relevance regions are finally evaluated for binary symmetric and Gaussian statistical models, showing theoretical tradeoffs between the complexity-constrained descriptions and their relevance with respect to the hidden variables.
Bibliografie:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
ISSN:0018-9448
1557-9654
DOI:10.1109/TIT.2018.2883295