Decomposing and Tracing Mutual Information by Quantifying Reachable Decision Regions

The idea of a partial information decomposition (PID) gained significant attention for attributing the components of mutual information from multiple variables about a target to being unique, redundant/shared or synergetic. Since the original measure for this analysis was criticized, several alterna...

Full description

Saved in:
Bibliographic Details
Published in:Entropy (Basel, Switzerland) Vol. 25; no. 7; p. 1014
Main Authors: Mages, Tobias, Rohner, Christian
Format: Journal Article
Language:English
Published: Switzerland MDPI AG 30.06.2023
MDPI
Subjects:
Algebra
Axioms
Communication networks
Computer Science with specialization in Computer Communication
Data processing
Datavetenskap med inriktning mot datorkommunikation
Decomposition
Information flow
information flow analysis
Information management
Markov chains
Markov processes
partial information decomposition
redundancy
synergy
Tracing
Uncertainty
Valuation
Variables
Taiwan
synergy
redundancy
information flow analysis
partial information decomposition
ISSN:1099-4300, 1099-4300
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:The idea of a partial information decomposition (PID) gained significant attention for attributing the components of mutual information from multiple variables about a target to being unique, redundant/shared or synergetic. Since the original measure for this analysis was criticized, several alternatives have been proposed but have failed to satisfy the desired axioms, an inclusion–exclusion principle or have resulted in negative partial information components. For constructing a measure, we interpret the achievable type I/II error pairs for predicting each state of a target variable (reachable decision regions) as notions of pointwise uncertainty. For this representation of uncertainty, we construct a distributive lattice with mutual information as consistent valuation and obtain an algebra for the constructed measure. The resulting definition satisfies the original axioms, an inclusion–exclusion principle and provides a non-negative decomposition for an arbitrary number of variables. We demonstrate practical applications of this approach by tracing the flow of information through Markov chains. This can be used to model and analyze the flow of information in communication networks or data processing systems.
Bibliography:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
content type line 23
ISSN:1099-4300
1099-4300
DOI:10.3390/e25071014