Information-Theoretic Limits of Selecting Binary Graphical Models in High Dimensions

The problem of graphical model selection is to estimate the graph structure of a Markov random field given samples from it. We analyze the information-theoretic limitations of the problem of graph selection for binary Markov random fields under high-dimensional scaling, in which the graph size and t...

Full description

Saved in:
Bibliographic Details
Published in:IEEE transactions on information theory Vol. 58; no. 7; pp. 4117 - 4134
Main Authors: Santhanam, Narayana P., Wainwright, Martin J.
Format: Journal Article
Language:English
Published: New York, NY IEEE 01.07.2012
Institute of Electrical and Electronics Engineers
The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Subjects:
Analytical models
Applied sciences
Coding theory
Coding, codes
Decoding
Exact sciences and technology
Graphical models
Graphs
High dimensional inference
Image edge detection
Information theory
Information, signal and communications theory
KL divergence between Ising models
Markov analysis
Markov processes
Markov random fields
Probability
Random variables
sample complexity
Sample size
Signal and communications theory
structure of Ising models
Telecommunications and information theory
Vectors
Error probability
Probabilistic approach
Necessary and sufficient condition
Model selection
structure of Ising models
Decoding
Markov model
Markov random fields
sample complexity
Random field
High dimensional inference
Pairwise Markov model
KL divergence between Ising models
Ising model
Information theory
ISSN:0018-9448, 1557-9654
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:The problem of graphical model selection is to estimate the graph structure of a Markov random field given samples from it. We analyze the information-theoretic limitations of the problem of graph selection for binary Markov random fields under high-dimensional scaling, in which the graph size and the number of edges k, and/or the maximal node degree d, are allowed to increase to infinity as a function of the sample size n. For pair-wise binary Markov random fields, we derive both necessary and sufficient conditions for correct graph selection over the class G p,k of graphs on vertices with at most k edges, and over the class G p,d of graphs on p vertices with maximum degree at most d. For the class G p,k , we establish the existence of constants c and c' such that if n <; ck log p, any method has error probability at least 1/2 uniformly over the family, and we demonstrate a graph decoder that succeeds with high probability uniformly over the family for sample sizes n >; c' k 2 log p. Similarly, for the class G p,d , we exhibit constants c and c' such that for n <; cd 2 log p, any method fails with probability at least 1/2, and we demonstrate a graph decoder that succeeds with high probability for n >; c' d 3 log p.
Bibliography:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
ISSN:0018-9448
1557-9654
DOI:10.1109/TIT.2012.2191659