On statistical inference when fixed points of belief propagation are unstable

Many statistical inference problems correspond to recovering the values of a set of hidden variables from sparse observations on them. For instance, in a planted constraint satisfaction problem such as planted 3-SAT, the clauses are sparse observations from which the hidden assignment is to be recov...

Full description

Saved in:
Bibliographic Details
Published in:Proceedings / annual Symposium on Foundations of Computer Science pp. 395 - 405
Main Authors: Liu, Siqi, Mohanty, Sidhanth, Raghavendra, Prasad
Format: Conference Proceeding
Language:English
Published: IEEE 01.02.2022
Subjects:
belief propagation
Computational modeling
Computer science
constraint satisfaction problems
Image edge detection
Inference algorithms
Prediction algorithms
Predictive models
ran-dom graphs
random matrices
spectral algorithms
statistical inference
statistical physics
Stochastic processes
ISSN:2575-8454
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Abstract Many statistical inference problems correspond to recovering the values of a set of hidden variables from sparse observations on them. For instance, in a planted constraint satisfaction problem such as planted 3-SAT, the clauses are sparse observations from which the hidden assignment is to be recovered. In the problem of community detection in a stochastic block model, the community labels are hidden variables that are to be recovered from the edges of the graph. Inspired by ideas from statistical physics, the presence of a stable fixed point for belief propogation has been widely conjectured to characterize the computational tractability of these problems. For community detection in stochastic block models, many of these predictions have been rigorously confirmed. In this work, we consider a general model of statistical inference problems that includes both community detection in stochastic block models, and all planted constraint satisfaction problems as special cases. We carry out the cavity method calculations from statistical physics to compute the regime of parameters where detection and recovery should be algorithmically tractable. At precisely the predicted tractable regime, we give: (i) a general polynomial-time algorithm for the problem of detection: distinguishing an input with a planted signal from one without; (ii) a general polynomial-time algorithm for the problem of recovery: outputting a vector that correlates with the hidden assignment significantly better than a random guess would. Analogous to the spectral algorithm for community detection [1], [2], the detection and recovery algorithms are based on the spectra of a matrix that arises as the derivatives of the belief propagation update rule. To devise a spectral algorithm in our general model, we obtain bounds on the spectral norms of certain families of random matrices with correlated and matrix valued entries. We then demonstrate how eigenvectors of various powers of the matrix can be used to partially recover the hidden variables.
AbstractList Many statistical inference problems correspond to recovering the values of a set of hidden variables from sparse observations on them. For instance, in a planted constraint satisfaction problem such as planted 3-SAT, the clauses are sparse observations from which the hidden assignment is to be recovered. In the problem of community detection in a stochastic block model, the community labels are hidden variables that are to be recovered from the edges of the graph. Inspired by ideas from statistical physics, the presence of a stable fixed point for belief propogation has been widely conjectured to characterize the computational tractability of these problems. For community detection in stochastic block models, many of these predictions have been rigorously confirmed. In this work, we consider a general model of statistical inference problems that includes both community detection in stochastic block models, and all planted constraint satisfaction problems as special cases. We carry out the cavity method calculations from statistical physics to compute the regime of parameters where detection and recovery should be algorithmically tractable. At precisely the predicted tractable regime, we give: (i) a general polynomial-time algorithm for the problem of detection: distinguishing an input with a planted signal from one without; (ii) a general polynomial-time algorithm for the problem of recovery: outputting a vector that correlates with the hidden assignment significantly better than a random guess would. Analogous to the spectral algorithm for community detection [1], [2], the detection and recovery algorithms are based on the spectra of a matrix that arises as the derivatives of the belief propagation update rule. To devise a spectral algorithm in our general model, we obtain bounds on the spectral norms of certain families of random matrices with correlated and matrix valued entries. We then demonstrate how eigenvectors of various powers of the matrix can be used to partially recover the hidden variables.
Author Raghavendra, Prasad
Liu, Siqi
Mohanty, Sidhanth
Author_xml – sequence: 1
  givenname: Siqi
  surname: Liu
  fullname: Liu, Siqi
  email: sliu18@berkeley.edu
  organization: UC Berkeley,Berkeley,CA,USA
– sequence: 2
  givenname: Sidhanth
  surname: Mohanty
  fullname: Mohanty, Sidhanth
  email: sidhanthm@cs.berkeley.edu
  organization: UC Berkeley,Berkeley,CA,USA
– sequence: 3
  givenname: Prasad
  surname: Raghavendra
  fullname: Raghavendra, Prasad
  email: prasad@cs.berkeley.edu
  organization: UC Berkeley,Berkeley,CA,USA
BookMark eNotjs1KxDAUhaMoOB19Al3kBTre_NwmWUpxRmGkC3U9pPVGIzUtTUV9ewu6Ootzvo9TsJM0JGLsSsBGCHDX26Z-ROmM20iQYgMA2hyxQlQVagmI6pitJBosrUZ9xoqc35cJIOgVe2gSz7OfY55j53seU6CJUkf8640SD_GbXvg4xDRnPgTeUh8p8HEaRv-6UEPifiL-mRZH29M5Ow2-z3Txn2v2vL19qu_KfbO7r2_2ZZSg5lIaCkKFlkynyBqqbOiMX65qrFQH0BrbhkqAskutoNUeUZJDL7S0XoNas8s_bySiwzjFDz_9HJwRzkhUv3ayT8E
CODEN IEEPAD
ContentType Conference Proceeding
DBID 6IE
6IH
CBEJK
RIE
RIO
DOI 10.1109/FOCS52979.2021.00047
DatabaseName IEEE Electronic Library (IEL) Conference Proceedings
IEEE Proceedings Order Plan (POP) 1998-present by volume
IEEE Xplore All Conference Proceedings
IEEE Electronic Library (IEL)
IEEE Proceedings Order Plans (POP) 1998-present
DatabaseTitleList
Database_xml – sequence: 1
  dbid: RIE
  name: IEEE Electronic Library (IEL)
  url: https://ieeexplore.ieee.org/
  sourceTypes: Publisher
DeliveryMethod fulltext_linktorsrc
Discipline Mathematics
Computer Science
EISBN 1665420553
9781665420556
EISSN 2575-8454
EndPage 405
ExternalDocumentID 9719725
Genre orig-research
GroupedDBID --Z
29O
6IE
6IH
6IK
ALMA_UNASSIGNED_HOLDINGS
CBEJK
RIE
RIO
ID FETCH-LOGICAL-i203t-27ef13fbe7c3e87e68fc7a2054563c00b78bf610383e830b4a552e95a1428a403
IEDL.DBID RIE
ISICitedReferencesCount 2
ISICitedReferencesURI http://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=Summon&SrcAuth=ProQuest&DestLinkType=CitingArticles&DestApp=WOS_CPL&KeyUT=000802209600037&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D
IngestDate Wed Aug 27 02:49:22 EDT 2025
IsPeerReviewed false
IsScholarly true
Language English
LinkModel DirectLink
MergedId FETCHMERGED-LOGICAL-i203t-27ef13fbe7c3e87e68fc7a2054563c00b78bf610383e830b4a552e95a1428a403
PageCount 11
ParticipantIDs ieee_primary_9719725
PublicationCentury 2000
PublicationDate 2022-Feb.
PublicationDateYYYYMMDD 2022-02-01
PublicationDate_xml – month: 02
  year: 2022
  text: 2022-Feb.
PublicationDecade 2020
PublicationTitle Proceedings / annual Symposium on Foundations of Computer Science
PublicationTitleAbbrev FOCS
PublicationYear 2022
Publisher IEEE
Publisher_xml – name: IEEE
SSID ssj0040504
Score 2.2326124
Snippet Many statistical inference problems correspond to recovering the values of a set of hidden variables from sparse observations on them. For instance, in a...
SourceID ieee
SourceType Publisher
StartPage 395
SubjectTerms belief propagation
Computational modeling
Computer science
constraint satisfaction problems
Image edge detection
Inference algorithms
Prediction algorithms
Predictive models
ran-dom graphs
random matrices
spectral algorithms
statistical inference
statistical physics
Stochastic processes
Title On statistical inference when fixed points of belief propagation are unstable
URI https://ieeexplore.ieee.org/document/9719725
WOSCitedRecordID wos000802209600037&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D
hasFullText 1
inHoldings 1
isFullTextHit
isPrint
link http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV1LSwMxEB5q8aCXalvxTQ4eXZsmu5vdc7F4sA9QobeSZCe4UHZLH-rPN0m3K4IXbyEJCUzIfJPH9w3AXRYbpTOuAyntESXUkQykRb5ASYtPSllQUF7E9VmMx8lslk4bcF9zYRDRfz7DB1f0b_lZqbfuqqyXCpckKzqAAyHiHVdr73Vt3EHDihrXp2lvOBm8RCwVjovC-l6T83cCFY8fw9b_Zj6B7g8Rj0xriDmFBhZtaO0zMZBqY7bheFSrr647MJoUxBGFvAazXJC8HuvzHQti8i_MyLLMi82alIYotHGoIdaVWufiF4rIFZKtCxzVArvwNnx8HTwFVdqEIGeUbwIm0PS5USg0x0RgnBgtJKMuVuKaUiUSZWInjG6bOVWhjCKGaSSd-JoMKT-DZlEWeA4kSmJlu8mUhjKMuYUuIwwqmdlq22YuoONsNV_ulDHmlZku_66-giPmyAP-z_M1NDerLd7Aof6w9ljd-uX8BqDKogA
linkProvider IEEE
linkToHtml http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV1LSwMxEB5qFdRLta34NgePrk2zj-yei6ViX2CF3kqyO8GFslv6UH--SbpdEbx4C0lIYELmmzy-bwDuk0DJOHFjRwh9RPFiXzhCI58jhcYnKTUoSCvi2ufDYTidRuMKPJRcGES0n8_w0RTtW36SxxtzVdaKuEmS5e_Bvu95jG7ZWju_qyMP6hXkuDaNWt1R59VnETdsFNa2qpy_U6hYBOnW_jf3CTR_qHhkXILMKVQwq0Ntl4uBFFuzDseDUn911YDBKCOGKmRVmMWcpOVYn--YEZV-YUIWeZqtVyRXRKKORBXRzlS7F7tURCyRbEzoKOfYhLfu06TTc4rECU7KqLt2GEfVdpVEHrsYcgxCFXPBqImW3JhSyUOpAiONrptdKj3h-wwjXxj5NeFR9wyqWZ7hORA_DKTuJiLqCS9wNXgprlCKRFfrNnUBDWOr2WKrjTErzHT5d_UdHPYmg_6s_zx8uYIjZqgE9gf0NVTXyw3ewEH8oW2zvLVL-w09UKVH
openUrl ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=proceeding&rft.title=Proceedings+%2F+annual+Symposium+on+Foundations+of+Computer+Science&rft.atitle=On+statistical+inference+when+fixed+points+of+belief+propagation+are+unstable&rft.au=Liu%2C+Siqi&rft.au=Mohanty%2C+Sidhanth&rft.au=Raghavendra%2C+Prasad&rft.date=2022-02-01&rft.pub=IEEE&rft.eissn=2575-8454&rft.spage=395&rft.epage=405&rft_id=info:doi/10.1109%2FFOCS52979.2021.00047&rft.externalDocID=9719725