Stochastic Scale Invariant Power Iteration for KL-divergence Nonnegative Matrix Factorization

We introduce a mini-batch stochastic variance-reduced algorithm to solve finite-sum scale invariant problems which cover several examples in machine learning and statistics such as principal component analysis (PCA) and estimation of mixture proportions. The algorithm is a stochastic generalization...

Full description

Saved in:
Bibliographic Details
Published in:IEEE International Conference on Big Data pp. 969 - 977
Main Authors: Kim, Cheolmin, Kim, Youngseok, Jambunath, Yegna Subramanian, Klabjan, Diego
Format: Conference Proceeding
Language:English
Published: IEEE 15.12.2024
Subjects:
Big Data
Convergence
Estimation
Machine learning
Machine learning algorithms
Principal component analysis
Synthetic data
ISSN:2573-2978
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Abstract We introduce a mini-batch stochastic variance-reduced algorithm to solve finite-sum scale invariant problems which cover several examples in machine learning and statistics such as principal component analysis (PCA) and estimation of mixture proportions. The algorithm is a stochastic generalization of scale invariant power iteration, specializing to power iteration when full-batch is used for the PCA problem. In convergence analysis, we show the expectation of the optimality gap decreases at a linear rate under some conditions on the step size, epoch length, batch size and initial iterate. Numerical experiments on the non-negative factorization problem with the KullbackLeibler divergence using real and synthetic datasets demonstrate that the proposed stochastic approach not only converges faster than state-of-the-art deterministic algorithms but also produces excellent quality robust solutions.
AbstractList We introduce a mini-batch stochastic variance-reduced algorithm to solve finite-sum scale invariant problems which cover several examples in machine learning and statistics such as principal component analysis (PCA) and estimation of mixture proportions. The algorithm is a stochastic generalization of scale invariant power iteration, specializing to power iteration when full-batch is used for the PCA problem. In convergence analysis, we show the expectation of the optimality gap decreases at a linear rate under some conditions on the step size, epoch length, batch size and initial iterate. Numerical experiments on the non-negative factorization problem with the KullbackLeibler divergence using real and synthetic datasets demonstrate that the proposed stochastic approach not only converges faster than state-of-the-art deterministic algorithms but also produces excellent quality robust solutions.
Author Jambunath, Yegna Subramanian
Kim, Cheolmin
Klabjan, Diego
Kim, Youngseok
Author_xml – sequence: 1
  givenname: Cheolmin
  surname: Kim
  fullname: Kim, Cheolmin
  organization: Northwestern University,Department of Industrial Engineering and Management Sciences,Evanston,USA
– sequence: 2
  givenname: Youngseok
  surname: Kim
  fullname: Kim, Youngseok
  organization: University of Chicago,Department of Statistics,Chicago,USA
– sequence: 3
  givenname: Yegna Subramanian
  surname: Jambunath
  fullname: Jambunath, Yegna Subramanian
  organization: Northwestern University,Center for Deep Learning,Evanston,USA
– sequence: 4
  givenname: Diego
  surname: Klabjan
  fullname: Klabjan, Diego
  organization: Northwestern University,Department of Industrial Engineering and Management Sciences,Evanston,USA
BookMark eNo1kEtPAjEURqvRRET-gYvG_eBt7_S1VBQl4iOBrSFl5hZrsGNKg49fr_Gx-hYn5yy-Q7aXukSMnQgYCgHu9DyuLnzxWqLEoQRZDwVYqVDIHTZwxllUgFrWFnZZTyqDlXTGHrDBZvMMACiMUUr02OOsdM2T35TY8Fnj18Qnaetz9Knwh-6NMp8Uyr7ELvHQZX4zrdq4pbyi1BC_61Ki1TfdEr_1Jcd3PvZN6XL8_FGO2H7w6w0N_rbP5uPL-ei6mt5fTUZn0yo6UaqlCSpIYRECaPLGBK0DLAEbE1pUrXW1B_QahW1gqS252mLdKuGs1MEG7LPj32wkosVrji8-fyz-D8EvXrVZAA
ContentType Conference Proceeding
DBID 6IE
6IL
CBEJK
RIE
RIL
DOI 10.1109/BigData62323.2024.10825312
DatabaseName IEEE Electronic Library (IEL) Conference Proceedings
IEEE Xplore POP ALL
IEEE Xplore All Conference Proceedings
IEEE Electronic Library (IEL)
IEEE Proceedings Order Plans (POP All) 1998-Present
DatabaseTitleList
Database_xml – sequence: 1
  dbid: RIE
  name: IEEE Electronic Library (IEL)
  url: https://ieeexplore.ieee.org/
  sourceTypes: Publisher
DeliveryMethod fulltext_linktorsrc
Discipline Computer Science
EISBN 9798350362480
EISSN 2573-2978
EndPage 977
ExternalDocumentID 10825312
Genre orig-research
GroupedDBID 6IE
6IF
6IK
6IL
6IN
AAJGR
ABLEC
ADZIZ
ALMA_UNASSIGNED_HOLDINGS
BEFXN
BFFAM
BGNUA
BKEBE
BPEOZ
CBEJK
CHZPO
IEGSK
IPLJI
M43
OCL
RIE
RIL
ID FETCH-LOGICAL-i91t-b7f5f21830f06ea77f66f0b03c7fd35d894a03a6318c0b68e94834d519826f8f3
IEDL.DBID RIE
IngestDate Wed Aug 27 01:57:58 EDT 2025
IsPeerReviewed false
IsScholarly false
Language English
LinkModel DirectLink
MergedId FETCHMERGED-LOGICAL-i91t-b7f5f21830f06ea77f66f0b03c7fd35d894a03a6318c0b68e94834d519826f8f3
PageCount 9
ParticipantIDs ieee_primary_10825312
PublicationCentury 2000
PublicationDate 2024-Dec.-15
PublicationDateYYYYMMDD 2024-12-15
PublicationDate_xml – month: 12
  year: 2024
  text: 2024-Dec.-15
  day: 15
PublicationDecade 2020
PublicationTitle IEEE International Conference on Big Data
PublicationTitleAbbrev BigData
PublicationYear 2024
Publisher IEEE
Publisher_xml – name: IEEE
SSID ssj0003177551
Score 1.8926382
Snippet We introduce a mini-batch stochastic variance-reduced algorithm to solve finite-sum scale invariant problems which cover several examples in machine learning...
SourceID ieee
SourceType Publisher
StartPage 969
SubjectTerms Big Data
Convergence
Estimation
Machine learning
Machine learning algorithms
Principal component analysis
Synthetic data
Title Stochastic Scale Invariant Power Iteration for KL-divergence Nonnegative Matrix Factorization
URI https://ieeexplore.ieee.org/document/10825312
hasFullText 1
inHoldings 1
isFullTextHit
isPrint
link http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV07TwMxDI6gYmAqjyLeysCakt4jj5VHRUWpKtGhC6pySVy6XFG5Vvx8nPRaxMDAdspwOtnyfbZjfx8hN1bmNisMZwAgWWa9YFp6jTWPTcAYqfMiUub35WCgxmM9rJfV4y6M9z4On_l2eIx3-W5ul6FVhhGO9UwaNIV3pRTrZa1tQwWBUCL818SiHa5v72bTB1MZBPgkxUowydqbF_ySUolI0m3-8xsOSOtnJ48Ot2hzSHZ8eUSaG1EGWsfoMXl7reb23QT6ZTzE3z_tlSssiNGCdBgk0WgvEimjPygmrPS5z1yYzYiknHQQ5l6mkQycvgT2_i_ajYo89bpmi4y6j6P7J1ZrKLCZ7lSskJBDyII4cOGNlCAE8IKnVoJLc6d0ZnhqBEa25YVQXofmosO0DssOUJCekEY5L_0poTZ33AhpEgXhtjBTRnkFhQONKQBP5RlpBWtNPtYsGZONoc7_OL8g-8EnYTSkk1-SRrVY-iuyZ1fV7HNxHX37DeCUpk8
linkProvider IEEE
linkToHtml http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV07T8MwELYQIMFUHkW88cCa4ubhx8qjatU0qkSHLqhyHF_pkqKSVvx8zm5axMDAFnmIrDudvrvz3fcRcm9EYuJcswAARBAbywMlrMKax4SgtVBJ7inzU5FlcjxWw3pZ3e_CWGv98JltuU__ll_MzdK1yjDCsZ6JnKbwXhLHIVuva21bKgiFAhOAmlq0zdTD42z6rCuNEB9GWAuGcWvzi19iKh5LOo1_3uKINH-28uhwizfHZMeWJ6SxkWWgdZSekrfXam7etSNgxkMEANorV1gSow3p0Imi0Z6nUkaPUExZaT8NCjed4Wk5aeYmX6aeDpwOHH__F-14TZ56YbNJRp2X0VM3qFUUgplqV0EuIAGXBzFg3GohgHNgOYuMgCJKCqlizSLNMbYNy7m0yrUXC0zssPAACdEZ2S3npT0n1CQF01zoUIJ7L4ylllZCXoDCJIBF4oI0nbUmH2uejMnGUJd_nN-Rg-5okE7SXta_IofOP25QpJ1ck91qsbQ3ZN-sqtnn4tb7-Ru1V6mW
openUrl ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Abook&rft.genre=proceeding&rft.title=IEEE+International+Conference+on+Big+Data&rft.atitle=Stochastic+Scale+Invariant+Power+Iteration+for+KL-divergence+Nonnegative+Matrix+Factorization&rft.au=Kim%2C+Cheolmin&rft.au=Kim%2C+Youngseok&rft.au=Jambunath%2C+Yegna+Subramanian&rft.au=Klabjan%2C+Diego&rft.date=2024-12-15&rft.pub=IEEE&rft.eissn=2573-2978&rft.spage=969&rft.epage=977&rft_id=info:doi/10.1109%2FBigData62323.2024.10825312&rft.externalDocID=10825312