Stochastic Scale Invariant Power Iteration for KL-divergence Nonnegative Matrix Factorization
We introduce a mini-batch stochastic variance-reduced algorithm to solve finite-sum scale invariant problems which cover several examples in machine learning and statistics such as principal component analysis (PCA) and estimation of mixture proportions. The algorithm is a stochastic generalization...
Uložené v:
| Vydané v: | IEEE International Conference on Big Data s. 969 - 977 |
|---|---|
| Hlavní autori: | , , , |
| Médium: | Konferenčný príspevok.. |
| Jazyk: | English |
| Vydavateľské údaje: |
IEEE
15.12.2024
|
| Predmet: | |
| ISSN: | 2573-2978 |
| On-line prístup: | Získať plný text |
| Tagy: |
Pridať tag
Žiadne tagy, Buďte prvý, kto otaguje tento záznam!
|
| Abstract | We introduce a mini-batch stochastic variance-reduced algorithm to solve finite-sum scale invariant problems which cover several examples in machine learning and statistics such as principal component analysis (PCA) and estimation of mixture proportions. The algorithm is a stochastic generalization of scale invariant power iteration, specializing to power iteration when full-batch is used for the PCA problem. In convergence analysis, we show the expectation of the optimality gap decreases at a linear rate under some conditions on the step size, epoch length, batch size and initial iterate. Numerical experiments on the non-negative factorization problem with the KullbackLeibler divergence using real and synthetic datasets demonstrate that the proposed stochastic approach not only converges faster than state-of-the-art deterministic algorithms but also produces excellent quality robust solutions. |
|---|---|
| AbstractList | We introduce a mini-batch stochastic variance-reduced algorithm to solve finite-sum scale invariant problems which cover several examples in machine learning and statistics such as principal component analysis (PCA) and estimation of mixture proportions. The algorithm is a stochastic generalization of scale invariant power iteration, specializing to power iteration when full-batch is used for the PCA problem. In convergence analysis, we show the expectation of the optimality gap decreases at a linear rate under some conditions on the step size, epoch length, batch size and initial iterate. Numerical experiments on the non-negative factorization problem with the KullbackLeibler divergence using real and synthetic datasets demonstrate that the proposed stochastic approach not only converges faster than state-of-the-art deterministic algorithms but also produces excellent quality robust solutions. |
| Author | Jambunath, Yegna Subramanian Kim, Cheolmin Klabjan, Diego Kim, Youngseok |
| Author_xml | – sequence: 1 givenname: Cheolmin surname: Kim fullname: Kim, Cheolmin organization: Northwestern University,Department of Industrial Engineering and Management Sciences,Evanston,USA – sequence: 2 givenname: Youngseok surname: Kim fullname: Kim, Youngseok organization: University of Chicago,Department of Statistics,Chicago,USA – sequence: 3 givenname: Yegna Subramanian surname: Jambunath fullname: Jambunath, Yegna Subramanian organization: Northwestern University,Center for Deep Learning,Evanston,USA – sequence: 4 givenname: Diego surname: Klabjan fullname: Klabjan, Diego organization: Northwestern University,Department of Industrial Engineering and Management Sciences,Evanston,USA |
| BookMark | eNo1kEtPAjEURqvRRET-gYvG_eBt7_S1VBQl4iOBrSFl5hZrsGNKg49fr_Gx-hYn5yy-Q7aXukSMnQgYCgHu9DyuLnzxWqLEoQRZDwVYqVDIHTZwxllUgFrWFnZZTyqDlXTGHrDBZvMMACiMUUr02OOsdM2T35TY8Fnj18Qnaetz9Knwh-6NMp8Uyr7ELvHQZX4zrdq4pbyi1BC_61Ki1TfdEr_1Jcd3PvZN6XL8_FGO2H7w6w0N_rbP5uPL-ei6mt5fTUZn0yo6UaqlCSpIYRECaPLGBK0DLAEbE1pUrXW1B_QahW1gqS252mLdKuGs1MEG7LPj32wkosVrji8-fyz-D8EvXrVZAA |
| ContentType | Conference Proceeding |
| DBID | 6IE 6IL CBEJK RIE RIL |
| DOI | 10.1109/BigData62323.2024.10825312 |
| DatabaseName | IEEE Electronic Library (IEL) Conference Proceedings IEEE Xplore POP ALL IEEE Xplore All Conference Proceedings IEEE Xplore IEEE Proceedings Order Plans (POP All) 1998-Present |
| DatabaseTitleList | |
| Database_xml | – sequence: 1 dbid: RIE name: IEEE Xplore url: https://ieeexplore.ieee.org/ sourceTypes: Publisher |
| DeliveryMethod | fulltext_linktorsrc |
| Discipline | Computer Science |
| EISBN | 9798350362480 |
| EISSN | 2573-2978 |
| EndPage | 977 |
| ExternalDocumentID | 10825312 |
| Genre | orig-research |
| GroupedDBID | 6IE 6IF 6IK 6IL 6IN AAJGR ABLEC ADZIZ ALMA_UNASSIGNED_HOLDINGS BEFXN BFFAM BGNUA BKEBE BPEOZ CBEJK CHZPO IEGSK IPLJI M43 OCL RIE RIL |
| ID | FETCH-LOGICAL-i91t-b7f5f21830f06ea77f66f0b03c7fd35d894a03a6318c0b68e94834d519826f8f3 |
| IEDL.DBID | RIE |
| IngestDate | Wed Aug 27 01:57:58 EDT 2025 |
| IsPeerReviewed | false |
| IsScholarly | false |
| Language | English |
| LinkModel | DirectLink |
| MergedId | FETCHMERGED-LOGICAL-i91t-b7f5f21830f06ea77f66f0b03c7fd35d894a03a6318c0b68e94834d519826f8f3 |
| PageCount | 9 |
| ParticipantIDs | ieee_primary_10825312 |
| PublicationCentury | 2000 |
| PublicationDate | 2024-Dec.-15 |
| PublicationDateYYYYMMDD | 2024-12-15 |
| PublicationDate_xml | – month: 12 year: 2024 text: 2024-Dec.-15 day: 15 |
| PublicationDecade | 2020 |
| PublicationTitle | IEEE International Conference on Big Data |
| PublicationTitleAbbrev | BigData |
| PublicationYear | 2024 |
| Publisher | IEEE |
| Publisher_xml | – name: IEEE |
| SSID | ssj0003177551 |
| Score | 1.8926382 |
| Snippet | We introduce a mini-batch stochastic variance-reduced algorithm to solve finite-sum scale invariant problems which cover several examples in machine learning... |
| SourceID | ieee |
| SourceType | Publisher |
| StartPage | 969 |
| SubjectTerms | Big Data Convergence Estimation Machine learning Machine learning algorithms Principal component analysis Synthetic data |
| Title | Stochastic Scale Invariant Power Iteration for KL-divergence Nonnegative Matrix Factorization |
| URI | https://ieeexplore.ieee.org/document/10825312 |
| hasFullText | 1 |
| inHoldings | 1 |
| isFullTextHit | |
| isPrint | |
| link | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV07T8MwELagYmDiVcRbHlhd7Dix45VHRUWpKlGhLqhynXPpkqKSVvx8zm5axMDAFnmIojtd7r7z3fcRcu0NiHAfxkCalKU28yxPCse08gAm09pHMp3Xru718uHQ9Otl9bgLAwBx-Axa4THe5RcztwitMoxwxDMyaApva61Wy1qbhgomQo3pvyYWFdzc3E4n97aymOATiUgwSVvrF_ySUomZpL33z2_YJ82fnTza32SbA7IF5SHZW4sy0DpGj8jbSzVz7zbQL-Mh_v5pp1wiIEYL0n6QRKOdSKSM_qBYsNKnLivCbEYk5aS9MPcyiWTg9Dmw93_RdlTkqdc1m2TQfhjcPbJaQ4FNjajYWPvMhyqIe67AoumV8nzMpdO-kFmRm9RyaRVGtuNjlYMJzcUCyzqEHT738pg0ylkJJ4Qm3gnEPsImAKlwiGwzm1qRc-uAu8Sdkmaw1uhjxZIxWhvq7I_zc7IbfBJGQ0R2QRrVfAGXZMctq-nn_Cr69htXmaXT |
| linkProvider | IEEE |
| linkToHtml | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV1LTwIxEG4MmugJHxjf9uC12O52H736IBAWQiIxXAwp3SlyWQwuxJ_vtCwYDx68bXrYNDOZzHzTme8j5M4qEO49jEGoJJM6siwNcsOS2AKoKEmsJ9N5zZJ-Px2N1KBaVve7MADgh8-g6T79W34-N0vXKsMIRzwTOk3h3UjKgK_XtbYtFUyFCRYAFbWo4Or-YTZ90qXGFB-EiAUD2dz84peYis8lrfo_b3FIGj9beXSwzTdHZAeKY1LfyDLQKkpPyNtLOTfv2hEw4yEmANopVgiJ0YZ04ETRaMdTKaNHKJastJux3E1neFpO2neTL1NPB057jr__i7a8Jk-1sNkgw9bz8LHNKhUFNlOiZJPERtbVQdzyGDQaP44tn_DQJDYPozxVUvNQxxjbhk_iFJRrL-ZY2CHwsKkNT0mtmBdwRmhgjUD0I3QAIIVBbBtpqUXKtQFuAnNOGs5a4481T8Z4Y6iLP85vyX572MvGWaffvSQHzj9uUEREV6RWLpZwTfbMqpx9Lm68n78BJSKpGg |
| openUrl | ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Abook&rft.genre=proceeding&rft.title=IEEE+International+Conference+on+Big+Data&rft.atitle=Stochastic+Scale+Invariant+Power+Iteration+for+KL-divergence+Nonnegative+Matrix+Factorization&rft.au=Kim%2C+Cheolmin&rft.au=Kim%2C+Youngseok&rft.au=Jambunath%2C+Yegna+Subramanian&rft.au=Klabjan%2C+Diego&rft.date=2024-12-15&rft.pub=IEEE&rft.eissn=2573-2978&rft.spage=969&rft.epage=977&rft_id=info:doi/10.1109%2FBigData62323.2024.10825312&rft.externalDocID=10825312 |