Stochastic Scale Invariant Power Iteration for KL-divergence Nonnegative Matrix Factorization
We introduce a mini-batch stochastic variance-reduced algorithm to solve finite-sum scale invariant problems which cover several examples in machine learning and statistics such as principal component analysis (PCA) and estimation of mixture proportions. The algorithm is a stochastic generalization...
Uloženo v:
| Vydáno v: | IEEE International Conference on Big Data s. 969 - 977 |
|---|---|
| Hlavní autoři: | , , , |
| Médium: | Konferenční příspěvek |
| Jazyk: | angličtina |
| Vydáno: |
IEEE
15.12.2024
|
| Témata: | |
| ISSN: | 2573-2978 |
| On-line přístup: | Získat plný text |
| Tagy: |
Přidat tag
Žádné tagy, Buďte první, kdo vytvoří štítek k tomuto záznamu!
|
| Abstract | We introduce a mini-batch stochastic variance-reduced algorithm to solve finite-sum scale invariant problems which cover several examples in machine learning and statistics such as principal component analysis (PCA) and estimation of mixture proportions. The algorithm is a stochastic generalization of scale invariant power iteration, specializing to power iteration when full-batch is used for the PCA problem. In convergence analysis, we show the expectation of the optimality gap decreases at a linear rate under some conditions on the step size, epoch length, batch size and initial iterate. Numerical experiments on the non-negative factorization problem with the KullbackLeibler divergence using real and synthetic datasets demonstrate that the proposed stochastic approach not only converges faster than state-of-the-art deterministic algorithms but also produces excellent quality robust solutions. |
|---|---|
| AbstractList | We introduce a mini-batch stochastic variance-reduced algorithm to solve finite-sum scale invariant problems which cover several examples in machine learning and statistics such as principal component analysis (PCA) and estimation of mixture proportions. The algorithm is a stochastic generalization of scale invariant power iteration, specializing to power iteration when full-batch is used for the PCA problem. In convergence analysis, we show the expectation of the optimality gap decreases at a linear rate under some conditions on the step size, epoch length, batch size and initial iterate. Numerical experiments on the non-negative factorization problem with the KullbackLeibler divergence using real and synthetic datasets demonstrate that the proposed stochastic approach not only converges faster than state-of-the-art deterministic algorithms but also produces excellent quality robust solutions. |
| Author | Jambunath, Yegna Subramanian Kim, Cheolmin Klabjan, Diego Kim, Youngseok |
| Author_xml | – sequence: 1 givenname: Cheolmin surname: Kim fullname: Kim, Cheolmin organization: Northwestern University,Department of Industrial Engineering and Management Sciences,Evanston,USA – sequence: 2 givenname: Youngseok surname: Kim fullname: Kim, Youngseok organization: University of Chicago,Department of Statistics,Chicago,USA – sequence: 3 givenname: Yegna Subramanian surname: Jambunath fullname: Jambunath, Yegna Subramanian organization: Northwestern University,Center for Deep Learning,Evanston,USA – sequence: 4 givenname: Diego surname: Klabjan fullname: Klabjan, Diego organization: Northwestern University,Department of Industrial Engineering and Management Sciences,Evanston,USA |
| BookMark | eNo1kEtPAjEURqvRRET-gYvG_eBt7_S1VBQl4iOBrSFl5hZrsGNKg49fr_Gx-hYn5yy-Q7aXukSMnQgYCgHu9DyuLnzxWqLEoQRZDwVYqVDIHTZwxllUgFrWFnZZTyqDlXTGHrDBZvMMACiMUUr02OOsdM2T35TY8Fnj18Qnaetz9Knwh-6NMp8Uyr7ELvHQZX4zrdq4pbyi1BC_61Ki1TfdEr_1Jcd3PvZN6XL8_FGO2H7w6w0N_rbP5uPL-ei6mt5fTUZn0yo6UaqlCSpIYRECaPLGBK0DLAEbE1pUrXW1B_QahW1gqS252mLdKuGs1MEG7LPj32wkosVrji8-fyz-D8EvXrVZAA |
| ContentType | Conference Proceeding |
| DBID | 6IE 6IL CBEJK RIE RIL |
| DOI | 10.1109/BigData62323.2024.10825312 |
| DatabaseName | IEEE Electronic Library (IEL) Conference Proceedings IEEE Xplore POP ALL IEEE Xplore All Conference Proceedings IEEE Electronic Library (IEL) IEEE Proceedings Order Plans (POP All) 1998-Present |
| DatabaseTitleList | |
| Database_xml | – sequence: 1 dbid: RIE name: IEEE Electronic Library (IEL) url: https://ieeexplore.ieee.org/ sourceTypes: Publisher |
| DeliveryMethod | fulltext_linktorsrc |
| Discipline | Computer Science |
| EISBN | 9798350362480 |
| EISSN | 2573-2978 |
| EndPage | 977 |
| ExternalDocumentID | 10825312 |
| Genre | orig-research |
| GroupedDBID | 6IE 6IF 6IK 6IL 6IN AAJGR ABLEC ADZIZ ALMA_UNASSIGNED_HOLDINGS BEFXN BFFAM BGNUA BKEBE BPEOZ CBEJK CHZPO IEGSK IPLJI M43 OCL RIE RIL |
| ID | FETCH-LOGICAL-i91t-b7f5f21830f06ea77f66f0b03c7fd35d894a03a6318c0b68e94834d519826f8f3 |
| IEDL.DBID | RIE |
| IngestDate | Wed Aug 27 01:57:58 EDT 2025 |
| IsPeerReviewed | false |
| IsScholarly | false |
| Language | English |
| LinkModel | DirectLink |
| MergedId | FETCHMERGED-LOGICAL-i91t-b7f5f21830f06ea77f66f0b03c7fd35d894a03a6318c0b68e94834d519826f8f3 |
| PageCount | 9 |
| ParticipantIDs | ieee_primary_10825312 |
| PublicationCentury | 2000 |
| PublicationDate | 2024-Dec.-15 |
| PublicationDateYYYYMMDD | 2024-12-15 |
| PublicationDate_xml | – month: 12 year: 2024 text: 2024-Dec.-15 day: 15 |
| PublicationDecade | 2020 |
| PublicationTitle | IEEE International Conference on Big Data |
| PublicationTitleAbbrev | BigData |
| PublicationYear | 2024 |
| Publisher | IEEE |
| Publisher_xml | – name: IEEE |
| SSID | ssj0003177551 |
| Score | 1.8925321 |
| Snippet | We introduce a mini-batch stochastic variance-reduced algorithm to solve finite-sum scale invariant problems which cover several examples in machine learning... |
| SourceID | ieee |
| SourceType | Publisher |
| StartPage | 969 |
| SubjectTerms | Big Data Convergence Estimation Machine learning Machine learning algorithms Principal component analysis Synthetic data |
| Title | Stochastic Scale Invariant Power Iteration for KL-divergence Nonnegative Matrix Factorization |
| URI | https://ieeexplore.ieee.org/document/10825312 |
| hasFullText | 1 |
| inHoldings | 1 |
| isFullTextHit | |
| isPrint | |
| link | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV07T8MwELagYmDiVcRbHlhdnDi245VHRQVUlejQBVWOY5cuKSppxc_nzk2LGBjYEiuKoruc7-G77yPk2mQqsVI6plPuWSa1YTbBxNU4Az9M6bLURrIJ3e_no5EZNMPqcRbGex-bz3wHL-NZfjlzCyyVgYVDPiOQU3hba7Ua1toUVMARanD_DbBows3N7XRyb2sLDj4VkAmmWWf9gl9UKtGTdPf--Q37pP0zk0cHG29zQLZ8dUj21qQMtLHRI_L2Ws_cu0X4ZViE7Z_2qiUkxCBBOkBKNNqLQMqgDwoBK316ZiX2ZkRQTtrHvpdJBAOnL4je_0W7kZGnGddsk2H3YXj3yBoOBTY1Sc0KHWTAKIgHrrzVOigVeMGF06EUssxNZrmwCizb8ULl3mBxsYSwDtKOkAdxTFrVrPInhEoINHSAfdVDzBKEK4RQEM7k8JQu4O6UtFFa448VSsZ4LaizP9bPyS7qBFtDEnlBWvV84S_JjlvW08_5VdTtN7X5pNI |
| linkProvider | IEEE |
| linkToHtml | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV07T8MwELYQIMFUHkW88cDq4sSxHa88qlZto0p06IIqx7FLlxSVtOLnc3bTIgYGtsSKosiXy313ufs-hO5VIiLNuSEyppYkXCqiI5-4KqPghSlMEusgNiGzLB2P1bAeVg-zMNba0HxmW_4w_Msv5mbpS2Xg4ZDPMK8pvMeTJKbrca1tSQVCoQQAUFOLRlQ9PM6mz7rSEOJjBrlgnLQ2t_glphJiSbvxz6c4Qs2fqTw83MabY7RjyxPU2Mgy4NpLT9HbazU379oTMMMiBADcLVeQEsMe4qEXRcPdQKUMFsEAWXGvTwrfnRFoOXHmO1-mgQ4cDzx__xduB02eemCziUbtl9FTh9QqCmSmoork0nHncRB1VFgtpRPC0ZwyI13BeJGqRFOmBfi2oblIrfLlxQKAHSQeLnXsDO2W89KeI8wBakgHX1YLqMUxkzMmANCkcJXM4ewCNf1uTT7WPBmTzUZd_rF-hw46o0F_0u9mvSt06O3jG0Uifo12q8XS3qB9s6pmn4vbYOdvdiCoGQ |
| openUrl | ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Abook&rft.genre=proceeding&rft.title=IEEE+International+Conference+on+Big+Data&rft.atitle=Stochastic+Scale+Invariant+Power+Iteration+for+KL-divergence+Nonnegative+Matrix+Factorization&rft.au=Kim%2C+Cheolmin&rft.au=Kim%2C+Youngseok&rft.au=Jambunath%2C+Yegna+Subramanian&rft.au=Klabjan%2C+Diego&rft.date=2024-12-15&rft.pub=IEEE&rft.eissn=2573-2978&rft.spage=969&rft.epage=977&rft_id=info:doi/10.1109%2FBigData62323.2024.10825312&rft.externalDocID=10825312 |