Single-Loop Variance-Reduced Stochastic Algorithm for Nonconvex-Concave Minimax Optimization
Nonconvex-concave (NC-C) finite-sum minimax problems have broad applications in decentralized optimization and various machine learning tasks. However, the nonsmooth nature of NC-C problems makes it challenging to design effective variance reduction techniques. Existing vanilla stochastic algorithms...
Uložené v:
| Vydané v: | Proceedings of the ... IEEE International Conference on Acoustics, Speech and Signal Processing (1998) s. 1 - 5 |
|---|---|
| Hlavní autori: | , , , |
| Médium: | Konferenčný príspevok.. |
| Jazyk: | English |
| Vydavateľské údaje: |
IEEE
06.04.2025
|
| Predmet: | |
| ISSN: | 2379-190X |
| On-line prístup: | Získať plný text |
| Tagy: |
Pridať tag
Žiadne tagy, Buďte prvý, kto otaguje tento záznam!
|
| Abstract | Nonconvex-concave (NC-C) finite-sum minimax problems have broad applications in decentralized optimization and various machine learning tasks. However, the nonsmooth nature of NC-C problems makes it challenging to design effective variance reduction techniques. Existing vanilla stochastic algorithms using uniform samples for gradient estimation often exhibit slow convergence rates and require bounded variance assumptions. In this paper, we develop a novel probabilistic variance reduction updating scheme and propose a single-loop algorithm called the probabilistic variance-reduced smoothed gradient descent-ascent (PVR-SGDA) algorithm. The proposed algorithm achieves an iteration complexity of {\mathcal{O}}\left({{\varepsilon ^{ - 4}}}\right), surpassing the best-known rates of stochastic algorithms for NC-C minimax problems and matching the performance of the best deterministic algorithms in this context. Finally, we demonstrate the effectiveness of the proposed algorithm through numerical simulations. |
|---|---|
| AbstractList | Nonconvex-concave (NC-C) finite-sum minimax problems have broad applications in decentralized optimization and various machine learning tasks. However, the nonsmooth nature of NC-C problems makes it challenging to design effective variance reduction techniques. Existing vanilla stochastic algorithms using uniform samples for gradient estimation often exhibit slow convergence rates and require bounded variance assumptions. In this paper, we develop a novel probabilistic variance reduction updating scheme and propose a single-loop algorithm called the probabilistic variance-reduced smoothed gradient descent-ascent (PVR-SGDA) algorithm. The proposed algorithm achieves an iteration complexity of {\mathcal{O}}\left({{\varepsilon ^{ - 4}}}\right), surpassing the best-known rates of stochastic algorithms for NC-C minimax problems and matching the performance of the best deterministic algorithms in this context. Finally, we demonstrate the effectiveness of the proposed algorithm through numerical simulations. |
| Author | So, Anthony Man-Cho Zheng, Taoli Jiang, Xia Zhu, Linglingzhi |
| Author_xml | – sequence: 1 givenname: Xia surname: Jiang fullname: Jiang, Xia email: xiajiang@cuhk.edu.hk organization: The Chinese University of Hong Kong,Department of Systems Engineering and Engineering Management – sequence: 2 givenname: Linglingzhi surname: Zhu fullname: Zhu, Linglingzhi email: llzzhu@se.cuhk.edu.hk organization: The Chinese University of Hong Kong,Department of Systems Engineering and Engineering Management – sequence: 3 givenname: Taoli surname: Zheng fullname: Zheng, Taoli email: tlzheng@se.cuhk.edu.hk organization: The Chinese University of Hong Kong,Department of Systems Engineering and Engineering Management – sequence: 4 givenname: Anthony Man-Cho surname: So fullname: So, Anthony Man-Cho email: manchoso@se.cuhk.edu.hk organization: The Chinese University of Hong Kong,Department of Systems Engineering and Engineering Management |
| BookMark | eNo1UFFLwzAYjKLgNvcPfIg_IPNL0zTJ4xg6henEDvFBGGn2dYu0yWjrmP56C-rL3XEHx3FDchZiQEKuOUw4B3PzMJvm-XNqsgwmCSRywkFrAzo7IWOjjBYSRKZVyk_JIBHKMG7g7YIM2_YDAPpAD8h77sO2QraIcU9fbeNtcMhecPPpcEPzLrqdbTvv6LTaxsZ3u5qWsaFPMbgYDnhks17ZA9JHH3xtj3S573ztv23nY7gk56WtWhz_8Yis7m5Xs3u2WM778QvmjehYKrkqDS-EScFIm1gtiwKFKDn26CRaUCALY0BKLnoTlILCuQydcoUGMSJXv7UeEdf7pt_RfK3_zxA_htFX0w |
| ContentType | Conference Proceeding |
| DBID | 6IE 6IH CBEJK RIE RIO |
| DOI | 10.1109/ICASSP49660.2025.10889086 |
| DatabaseName | IEEE Electronic Library (IEL) Conference Proceedings IEEE Proceedings Order Plan (POP) 1998-present by volume IEEE Xplore All Conference Proceedings IEEE Electronic Library (IEL) IEEE Proceedings Order Plans (POP) 1998-present |
| DatabaseTitleList | |
| Database_xml | – sequence: 1 dbid: RIE name: IEEE Electronic Library (IEL) url: https://ieeexplore.ieee.org/ sourceTypes: Publisher |
| DeliveryMethod | fulltext_linktorsrc |
| Discipline | Engineering |
| EISBN | 9798350368741 |
| EISSN | 2379-190X |
| EndPage | 5 |
| ExternalDocumentID | 10889086 |
| Genre | orig-research |
| GroupedDBID | 23M 6IE 6IF 6IH 6IK 6IL 6IM 6IN AAJGR AAWTH ABLEC ACGFS ADZIZ ALMA_UNASSIGNED_HOLDINGS BEFXN BFFAM BGNUA BKEBE BPEOZ CBEJK CHZPO IEGSK IJVOP IPLJI M43 OCL RIE RIL RIO RNS |
| ID | FETCH-LOGICAL-i93t-4517f91b394095a2a85bbe33f1ee33c5ea0705b99055131ee0770bcc6ec7cb803 |
| IEDL.DBID | RIE |
| IngestDate | Wed Nov 19 08:27:13 EST 2025 |
| IsPeerReviewed | false |
| IsScholarly | true |
| Language | English |
| LinkModel | DirectLink |
| MergedId | FETCHMERGED-LOGICAL-i93t-4517f91b394095a2a85bbe33f1ee33c5ea0705b99055131ee0770bcc6ec7cb803 |
| PageCount | 5 |
| ParticipantIDs | ieee_primary_10889086 |
| PublicationCentury | 2000 |
| PublicationDate | 2025-April-6 |
| PublicationDateYYYYMMDD | 2025-04-06 |
| PublicationDate_xml | – month: 04 year: 2025 text: 2025-April-6 day: 06 |
| PublicationDecade | 2020 |
| PublicationTitle | Proceedings of the ... IEEE International Conference on Acoustics, Speech and Signal Processing (1998) |
| PublicationTitleAbbrev | ICASSP |
| PublicationYear | 2025 |
| Publisher | IEEE |
| Publisher_xml | – name: IEEE |
| SSID | ssj0008748 |
| Score | 2.2946823 |
| Snippet | Nonconvex-concave (NC-C) finite-sum minimax problems have broad applications in decentralized optimization and various machine learning tasks. However, the... |
| SourceID | ieee |
| SourceType | Publisher |
| StartPage | 1 |
| SubjectTerms | Complexity theory Convergence Machine learning algorithms nonconvex-concave minimax optimization Numerical simulation Optimization Probabilistic logic Signal processing Signal processing algorithms single-loop Smoothing methods Speech processing stochastic algorithm variance reduction |
| Title | Single-Loop Variance-Reduced Stochastic Algorithm for Nonconvex-Concave Minimax Optimization |
| URI | https://ieeexplore.ieee.org/document/10889086 |
| hasFullText | 1 |
| inHoldings | 1 |
| isFullTextHit | |
| isPrint | |
| link | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV1bS8MwFA5uiOiLt4l3Ivia2UvSpI9jOBR0DjtkD8JIstQVtnbMbuzne5Jd1AcffGlLoA2cw8l30pzzfQjdxjQ0mlFOeOqHhPrUJ5CFKALY6OlAACZ60olN8HZb9HpxZ9Ws7nphjDGu-MzU7aM7yx8UemZ_lUGEC_iEiCqowjlfNmttll3BqdhBNysSzbvHZiNJOtSST8IuMGD19cu_ZFQcirT2_zn_Aap99-PhzgZpDtGWyY_Q3g8qwWP0nsBtZMhTUUzwG2yArTfJqyVmNQOclIUeSkvJjBujj2KalcMxhnQVt4vc1Z0vCEyl5dzg5yzPxnKBX2AtGa-aNGuo27rvNh_ISjmBZHFYEsp8noLFrep5zGQgBVPKhGHqG7hqZiQEOlMARFbeBQY9zj2ldWQ010p44Qmq5kVuThGWIpIRpBE8pIzqiMrAVzaIFQt1mgbmDNWsnfqTJTdGf22i8z_GL9Cu9YarfYkuUbWczswV2tbzMvucXjuPfgFLFKGL |
| linkProvider | IEEE |
| linkToHtml | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV1bT8IwFG4UjZcXbxjv1sTX4i7t2j0SIpEIkwgxPJiQrnSyBDaCg_DzPR0D9cEHX7alyS45J6ffaXfO9yF071NXK0Y54ZHtEmpTm0AWEhLARks5AjDRkrnYBA8C0ev57aJZPe-F0VrnxWe6Yi7zf_mDVM3MVhlEuIBHCG8TbTFKHXvZrrWeeAWnYgfdFTSaD41atdNpU0M_CetAh1VWt_8SUslxpH7wzy84ROXvjjzcXmPNEdrQyTHa_0EmeILeO3AaadJM0wl-gyWw8Sd5NdSseoA7WaqG0pAy4-roI53G2XCMIWHFQZrklecLAq9Scq5xK07isVzgF5hNxkWbZhl164_d2hMptBNI7LsZoczmEdjc6J77TDpSsDDUrhvZGo6KaQmhzkKAIiPwAoMW51aolKcVV6Gw3FNUStJEnyEshSc9SCS4SxlVHpWOHZowDpmrosjR56hs7NSfLNkx-isTXfwxfot2n7qtZr_ZCJ4v0Z7xTF4J412hUjad6Wu0reZZ_Dm9yb37Bd-apNI |
| openUrl | ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Abook&rft.genre=proceeding&rft.title=Proceedings+of+the+...+IEEE+International+Conference+on+Acoustics%2C+Speech+and+Signal+Processing+%281998%29&rft.atitle=Single-Loop+Variance-Reduced+Stochastic+Algorithm+for+Nonconvex-Concave+Minimax+Optimization&rft.au=Jiang%2C+Xia&rft.au=Zhu%2C+Linglingzhi&rft.au=Zheng%2C+Taoli&rft.au=So%2C+Anthony+Man-Cho&rft.date=2025-04-06&rft.pub=IEEE&rft.eissn=2379-190X&rft.spage=1&rft.epage=5&rft_id=info:doi/10.1109%2FICASSP49660.2025.10889086&rft.externalDocID=10889086 |