Sparse Generalized Eigenvalue Problem Via Smooth Optimization
In this paper, we consider an ℓ 0 -norm penalized formulation of the generalized eigenvalue problem (GEP), aimed at extracting the leading sparse generalized eigenvector of a matrix pair. The formulation involves maximization of a discontinuous nonconcave objective function over a nonconvex constrai...
Uloženo v:
| Vydáno v: | IEEE transactions on signal processing Ročník 63; číslo 7; s. 1627 - 1642 |
|---|---|
| Hlavní autoři: | , , |
| Médium: | Journal Article |
| Jazyk: | angličtina |
| Vydáno: |
IEEE
01.04.2015
|
| Témata: | |
| ISSN: | 1053-587X, 1941-0476 |
| On-line přístup: | Získat plný text |
| Tagy: |
Přidat tag
Žádné tagy, Buďte první, kdo vytvoří štítek k tomuto záznamu!
|
| Abstract | In this paper, we consider an ℓ 0 -norm penalized formulation of the generalized eigenvalue problem (GEP), aimed at extracting the leading sparse generalized eigenvector of a matrix pair. The formulation involves maximization of a discontinuous nonconcave objective function over a nonconvex constraint set, and is therefore computationally intractable. To tackle the problem, we first approximate the ℓ 0 -norm by a continuous surrogate function. Then an algorithm is developed via iteratively majorizing the surrogate function by a quadratic separable function, which at each iteration reduces to a regular generalized eigenvalue problem. A preconditioned steepest ascent algorithm for finding the leading generalized eigenvector is provided. A systematic way based on smoothing is proposed to deal with the "singularity issue" that arises when a quadratic function is used to majorize the nondifferentiable surrogate function. For sparse GEPs with special structure, algorithms that admit a closed-form solution at every iteration are derived. Numerical experiments show that the proposed algorithms match or outperform existing algorithms in terms of computational complexity and support recovery. |
|---|---|
| AbstractList | In this paper, we consider an ℓ 0 -norm penalized formulation of the generalized eigenvalue problem (GEP), aimed at extracting the leading sparse generalized eigenvector of a matrix pair. The formulation involves maximization of a discontinuous nonconcave objective function over a nonconvex constraint set, and is therefore computationally intractable. To tackle the problem, we first approximate the ℓ 0 -norm by a continuous surrogate function. Then an algorithm is developed via iteratively majorizing the surrogate function by a quadratic separable function, which at each iteration reduces to a regular generalized eigenvalue problem. A preconditioned steepest ascent algorithm for finding the leading generalized eigenvector is provided. A systematic way based on smoothing is proposed to deal with the "singularity issue" that arises when a quadratic function is used to majorize the nondifferentiable surrogate function. For sparse GEPs with special structure, algorithms that admit a closed-form solution at every iteration are derived. Numerical experiments show that the proposed algorithms match or outperform existing algorithms in terms of computational complexity and support recovery. |
| Author | Junxiao Song Babu, Prabhu Palomar, Daniel P. |
| Author_xml | – sequence: 1 surname: Junxiao Song fullname: Junxiao Song email: jsong@ust.hk organization: Dept. of Electron. & Comput. Eng., Hong Kong Univ. of Sci. & Technol. (HKUST), Hong Kong, China – sequence: 2 givenname: Prabhu surname: Babu fullname: Babu, Prabhu email: eeprabhubabu@ust.hk organization: Dept. of Electron. & Comput. Eng., Hong Kong Univ. of Sci. & Technol. (HKUST), Hong Kong, China – sequence: 3 givenname: Daniel P. surname: Palomar fullname: Palomar, Daniel P. email: palomar@ust.hk organization: Dept. of Electron. & Comput. Eng., Hong Kong Univ. of Sci. & Technol. (HKUST), Hong Kong, China |
| BookMark | eNp9kE1Lw0AQhhepYFu9C17yBxJ3sptN9uBBSq1CoYVW8RYm21ldyUfZRKH99aa2ePAgc3jn8D7D8IzYoG5qYuwaeATA9e16tYxiDkkUCy2lFGdsCFpCyGWqBv3OExEmWfp6wUZt-8E5SKnVkN2ttuhbCmZUk8fS7WkTTN0b1V9YflKw9E1RUhW8OAxWVdN078Fi27nK7bFzTX3Jzi2WLV2dcsyeH6bryWM4X8yeJvfz0MRKdKFKSaOQsZVKZbEqVCJRFwigMFOZsklmuJVkCiOAoADRD2yM3RBq25fFmKnjXeObtvVkc-O6nw86j67MgecHCXkvIT9IyE8SepD_AbfeVeh3_yE3R8QR0W895ZD2-sQ34CFpyA |
| CODEN | ITPRED |
| CitedBy_id | crossref_primary_10_1109_LCOMM_2024_3438876 crossref_primary_10_1109_LSP_2021_3108675 crossref_primary_10_1109_TSP_2018_2799193 crossref_primary_10_1109_LSP_2016_2593589 crossref_primary_10_1137_20M1325381 crossref_primary_10_1109_TVT_2019_2962542 crossref_primary_10_1109_TSP_2017_2762286 crossref_primary_10_1088_1361_6420_acbe5e crossref_primary_10_1111_biom_12886 crossref_primary_10_1109_TVT_2023_3298223 crossref_primary_10_1016_j_sigpro_2019_107433 crossref_primary_10_1109_TCOMM_2018_2854175 crossref_primary_10_1080_10618600_2019_1568014 crossref_primary_10_1109_TSP_2021_3058442 crossref_primary_10_1109_LSP_2017_2698721 crossref_primary_10_1109_TSP_2015_2452219 crossref_primary_10_1016_j_sigpro_2018_12_017 crossref_primary_10_1016_j_sigpro_2022_108914 crossref_primary_10_1109_TSP_2016_2601299 crossref_primary_10_1016_j_conengprac_2019_07_007 crossref_primary_10_1007_s11222_025_10610_0 crossref_primary_10_1109_TNSE_2019_2952454 crossref_primary_10_1109_TSP_2016_2605073 crossref_primary_10_1109_TCYB_2020_2982901 crossref_primary_10_1109_LSP_2021_3132276 crossref_primary_10_1137_22M1472000 crossref_primary_10_1016_j_jfranklin_2020_03_034 |
| Cites_doi | 10.1137/050645506 10.1137/090756855 10.1038/nature04296 10.1109/TSP.2013.2293126 10.1007/s10994-010-5226-3 10.1137/120891009 10.1080/03610927708827533 10.1007/s10107-004-0552-5 10.1109/ICASSP.2008.4518498 10.1109/TSP.2009.2031732 10.1137/1.9780898719581 10.1080/757584614 10.1137/110839072 10.1007/s10898-004-3134-4 10.1287/moor.1070.0268 10.1109/78.558475 10.1080/01621459.1973.10481436 10.1198/106186006X113430 10.1093/biostatistics/kxp008 10.1109/TIP.2007.909318 10.1007/s00041-008-9045-x 10.2307/2333955 10.1007/978-3-642-99789-1_13 10.1198/0003130042836 10.1016/j.jmva.2007.06.007 10.1145/1273496.1273601 10.1016/j.compbiolchem.2005.08.006 10.1109/78.738251 |
| ContentType | Journal Article |
| DBID | 97E RIA RIE AAYXX CITATION |
| DOI | 10.1109/TSP.2015.2394443 |
| DatabaseName | IEEE Xplore (IEEE) IEEE All-Society Periodicals Package (ASPP) 1998–Present IEEE Electronic Library (IEL) CrossRef |
| DatabaseTitle | CrossRef |
| DatabaseTitleList | |
| Database_xml | – sequence: 1 dbid: RIE name: IEEE Electronic Library (IEL) url: https://ieeexplore.ieee.org/ sourceTypes: Publisher |
| DeliveryMethod | fulltext_linktorsrc |
| Discipline | Engineering |
| EISSN | 1941-0476 |
| EndPage | 1642 |
| ExternalDocumentID | 10_1109_TSP_2015_2394443 7017587 |
| Genre | orig-research |
| GrantInformation_xml | – fundername: Hong Kong RGC 617312 |
| GroupedDBID | -~X .DC 0R~ 29I 4.4 5GY 6IK 85S 97E AAJGR AARMG AASAJ AAWTH ABAZT ABQJQ ABVLG ACGFO ACIWK ACNCT AENEX AGQYO AGSQL AHBIQ AJQPL AKQYR ALMA_UNASSIGNED_HOLDINGS ASUFR ATWAV BEFXN BFFAM BGNUA BKEBE BPEOZ CS3 EBS EJD F5P HZ~ IFIPE IPLJI JAVBF LAI MS~ O9- OCL P2P RIA RIE RNS TAE TN5 3EH 53G 5VS AAYXX ABFSI ACKIV AETIX AI. AIBXA AKJIK ALLEH CITATION E.L H~9 ICLAB IFJZH VH1 |
| ID | FETCH-LOGICAL-c263t-67e9a342f466826b654a9ba116a8686f58c0f4ecbc31e1b131311dcfdea9f6543 |
| IEDL.DBID | RIE |
| ISICitedReferencesCount | 53 |
| ISICitedReferencesURI | http://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=Summon&SrcAuth=ProQuest&DestLinkType=CitingArticles&DestApp=WOS_CPL&KeyUT=000350880900001&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| ISSN | 1053-587X |
| IngestDate | Tue Nov 18 22:32:10 EST 2025 Sat Nov 29 04:10:34 EST 2025 Tue Aug 26 16:40:39 EDT 2025 |
| IsPeerReviewed | true |
| IsScholarly | true |
| Issue | 7 |
| Language | English |
| License | https://ieeexplore.ieee.org/Xplorehelp/downloads/license-information/IEEE.html |
| LinkModel | DirectLink |
| MergedId | FETCHMERGED-LOGICAL-c263t-67e9a342f466826b654a9ba116a8686f58c0f4ecbc31e1b131311dcfdea9f6543 |
| PageCount | 16 |
| ParticipantIDs | ieee_primary_7017587 crossref_citationtrail_10_1109_TSP_2015_2394443 crossref_primary_10_1109_TSP_2015_2394443 |
| PublicationCentury | 2000 |
| PublicationDate | 2015-April1, 2015-4-00 |
| PublicationDateYYYYMMDD | 2015-04-01 |
| PublicationDate_xml | – month: 04 year: 2015 text: 2015-April1, day: 01 |
| PublicationDecade | 2010 |
| PublicationTitle | IEEE transactions on signal processing |
| PublicationTitleAbbrev | TSP |
| PublicationYear | 2015 |
| Publisher | IEEE |
| Publisher_xml | – name: IEEE |
| References | ref13 ref12 ref15 ref14 ref31 ref30 ref11 ref10 ref2 ref17 ref16 ref19 ref18 jolliffe (ref1) 2002 journ e (ref7) 2010; 11 ref24 ref23 ref26 ref25 ref20 ref22 ref21 ref28 rockafellar (ref27) 1997; 28 ref29 ref8 ref9 ref4 ref3 ref6 ref5 |
| References_xml | – ident: ref5 doi: 10.1137/050645506 – ident: ref24 doi: 10.1137/090756855 – ident: ref29 doi: 10.1038/nature04296 – ident: ref21 doi: 10.1109/TSP.2013.2293126 – ident: ref11 doi: 10.1007/s10994-010-5226-3 – ident: ref20 doi: 10.1137/120891009 – ident: ref22 doi: 10.1080/03610927708827533 – ident: ref14 doi: 10.1007/s10107-004-0552-5 – ident: ref16 doi: 10.1109/ICASSP.2008.4518498 – year: 2002 ident: ref1 publication-title: Principal Component Analysis – volume: 11 start-page: 517 year: 2010 ident: ref7 article-title: Generalized power method for sparse principal component analysis publication-title: J Mach Learn Res – volume: 28 year: 1997 ident: ref27 publication-title: Convex Analysis – ident: ref31 doi: 10.1109/TSP.2009.2031732 – ident: ref25 doi: 10.1137/1.9780898719581 – ident: ref3 doi: 10.1080/757584614 – ident: ref8 doi: 10.1137/110839072 – ident: ref26 doi: 10.1007/s10898-004-3134-4 – ident: ref30 doi: 10.1287/moor.1070.0268 – ident: ref15 doi: 10.1109/78.558475 – ident: ref12 doi: 10.1080/01621459.1973.10481436 – ident: ref4 doi: 10.1198/106186006X113430 – ident: ref9 doi: 10.1093/biostatistics/kxp008 – ident: ref13 doi: 10.1109/TIP.2007.909318 – ident: ref17 doi: 10.1007/s00041-008-9045-x – ident: ref2 doi: 10.2307/2333955 – ident: ref18 doi: 10.1007/978-3-642-99789-1_13 – ident: ref19 doi: 10.1198/0003130042836 – ident: ref6 doi: 10.1016/j.jmva.2007.06.007 – ident: ref10 doi: 10.1145/1273496.1273601 – ident: ref28 doi: 10.1016/j.compbiolchem.2005.08.006 – ident: ref23 doi: 10.1109/78.738251 |
| SSID | ssj0014496 |
| Score | 2.4056957 |
| Snippet | In this paper, we consider an ℓ 0 -norm penalized formulation of the generalized eigenvalue problem (GEP), aimed at extracting the leading sparse generalized... |
| SourceID | crossref ieee |
| SourceType | Enrichment Source Index Database Publisher |
| StartPage | 1627 |
| SubjectTerms | Approximation methods Eigenvalues and eigenfunctions Minorization-maximization Principal component analysis Signal processing algorithms smooth optimization sparse generalized eigenvalue problem Sparse matrices sparse PCA Tin Vectors |
| Title | Sparse Generalized Eigenvalue Problem Via Smooth Optimization |
| URI | https://ieeexplore.ieee.org/document/7017587 |
| Volume | 63 |
| WOSCitedRecordID | wos000350880900001&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| hasFullText | 1 |
| inHoldings | 1 |
| isFullTextHit | |
| isPrint | |
| journalDatabaseRights | – providerCode: PRVIEE databaseName: IEEE Electronic Library (IEL) customDbUrl: eissn: 1941-0476 dateEnd: 99991231 omitProxy: false ssIdentifier: ssj0014496 issn: 1053-587X databaseCode: RIE dateStart: 19910101 isFulltext: true titleUrlDefault: https://ieeexplore.ieee.org/ providerName: IEEE |
| link | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV1LSwMxEB5q8aAHX1WsL_bgRTDtbpNNNkeRFg9SC63S25JNJlCwrfThwV9vsrtdKojgbVlmYPkSMjM7me8DuLWZcIeepsS4cEWYoIK4sjkmFKUWGU0ML9j1n0W_n4zHclCD-2oWBhHzy2fY8o95L9_M9dr_KmsLt33iROzAjhC8mNWqOgaM5VpcLl2gxNmMNy3JULZHw4G_wxW3vAw4Y_RHCNrSVMlDSu_wfx9zBAdl6hg8FGt9DDWcncD-FqFgA7zC8GKJQUkmPflCE3Q936bn9MZgUKjHBG8TFQync7dIwYs7MqblLOYpvPa6o8cnUgokEN3hdEW4QKko61jGuSsTMh4zJTMVRVwlPOE2TnRoGepM0wijLKKeW8doa1BJ64dKz6A-m8_wHAIupNAdExp0Hkr6yTujXKoVW86tSyGa0N5gluqSPdyLWLyneRURytShnHqU0xLlJtxVHh8Fc8Yftg0PcGVXYnvx--tL2PPOxf2ZK6ivFmu8hl39uZosFzf5tvgGIaaz2Q |
| linkProvider | IEEE |
| linkToHtml | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV1bS8MwFD7MKagP3qY4r3nwRbBbu6RJ8yiyMXHOwabsraRJCgO3yS4--OtN2qxMEMG3Uk5C-RJyzmnO-T6AmzRh5tCT2FPGXXmEYeaZtDn0sOaSJThSNGfX77BuNxoOea8Ed0UvjNY6Kz7TNfuY3eWrqVzaX2V1ZrZPGLEN2LTKWa5bq7gzICRT4zIBA_aM1XB1Kenz-qDfs1VcYc0KgROCfzihNVWVzKm09v_3OQew54JHdJ-v9iGU9OQIdtcoBStgNYZnc40cnfToSyvUtIybltVbo16uH4PeRgL1x1OzTOjFHBpj1415DK-t5uCh7TmJBE82KF54lGkuMGmkhFKTKCQ0JIInIgioiGhE0zCSfkq0TCQOdJAE2LLrKJkqLXhq20pPoDyZTvQpIMo4kw3lK21GCG5775QwwVaYUpqaIKIK9RVmsXT84VbG4j3O8gifxwbl2KIcO5SrcFuM-Mi5M_6wrViACzuH7dnvr69huz147sSdx-7TOezYifJqmgsoL2ZLfQlb8nMxms-usi3yDbADtyI |
| openUrl | ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Sparse+Generalized+Eigenvalue+Problem+Via+Smooth+Optimization&rft.jtitle=IEEE+transactions+on+signal+processing&rft.au=Junxiao+Song&rft.au=Babu%2C+Prabhu&rft.au=Palomar%2C+Daniel+P.&rft.date=2015-04-01&rft.pub=IEEE&rft.issn=1053-587X&rft.volume=63&rft.issue=7&rft.spage=1627&rft.epage=1642&rft_id=info:doi/10.1109%2FTSP.2015.2394443&rft.externalDocID=7017587 |
| thumbnail_l | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/lc.gif&issn=1053-587X&client=summon |
| thumbnail_m | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/mc.gif&issn=1053-587X&client=summon |
| thumbnail_s | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/sc.gif&issn=1053-587X&client=summon |