Classification and Regression Using an Outer Approximation Projection-Gradient Method
This paper deals with sparse feature selection and grouping for classification and regression. The classification or regression problems under consideration consists of minimizing a convex empirical risk function subject to an ℓ 1 constraint, a pairwise ℓ ∞ constraint, or a pairwise ℓ 1 constraint....
Uloženo v:
| Vydáno v: | IEEE transactions on signal processing Ročník 65; číslo 17; s. 4635 - 4644 |
|---|---|
| Hlavní autoři: | , , , |
| Médium: | Journal Article |
| Jazyk: | angličtina |
| Vydáno: |
IEEE
01.09.2017
|
| Témata: | |
| ISSN: | 1053-587X, 1941-0476 |
| On-line přístup: | Získat plný text |
| Tagy: |
Přidat tag
Žádné tagy, Buďte první, kdo vytvoří štítek k tomuto záznamu!
|
| Abstract | This paper deals with sparse feature selection and grouping for classification and regression. The classification or regression problems under consideration consists of minimizing a convex empirical risk function subject to an ℓ 1 constraint, a pairwise ℓ ∞ constraint, or a pairwise ℓ 1 constraint. Existing work, such as the Lasso formulation, has focused mainly on Lagrangian penalty approximations, which often require ad hoc or computationally expensive procedures to determine the penalization parameter. We depart from this approach and address the constrained problem directly via a splitting method. The structure of the method is that of the classical gradient-projection algorithm, which alternates a gradient step on the objective and a projection step onto the lower level set modeling the constraint. The novelty of our approach is that the projection step is implemented via an outer approximation scheme in which the constraint set is approximated by a sequence of simple convex sets consisting of the intersection of two half-spaces. Convergence of the iterates generated by the algorithm is established for a general smooth convex minimization problem with inequality constraints. Experiments on both synthetic and biological data show that our method outperforms penalty methods. |
|---|---|
| AbstractList | This paper deals with sparse feature selection and grouping for classification and regression. The classification or regression problems under consideration consists of minimizing a convex empirical risk function subject to an ℓ 1 constraint, a pairwise ℓ ∞ constraint, or a pairwise ℓ 1 constraint. Existing work, such as the Lasso formulation, has focused mainly on Lagrangian penalty approximations, which often require ad hoc or computationally expensive procedures to determine the penalization parameter. We depart from this approach and address the constrained problem directly via a splitting method. The structure of the method is that of the classical gradient-projection algorithm, which alternates a gradient step on the objective and a projection step onto the lower level set modeling the constraint. The novelty of our approach is that the projection step is implemented via an outer approximation scheme in which the constraint set is approximated by a sequence of simple convex sets consisting of the intersection of two half-spaces. Convergence of the iterates generated by the algorithm is established for a general smooth convex minimization problem with inequality constraints. Experiments on both synthetic and biological data show that our method outperforms penalty methods. |
| Author | Combettes, Patrick L. Barlaud, Michel Fillatre, Lionel Belhajali, Wafa |
| Author_xml | – sequence: 1 givenname: Michel surname: Barlaud fullname: Barlaud, Michel email: barlaud@i3s.unice.fr organization: I3S, Univ. Cote d'Azur, Sophia Antipolis, France – sequence: 2 givenname: Wafa surname: Belhajali fullname: Belhajali, Wafa email: wafa.ibnelhajali@gmail.com organization: I3S, Univ. Cote d'Azur, Sophia Antipolis, France – sequence: 3 givenname: Patrick L. surname: Combettes fullname: Combettes, Patrick L. email: plc@math.ncsu.edu organization: Dept. of Math., North Carolina State Univ., Raleigh, NC, USA – sequence: 4 givenname: Lionel surname: Fillatre fullname: Fillatre, Lionel email: fillatre@i3s.unice.fr organization: I3S, Univ. Cote d'Azur, Sophia Antipolis, France |
| BookMark | eNp9UMFOAjEQbQwmAno38bI_sNh2Z1t6JETRBANRSLxt2u4US3CXtGuif28R4sGDp3nzMm9m3huQXtM2SMg1oyPGqLpdvSxHnDI54pIqLvgZ6TMFLKcgRS9hWhZ5OZavF2QQ45ZSBqBEn6ynOx2jd97qzrdNpps6e8ZNwESmdh19s0lktvjoMGST_T60n_79OLsM7RbtAeazoGuPTZc9YffW1pfk3OldxKtTHZL1_d1q-pDPF7PH6WSeWy6KLmdjZtFxUwu0aKzRANII4FBYK0tuDTUaFZQsOXRKUlcD5wq1cU6YEupiSOhxrw1tjAFdtQ_pu_BVMVodYqlSLNUhluoUS5KIPxLrux8_XdB-95_w5ij0iPh7R6oCAHjxDS67dAM |
| CODEN | ITPRED |
| CitedBy_id | crossref_primary_10_1080_02331934_2018_1474470 crossref_primary_10_1109_TSP_2025_3544463 crossref_primary_10_1287_ijoc_2022_0328 crossref_primary_10_1016_j_aeue_2018_02_003 crossref_primary_10_1186_s12859_021_04478_w crossref_primary_10_1007_s10589_019_00060_6 crossref_primary_10_1109_TSP_2019_2924580 crossref_primary_10_1186_s12859_022_04900_x crossref_primary_10_1007_s10107_019_01401_3 |
| Cites_doi | 10.1007/BF02868578 10.1145/1553374.1553431 10.1056/NEJMoa021967 10.1111/j.1467-9868.2005.00490.x 10.1111/j.2517-6161.1996.tb02080.x 10.1016/0041-5553(66)90114-5 10.1137/0149053 10.1007/978-3-642-15883-4_27 10.1093/bioinformatics/btn081 10.1016/j.crma.2008.03.014 10.1109/83.563316 10.1007/BF01585107 10.1137/S036301299732626X 10.1007/s10107-015-0946-6 10.1002/cpa.20303 10.1038/msb4100180 10.1137/060669498 10.1137/0152031 10.1109/TIP.2004.832922 10.18637/jss.v033.i01 10.1007/978-1-4419-9569-8_10 10.1137/050626090 10.1007/978-1-4613-3341-8_3 10.1111/j.1541-0420.2007.00843.x 10.1073/pnas.0437847100 10.1007/978-3-319-48311-5 |
| ContentType | Journal Article |
| DBID | 97E RIA RIE AAYXX CITATION |
| DOI | 10.1109/TSP.2017.2709262 |
| DatabaseName | IEEE All-Society Periodicals Package (ASPP) 2005–Present IEEE All-Society Periodicals Package (ASPP) 1998–Present IEEE Electronic Library (IEL) CrossRef |
| DatabaseTitle | CrossRef |
| DatabaseTitleList | |
| Database_xml | – sequence: 1 dbid: RIE name: IEEE Electronic Library (IEL) url: https://ieeexplore.ieee.org/ sourceTypes: Publisher |
| DeliveryMethod | fulltext_linktorsrc |
| Discipline | Engineering |
| EISSN | 1941-0476 |
| EndPage | 4644 |
| ExternalDocumentID | 10_1109_TSP_2017_2709262 7934442 |
| Genre | orig-research |
| GroupedDBID | -~X .DC 0R~ 29I 4.4 5GY 6IK 85S 97E AAJGR AARMG AASAJ AAWTH ABAZT ABQJQ ABVLG ACGFO ACIWK ACNCT AENEX AGQYO AHBIQ AJQPL AKJIK AKQYR ALMA_UNASSIGNED_HOLDINGS ASUFR ATWAV BEFXN BFFAM BGNUA BKEBE BPEOZ CS3 EBS EJD F5P HZ~ IFIPE IPLJI JAVBF LAI MS~ O9- OCL P2P RIA RIE RNS TAE TN5 AAYXX CITATION |
| ID | FETCH-LOGICAL-c263t-181cef2bd6ecebcba447b64243cc752cb0bae9451110f970fd4229eabff6b54d3 |
| IEDL.DBID | RIE |
| ISICitedReferencesCount | 17 |
| ISICitedReferencesURI | http://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=Summon&SrcAuth=ProQuest&DestLinkType=CitingArticles&DestApp=WOS_CPL&KeyUT=000405394000016&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| ISSN | 1053-587X |
| IngestDate | Sat Nov 29 04:10:43 EST 2025 Tue Nov 18 21:27:22 EST 2025 Tue Aug 26 17:00:18 EDT 2025 |
| IsPeerReviewed | true |
| IsScholarly | true |
| Issue | 17 |
| Language | English |
| License | https://ieeexplore.ieee.org/Xplorehelp/downloads/license-information/IEEE.html |
| LinkModel | DirectLink |
| MergedId | FETCHMERGED-LOGICAL-c263t-181cef2bd6ecebcba447b64243cc752cb0bae9451110f970fd4229eabff6b54d3 |
| PageCount | 10 |
| ParticipantIDs | ieee_primary_7934442 crossref_primary_10_1109_TSP_2017_2709262 crossref_citationtrail_10_1109_TSP_2017_2709262 |
| PublicationCentury | 2000 |
| PublicationDate | 2017-Sept.1,-1 2017-9-1 |
| PublicationDateYYYYMMDD | 2017-09-01 |
| PublicationDate_xml | – month: 09 year: 2017 text: 2017-Sept.1,-1 day: 01 |
| PublicationDecade | 2010 |
| PublicationTitle | IEEE transactions on signal processing |
| PublicationTitleAbbrev | TSP |
| PublicationYear | 2017 |
| Publisher | IEEE |
| Publisher_xml | – name: IEEE |
| References | ref35 ref13 ref12 ref15 ref36 ref14 belhajali (ref4) 2014 ref30 ref11 ref10 ref16 qian (ref31) 2013 ref19 schmidt (ref32) 2011 ref18 (ref1) 0 mairal (ref28) 2012 ref24 donoho (ref17) 2003; 100 ref26 ref25 hastie (ref22) 2004; 5 tibshirani (ref34) 1996; b58 friedman (ref21) 2010; 33 ref27 haugazeau (ref23) 1968 ref29 ref8 ref7 (ref2) 0 ref9 ref3 ref6 sra (ref33) 2012 ref5 figueiredo (ref20) 2016 |
| References_xml | – ident: ref29 doi: 10.1007/BF02868578 – ident: ref24 doi: 10.1145/1553374.1553431 – ident: ref36 doi: 10.1056/NEJMoa021967 – ident: ref35 doi: 10.1111/j.1467-9868.2005.00490.x – volume: b58 start-page: 267 year: 1996 ident: ref34 article-title: Regression shrinkage and selection via the Lasso publication-title: J Roy Stat Soc doi: 10.1111/j.2517-6161.1996.tb02080.x – start-page: 353 year: 2012 ident: ref28 article-title: Complexity analysis of the Lasso regularization path publication-title: Proc 29th Int Conf Mach Learn – ident: ref25 doi: 10.1016/0041-5553(66)90114-5 – start-page: 1458 year: 2011 ident: ref32 article-title: Convergence rates of inexact proximal-gradient methods for convex optimization publication-title: Proc Adv Neural Inf Process Syst – year: 2013 ident: ref31 article-title: Glmnet for MatLab – volume: 5 start-page: 1391 year: 2004 ident: ref22 article-title: The entire regularization path for the support vector machine publication-title: J Mach Learn Res – start-page: 930 year: 2016 ident: ref20 article-title: Ordered weighted $\ell _1$ regularized regression with strongly correlated covariates: Theoretical aspects publication-title: Proc 19th Int Conf Artif Intell – ident: ref19 doi: 10.1137/0149053 – year: 0 ident: ref2 – year: 1968 ident: ref23 article-title: Sur les inéquations variationnelles et la minimisation de fonctionnelles convexes – ident: ref30 doi: 10.1007/978-3-642-15883-4_27 – year: 0 ident: ref1 – ident: ref27 doi: 10.1093/bioinformatics/btn081 – ident: ref6 doi: 10.1016/j.crma.2008.03.014 – ident: ref9 doi: 10.1109/83.563316 – start-page: 232 year: 2014 ident: ref4 article-title: Boosting stochastic Newton with entropy constraint for large-scale image classification publication-title: Proc 22nd Int Conf Pattern Recognit – ident: ref7 doi: 10.1007/BF01585107 – ident: ref10 doi: 10.1137/S036301299732626X – ident: ref15 doi: 10.1007/s10107-015-0946-6 – ident: ref16 doi: 10.1002/cpa.20303 – ident: ref8 doi: 10.1038/msb4100180 – year: 2012 ident: ref33 publication-title: Optimization for Machine Learning – ident: ref12 doi: 10.1137/060669498 – ident: ref18 doi: 10.1137/0152031 – ident: ref11 doi: 10.1109/TIP.2004.832922 – volume: 33 start-page: 1 year: 2010 ident: ref21 article-title: Regularization paths for generalized linear models via coordinate descent publication-title: J Stat Softw doi: 10.18637/jss.v033.i01 – ident: ref13 doi: 10.1007/978-1-4419-9569-8_10 – ident: ref14 doi: 10.1137/050626090 – ident: ref26 doi: 10.1007/978-1-4613-3341-8_3 – ident: ref5 doi: 10.1111/j.1541-0420.2007.00843.x – volume: 100 start-page: 2197 year: 2003 ident: ref17 article-title: Optimally sparse representation in general (nonorthogonal) dictionaries via $\ell ^1$ minimization publication-title: Proc Nat Acad Sci USA doi: 10.1073/pnas.0437847100 – ident: ref3 doi: 10.1007/978-3-319-48311-5 |
| SSID | ssj0014496 |
| Score | 2.3617165 |
| Snippet | This paper deals with sparse feature selection and grouping for classification and regression. The classification or regression problems under consideration... |
| SourceID | crossref ieee |
| SourceType | Enrichment Source Index Database Publisher |
| StartPage | 4635 |
| SubjectTerms | Biological system modeling Convergence Convex functions Convex optimization Level set Optimization outer approximation projection-gradient algorithm Signal processing algorithms |
| Title | Classification and Regression Using an Outer Approximation Projection-Gradient Method |
| URI | https://ieeexplore.ieee.org/document/7934442 |
| Volume | 65 |
| WOSCitedRecordID | wos000405394000016&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| hasFullText | 1 |
| inHoldings | 1 |
| isFullTextHit | |
| isPrint | |
| journalDatabaseRights | – providerCode: PRVIEE databaseName: IEEE Electronic Library (IEL) customDbUrl: eissn: 1941-0476 dateEnd: 99991231 omitProxy: false ssIdentifier: ssj0014496 issn: 1053-587X databaseCode: RIE dateStart: 19910101 isFulltext: true titleUrlDefault: https://ieeexplore.ieee.org/ providerName: IEEE |
| link | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV1bS8MwFA5z-KAP3qY4b-TBF8FuaZpL-zjEy4tz6AZ7K01yKgPpZHbizzdJuzJBBF9KCAmULyHnnOQ750PoMhOZ4HloY5Mstx-e8EABqCA0lHBquIhU7MUm5HAYT6fJqIWum1wYAPDkM-i5pn_LN3O9dFdlfbuXGGP2wN2QUlS5Ws2LAWNei8u6C1HAYzldPUmSpD9-GTkOl-xRSVx5vB8maE1TxZuUu93__cwe2qldRzyo1noftaA4QNtrBQU7aOI1Lh37xwOOs8LgZ3ituK4F9vwA24mfnJADHrh64l-zKnkRj6o7GdsM7heeCFbiR68vfYgmd7fjm4egFk4INBVRGVirrSGnygjQoLTKGJPKBhos0lpyqhVRGSSuMllI8kSS3DBKE8hUngvFmYmOULuYF3CMMAEqMutVEBAxM3YtickTGzWHOiaGg-mi_grLVNdVxZ24xVvqowuSpBb91KGf1uh30VUz472qqPHH2I4DvhlXY37ye_cp2nKTKwbYGWqXiyWco039Wc4-Fhd-u3wDHVe-rw |
| linkProvider | IEEE |
| linkToHtml | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV1bS8MwFA5DBfXB2xTnNQ--CHZL06SXxyHOidscusHeSpOcykA6mZ34803SrkwQwZc2hLSUL6EnJ-c750PoKvETn6eu9k2SVF94xB0BIBxXUcKp4r4nQis2EQwG4WQSDWvopsqFAQBLPoOmadpYvprJhTkqa-m1xBjTP9x1rm-kyNaqYgaMWTUuvWHwHB4Gk2VQkkSt0cvQsLiCJg2IKZD3wwitqKpYo9LZ_d_n7KGdcvOI28Vs76MaZAdoe6WkYB2Nrcql4f9YyHGSKfwMrwXbNcOWIaA78ZORcsBtU1H8a1qkL-JhcSqjm8793FLBcty3CtOHaNy5G912nVI6wZHU93JH220JKRXKBwlCioSxQGhXg3lSBpxKQUQCkalN5pI0CkiqGKURJCJNfcGZ8o7QWjbL4BhhAtRP9L6CgB8ypWeTqDTSfrMrQ6I4qAZqLbGMZVlX3MhbvMXWvyBRrNGPDfpxiX4DXVdPvBc1Nf4YWzfAV-NKzE9-775Em91Rvxf3HgaPp2jLvKjgg52htXy-gHO0IT_z6cf8wi6dbzMMwfY |
| openUrl | ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Classification+and+Regression+Using+an+Outer+Approximation+Projection-Gradient+Method&rft.jtitle=IEEE+transactions+on+signal+processing&rft.au=Barlaud%2C+Michel&rft.au=Belhajali%2C+Wafa&rft.au=Combettes%2C+Patrick+L.&rft.au=Fillatre%2C+Lionel&rft.date=2017-09-01&rft.pub=IEEE&rft.issn=1053-587X&rft.volume=65&rft.issue=17&rft.spage=4635&rft.epage=4644&rft_id=info:doi/10.1109%2FTSP.2017.2709262&rft.externalDocID=7934442 |
| thumbnail_l | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/lc.gif&issn=1053-587X&client=summon |
| thumbnail_m | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/mc.gif&issn=1053-587X&client=summon |
| thumbnail_s | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/sc.gif&issn=1053-587X&client=summon |