Exact continuous relaxations of ℓ0-regularized criteria with non-quadratic data terms
We consider the minimization of ℓ0-regularized criteria involving non-quadratic data terms such as the Kullback-Leibler divergence and the logistic regression, possibly combined with an ℓ2 regularization. We first prove the existence of global minimizers for such problems and characterize their loca...
Saved in:
| Published in: | Journal of global optimization Vol. 93; no. 3; pp. 651 - 699 |
|---|---|
| Main Authors: | , , |
| Format: | Journal Article |
| Language: | English |
| Published: |
Dordrecht
Springer Nature B.V
01.11.2025
Springer Verlag |
| Subjects: | |
| ISSN: | 0925-5001, 1573-2916 |
| Online Access: | Get full text |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| Abstract | We consider the minimization of ℓ0-regularized criteria involving non-quadratic data terms such as the Kullback-Leibler divergence and the logistic regression, possibly combined with an ℓ2 regularization. We first prove the existence of global minimizers for such problems and characterize their local minimizers. Then, we propose a new class of continuous relaxations of the ℓ0 pseudo-norm, termed as ℓ0 Bregman Relaxations (B-rex). They are defined in terms of suitable Bregman distances and lead to exact continuous relaxations of the original ℓ0-regularized problem in the sense that they do not alter its set of global minimizers and reduce its non-convexity by eliminating certain local minimizers. Both features make such relaxed problems more amenable to be solved by standard non-convex optimization algorithms. In this spirit, we consider the proximal gradient algorithm and provide explicit computation of proximal points for the B-rex penalty in several cases. Finally, we report a set of numerical results illustrating the geometrical behavior of the proposed B-rex penalty for different choices of the underlying Bregman distance, its relation with convex envelopes, as well as its exact relaxation properties in 1D/2D and higher dimensions. |
|---|---|
| AbstractList | We consider the minimization of ℓ0-regularized criteria involving non-quadratic data terms such as the Kullback-Leibler divergence and the logistic regression, possibly combined with an ℓ2 regularization. We first prove the existence of global minimizers for such problems and characterize their local minimizers. Then, we propose a new class of continuous relaxations of the ℓ0 pseudo-norm, termed as ℓ0 Bregman Relaxations (B-rex). They are defined in terms of suitable Bregman distances and lead to exact continuous relaxations of the original ℓ0-regularized problem in the sense that they do not alter its set of global minimizers and reduce its non-convexity by eliminating certain local minimizers. Both features make such relaxed problems more amenable to be solved by standard non-convex optimization algorithms. In this spirit, we consider the proximal gradient algorithm and provide explicit computation of proximal points for the B-rex penalty in several cases. Finally, we report a set of numerical results illustrating the geometrical behavior of the proposed B-rex penalty for different choices of the underlying Bregman distance, its relation with convex envelopes, as well as its exact relaxation properties in 1D/2D and higher dimensions. We propose a new class of exact continuous relaxations of l0-regularized criteria involving non-quadratic data terms such as the Kullback-Leibler divergence and the logistic regression, possibly combined with an l2 regularization. We first prove the existence of global minimizers for such problems and characterize their local minimizers.Then, we propose the l0 Bregman Relaxation (B-rex), a continuous approximation of the l0 pseudo-norm defined in terms of suitable Bregman distances, which leads to an exact continuous relaxations of the original l0-regularized problem in the sense that it does not alter its set of global minimizers and reduces the non-convexity by eliminating certain local minimizers. Both features make the relaxed problem more amenable to be solved by standard non-convex optimization algorithms. In this spirit, we consider the proximal gradient algorithm and provide explicit computation of proximal points for the B-rex penalty in several cases. Finally, we report a set of numerical results illustrating the geometrical behavior of the proposed B-rex penalty for different choices of the underlying Bregman distance, its relation with convex envelopes, as well as its exact relaxation properties in 1D/2D and higher dimensions. |
| Author | Soubies, Emmanuel Essafri, Mhamed Calatroni, Luca |
| Author_xml | – sequence: 1 givenname: Mhamed surname: Essafri fullname: Essafri, Mhamed – sequence: 2 givenname: Luca surname: Calatroni fullname: Calatroni, Luca – sequence: 3 givenname: Emmanuel surname: Soubies fullname: Soubies, Emmanuel |
| BackLink | https://hal.science/hal-04451909$$DView record in HAL |
| BookMark | eNotjsFOwzAQRC1UJNrCD3CyxImDYdeOm_pYVYUiVeIC4hhtE5umSuPWcaD0zB_wh3wJqcpptDNvRzNgvdrXlrFrhDsESO8bhLEZC5BaAGqZisMZ66NOlZAGRz3WB9NFGgAv2KBp1gBgxlr22dtsT3nkua9jWbe-bXiwFe0plr5uuHf89_sHRLDvbUWhPNiC56GMNpTEP8u44t0OsWupCN1HzguKxLt001yyc0dVY6_-dcheH2Yv07lYPD8-TScLsZJoonCpRRg5XTjpELVOlCsS0xlmlObdnSoJzlor84J0vsRCqyVRYSjVDgiNGrLbU--Kqmwbyg2Fr8xTmc0ni-zoQZJoNGA-sGNvTuw2-F1rm5itfRvqbl6mZJocQYXqD7YYZbs |
| ContentType | Journal Article |
| Copyright | The Author’s 2025. This work is published under http://creativecommons.org/licenses/by/4.0/ (the “License”). Notwithstanding the ProQuest Terms and Conditions, you may use this content in accordance with the terms of the License. licence_http://creativecommons.org/publicdomain/zero |
| Copyright_xml | – notice: The Author’s 2025. This work is published under http://creativecommons.org/licenses/by/4.0/ (the “License”). Notwithstanding the ProQuest Terms and Conditions, you may use this content in accordance with the terms of the License. – notice: licence_http://creativecommons.org/publicdomain/zero |
| DBID | 7SC 8FD JQ2 L7M L~C L~D 1XC |
| DOI | 10.1007/s10898-025-01527-z |
| DatabaseName | Computer and Information Systems Abstracts Technology Research Database ProQuest Computer Science Collection Advanced Technologies Database with Aerospace Computer and Information Systems Abstracts Academic Computer and Information Systems Abstracts Professional Hyper Article en Ligne (HAL) |
| DatabaseTitle | Computer and Information Systems Abstracts Technology Research Database Computer and Information Systems Abstracts – Academic Advanced Technologies Database with Aerospace ProQuest Computer Science Collection Computer and Information Systems Abstracts Professional |
| DatabaseTitleList | Computer and Information Systems Abstracts |
| DeliveryMethod | fulltext_linktorsrc |
| Discipline | Engineering Sciences (General) Mathematics |
| EISSN | 1573-2916 |
| EndPage | 699 |
| ExternalDocumentID | oai:HAL:hal-04451909v1 |
| GroupedDBID | -~C .4S .86 .DC .VR 06D 0R~ 0VY 199 1N0 203 29K 2J2 2JN 2JY 2KG 2LR 2~H 30V 4.4 406 408 409 40D 40E 5GY 5VS 67Z 6NX 78A 7SC 7WY 8FD 8FL 8TC 8UJ 95- 95. 95~ 96X AABHQ AACDK AAHNG AAIAL AAJBT AAJKR AANZL AAPKM AARTL AASML AATNV AATVU AAUYE AAWCG AAYIU AAYQN ABAKF ABBBX ABBRH ABBXA ABDBE ABDZT ABECU ABFSG ABFTV ABHLI ABHQN ABJNI ABJOX ABKCH ABKTR ABMNI ABMQK ABNWP ABQBU ABRTQ ABSXP ABTEG ABTHY ABTKH ABTMW ABWNU ABXPI ACAOD ACDTI ACGFS ACGOD ACHSB ACHXU ACIWK ACKNC ACMDZ ACMLO ACOKC ACOMO ACPIV ACSTC ACZOJ ADHHG ADHIR ADIMF ADKNI ADKPE ADRFC ADTPH ADURQ ADYFF ADZKW AEFQL AEGAL AEGNC AEJHL AEJRE AEMSY AENEX AEOHA AEPYU AETLH AEVLU AEXYK AEZWR AFBBN AFDZB AFHIU AFLOW AFOHR AFQWF AFWTZ AFZKB AGAYW AGDGC AGJBK AGMZJ AGQEE AGQMX AGRTI AGWIL AGWZB AGYKE AHAVH AHBYD AHKAY AHPBZ AHSBF AHWEU AHYZX AIAKS AIGIU AIIXL AILAN AITGF AIXLP AJRNO AJZVZ ALMA_UNASSIGNED_HOLDINGS ALWAN AMKLP AMXSW AMYLF AMYQR AOCGG ARAPS ARMRJ ASPBG ATHPR AVWKF AXYYD AYFIA AYQZM AZFZN B-. BA0 BAPOH BENPR BSONS CS3 CSCUP D-I DDRTE DL5 DNIVK DPUIP DU5 EBLON EBS EIOEI ESBYG FEDTE FERAY FFXSO FIGPU FNLPD FRRFC FWDCC GGCAI GGRSB GJIRD GNWQR GQ7 GQ8 GXS HCIFZ HF~ HG5 HG6 HMJXF HQYDN HRMNR HVGLF HZ~ I09 IAO IHE IJ- IKXTQ ITM IWAJR IXC IZIGR IZQ I~X I~Z J-C J0Z JBSCW JCJTX JQ2 JZLTJ K60 K6~ KDC KOV L7M LAK LLZTM L~C L~D M2O MA- N9A NB0 NPVJJ NQJWS O93 O9G O9I O9J OAM P19 P2P P9M PF0 PT4 PT5 QOK QOS R89 R9I RHV RNS ROL RPX RSV S16 S1Z S27 S3B SAP SBE SDD SDH SDM SHX SISQX SJYHP SNE SNPRN SNX SOHCF SOJ SPISZ SRMVM SSLCW STPWE SZN T13 TN5 TSG TSK TSV TUC TUS U2A UG4 UOJIU UTJUX VC2 W23 W48 WK8 YLTOR Z45 ZMTXR ~EX -Y2 1SB 1XC 2.D 28- 2P1 2VQ 5QI 88I 8AO 8FE 8FG 8G5 AARHV AAYTO ABJCF ABQSL ABULA ABUWG ACBXY ADHKG AEBTG AEFIE AEKMD AFEXP AFFHD AFGCZ AFKRA AGGDS AGQPQ AI. AJBLW AMVHM ARCSS AZQEC BBWZM BDATZ BEZIV BGLVJ BGNMA BPHCQ CAG CCPQU COF DWQXO EDO EJD FINBP FRNLG FSGXE GNUQQ GUQSH H13 ITC K6V K7- KOW L6V M0C M2P M4Y M7S N2Q NDZJH NU0 O9- OVD P62 PHGZM PHGZT PQBIZ PQBZA PQGLB PQQKQ PROAC PTHSS Q2X R4E RNI RZC RZE RZK S26 S28 SCLPG T16 TEORI UZXMN VFIZW VH1 ZWQNP ZY4 |
| ID | FETCH-LOGICAL-h219t-f7e106f5df2f115543fd496f5967c1557320feee2cda5cb1d53baad9a75f0a193 |
| ISICitedReferencesCount | 0 |
| ISICitedReferencesURI | http://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=Summon&SrcAuth=ProQuest&DestLinkType=CitingArticles&DestApp=WOS_CPL&KeyUT=001598957500001&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| ISSN | 0925-5001 |
| IngestDate | Sun Nov 23 06:21:08 EST 2025 Mon Nov 24 04:10:54 EST 2025 |
| IsDoiOpenAccess | false |
| IsOpenAccess | true |
| IsPeerReviewed | true |
| IsScholarly | true |
| Issue | 3 |
| Language | English |
| License | licence_http://creativecommons.org/publicdomain/zero/: http://creativecommons.org/publicdomain/zero |
| LinkModel | OpenURL |
| MergedId | FETCHMERGED-LOGICAL-h219t-f7e106f5df2f115543fd496f5967c1557320feee2cda5cb1d53baad9a75f0a193 |
| Notes | ObjectType-Article-1 SourceType-Scholarly Journals-1 ObjectType-Feature-2 content type line 14 |
| ORCID | 0000-0003-0571-6983 0000-0003-3887-1859 |
| PQID | 3274445131 |
| PQPubID | 29930 |
| PageCount | 49 |
| ParticipantIDs | hal_primary_oai_HAL_hal_04451909v1 proquest_journals_3274445131 |
| PublicationCentury | 2000 |
| PublicationDate | 2025-11-01 |
| PublicationDateYYYYMMDD | 2025-11-01 |
| PublicationDate_xml | – month: 11 year: 2025 text: 2025-11-01 day: 01 |
| PublicationDecade | 2020 |
| PublicationPlace | Dordrecht |
| PublicationPlace_xml | – name: Dordrecht |
| PublicationTitle | Journal of global optimization |
| PublicationYear | 2025 |
| Publisher | Springer Nature B.V Springer Verlag |
| Publisher_xml | – name: Springer Nature B.V – name: Springer Verlag |
| SSID | ssj0009852 |
| Score | 2.4302945 |
| Snippet | We consider the minimization of ℓ0-regularized criteria involving non-quadratic data terms such as the Kullback-Leibler divergence and the logistic regression,... We propose a new class of exact continuous relaxations of l0-regularized criteria involving non-quadratic data terms such as the Kullback-Leibler divergence... |
| SourceID | hal proquest |
| SourceType | Open Access Repository Aggregation Database |
| StartPage | 651 |
| SubjectTerms | Algorithms Approximation Convex analysis Convexity Criteria Fines & penalties Mathematics Optimization Regression analysis Regularization Sparsity |
| Title | Exact continuous relaxations of ℓ0-regularized criteria with non-quadratic data terms |
| URI | https://www.proquest.com/docview/3274445131 https://hal.science/hal-04451909 |
| Volume | 93 |
| WOSCitedRecordID | wos001598957500001&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| hasFullText | 1 |
| inHoldings | 1 |
| isFullTextHit | |
| isPrint | |
| journalDatabaseRights | – providerCode: PRVAVX databaseName: SpringerLINK Contemporary 1997-Present customDbUrl: eissn: 1573-2916 dateEnd: 99991231 omitProxy: false ssIdentifier: ssj0009852 issn: 0925-5001 databaseCode: RSV dateStart: 19970101 isFulltext: true titleUrlDefault: https://link.springer.com/search?facet-content-type=%22Journal%22 providerName: Springer Nature |
| link | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwtV1Rb9MwELbK4AEeEBugDQayECAQsuQkTZ08VrCpiFFNdMDeIieO1Uo07ZKmqvbMP-An8M_4JdzZTtMVgeCBl6h1rEbNfbm72Pd9R8jTSCiIapyzlKcZ63pZziAopOAMVSZQkEppZZpNiOEwOj-PTzud7w0XZvlFFEW0WsXz_2pqGANjI3X2H8y9_lEYgM9gdDiC2eH4V4Y_WiHtEUvQJ0WNBa5IV1m1FW-mvKH7LA44K00j-nJyCVkneA-UbZZ2ZbaYFeyilqo0gq5YRvoKfXj1m1zWyYrMwP9MHbFznapXldSWzv5-LKeOS2V2PQwZqbD07LotGRrN6nTiaBLTqSxqV9Xv1ib80JH0ttcmsfAat0PW3Bm7AAnzQ-6m5879CuwsZ9mXjX-2HRQdDoMNZ9tzUrU2bvdso6VfQgJvKNIRkgl9LFUMfcEu2wDYbPoP-qPk9M1xcvJ2-O75_IJhazLcwnd9Wq6R674IY3SdH0afWmXnyLR1Wv8dx8xy_Myti0IGM8aC2624b5KZszvktrMc7Vv07JJOXuyRWxvalHtk13n9ir5w0uQv75LPBly0BRfdABedafrj67crsKINrCjCil6BFUVYUQOre-Tj8dHZ6wFzrTnYGELcgmmRe7ynQ6V97WFKGmjVjWEg7okMvovA5zrPcz9TMsxST8HzL6WKpQg1l_DScJ_swCXzfUIlV2EeQRaNnJIw0mkEr_xRV4d-6qdw5oA8gVuWzK34SoJy6IP-SYJjHNX1Yh4vvQNy2NzRxD2LVRKg-iXMCLwHfz79kNxs0XtIdhZlnT8iN7LlYlKVj425fwKaMoWg |
| linkProvider | Springer Nature |
| openUrl | ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Exact+continuous+relaxations+of+%E2%84%930-regularized+criteria+with+non-quadratic+data+terms&rft.jtitle=Journal+of+global+optimization&rft.au=Essafri%2C+Mhamed&rft.au=Calatroni%2C+Luca&rft.au=Soubies%2C+Emmanuel&rft.date=2025-11-01&rft.pub=Springer+Nature+B.V&rft.issn=0925-5001&rft.eissn=1573-2916&rft.volume=93&rft.issue=3&rft.spage=651&rft.epage=699&rft_id=info:doi/10.1007%2Fs10898-025-01527-z&rft.externalDBID=HAS_PDF_LINK |
| thumbnail_l | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/lc.gif&issn=0925-5001&client=summon |
| thumbnail_m | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/mc.gif&issn=0925-5001&client=summon |
| thumbnail_s | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/sc.gif&issn=0925-5001&client=summon |