Robust Multivariate Lasso Regression with Covariance Estimation
Multivariate regression with covariance estimation (MRCE) is a method that performs sparse estimation of multivariate regression coefficients, while taking account the covariance structure of the response variables. MRCE uses a penalized likelihood approach to simultaneously estimate the regression...
Uloženo v:
| Vydáno v: | Journal of computational and graphical statistics Ročník 32; číslo 3; s. 961 - 973 |
|---|---|
| Hlavní autoři: | , |
| Médium: | Journal Article |
| Jazyk: | angličtina |
| Vydáno: |
Alexandria
Taylor & Francis
03.07.2023
Taylor & Francis Ltd |
| Témata: | |
| ISSN: | 1061-8600, 1537-2715 |
| On-line přístup: | Získat plný text |
| Tagy: |
Přidat tag
Žádné tagy, Buďte první, kdo vytvoří štítek k tomuto záznamu!
|
| Abstract | Multivariate regression with covariance estimation (MRCE) is a method that performs sparse estimation of multivariate regression coefficients, while taking account the covariance structure of the response variables. MRCE uses a penalized likelihood approach to simultaneously estimate the regression coefficients and the inverse covariance matrix so that prediction accuracy can be significantly improved. However, traditional likelihood-based methods such as MRCE can produce very misleading results in the presence of outliers. In this work, we propose an extension of MRCE, namely, a robust multivariate lasso regression with covariance estimation (RMLC) to handle potential outliers within the data. By using Huber's loss or Tukey's biweight loss, RMLC can be resistant to outliers in the responses or in both the responses and the covariates. A novel optimization algorithm that incorporates a 2-fold accelerated proximal gradient (APG) algorithm is developed to solve RMLC efficiently. We also demonstrate that our proposed RMLC enjoys the oracle property. Our simulation study shows that RMLC produces very reliable results for both the regression coefficients and the correlation structure of the responses, even if the data are contaminated. A real analysis on hyperspectral data further demonstrates the utility of RMLC.
Supplementary materials
for this article are available online. |
|---|---|
| AbstractList | Multivariate regression with covariance estimation (MRCE) is a method that performs sparse estimation of multivariate regression coefficients, while taking account the covariance structure of the response variables. MRCE uses a penalized likelihood approach to simultaneously estimate the regression coefficients and the inverse covariance matrix so that prediction accuracy can be significantly improved. However, traditional likelihood-based methods such as MRCE can produce very misleading results in the presence of outliers. In this work, we propose an extension of MRCE, namely, a robust multivariate lasso regression with covariance estimation (RMLC) to handle potential outliers within the data. By using Huber's loss or Tukey's biweight loss, RMLC can be resistant to outliers in the responses or in both the responses and the covariates. A novel optimization algorithm that incorporates a 2-fold accelerated proximal gradient (APG) algorithm is developed to solve RMLC efficiently. We also demonstrate that our proposed RMLC enjoys the oracle property. Our simulation study shows that RMLC produces very reliable results for both the regression coefficients and the correlation structure of the responses, even if the data are contaminated. A real analysis on hyperspectral data further demonstrates the utility of RMLC.
Supplementary materials
for this article are available online. Multivariate regression with covariance estimation (MRCE) is a method that performs sparse estimation of multivariate regression coefficients, while taking account the covariance structure of the response variables. MRCE uses a penalized likelihood approach to simultaneously estimate the regression coefficients and the inverse covariance matrix so that prediction accuracy can be significantly improved. However, traditional likelihood-based methods such as MRCE can produce very misleading results in the presence of outliers. In this work, we propose an extension of MRCE, namely, a robust multivariate lasso regression with covariance estimation (RMLC) to handle potential outliers within the data. By using Huber’s loss or Tukey’s biweight loss, RMLC can be resistant to outliers in the responses or in both the responses and the covariates. A novel optimization algorithm that incorporates a 2-fold accelerated proximal gradient (APG) algorithm is developed to solve RMLC efficiently. We also demonstrate that our proposed RMLC enjoys the oracle property. Our simulation study shows that RMLC produces very reliable results for both the regression coefficients and the correlation structure of the responses, even if the data are contaminated. A real analysis on hyperspectral data further demonstrates the utility of RMLC. Supplementary materials for this article are available online. |
| Author | Chang, Le Welsh, A. H. |
| Author_xml | – sequence: 1 givenname: Le orcidid: 0000-0001-6045-9727 surname: Chang fullname: Chang, Le organization: Research School of Finance, Actuarial Studies, and Statistics, Australian National University – sequence: 2 givenname: A. H. orcidid: 0000-0002-3165-9559 surname: Welsh fullname: Welsh, A. H. organization: Research School of Finance, Actuarial Studies, and Statistics, Australian National University |
| BookMark | eNqFkE9LAzEQxYNUsK1-BGHB89aZpLvZ4kGl1D9QEYqeQzab1ZTtpiZZS7-9aasXD3qagXlv5s1vQHqtbTUh5wgjhAIuEXIscoARBUpHFLHgGT0ifcwYTynHrBf7qEl3ohMy8H4JAJhPeJ9cL2zZ-ZA8dU0wn9IZGXQyl97bZKHfnPbe2DbZmPCeTO1-3iqdzHwwKxni6JQc17Lx-uy7Dsnr3exl-pDOn-8fp7fzVDFWhJRNUIGWQFEypvOqBlWxCjgWtFQ5z7MMeMW44ljSvGTIkMeEoItClaxGZENycdi7dvaj0z6Ipe1cG08KWuRjNsk4G0fV1UGlnPXe6VooE_Y5g5OmEQhiR0z8EBM7YuKbWHRnv9xrF9902399NwefaWvrVnJjXVOJILeNdbWLwIwX7O8VX_pvgqc |
| CitedBy_id | crossref_primary_10_1002_wics_70021 crossref_primary_10_3390_sym15122155 crossref_primary_10_3390_horticulturae10121321 crossref_primary_10_1007_s42081_025_00312_2 |
| Cites_doi | 10.1214/07-AOAS131 10.1016/j.csda.2015.02.005 10.1137/080716542 10.1214/aos/1176350366 10.1002/cpa.20042 10.1198/073500106000000251 10.1198/TECH.2010.09114 10.1093/biomet/asm018 10.1111/rssb.12033 10.1214/18-EJS1427 10.1198/016214506000000735 10.1016/j.agrformet.2019.03.020 10.1214/11-EJS635 10.1093/biostatistics/kxm045 10.1016/j.jmva.2012.03.013 10.1214/07-AOS588 10.1017/CBO9780511804458.003 10.1007/978-3-319-22404-6_19 10.1214/088342307000000087 10.1016/j.csda.2017.02.002 10.1007/978-1-4615-7821-5_15 10.1090/conm/443/08555 10.1080/00401706.2017.1305299 10.1016/j.csda.2021.107315 10.1214/08-AOS625 10.1198/016214501753382273 10.1198/004017004000000329 10.1198/jcgs.2010.09188 10.1016/0034-4257(92)90059-S 10.1093/biomet/81.3.425 10.1137/0601049 10.1111/j.2517-6161.1996.tb02080.x 10.5705/ss.2013.192 10.1214/12-AOAS575 10.1214/aoms/1177703732 10.1214/11-AOAS494 |
| ContentType | Journal Article |
| Copyright | 2022 American Statistical Association and Institute of Mathematical Statistics 2022 2022 American Statistical Association and Institute of Mathematical Statistics |
| Copyright_xml | – notice: 2022 American Statistical Association and Institute of Mathematical Statistics 2022 – notice: 2022 American Statistical Association and Institute of Mathematical Statistics |
| DBID | AAYXX CITATION JQ2 |
| DOI | 10.1080/10618600.2022.2118752 |
| DatabaseName | CrossRef ProQuest Computer Science Collection |
| DatabaseTitle | CrossRef ProQuest Computer Science Collection |
| DatabaseTitleList | ProQuest Computer Science Collection |
| DeliveryMethod | fulltext_linktorsrc |
| Discipline | Statistics Mathematics |
| EISSN | 1537-2715 |
| EndPage | 973 |
| ExternalDocumentID | 10_1080_10618600_2022_2118752 2118752 |
| Genre | Research Article |
| GroupedDBID | -~X .4S .7F .DC .QJ 0BK 0R~ 30N 4.4 5GY AAENE AAGDL AAHIA AAJMT AALDU AAMIU AAPUL AAQRR ABCCY ABFAN ABFIM ABJNI ABLIJ ABLJU ABPAQ ABPEM ABTAI ABXUL ABXYU ABYWD ACGFO ACGFS ACIWK ACMTB ACTIO ACTMH ADCVX ADGTB AEGXH AELLO AENEX AEOZL AEPSL AEYOC AFRVT AFVYC AGDLA AGMYJ AHDZW AIAGR AIJEM AKBRZ AKBVH AKOOK ALMA_UNASSIGNED_HOLDINGS ALQZU AMVHM AQRUH AQTUD ARCSS AVBZW AWYRJ BLEHA CCCUG CS3 D0L DGEBU DKSSO DU5 EBS E~A E~B F5P GTTXZ H13 HF~ HZ~ H~P IPNFZ J.P JAA KYCEM LJTGL M4Z MS~ NA5 NY~ O9- P2P PQQKQ RIG RNANH ROSJB RTWRZ RWL RXW S-T SNACF TAE TASJS TBQAZ TDBHL TEJ TFL TFT TFW TN5 TTHFI TUROJ TUS UT5 UU3 WZA XWC ZGOLN ~S~ AAYXX CITATION JQ2 |
| ID | FETCH-LOGICAL-c338t-391c0ea021a33e6df0cd3d07182bc6765507d37c71b26b313170010e88cb3f113 |
| IEDL.DBID | TFW |
| ISICitedReferencesCount | 6 |
| ISICitedReferencesURI | http://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=Summon&SrcAuth=ProQuest&DestLinkType=CitingArticles&DestApp=WOS_CPL&KeyUT=000876591400001&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| ISSN | 1061-8600 |
| IngestDate | Thu Oct 16 01:25:24 EDT 2025 Thu Oct 16 04:38:52 EDT 2025 Tue Nov 18 20:48:58 EST 2025 Mon Oct 20 23:45:10 EDT 2025 |
| IsPeerReviewed | true |
| IsScholarly | true |
| Issue | 3 |
| Language | English |
| LinkModel | DirectLink |
| MergedId | FETCHMERGED-LOGICAL-c338t-391c0ea021a33e6df0cd3d07182bc6765507d37c71b26b313170010e88cb3f113 |
| Notes | ObjectType-Article-1 SourceType-Scholarly Journals-1 ObjectType-Feature-2 content type line 14 |
| ORCID | 0000-0001-6045-9727 0000-0002-3165-9559 |
| PQID | 2864395734 |
| PQPubID | 29738 |
| PageCount | 13 |
| ParticipantIDs | informaworld_taylorfrancis_310_1080_10618600_2022_2118752 proquest_journals_2864395734 crossref_citationtrail_10_1080_10618600_2022_2118752 crossref_primary_10_1080_10618600_2022_2118752 |
| PublicationCentury | 2000 |
| PublicationDate | 2023-07-03 |
| PublicationDateYYYYMMDD | 2023-07-03 |
| PublicationDate_xml | – month: 07 year: 2023 text: 2023-07-03 day: 03 |
| PublicationDecade | 2020 |
| PublicationPlace | Alexandria |
| PublicationPlace_xml | – name: Alexandria |
| PublicationTitle | Journal of computational and graphical statistics |
| PublicationYear | 2023 |
| Publisher | Taylor & Francis Taylor & Francis Ltd |
| Publisher_xml | – name: Taylor & Francis – name: Taylor & Francis Ltd |
| References | e_1_3_3_30_1 Rousseeuw P. J. (e_1_3_3_29_1) 2005 e_1_3_3_18_1 e_1_3_3_17_1 e_1_3_3_39_1 e_1_3_3_19_1 e_1_3_3_14_1 e_1_3_3_37_1 e_1_3_3_13_1 e_1_3_3_38_1 e_1_3_3_16_1 e_1_3_3_35_1 e_1_3_3_15_1 e_1_3_3_36_1 e_1_3_3_10_1 e_1_3_3_33_1 e_1_3_3_31_1 e_1_3_3_11_1 e_1_3_3_32_1 e_1_3_3_40_1 e_1_3_3_41_1 Banerjee O. (e_1_3_3_4_1) 2008; 9 e_1_3_3_7_1 e_1_3_3_6_1 e_1_3_3_9_1 e_1_3_3_8_1 e_1_3_3_28_1 Freue G. V. C. (e_1_3_3_12_1) 2019; 13 e_1_3_3_25_1 e_1_3_3_24_1 e_1_3_3_27_1 e_1_3_3_3_1 e_1_3_3_21_1 Rolfs B. (e_1_3_3_26_1) 2012 Van Aelst S. (e_1_3_3_34_1) 2005; 15 e_1_3_3_2_1 e_1_3_3_20_1 e_1_3_3_5_1 e_1_3_3_23_1 e_1_3_3_42_1 e_1_3_3_22_1 |
| References_xml | – ident: e_1_3_3_13_1 doi: 10.1214/07-AOAS131 – ident: e_1_3_3_32_1 doi: 10.1016/j.csda.2015.02.005 – ident: e_1_3_3_5_1 doi: 10.1137/080716542 – ident: e_1_3_3_39_1 doi: 10.1214/aos/1176350366 – ident: e_1_3_3_9_1 doi: 10.1002/cpa.20042 – volume-title: Robust Regression and Outlier Detection year: 2005 ident: e_1_3_3_29_1 – ident: e_1_3_3_35_1 doi: 10.1198/073500106000000251 – ident: e_1_3_3_22_1 doi: 10.1198/TECH.2010.09114 – ident: e_1_3_3_40_1 doi: 10.1093/biomet/asm018 – ident: e_1_3_3_8_1 doi: 10.1111/rssb.12033 – ident: e_1_3_3_20_1 doi: 10.1214/18-EJS1427 – ident: e_1_3_3_41_1 doi: 10.1198/016214506000000735 – ident: e_1_3_3_37_1 doi: 10.1016/j.agrformet.2019.03.020 – ident: e_1_3_3_18_1 doi: 10.1214/11-EJS635 – ident: e_1_3_3_14_1 doi: 10.1093/biostatistics/kxm045 – ident: e_1_3_3_19_1 doi: 10.1016/j.jmva.2012.03.013 – ident: e_1_3_3_3_1 doi: 10.1214/07-AOS588 – volume: 9 start-page: 485 year: 2008 ident: e_1_3_3_4_1 article-title: “Model Selection through Sparse Maximum Likelihood Estimation for Multivariate Gaussian or Binary Data publication-title: Journal of Machine Learning Research – ident: e_1_3_3_6_1 doi: 10.1017/CBO9780511804458.003 – ident: e_1_3_3_24_1 doi: 10.1007/978-3-319-22404-6_19 – volume: 13 start-page: 2065 year: 2019 ident: e_1_3_3_12_1 article-title: “Robust Elastic Net Estimators for Variable Selection and Identification of Proteomic Biomarkers publication-title: The Annals of Applied Statistics – start-page: 1574 year: 2012 ident: e_1_3_3_26_1 article-title: “Iterative Thresholding Algorithm for Sparse Inverse Covariance Estimation publication-title: Advances in Neural Information Processing Systems – ident: e_1_3_3_17_1 doi: 10.1214/088342307000000087 – ident: e_1_3_3_31_1 doi: 10.1016/j.csda.2017.02.002 – ident: e_1_3_3_28_1 doi: 10.1007/978-1-4615-7821-5_15 – ident: e_1_3_3_25_1 doi: 10.1090/conm/443/08555 – ident: e_1_3_3_7_1 doi: 10.1080/00401706.2017.1305299 – ident: e_1_3_3_23_1 doi: 10.1016/j.csda.2021.107315 – ident: e_1_3_3_42_1 doi: 10.1214/08-AOS625 – ident: e_1_3_3_11_1 doi: 10.1198/016214501753382273 – ident: e_1_3_3_30_1 doi: 10.1198/004017004000000329 – volume: 15 start-page: 981 year: 2005 ident: e_1_3_3_34_1 article-title: “Multivariate Regression s-estimators for Robust Estimation and Inference publication-title: Statistica Sinica – ident: e_1_3_3_27_1 doi: 10.1198/jcgs.2010.09188 – ident: e_1_3_3_15_1 doi: 10.1016/0034-4257(92)90059-S – ident: e_1_3_3_10_1 doi: 10.1093/biomet/81.3.425 – ident: e_1_3_3_21_1 doi: 10.1137/0601049 – ident: e_1_3_3_33_1 doi: 10.1111/j.2517-6161.1996.tb02080.x – ident: e_1_3_3_36_1 doi: 10.5705/ss.2013.192 – ident: e_1_3_3_2_1 doi: 10.1214/12-AOAS575 – ident: e_1_3_3_16_1 doi: 10.1214/aoms/1177703732 – ident: e_1_3_3_38_1 doi: 10.1214/11-AOAS494 |
| SSID | ssj0001697 |
| Score | 2.393826 |
| Snippet | Multivariate regression with covariance estimation (MRCE) is a method that performs sparse estimation of multivariate regression coefficients, while taking... |
| SourceID | proquest crossref informaworld |
| SourceType | Aggregation Database Enrichment Source Index Database Publisher |
| StartPage | 961 |
| SubjectTerms | Algorithms Covariance matrix Graphical lasso Lasso Multivariate analysis Multivariate regression Optimization Oracal property Outliers (statistics) Proximal gradient algorithm Regression coefficients Robust estimation Robustness (mathematics) |
| Title | Robust Multivariate Lasso Regression with Covariance Estimation |
| URI | https://www.tandfonline.com/doi/abs/10.1080/10618600.2022.2118752 https://www.proquest.com/docview/2864395734 |
| Volume | 32 |
| WOSCitedRecordID | wos000876591400001&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| hasFullText | 1 |
| inHoldings | 1 |
| isFullTextHit | |
| isPrint | |
| journalDatabaseRights | – providerCode: PRVAWR databaseName: Taylor & Francis Online Journals customDbUrl: eissn: 1537-2715 dateEnd: 99991231 omitProxy: false ssIdentifier: ssj0001697 issn: 1061-8600 databaseCode: TFW dateStart: 19920301 isFulltext: true titleUrlDefault: https://www.tandfonline.com providerName: Taylor & Francis |
| link | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwrV1NSwMxEA1SPNSDH1WxWiUHr1u7SfbrJFJaPGiRUrG3sMluvMhWutv-fmey2WIR6UHPy4RlMpl5EybvEXIbptDdQBnxTB4IT4Qs8ZJAQSybNBHCxJnJhBWbiCaTeD5PXtw0YenGKrGHNjVRhM3VeLhTVTYTcXfYxcRQqKG7Y6zPUDA7wCwMpR81DGbjt00u9p28Clh4aNK84fltla3qtMVd-iNX2wI0PvqHXz8mhw590oc6XE7IXl50yMHzhrq17JA2ws-avfmU3E8XalVW1D7TXUNbDciUPgHeXtBp_l6P0BYU73LpcGG_QwzREVjXLyLPyOt4NBs-ek5ywdPQq1YeT3w9yFMo_CnneZiZgc54BjAkZkqHUYjsZxmPdOQrFiruc6T38wd5HGvFje_zc9IqFkV-QSgLoZVCXXqmhcgA97BMxcIwhZeOTAVdIhpXS-34yFEW40P6jra0cZZEZ0nnrC7pb8w-a0KOXQbJ932Ulb0JMbVsieQ7bHvNpkt3tkvJYkRxQcTF5R-WviJtVK63k7-8R1rVcpVfk329hi1e3tgo_gITlukd |
| linkProvider | Taylor & Francis |
| linkToHtml | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwrV1LT8JAEN4YNBEPPlAjitqD1yL76OtkDIFgBA4EI7dNd9v1YloDhd_vTB8EYgwHPTezaWZn55WZ7yPkwQ2huoEwYpvYEbZwWWAHjgJbNmEghPEjE4mcbMIbj_3ZLNjchcGxSqyhTQEUkftqfNzYjK5G4h6xjPEhUkN5x1ibIWO2A25434FYi1Y-7b-vvTEtCVZAxEaZaovnt2O24tMWeukPb52HoP7Jf_z8KTkuE1DrubCYM7IXJw1yNFqjty4apI4ZaAHgfE6eJqlaLjIr39RdQWUNyak1hJQ7tSbxRzFFm1jYzrW6af4dzMjqgXSxFHlB3vq9aXdgl6wLtoZyNbN5QHUnDiH2h5zHbmQ6OuIRZCI-U9r1XARAi7inPaqYqzjliPBHO7Hva8UNpfyS1JI0ia-IxVyoppCanmkhIkh9WKR8YZjCviNTTpOIStdSl5DkyIzxKWmJXFopS6KyZKmsJmmvxb4KTI5dAsHmRcosb4aYgrlE8h2yrerWZfm8F5L5mMg5HhfXfzj6nhwOpqOhHL6MX29IHYns80Fg3iK1bL6Mb8mBXsF1z-9yk_4G90ntSA |
| linkToPdf | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwrV3PS8MwFA4yRebBH1NxOrUHr51rkv46icwVxTnGmLhbaJLGi7Rj7fb3-9KmQxHZQc_lhfLy8t73wsv3IXTjxdDdQBmxVeJSm3o4tEOXQyyrOKRUBVJJWopN-KNRMJuFYzNNmJuxSt1Dq4oooszV-nDPpaon4m51FxNAoYbuDuMu1oLZLmThbYDOnu6_ptHbOhk7Rl8FTGxtUz_i-W2Zb-XpG3npj2RdVqDo4B_-_RDtG_hp3VfxcoS2krSF9l7W3K15CzU1_qzom4_R3STjy7ywyne6K-irAZpaQwDcmTVJ3qsZ2tTSl7lWPyu_QxBZA7CunkSeoNdoMO0_2kZzwRbQrBY2CR3RS2Ko_DEhiSdVT0giAYcEmAvP9zT9mSS-8B2OPU4covn9nF4SBIIT5TjkFDXSLE3OkIU96KW0MD0WlEoAPljygCrM9a0j5m4b0drVTBhCcq2L8cEcw1taO4tpZzHjrDbqrs3mFSPHJoPw6z6yorwKUZVuCSMbbDv1pjNzuHOGAw3jXJ_Q8z8sfY12xw8RGz6Nni9QU6vYl1PApIMaxWKZXKIdsYLdXlyVAf0JJePr-Q |
| openUrl | ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Robust+Multivariate+Lasso+Regression+with+Covariance+Estimation&rft.jtitle=Journal+of+computational+and+graphical+statistics&rft.au=Chang%2C+Le&rft.au=Welsh%2C+A+H&rft.date=2023-07-03&rft.pub=Taylor+%26+Francis+Ltd&rft.issn=1061-8600&rft.eissn=1537-2715&rft.volume=32&rft.issue=3&rft.spage=961&rft.epage=973&rft_id=info:doi/10.1080%2F10618600.2022.2118752&rft.externalDBID=NO_FULL_TEXT |
| thumbnail_l | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/lc.gif&issn=1061-8600&client=summon |
| thumbnail_m | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/mc.gif&issn=1061-8600&client=summon |
| thumbnail_s | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/sc.gif&issn=1061-8600&client=summon |