Cross validation in LASSO and its acceleration
We investigate leave-one-out cross validation (CV) as a determinator of the weight of the penalty term in the least absolute shrinkage and selection operator (LASSO). First, on the basis of the message passing algorithm and a perturbative discussion assuming that the number of observations is suffic...
Gespeichert in:
| Veröffentlicht in: | Journal of statistical mechanics Jg. 2016; H. 5 |
|---|---|
| Hauptverfasser: | , |
| Format: | Journal Article |
| Sprache: | Englisch |
| Veröffentlicht: |
IOP Publishing and SISSA
31.05.2016
|
| ISSN: | 1742-5468 |
| Online-Zugang: | Volltext |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| Abstract | We investigate leave-one-out cross validation (CV) as a determinator of the weight of the penalty term in the least absolute shrinkage and selection operator (LASSO). First, on the basis of the message passing algorithm and a perturbative discussion assuming that the number of observations is sufficiently large, we provide simple formulas for approximately assessing two types of CV errors, which enable us to significantly reduce the necessary cost of computation. These formulas also provide a simple connection of the CV errors to the residual sums of squares between the reconstructed and the given measurements. Second, on the basis of this finding, we analytically evaluate the CV errors when the design matrix is given as a simple random matrix in the large size limit by using the replica method. Finally, these results are compared with those of numerical simulations on finite-size systems and are confirmed to be correct. We also apply the simple formulas of the first type of CV error to an actual dataset of the supernovae. |
|---|---|
| AbstractList | We investigate leave-one-out cross validation (CV) as a determinator of the weight of the penalty term in the least absolute shrinkage and selection operator (LASSO). First, on the basis of the message passing algorithm and a perturbative discussion assuming that the number of observations is sufficiently large, we provide simple formulas for approximately assessing two types of CV errors, which enable us to significantly reduce the necessary cost of computation. These formulas also provide a simple connection of the CV errors to the residual sums of squares between the reconstructed and the given measurements. Second, on the basis of this finding, we analytically evaluate the CV errors when the design matrix is given as a simple random matrix in the large size limit by using the replica method. Finally, these results are compared with those of numerical simulations on finite-size systems and are confirmed to be correct. We also apply the simple formulas of the first type of CV error to an actual dataset of the supernovae. |
| Author | Obuchi, Tomoyuki Kabashima, Yoshiyuki |
| Author_xml | – sequence: 1 givenname: Tomoyuki surname: Obuchi fullname: Obuchi, Tomoyuki email: obuchi@c.titech.ac.jp organization: Tokyo Institute of Technology Department of Mathematical and Computing Science, Yokohama, Kanagawa, 226-8502, Japan – sequence: 2 givenname: Yoshiyuki surname: Kabashima fullname: Kabashima, Yoshiyuki email: kaba@c.titech.ac.jp organization: Tokyo Institute of Technology Department of Mathematical and Computing Science, Yokohama, Kanagawa, 226-8502, Japan |
| BookMark | eNo9z91qhDAQBeBQutD96TvkBVwnmYkml4v0D4S92PY6aIzgIkkxts9f7ZbCwFwcOIdvx-5DDJ4xLuAoQOtclCQzRYXOJYgiB7UcItAd2_5HD2yX0hUAJZDesmM1xZT4dzMOXTMPMfAh8Pp0uZx5Ezo-zIk3zvnRT7_pgW36Zkz-8e_v2cfz03v1mtXnl7fqVGcOjZ4zVKQ0OEW6XWbaFgX1vey9KDtUBYDTCqWXnoCML6QxJLXpgEpdQK8M4p7RrXeIn_Yav6awrFkBdnXaFWNXjF2dFpS9OfEH41tGug |
| CODEN | JSMTC6 |
| CitedBy_id | crossref_primary_10_1007_s43545_021_00076_y crossref_primary_10_3389_fams_2023_1282975 crossref_primary_10_1109_TIT_2021_3095375 crossref_primary_10_1088_1751_8121_ab3e89 crossref_primary_10_1088_1751_8121_acb530 crossref_primary_10_1007_s11294_024_09898_x crossref_primary_10_1002_acs_3884 crossref_primary_10_1016_j_anucene_2023_109686 crossref_primary_10_7566_JPSJ_87_044801 crossref_primary_10_1016_j_pnucene_2023_104792 crossref_primary_10_1109_TIT_2024_3450002 crossref_primary_10_1186_s12967_020_02387_9 crossref_primary_10_7566_JPSJ_89_094804 crossref_primary_10_1111_pce_15282 crossref_primary_10_7566_JPSJ_89_012001 crossref_primary_10_1162_neco_a_01581 crossref_primary_10_1007_s42979_022_01051_x crossref_primary_10_1093_bioinformatics_btx049 crossref_primary_10_1139_cgj_2024_0181 crossref_primary_10_1186_s12931_023_02592_2 crossref_primary_10_1007_s11042_022_13568_7 crossref_primary_10_1016_j_compgeo_2025_107072 crossref_primary_10_1080_07853890_2022_2105391 crossref_primary_10_3847_1538_3881_aa6302 crossref_primary_10_1088_1742_5468_2016_12_123302 crossref_primary_10_1016_j_rser_2024_114854 crossref_primary_10_1016_j_ijpe_2025_109731 crossref_primary_10_1214_17_AOS1630 crossref_primary_10_7566_JPSJ_89_124802 crossref_primary_10_3390_e23010033 crossref_primary_10_1080_03610926_2024_2322616 crossref_primary_10_3847_1538_4357_aa6305 crossref_primary_10_1016_j_eswa_2022_116936 crossref_primary_10_1007_s42519_023_00341_8 crossref_primary_10_1016_j_pnucene_2025_105710 crossref_primary_10_3389_fmars_2023_1185106 crossref_primary_10_1016_j_jcsr_2024_109034 crossref_primary_10_7566_JPSJ_94_074002 |
| ContentType | Journal Article |
| Copyright | 2016 The Author(s). Published by IOP Publishing Ltd on behalf of SISSA Medialab srl |
| Copyright_xml | – notice: 2016 The Author(s). Published by IOP Publishing Ltd on behalf of SISSA Medialab srl |
| DBID | O3W TSCCA |
| DOI | 10.1088/1742-5468/2016/05/053304 |
| DatabaseName | Institute of Physics Open Access Journals (Activated by CARLI) IOPscience (Open Access) |
| DatabaseTitleList | |
| Database_xml | – sequence: 1 dbid: O3W name: Institute of Physics Open Access Journal Titles url: http://iopscience.iop.org/ sourceTypes: Publisher |
| DeliveryMethod | fulltext_linktorsrc |
| Discipline | Physics |
| DocumentTitleAlternate | Cross validation in LASSO and its acceleration |
| EISSN | 1742-5468 |
| ExternalDocumentID | jstataa1690 |
| GroupedDBID | 1JI 4.4 5B3 5GY 5VS 5ZH 7.M 7.Q AAGCD AAGID AAJIO AAJKP AALHV AATNI ABCXL ABJNI ABQJV ABVAM ACAFW ACGFO ACGFS ACHIP ADWVK AEFHF AFYNE AKPSB ALMA_UNASSIGNED_HOLDINGS AOAED ASPBG ATQHT AVWKF AZFZN CBCFC CEBXE CJUJL CRLBU CS3 EBS EDWGO EJD EMSAF EPQRW EQZZN HAK IHE IJHAN IOP IZVLO J9A KOT LAP M45 MV1 N5L N9A NT- NT. O3W P2P PJBAE RIN RNS ROL RPA S3P SY9 TSCCA VSI W28 XPP ZMT |
| ID | FETCH-LOGICAL-c398t-354580c548b204bb314ff2fe17d35600c8532e2e4049e62994289d047860f5933 |
| IEDL.DBID | O3W |
| ISICitedReferencesCount | 44 |
| ISICitedReferencesURI | http://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=Summon&SrcAuth=ProQuest&DestLinkType=CitingArticles&DestApp=WOS_CPL&KeyUT=000381357800004&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| IngestDate | Wed Aug 21 03:41:09 EDT 2024 |
| IsDoiOpenAccess | true |
| IsOpenAccess | true |
| IsPeerReviewed | true |
| IsScholarly | true |
| Issue | 5 |
| Language | English |
| License | Original content from this work may be used under the terms of the Creative Commons Attribution 3.0 licence. Any further distribution of this work must maintain attribution to the author(s) and the title of the work, journal citation and DOI. |
| LinkModel | DirectLink |
| MergedId | FETCHMERGED-LOGICAL-c398t-354580c548b204bb314ff2fe17d35600c8532e2e4049e62994289d047860f5933 |
| Notes | JSTAT_043P_1215 |
| OpenAccessLink | https://iopscience.iop.org/article/10.1088/1742-5468/2016/05/053304 |
| PageCount | 36 |
| ParticipantIDs | iop_journals_10_1088_1742_5468_2016_05_053304 |
| PublicationCentury | 2000 |
| PublicationDate | 20160531 |
| PublicationDateYYYYMMDD | 2016-05-31 |
| PublicationDate_xml | – month: 5 year: 2016 text: 20160531 day: 31 |
| PublicationDecade | 2010 |
| PublicationTitle | Journal of statistical mechanics |
| PublicationTitleAbbrev | JSTAT |
| PublicationTitleAlternate | J. Stat. Mech |
| PublicationYear | 2016 |
| Publisher | IOP Publishing and SISSA |
| Publisher_xml | – name: IOP Publishing and SISSA |
| SSID | ssj0032048 |
| Score | 2.4158967 |
| Snippet | We investigate leave-one-out cross validation (CV) as a determinator of the weight of the penalty term in the least absolute shrinkage and selection operator... |
| SourceID | iop |
| SourceType | Publisher |
| Title | Cross validation in LASSO and its acceleration |
| URI | https://iopscience.iop.org/article/10.1088/1742-5468/2016/05/053304 |
| Volume | 2016 |
| WOSCitedRecordID | wos000381357800004&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| hasFullText | 1 |
| inHoldings | 1 |
| isFullTextHit | |
| isPrint | |
| link | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwrV1NS8QwEA3LquDFb_GbHLxmN23SNDnK4uJBVg-KewtNmkAvXdl2_f3OtFU8CYK3EBqavg55M8zkDSG3YAKR-9yxMqQJtjDjTBeFZ8HlARwKj83Nu2YT-WKhl0vzPCKz77swq_fh6J_AsBcK7iEcCuL0FHxolO9XGgL3RE15NsX7pKgJuiV0ptDYn8Tb13EsUJl2KOH5bTVwC7zwB7fM9_9lVwdkb3At6V2_5JCMQn1EdroST98ck8kMCZGCZVV9HyVa1fQRZR1pUZe0ahtaeA8s1NvECXmd37_MHtjQLYF5YXTLBKbAuIcIxMEnOycSGWMaQ5KXAt0aD8SchjRIiAmCAhaCwMOUKM6jeMyMEKdkXK_qcEZo5KXigefGySC9lM5ElRR5CmNTaOnPCQMY7GDtje0S2VpbxMAiBhYxsDyzPQYXf3z-kuziRJ-kvyLjdr0J12Tbf7RVs77pfu8nbaGelw |
| linkProvider | IOP Publishing |
| linkToPdf | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwrV1LT4QwEJ6s6yNefBvf9uAVt9AC5WhWV42bdQ8a99ZAaRMu7GZBf78zgMaTiYm3JkCBr5N-M5npfABXaAKOmzjzchv4JGHGPZWmxrNZbNGhMCRu3ohNxJOJms2SaQ_uv8_CzBfd1n-Nw7ZRcAthVxCnBuhDU_v-SGHg7kcDHg7oPCmXg0XuVmA1FKEgDYdn8fa1JQvqTtuV8fw2A_ILvvQHv4y2_-3LdmCrczHZTfvYLvRsuQfrTamnqfbhekjEyNDCilZPiRUlG1N7R5aWOSvqiqXGIBu1tnEAr6O7l-GD16kmeEYkqvYEpcK4wUgkw9_OMuFL5wJn_TgX5N4YJOjABlZibGAjZCMMQJKcmvRE3IWJEIfQL-elPQLmeB5xy-Mkk1YaKbPERX4aBzhOUiXNMXgIhe6svtJNQlspTThowkETDpqHusXh5I_3X8LG9Hakx4-Tp1PYpGtt3v4M-vXy3Z7Dmvmoi2p50az2J5n9o_Y |
| openUrl | ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Cross+validation+in+LASSO+and+its+acceleration&rft.jtitle=Journal+of+statistical+mechanics&rft.au=Obuchi%2C+Tomoyuki&rft.au=Kabashima%2C+Yoshiyuki&rft.date=2016-05-31&rft.pub=IOP+Publishing+and+SISSA&rft.eissn=1742-5468&rft.volume=2016&rft.issue=5&rft_id=info:doi/10.1088%2F1742-5468%2F2016%2F05%2F053304&rft.externalDocID=jstataa1690 |