Deep Unlearning via Randomized Conditionally Independent Hessians
Recent legislation has led to interest in machine unlearning, i. e., removing specific training samples from a predictive model as if they never existed in the training dataset. Unlearning may also be required due to corrupted/adversarial data or simply a user's updated privacy requirement. For...
Gespeichert in:
| Veröffentlicht in: | Proceedings (IEEE Computer Society Conference on Computer Vision and Pattern Recognition. Online) Jg. 2022; S. 10412 - 10421 |
|---|---|
| Hauptverfasser: | , , , |
| Format: | Tagungsbericht Journal Article |
| Sprache: | Englisch |
| Veröffentlicht: |
United States
IEEE
01.06.2022
|
| Schlagworte: | |
| ISSN: | 1063-6919, 1063-6919 |
| Online-Zugang: | Volltext |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| Abstract | Recent legislation has led to interest in machine unlearning, i. e., removing specific training samples from a predictive model as if they never existed in the training dataset. Unlearning may also be required due to corrupted/adversarial data or simply a user's updated privacy requirement. For models which require no training (k-NN), simply deleting the closest original sample can be effective. But this idea is inapplicable to models which learn richer representations. Recent ideas leveraging optimization-based updates scale poorly with the model dimension d, due to inverting the Hessian of the loss function. We use a variant of a new conditional independence coefficient, L-CODEC, to identify a subset of the model parameters with the most semantic overlap on an individual sample level. Our approach completely avoids the need to invert a (possibly) huge matrix. By utilizing a Markov blanket selection, we premise that L-CODEC is also suitable for deep unlearning, as well as other applications in vision. Compared to alternatives, L-CODEC makes approximate unlearning possible in settings that would otherwise be infeasible, including vision models used for face recognition, person reidentification and NLP models that may require unlearning samples identified for exclusion. Code is available at https://github.com/vsingh-group/LCODEC-deep-unlearning |
|---|---|
| AbstractList | Recent legislation has led to interest in machine unlearning, i.e., removing specific training samples from a predictive model as if they never existed in the training dataset. Unlearning may also be required due to corrupted/adversarial data or simply a user’s updated privacy requirement. For models which require no training (k-NN), simply deleting the closest original sample can be effective. But this idea is inapplicable to models which learn richer representations. Recent ideas leveraging optimization-based updates scale poorly with the model dimension d, due to inverting the Hessian of the loss function. We use a variant of a new conditional independence coefficient, L-CODEC, to identify a subset of the model parameters with the most semantic overlap on an individual sample level. Our approach completely avoids the need to invert a (possibly) huge matrix. By utilizing a Markov blanket selection, we premise that L-CODEC is also suitable for deep unlearning, as well as other applications in vision. Compared to alternatives, L-CODEC makes approximate unlearning possible in settings that would otherwise be infeasible, including vision models used for face recognition, person re-identification and NLP models that may require unlearning samples identified for exclusion. Code is available at
https://github.com/vsingh-group/LCODEC-deep-unlearning Recent legislation has led to interest in machine unlearning, i. e., removing specific training samples from a predictive model as if they never existed in the training dataset. Unlearning may also be required due to corrupted/adversarial data or simply a user's updated privacy requirement. For models which require no training (k-NN), simply deleting the closest original sample can be effective. But this idea is inapplicable to models which learn richer representations. Recent ideas leveraging optimization-based updates scale poorly with the model dimension d, due to inverting the Hessian of the loss function. We use a variant of a new conditional independence coefficient, L-CODEC, to identify a subset of the model parameters with the most semantic overlap on an individual sample level. Our approach completely avoids the need to invert a (possibly) huge matrix. By utilizing a Markov blanket selection, we premise that L-CODEC is also suitable for deep unlearning, as well as other applications in vision. Compared to alternatives, L-CODEC makes approximate unlearning possible in settings that would otherwise be infeasible, including vision models used for face recognition, person reidentification and NLP models that may require unlearning samples identified for exclusion. Code is available at https://github.com/vsingh-group/LCODEC-deep-unlearning machine unlearning, predictive https://github.com/vsingh-group/LCODEC-deep-unlearning. Recent legislation has led to interest in machine unlearning, i.e., removing specific training samples from a predictive model as if they never existed in the training dataset. Unlearning may also be required due to corrupted/adversarial data or simply a user's updated privacy requirement. For models which require no training (k-NN), simply deleting the closest original sample can be effective. But this idea is inapplicable to models which learn richer representations. Recent ideas leveraging optimization-based updates scale poorly with the model dimension d, due to inverting the Hessian of the loss function. We use a variant of a new conditional independence coefficient, L-CODEC, to identify a subset of the model parameters with the most semantic overlap on an individual sample level. Our approach completely avoids the need to invert a (possibly) huge matrix. By utilizing a Markov blanket selection, we premise that L-CODEC is also suitable for deep unlearning, as well as other applications in vision. Compared to alternatives, L-CODEC makes approximate unlearning possible in settings that would otherwise be infeasible, including vision models used for face recognition, person re-identification and NLP models that may require unlearning samples identified for exclusion. Code is available at https://github.com/vsingh-group/LCODEC-deep-unlearning.Recent legislation has led to interest in machine unlearning, i.e., removing specific training samples from a predictive model as if they never existed in the training dataset. Unlearning may also be required due to corrupted/adversarial data or simply a user's updated privacy requirement. For models which require no training (k-NN), simply deleting the closest original sample can be effective. But this idea is inapplicable to models which learn richer representations. Recent ideas leveraging optimization-based updates scale poorly with the model dimension d, due to inverting the Hessian of the loss function. We use a variant of a new conditional independence coefficient, L-CODEC, to identify a subset of the model parameters with the most semantic overlap on an individual sample level. Our approach completely avoids the need to invert a (possibly) huge matrix. By utilizing a Markov blanket selection, we premise that L-CODEC is also suitable for deep unlearning, as well as other applications in vision. Compared to alternatives, L-CODEC makes approximate unlearning possible in settings that would otherwise be infeasible, including vision models used for face recognition, person re-identification and NLP models that may require unlearning samples identified for exclusion. Code is available at https://github.com/vsingh-group/LCODEC-deep-unlearning. |
| Author | Mehta, Ronak Ravi, Sathya N. Singh, Vikas Pal, Sourav |
| AuthorAffiliation | 2 University of Illinois at Chicago 1 University of Wisconsin-Madison |
| AuthorAffiliation_xml | – name: 2 University of Illinois at Chicago – name: 1 University of Wisconsin-Madison |
| Author_xml | – sequence: 1 givenname: Ronak surname: Mehta fullname: Mehta, Ronak email: ronakrm@cs.wisc.edu organization: University of Wisconsin-Madison – sequence: 2 givenname: Sourav surname: Pal fullname: Pal, Sourav email: spal9@wisc.edu organization: University of Wisconsin-Madison – sequence: 3 givenname: Vikas surname: Singh fullname: Singh, Vikas email: vsingh@biostat.wisc.edu organization: University of Wisconsin-Madison – sequence: 4 givenname: Sathya N. surname: Ravi fullname: Ravi, Sathya N. email: sathya@uic.edu organization: University of Illinois at Chicago |
| BackLink | https://www.ncbi.nlm.nih.gov/pubmed/37441105$$D View this record in MEDLINE/PubMed |
| BookMark | eNpVkFtLw0AQhVepaL38AkXy6Evqzu5mL08i9dKCoIj6GjbJpK6kuzHbCvXXG1FLfZkZOIfvHGafDHzwSMgJ0BEANefjl4fHjEmtR4wyNqJAQW2RfZAyE9IIybfJEKjkqTRgBhv3HjmK8Y1SyhmANHqX7HElRA_NhuTyCrFNnn2DtvPOz5IPZ5NH66swd59YJePgK7dwwdumWSVTX2GL_fCLZIIxOuvjIdmpbRPx6HcfkOeb66fxJL27v52OL-9SJ5hapBkYq7XA7xolmKxQphY1iEIbKlEAy0yhClA1FLwqqeG1LTXTpRAlUGoUPyAXP9x2WcyxKvsOnW3ytnNz263yYF3-X_HuNZ-Fjxwo50qB7glnv4QuvC8xLvK5iyU2jfUYljFnmmsmFJXQW083w9Ypf4_rDcc_BoeIa9loTblk_Ascv34h |
| CODEN | IEEPAD |
| ContentType | Conference Proceeding Journal Article |
| DBID | 6IE 6IH CBEJK RIE RIO NPM 7X8 5PM |
| DOI | 10.1109/CVPR52688.2022.01017 |
| DatabaseName | IEEE Electronic Library (IEL) Conference Proceedings IEEE Proceedings Order Plan (POP) 1998-present by volume IEEE Xplore All Conference Proceedings IEEE Electronic Library (IEL) IEEE Proceedings Order Plans (POP) 1998-present PubMed MEDLINE - Academic PubMed Central (Full Participant titles) |
| DatabaseTitle | PubMed MEDLINE - Academic |
| DatabaseTitleList | PubMed MEDLINE - Academic |
| Database_xml | – sequence: 1 dbid: NPM name: PubMed url: http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed sourceTypes: Index Database – sequence: 2 dbid: RIE name: IEEE/IET Electronic Library url: https://ieeexplore.ieee.org/ sourceTypes: Publisher – sequence: 3 dbid: 7X8 name: MEDLINE - Academic url: https://search.proquest.com/medline sourceTypes: Aggregation Database |
| DeliveryMethod | fulltext_linktorsrc |
| Discipline | Applied Sciences Law Computer Science |
| EISBN | 1665469463 9781665469463 |
| EISSN | 1063-6919 |
| EndPage | 10421 |
| ExternalDocumentID | PMC10337718 37441105 9880362 |
| Genre | orig-research Journal Article |
| GrantInformation_xml | – fundername: NIH grantid: RF1AG059312,RF1AG062336,RF1AG059869 funderid: 10.13039/100000002 – fundername: NSF grantid: CCF 1918211 funderid: 10.13039/100000001 – fundername: NIA NIH HHS grantid: RF1 AG059869 – fundername: NIA NIH HHS grantid: RF1 AG059312 – fundername: NIA NIH HHS grantid: RF1 AG062336 |
| GroupedDBID | 6IE 6IH 6IL 6IN AAWTH ABLEC ADZIZ ALMA_UNASSIGNED_HOLDINGS BEFXN BFFAM BGNUA BKEBE BPEOZ CBEJK CHZPO IEGSK IJVOP OCL RIE RIL RIO 23M 29F 29O 6IK ABDPE ACGFS IPLJI M43 NPM RIG RNS 7X8 5PM |
| ID | FETCH-LOGICAL-i427t-519a884e0032c195b79f4f14b8906e41259b7b17f1b3dc093fac828c44c100973 |
| IEDL.DBID | RIE |
| ISICitedReferencesCount | 47 |
| ISICitedReferencesURI | http://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=Summon&SrcAuth=ProQuest&DestLinkType=CitingArticles&DestApp=WOS_CPL&KeyUT=000870759103048&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| ISSN | 1063-6919 |
| IngestDate | Tue Sep 30 17:12:47 EDT 2025 Fri Sep 05 08:09:10 EDT 2025 Wed Feb 19 02:06:54 EST 2025 Wed Aug 27 02:15:09 EDT 2025 |
| IsDoiOpenAccess | false |
| IsOpenAccess | true |
| IsPeerReviewed | false |
| IsScholarly | true |
| Language | English |
| LinkModel | DirectLink |
| MergedId | FETCHMERGED-LOGICAL-i427t-519a884e0032c195b79f4f14b8906e41259b7b17f1b3dc093fac828c44c100973 |
| Notes | ObjectType-Article-1 SourceType-Scholarly Journals-1 ObjectType-Feature-2 content type line 23 Joint First Authors. |
| OpenAccessLink | https://pmc.ncbi.nlm.nih.gov/articles/PMC10337718/pdf/nihms-1894549.pdf |
| PMID | 37441105 |
| PQID | 2838247061 |
| PQPubID | 23479 |
| PageCount | 10 |
| ParticipantIDs | proquest_miscellaneous_2838247061 pubmed_primary_37441105 pubmedcentral_primary_oai_pubmedcentral_nih_gov_10337718 ieee_primary_9880362 |
| PublicationCentury | 2000 |
| PublicationDate | 20220601 |
| PublicationDateYYYYMMDD | 2022-06-01 |
| PublicationDate_xml | – month: 6 year: 2022 text: 20220601 day: 1 |
| PublicationDecade | 2020 |
| PublicationPlace | United States |
| PublicationPlace_xml | – name: United States |
| PublicationTitle | Proceedings (IEEE Computer Society Conference on Computer Vision and Pattern Recognition. Online) |
| PublicationTitleAbbrev | CVPR |
| PublicationTitleAlternate | Proc IEEE Comput Soc Conf Comput Vis Pattern Recognit |
| PublicationYear | 2022 |
| Publisher | IEEE |
| Publisher_xml | – name: IEEE |
| SSID | ssj0003211698 ssj0023720 |
| Score | 2.6613493 |
| Snippet | Recent legislation has led to interest in machine unlearning, i. e., removing specific training samples from a predictive model as if they never existed in the... machine unlearning, predictive https://github.com/vsingh-group/LCODEC-deep-unlearning. Recent legislation has led to interest in machine unlearning, i.e., removing specific training samples from a predictive model as if they never existed in the... |
| SourceID | pubmedcentral proquest pubmed ieee |
| SourceType | Open Access Repository Aggregation Database Index Database Publisher |
| StartPage | 10412 |
| SubjectTerms | accountability Computational modeling Face recognition fairness Law Legislation Predictive models privacy and ethics in vision; Machine learning; Optimization methods; Statistical methods Semantics Training Transparency |
| Title | Deep Unlearning via Randomized Conditionally Independent Hessians |
| URI | https://ieeexplore.ieee.org/document/9880362 https://www.ncbi.nlm.nih.gov/pubmed/37441105 https://www.proquest.com/docview/2838247061 https://pubmed.ncbi.nlm.nih.gov/PMC10337718 |
| Volume | 2022 |
| WOSCitedRecordID | wos000870759103048&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| hasFullText | 1 |
| inHoldings | 1 |
| isFullTextHit | |
| isPrint | |
| link | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV1LS8NAEB5s8eBJbavGR4ng0dhust3HUaqiIFLESm9ls9loQNPSl-ivdzZJo5VevAWyCcvsJPPN6xuAMx5rQpghnka079HIcE_5LPL8UDPOtG90lsF_vucPD2IwkL0NOC97YYwxWfGZubCXWS4_Gum5DZW1JCpb9sOtcM7zXq0ynhKgJ8OkKLrjSFu2us-9R0tmYgu4fP_CkqktZ6isg5N_qyJ_mZmb7f9tcAcaP_16bq-0RLuwYdIabBcA0y0-32kNKvfqow6XV8aM3X5aDIx4cReJch9VGo3eky9cjy-NkjxE-Pbp3pVzcmfura2ZRdvWgP7N9VP31ismKXgJ9fnMQ5imhKDGikkT2Qm5jGlMaChkmxmKIEeGPCQ8JmEQ6bYMYqXRFdOUapIR-uxBNR2l5gDcDqNMxCFHW6_QN4sV16zjxyHtSC0EUQ7UrVyG45wsY1iIxIHTpcSHqMA2K6FSM5pPh4hvhE854goH9vMTKB8OOKI1RIAOiJWzKRdYcuzVO2nympFkk3YQcDS8h-v3cwRbVivyoq9jqM4mc3MCm3oxS6aTJirYQDQzBfsGeADP3A |
| linkProvider | IEEE |
| linkToHtml | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV1LS8NAEB60CnryrfUZwaPR7mazj6NUpWItRVS8hc1mowFNxbaK_npnkxgfePEWyCYss5PMN69vAPZEagjhlvgG0b7PEit8TXni09hwwQ21psjg33RFrydvb1V_AvbrXhhrbVF8Zg_cZZHLTwZm7EJlhwqVrfjhToWMUVJ2a9URlQB9Ga5k1R9HWuqwfdO_dHQmroSL0gNHp_Y5ReUvQPm7LvKboTmd-98W52H5q2PP69e2aAEmbL4IcxXE9KoPeLgIk139ugRHx9Y-edd5NTLiznvJtHep82TwmL3jenxpkpVBwoc376yelDvyOq5qFq3bMlyfnly1O341S8HPGBUjH4GalpJZJyZDVBgLlbKUsFiqFrcMYY6KRUxESuIgMS0VpNqgM2YYM6Sg9FmBRj7I7Rp4IWdcprFAa6_RO0u1MDykacxCZaQkuglLTi7RU0mXEVUiacLup8QjVGGXl9C5HYyHESIcSZlAZNGE1fIE6ocDgXgNMWAT5I-zqRc4euyfd_LsvqDJJq0gEGh61__ezw7MdK4uulH3rHe-AbNOQ8oSsE1ojJ7HdgumzcsoGz5vF2r2AYYC0js |
| openUrl | ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=proceeding&rft.title=Proceedings+%28IEEE+Computer+Society+Conference+on+Computer+Vision+and+Pattern+Recognition.+Online%29&rft.atitle=Deep+Unlearning+via+Randomized+Conditionally+Independent+Hessians&rft.au=Mehta%2C+Ronak&rft.au=Pal%2C+Sourav&rft.au=Singh%2C+Vikas&rft.au=Ravi%2C+Sathya+N.&rft.date=2022-06-01&rft.pub=IEEE&rft.eissn=1063-6919&rft.spage=10412&rft.epage=10421&rft_id=info:doi/10.1109%2FCVPR52688.2022.01017&rft.externalDocID=9880362 |
| thumbnail_l | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/lc.gif&issn=1063-6919&client=summon |
| thumbnail_m | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/mc.gif&issn=1063-6919&client=summon |
| thumbnail_s | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/sc.gif&issn=1063-6919&client=summon |