Distribution Shift Inversion for Out-of-Distribution Prediction
Machine learning society has witnessed the emergence of a myriad of Out-of-Distribution (OoD) algorithms, which address the distribution shift between the training and the testing distribution by searching for a unified predictor or invariant feature representation. However, the task of directly mit...
Saved in:
| Published in: | Proceedings (IEEE Computer Society Conference on Computer Vision and Pattern Recognition. Online) pp. 3592 - 3602 |
|---|---|
| Main Authors: | , , , |
| Format: | Conference Proceeding |
| Language: | English |
| Published: |
IEEE
01.06.2023
|
| Subjects: | |
| ISSN: | 1063-6919 |
| Online Access: | Get full text |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| Abstract | Machine learning society has witnessed the emergence of a myriad of Out-of-Distribution (OoD) algorithms, which address the distribution shift between the training and the testing distribution by searching for a unified predictor or invariant feature representation. However, the task of directly mitigating the distribution shift in the unseen testing set is rarely investigated, due to the unavailability of the testing distribution during the training phase and thus the impossibility of training a distribution translator mapping between the training and testing distribution. In this paper, we explore how to bypass the requirement of testing distribution for distribution translator training and make the distribution translation useful for OoD prediction. We propose a portable Distribution Shift Inversion (DSI) algorithm, in which, before being fed into the prediction model, the OoD testing samples are first linearly combined with additional Gaussian noise and then transferred back towards the training distribution using a diffusion model trained only on the source distribution. Theoretical analysis reveals the feasibility of our method. Experimental results, on both multiple-domain generalization datasets and single-domain generalization datasets, show that our method provides a general performance gain when plugged into a wide range of commonly used OoD algorithms. Our code is available at https://github.com/yu-rp/Distribution-Shift-Iverson. |
|---|---|
| AbstractList | Machine learning society has witnessed the emergence of a myriad of Out-of-Distribution (OoD) algorithms, which address the distribution shift between the training and the testing distribution by searching for a unified predictor or invariant feature representation. However, the task of directly mitigating the distribution shift in the unseen testing set is rarely investigated, due to the unavailability of the testing distribution during the training phase and thus the impossibility of training a distribution translator mapping between the training and testing distribution. In this paper, we explore how to bypass the requirement of testing distribution for distribution translator training and make the distribution translation useful for OoD prediction. We propose a portable Distribution Shift Inversion (DSI) algorithm, in which, before being fed into the prediction model, the OoD testing samples are first linearly combined with additional Gaussian noise and then transferred back towards the training distribution using a diffusion model trained only on the source distribution. Theoretical analysis reveals the feasibility of our method. Experimental results, on both multiple-domain generalization datasets and single-domain generalization datasets, show that our method provides a general performance gain when plugged into a wide range of commonly used OoD algorithms. Our code is available at https://github.com/yu-rp/Distribution-Shift-Iverson. |
| Author | Liu, Songhua Yu, Runpeng Yang, Xingyi Wang, Xinchao |
| Author_xml | – sequence: 1 givenname: Runpeng surname: Yu fullname: Yu, Runpeng email: r.yu@u.nus.edu organization: National University of Singapore – sequence: 2 givenname: Songhua surname: Liu fullname: Liu, Songhua email: songhua.liu@u.nus.edu organization: National University of Singapore – sequence: 3 givenname: Xingyi surname: Yang fullname: Yang, Xingyi email: xyang@u.nus.edu organization: National University of Singapore – sequence: 4 givenname: Xinchao surname: Wang fullname: Wang, Xinchao email: xinchao@nus.edu.sg organization: National University of Singapore |
| BookMark | eNpVj91Kw0AUhFdRsNa8QS_yAhvP2ZNsslci0Wqh0OLfbcluzuKKJrJJBd_eFL3xambgY5g5Fydd37EQC4QMEcxl_bJ9KFSpTKZAUQZABRyJxJSmmhwBKlMdixmCJqkNmjORDMMbTJxC1KaaiaubMIwx2P0Y-i59fA1-TFfdF8fhkH0f081-lL2X_7ht5Da4g70Qp755Hzj507l4Xt4-1fdyvblb1ddrGRTko3TOAkKbtxpNA2xL64rG2QqQi0p7dJoKbpWzuSNFmoCniUpPr7xm2xqai8Vvb2Dm3WcMH0383iEoIINEP3HiTOU |
| CODEN | IEEPAD |
| ContentType | Conference Proceeding |
| DBID | 6IE 6IH CBEJK RIE RIO |
| DOI | 10.1109/CVPR52729.2023.00350 |
| DatabaseName | IEEE Electronic Library (IEL) Conference Proceedings IEEE Proceedings Order Plan (POP) 1998-present by volume IEEE Xplore All Conference Proceedings IEEE/IET Electronic Library IEEE Proceedings Order Plans (POP) 1998-present |
| DatabaseTitleList | |
| Database_xml | – sequence: 1 dbid: RIE name: IEEE/IET Electronic Library url: https://ieeexplore.ieee.org/ sourceTypes: Publisher |
| DeliveryMethod | fulltext_linktorsrc |
| Discipline | Applied Sciences |
| EISBN | 9798350301298 |
| EISSN | 1063-6919 |
| EndPage | 3602 |
| ExternalDocumentID | 10203913 |
| Genre | orig-research |
| GroupedDBID | 6IE 6IH 6IL 6IN AAWTH ABLEC ADZIZ ALMA_UNASSIGNED_HOLDINGS BEFXN BFFAM BGNUA BKEBE BPEOZ CBEJK CHZPO IEGSK IJVOP OCL RIE RIL RIO |
| ID | FETCH-LOGICAL-i204t-ccb010d4d619a0eb7bc5acb801e586f1c635ed2cb4c323630e03226202f6ebd93 |
| IEDL.DBID | RIE |
| ISICitedReferencesCount | 6 |
| ISICitedReferencesURI | http://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=Summon&SrcAuth=ProQuest&DestLinkType=CitingArticles&DestApp=WOS_CPL&KeyUT=001058542603085&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| IngestDate | Wed Aug 27 02:56:29 EDT 2025 |
| IsPeerReviewed | false |
| IsScholarly | true |
| Language | English |
| LinkModel | DirectLink |
| MergedId | FETCHMERGED-LOGICAL-i204t-ccb010d4d619a0eb7bc5acb801e586f1c635ed2cb4c323630e03226202f6ebd93 |
| PageCount | 11 |
| ParticipantIDs | ieee_primary_10203913 |
| PublicationCentury | 2000 |
| PublicationDate | 2023-June |
| PublicationDateYYYYMMDD | 2023-06-01 |
| PublicationDate_xml | – month: 06 year: 2023 text: 2023-June |
| PublicationDecade | 2020 |
| PublicationTitle | Proceedings (IEEE Computer Society Conference on Computer Vision and Pattern Recognition. Online) |
| PublicationTitleAbbrev | CVPR |
| PublicationYear | 2023 |
| Publisher | IEEE |
| Publisher_xml | – name: IEEE |
| SSID | ssj0003211698 |
| Score | 2.3811908 |
| Snippet | Machine learning society has witnessed the emergence of a myriad of Out-of-Distribution (OoD) algorithms, which address the distribution shift between the... |
| SourceID | ieee |
| SourceType | Publisher |
| StartPage | 3592 |
| SubjectTerms | continual Gaussian noise low-shot Machine learning Machine learning algorithms meta or long-tail learning Performance gain Prediction algorithms Predictive models Training Transfer |
| Title | Distribution Shift Inversion for Out-of-Distribution Prediction |
| URI | https://ieeexplore.ieee.org/document/10203913 |
| WOSCitedRecordID | wos001058542603085&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| hasFullText | 1 |
| inHoldings | 1 |
| isFullTextHit | |
| isPrint | |
| link | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV25TgMxEB1BREEVjiBuuaB18Np7eCuKQEQVVlxKF63tsUiToGTD9zN2lqAUFGxlrSxZGsuaN8d7A3AjbJo6JxUnrICcPL7kdeh1TbQXiS6lV9rHYRPFaKTH47JqyeqRC4OIsfkM-2EZa_lublchVUYvXAZBc7ULu0WRr8lam4SKolAmL3VLj0tEeTt4r54zSeixH2aE92MRbWuISvQhw-4_Tz-A3i8bj1UbP3MIOzg7gm4LH1n7OJfHcHcfRHDb-VXs5WPqGxZkNGJCjBE4ZU-rhs8939pXLUKpJix78DZ8eB088nY-Ap9KkTbcWkPRlEsdBUG1QFMYm9XWkM_BTOc-sQQm0ElrUqukypVAMlIQoJc-R-NKdQKd2XyGp8DQZfTVxtcEL6QWxgZglRYSramlUmfQCwaZfK4lMCY_tjj_4_8F7Aebr3uqLqHTLFZ4BXv2q5kuF9fx4r4BnC6Y3g |
| linkProvider | IEEE |
| linkToHtml | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV1LTwIxEJ4omugJHxjf7sFrsdt2l-7JA0owIm4UDTeybaeRCxhY_P22ZcVw8OCemk2TJtM0883j-wbgmmohjGGcOKyAxHl8Rgrf6xpLS2OZMculDcMmWv2-HA6zvCKrBy4MIobmM2z6Zajlm6le-FSZe-HMC5rzTdhKhGB0SddapVS4C2bSTFYEuZhmN-33_CVhDj82_ZTwZiijrY1RCV6kU__n-XvQ-OXjRfnK0-zDBk4OoF4ByKh6nvNDuL3zMrjVBKvo9WNsy8gLaYSUWOTgafS8KMnUkrV9-cwXa_yyAW-d-0G7S6oJCWTMqCiJ1srFU0YYFwYVFFVL6aTQynkdTGRqY-3gBBqmldCc8ZRTdEbyEvTMpqhMxo-gNplO8BgiNIn7CmULBzCYpEp7aCVaDLUqGOcn0PAGGX0uRTBGP7Y4_eP_Fex0B0-9Ue-h_3gGu97-yw6rc6iVswVewLb-Ksfz2WW4xG_AWZwl |
| openUrl | ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Abook&rft.genre=proceeding&rft.title=Proceedings+%28IEEE+Computer+Society+Conference+on+Computer+Vision+and+Pattern+Recognition.+Online%29&rft.atitle=Distribution+Shift+Inversion+for+Out-of-Distribution+Prediction&rft.au=Yu%2C+Runpeng&rft.au=Liu%2C+Songhua&rft.au=Yang%2C+Xingyi&rft.au=Wang%2C+Xinchao&rft.date=2023-06-01&rft.pub=IEEE&rft.eissn=1063-6919&rft.spage=3592&rft.epage=3602&rft_id=info:doi/10.1109%2FCVPR52729.2023.00350&rft.externalDocID=10203913 |