Transfer Knowledge from Head to Tail: Uncertainty Calibration under Long-tailed Distribution
How to estimate the uncertainty of a given model is a crucial problem. Current calibration techniques treat different classes equally and thus implicitly assume that the distribution of training data is balanced, but ignore the fact that real-world data often follows a long-tailed distribution. In t...
Saved in:
| Published in: | Proceedings (IEEE Computer Society Conference on Computer Vision and Pattern Recognition. Online) pp. 19978 - 19987 |
|---|---|
| Main Authors: | , |
| Format: | Conference Proceeding |
| Language: | English |
| Published: |
IEEE
01.06.2023
|
| Subjects: | |
| ISSN: | 1063-6919 |
| Online Access: | Get full text |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| Abstract | How to estimate the uncertainty of a given model is a crucial problem. Current calibration techniques treat different classes equally and thus implicitly assume that the distribution of training data is balanced, but ignore the fact that real-world data often follows a long-tailed distribution. In this paper, we explore the problem of calibrating the model trained from a long-tailed distribution. Due to the difference between the imbalanced training distribution and balanced test distribution, existing calibration methods such as temperature scaling can not generalize well to this problem. Specific calibration methods for domain adaptation are also not applicable because they rely on unlabeled target domain instances which are not available. Models trained from a long-tailed distribution tend to be more over-confident to head classes. To this end, we propose a novel knowledge-transferring-based calibration method by estimating the importance weights for samples of tail classes to realize long-tailed calibration. Our method models the distribution of each class as a Gaussian distribution and views the source statistics of head classes as a prior to calibrate the target distributions of tail classes. We adaptively transfer knowledge from head classes to get the target probability density of tail classes. The importance weight is estimated by the ratio of the target probability density over the source probability density. Extensive experiments on CIFAR-10-LT, MNIST-LT, CIFAR-100-LT, and ImageNet-LT datasets demonstrate the effectiveness of our method. |
|---|---|
| AbstractList | How to estimate the uncertainty of a given model is a crucial problem. Current calibration techniques treat different classes equally and thus implicitly assume that the distribution of training data is balanced, but ignore the fact that real-world data often follows a long-tailed distribution. In this paper, we explore the problem of calibrating the model trained from a long-tailed distribution. Due to the difference between the imbalanced training distribution and balanced test distribution, existing calibration methods such as temperature scaling can not generalize well to this problem. Specific calibration methods for domain adaptation are also not applicable because they rely on unlabeled target domain instances which are not available. Models trained from a long-tailed distribution tend to be more over-confident to head classes. To this end, we propose a novel knowledge-transferring-based calibration method by estimating the importance weights for samples of tail classes to realize long-tailed calibration. Our method models the distribution of each class as a Gaussian distribution and views the source statistics of head classes as a prior to calibrate the target distributions of tail classes. We adaptively transfer knowledge from head classes to get the target probability density of tail classes. The importance weight is estimated by the ratio of the target probability density over the source probability density. Extensive experiments on CIFAR-10-LT, MNIST-LT, CIFAR-100-LT, and ImageNet-LT datasets demonstrate the effectiveness of our method. |
| Author | Su, Bing Chen, Jiahao |
| Author_xml | – sequence: 1 givenname: Jiahao surname: Chen fullname: Chen, Jiahao email: nicelemon666@gmail.com organization: Gaoling School of Artificial Intelligence, Renmin University of China – sequence: 2 givenname: Bing surname: Su fullname: Su, Bing email: subingats@gmail.com organization: Gaoling School of Artificial Intelligence, Renmin University of China |
| BookMark | eNotjNFKwzAUQKMoOOf-YA_5gc7cZE1zfZPqnDhQZPNJGDdNOiJdKmmH7O_t0KfzcDjnml3ENnrGpiBmAAJvy4-391wWEmdSSDUTgKDO2AQLNCoXSoBEc85GILTKNAJesUnXfQkhlATQaEbsc50odrVP_CW2P413O8_r1O750pPjfcvXFJo7vomVTz2F2B95SU2wifrQRn6IbkhXbdxlgx1y_hC6PgV7OOkbdllT0_nJP8dss3hcl8ts9fr0XN6vsiDFvM9sRUo7r2xeKARSdSWdk2puKoNUGC3nJ29ljSgdAFW1yjWQ1YRoDVg1ZtO_b_Deb79T2FM6bkEMdzBC_QLfKldQ |
| CODEN | IEEPAD |
| ContentType | Conference Proceeding |
| DBID | 6IE 6IH CBEJK RIE RIO |
| DOI | 10.1109/CVPR52729.2023.01913 |
| DatabaseName | IEEE Electronic Library (IEL) Conference Proceedings IEEE Proceedings Order Plan (POP) 1998-present by volume IEEE Xplore All Conference Proceedings IEEE Electronic Library (IEL) IEEE Proceedings Order Plans (POP) 1998-present |
| DatabaseTitleList | |
| Database_xml | – sequence: 1 dbid: RIE name: IEEE/IET Electronic Library (IEL) (UW System Shared) url: https://ieeexplore.ieee.org/ sourceTypes: Publisher |
| DeliveryMethod | fulltext_linktorsrc |
| Discipline | Applied Sciences |
| EISBN | 9798350301298 |
| EISSN | 1063-6919 |
| EndPage | 19987 |
| ExternalDocumentID | 10204180 |
| Genre | orig-research |
| GrantInformation_xml | – fundername: Research Funds of Renmin University of China grantid: 21XNLG05 funderid: 10.13039/501100004260 – fundername: National Natural Science Foundation of China grantid: 61976206,61832017 funderid: 10.13039/501100001809 – fundername: Fundamental Research Funds for the Central Universities funderid: 10.13039/501100012226 |
| GroupedDBID | 6IE 6IH 6IL 6IN AAWTH ABLEC ADZIZ ALMA_UNASSIGNED_HOLDINGS BEFXN BFFAM BGNUA BKEBE BPEOZ CBEJK CHZPO IEGSK IJVOP OCL RIE RIL RIO |
| ID | FETCH-LOGICAL-i204t-bca36de3b57391a3fc2dd2348c89a78624a36db2f992d11acf3561ab6a99b81b3 |
| IEDL.DBID | RIE |
| ISICitedReferencesCount | 28 |
| ISICitedReferencesURI | http://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=Summon&SrcAuth=ProQuest&DestLinkType=CitingArticles&DestApp=WOS_CPL&KeyUT=001062531304029&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| IngestDate | Wed Aug 27 02:56:33 EDT 2025 |
| IsPeerReviewed | false |
| IsScholarly | true |
| Language | English |
| LinkModel | DirectLink |
| MergedId | FETCHMERGED-LOGICAL-i204t-bca36de3b57391a3fc2dd2348c89a78624a36db2f992d11acf3561ab6a99b81b3 |
| PageCount | 10 |
| ParticipantIDs | ieee_primary_10204180 |
| PublicationCentury | 2000 |
| PublicationDate | 2023-June |
| PublicationDateYYYYMMDD | 2023-06-01 |
| PublicationDate_xml | – month: 06 year: 2023 text: 2023-June |
| PublicationDecade | 2020 |
| PublicationTitle | Proceedings (IEEE Computer Society Conference on Computer Vision and Pattern Recognition. Online) |
| PublicationTitleAbbrev | CVPR |
| PublicationYear | 2023 |
| Publisher | IEEE |
| Publisher_xml | – name: IEEE |
| SSID | ssj0003211698 |
| Score | 2.388829 |
| Snippet | How to estimate the uncertainty of a given model is a crucial problem. Current calibration techniques treat different classes equally and thus implicitly... |
| SourceID | ieee |
| SourceType | Publisher |
| StartPage | 19978 |
| SubjectTerms | Adaptation models continual Head low-shot meta or long-tail learning Tail Temperature distribution Training Training data Transfer Uncertainty |
| Title | Transfer Knowledge from Head to Tail: Uncertainty Calibration under Long-tailed Distribution |
| URI | https://ieeexplore.ieee.org/document/10204180 |
| WOSCitedRecordID | wos001062531304029&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| hasFullText | 1 |
| inHoldings | 1 |
| isFullTextHit | |
| isPrint | |
| link | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV3LSsNAFB20uHBVHxXfzMLt1GQmycy4rZaCpRRpSxdCmVekIIm0qeDfO3eSVjcu3IW8mZvhnrk55x6E7mjkUbiMHEnyVBJo6E20zhVxVgrloO4f2O6zIR-NxHwux41YPWhhnHOBfOa6sBn-5dvSbKBU5mc4jZJY-BX6PudZLdbaFVSYX8pkUjTyuDiS973Z-CWlHj12wSO868EMmBj8MlEJOaTf_ufTj1DnR42Hx7s8c4z2XHGC2g18xM3kXJ-i15B3_BX4eVsow6AewQMfR1yVeKKW7w946s8ONIDqC4MyS9ffAAY12QoPy-KNAKvU3_oRmuo2flgdNO0_TXoD0pgnkKV_zYpoo1hmHdMpZzJWLDfUWsoSYYRUHGQhcFzTXEpq41iZnHkopXSmpNQey7Iz1CrKwp0jrKjQNou0UsYmueLSOu5Tu8kkS7mOkwvUgdFafNT9MRbbgbr8Y_8VOoSA1ISra9SqVht3gw7MZ7Vcr25DVL8Bl2KkCw |
| linkProvider | IEEE |
| linkToHtml | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV3LSsNAFB2kCrqqj4pvZ-F2ajKT17itlkpjKdKWLoQyr0ihJNKmgn_v3Ela3bhwF_JmboZ75uacexC6o55F4dwzJMhCTqChN5EyE8RonggDdX_Hdp-k8WCQTKd8WIvVnRbGGOPIZ6YNm-5fvi7UGkpldoZTL_ATu0LfDYOAepVca1tSYXYxE_GkFsj5Hr_vTIavIbX4sQ0u4W0LZ8DG4JeNissi3eY_n3-IWj96PDzcZpojtGPyY9SsASSup-fqBL25zGOvwP1NqQyDfgT3bCRxWeCRmC8e8Nie7YgA5RcGbZasvgIMerIlTov8nQCv1N76Edrq1o5YLTTuPo06PVLbJ5C5fc2SSCVYpA2TYcy4L1imqNaUBYlKuIhBGALHJc04p9r3hcqYBVNCRoJzadEsO0WNvMjNGcKCJlJHnhRC6SATMdcmtsldRZyFsfSDc9SC0Zp9VB0yZpuBuvhj_y3a741e0ln6POhfogMITkW_ukKNcrk212hPfZbz1fLGRfgbTQOnUg |
| openUrl | ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Abook&rft.genre=proceeding&rft.title=Proceedings+%28IEEE+Computer+Society+Conference+on+Computer+Vision+and+Pattern+Recognition.+Online%29&rft.atitle=Transfer+Knowledge+from+Head+to+Tail%3A+Uncertainty+Calibration+under+Long-tailed+Distribution&rft.au=Chen%2C+Jiahao&rft.au=Su%2C+Bing&rft.date=2023-06-01&rft.pub=IEEE&rft.eissn=1063-6919&rft.spage=19978&rft.epage=19987&rft_id=info:doi/10.1109%2FCVPR52729.2023.01913&rft.externalDocID=10204180 |