Learning label smoothing for text classification

Training with soft labels instead of hard labels can effectively improve the robustness and generalization of deep learning models. Label smoothing often provides uniformly distributed soft labels during the training process, whereas it does not take the semantic difference of labels into account. T...

Celý popis

Uloženo v:
Podrobná bibliografie
Vydáno v:PeerJ. Computer science Ročník 10; s. e2005
Hlavní autoři: Ren, Han, Zhao, Yajie, Zhang, Yong, Sun, Wei
Médium: Journal Article
Jazyk:angličtina
Vydáno: United States PeerJ. Ltd 23.04.2024
PeerJ Inc
Témata:
Algorithms and Analysis of Algorithms
Analysis
Computational Linguistics
Data Mining and Machine Learning
Excessive regularization
Label smoothing
Language processing
Natural language interfaces
Neural network
Neural Networks
Soft label
Text classification
Text Mining
New York
Neural network
Label smoothing
Excessive regularization
Soft label
Text classification
ISSN:2376-5992, 2376-5992
On-line přístup:Získat plný text
Tagy: Přidat tag
Žádné tagy, Buďte první, kdo vytvoří štítek k tomuto záznamu!
Popis
Shrnutí:Training with soft labels instead of hard labels can effectively improve the robustness and generalization of deep learning models. Label smoothing often provides uniformly distributed soft labels during the training process, whereas it does not take the semantic difference of labels into account. This article introduces discrimination-aware label smoothing, an adaptive label smoothing approach that learns appropriate distributions of labels for iterative optimization objectives. In this approach, positive and negative samples are employed to provide experience from both sides, and the performances of regularization and model calibration are improved through an iterative learning method. Experiments on five text classification datasets demonstrate the effectiveness of the proposed method.
Bibliografie:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 23
ISSN:2376-5992
2376-5992
DOI:10.7717/peerj-cs.2005