Decoupled Knowledge Distillation

State-of-the-art distillation methods are mainly based on distilling deep features from intermediate layers, while the significance of logit distillation is greatly overlooked. To provide a novel viewpoint to study logit distillation, we re-formulate the classical KD loss into two parts, i.e., targe...

Ausführliche Beschreibung

Bibliographische Detailangaben

Veröffentlicht in:	Proceedings (IEEE Computer Society Conference on Computer Vision and Pattern Recognition. Online) S. 11943 - 11952
Hauptverfasser:	Zhao, Borui, Cui, Quan, Song, Renjie, Qiu, Yiyu, Liang, Jiajun
Format:	Tagungsbericht
Sprache:	Englisch
Veröffentlicht:	IEEE 01.06.2022
Schlagworte:	categorization Codes Computer architecture Computer vision Deep learning Deep learning architectures and techniques; Efficient learning and inferences; Recognition: detection Feature extraction Object detection retrieval Training
ISSN:	1063-6919
Online-Zugang:	Volltext
Tags:	Tag hinzufügen Keine Tags, Fügen Sie den ersten Tag hinzu!