Neural collapse under cross-entropy loss

We consider the variational problem of cross-entropy loss with n feature vectors on a unit hypersphere in Rd. We prove that when d≥n−1, the global minimum is given by the simplex equiangular tight frame, which justifies the neural collapse behavior. We also prove that, as n→∞ with fixed d, the minim...

Celý popis

Podrobná bibliografia

Vydané v:	Applied and computational harmonic analysis Ročník 59; s. 224 - 241
Hlavní autori:	Lu, Jianfeng, Steinerberger, Stefan
Médium:	Journal Article
Jazyk:	English
Vydavateľské údaje:	Elsevier Inc 01.07.2022
Predmet:	Cross-entropy loss Frame potential Neural collapse Neural networks Neural collapse Frame potential Cross-entropy loss Neural networks
ISSN:	1063-5203
On-line prístup:	Získať plný text
Tagy:	Pridať tag Žiadne tagy, Buďte prvý, kto otaguje tento záznam!