P2SGrad: Refined Gradients for Optimizing Deep Face Models

Cosine-based softmax losses significantly improve the performance of deep face recognition networks. However, these losses always include sensitive hyper-parameters which can make training process unstable, and it is very tricky to set suitable hyper parameters for a specific dataset. This paper add...

Full description

Saved in:
Bibliographic Details
Published in:Proceedings (IEEE Computer Society Conference on Computer Vision and Pattern Recognition. Online) pp. 9898 - 9906
Main Authors: Zhang, Xiao, Zhao, Rui, Yan, Junjie, Gao, Mengya, Qiao, Yu, Wang, Xiaogang, Li, Hongsheng
Format: Conference Proceeding
Language:English
Published: IEEE 01.06.2019
Subjects:
and Body Pose
Benchmark testing
Biometrics ; Deep Learning ; Optimization Methods; Recognition: Detection
Categorization
Convergence
Environmentally friendly manufacturing techniques
Face
Face recognition
Gesture
Gradient methods
Neural networks
Noise
Optimization
Retrieval
Robustness
Training
ISSN:1063-6919
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Abstract Cosine-based softmax losses significantly improve the performance of deep face recognition networks. However, these losses always include sensitive hyper-parameters which can make training process unstable, and it is very tricky to set suitable hyper parameters for a specific dataset. This paper addresses this challenge by directly designing the gradients for training in an adaptive manner. We first investigate and unify previous cosine softmax losses from the perspective of gradients. This unified view inspires us to propose a novel gradient called P2SGrad (Probability-to-Similarity Gradient), which leverages a cosine similarity instead of classification probability to control the gradients for updating neural network parameters. P2SGrad is adaptive and hyper-parameter free, which makes training process more efficient and faster. We evaluate our P2SGrad on three face recognition benchmarks, LFW, MegaFace, and IJB-C. The results show that P2SGrad is stable in training, robust to noise, and achieves state-of-the-art performance on all the three benchmarks.
AbstractList Cosine-based softmax losses significantly improve the performance of deep face recognition networks. However, these losses always include sensitive hyper-parameters which can make training process unstable, and it is very tricky to set suitable hyper parameters for a specific dataset. This paper addresses this challenge by directly designing the gradients for training in an adaptive manner. We first investigate and unify previous cosine softmax losses from the perspective of gradients. This unified view inspires us to propose a novel gradient called P2SGrad (Probability-to-Similarity Gradient), which leverages a cosine similarity instead of classification probability to control the gradients for updating neural network parameters. P2SGrad is adaptive and hyper-parameter free, which makes training process more efficient and faster. We evaluate our P2SGrad on three face recognition benchmarks, LFW, MegaFace, and IJB-C. The results show that P2SGrad is stable in training, robust to noise, and achieves state-of-the-art performance on all the three benchmarks.
Author Zhang, Xiao
Gao, Mengya
Qiao, Yu
Wang, Xiaogang
Zhao, Rui
Yan, Junjie
Li, Hongsheng
Author_xml – sequence: 1
  givenname: Xiao
  surname: Zhang
  fullname: Zhang, Xiao
  organization: Chinese Univ. of Hong Kong
– sequence: 2
  givenname: Rui
  surname: Zhao
  fullname: Zhao, Rui
  organization: SenseTime Group Limited
– sequence: 3
  givenname: Junjie
  surname: Yan
  fullname: Yan, Junjie
  organization: Sensetime Group Limited
– sequence: 4
  givenname: Mengya
  surname: Gao
  fullname: Gao, Mengya
  organization: Tianjin Univ
– sequence: 5
  givenname: Yu
  surname: Qiao
  fullname: Qiao, Yu
  organization: Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences
– sequence: 6
  givenname: Xiaogang
  surname: Wang
  fullname: Wang, Xiaogang
  organization: Chinese Univ. of Hong Kong
– sequence: 7
  givenname: Hongsheng
  surname: Li
  fullname: Li, Hongsheng
  organization: Chinese Univ. of Hong Kong
BookMark eNotzL1OwzAUQGGDQKKUzAwsfoGEe-0ktruhQAtSUavys1aOfY2MWidKssDTIwTT0becS3aWukSMXSMUiGBum_ftrhCApgAELE9YZpRGJTRKYaQ-ZTOEWua1QXPBsnH8BAApEGujZ2yxFS-rwfoF31GIiTz_VaQ0jTx0A9_0UzzG75g--D1Rz5fWEX_uPB3GK3Ye7GGk7L9z9rZ8eG0e8_Vm9dTcrfMoQE65EU6AC6AVhrZC5RSVOtRetAItGFdpLzVp4SoKUFrfmjYgKZC-llAqLefs5u8biWjfD_Foh6-9NpVUiPIHEXhH9Q
ContentType Conference Proceeding
DBID 6IE
6IH
CBEJK
RIE
RIO
DOI 10.1109/CVPR.2019.01014
DatabaseName IEEE Electronic Library (IEL) Conference Proceedings
IEEE Proceedings Order Plan (POP) 1998-present by volume
IEEE Xplore All Conference Proceedings
IEEE Electronic Library (IEL)
IEEE Proceedings Order Plans (POP) 1998-present
DatabaseTitleList
Database_xml – sequence: 1
  dbid: RIE
  name: IEEE Electronic Library (IEL)
  url: https://ieeexplore.ieee.org/
  sourceTypes: Publisher
DeliveryMethod fulltext_linktorsrc
Discipline Applied Sciences
EISBN 9781728132938
1728132932
EISSN 1063-6919
EndPage 9906
ExternalDocumentID 8953711
Genre orig-research
GroupedDBID 6IE
6IH
6IL
6IN
AAWTH
ABLEC
ADZIZ
ALMA_UNASSIGNED_HOLDINGS
BEFXN
BFFAM
BGNUA
BKEBE
BPEOZ
CBEJK
CHZPO
IEGSK
IJVOP
OCL
RIE
RIL
RIO
ID FETCH-LOGICAL-i203t-92c20cf0871fb517c7e48f6d2b21a09c58d38e82c5ef04adb9bf1e703d6304783
IEDL.DBID RIE
ISICitedReferencesCount 22
ISICitedReferencesURI http://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=Summon&SrcAuth=ProQuest&DestLinkType=CitingArticles&DestApp=WOS_CPL&KeyUT=000542649303053&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D
IngestDate Wed Aug 27 07:44:55 EDT 2025
IsPeerReviewed false
IsScholarly true
Language English
LinkModel DirectLink
MergedId FETCHMERGED-LOGICAL-i203t-92c20cf0871fb517c7e48f6d2b21a09c58d38e82c5ef04adb9bf1e703d6304783
PageCount 9
ParticipantIDs ieee_primary_8953711
PublicationCentury 2000
PublicationDate 2019-June
PublicationDateYYYYMMDD 2019-06-01
PublicationDate_xml – month: 06
  year: 2019
  text: 2019-June
PublicationDecade 2010
PublicationTitle Proceedings (IEEE Computer Society Conference on Computer Vision and Pattern Recognition. Online)
PublicationTitleAbbrev CVPR
PublicationYear 2019
Publisher IEEE
Publisher_xml – name: IEEE
SSID ssj0003211698
Score 2.3102152
Snippet Cosine-based softmax losses significantly improve the performance of deep face recognition networks. However, these losses always include sensitive...
SourceID ieee
SourceType Publisher
StartPage 9898
SubjectTerms and Body Pose
Benchmark testing
Biometrics ; Deep Learning ; Optimization Methods; Recognition: Detection
Categorization
Convergence
Environmentally friendly manufacturing techniques
Face
Face recognition
Gesture
Gradient methods
Neural networks
Noise
Optimization
Retrieval
Robustness
Training
Title P2SGrad: Refined Gradients for Optimizing Deep Face Models
URI https://ieeexplore.ieee.org/document/8953711
WOSCitedRecordID wos000542649303053&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D
hasFullText 1
inHoldings 1
isFullTextHit
isPrint
link http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV1LTwIxEG6QePDkA4zv9ODRhW7b7YMrih4MbvARbmTbThMSBcKCB3-97bLBmHjx1vbQJtO0883jm0Ho2gWTy3jOEhYAfsKFz5KCaJIY7yUX0fHvqiKuj3I4VOOxzhvoZsuFAYAq-Qw6cVjF8t3crqOrrKt0xmQk8u5IKTZcra0_hQVLRmhVV-9Jie723_JRzN3Snaoh7a_2KZX2GOz_79wD1P6h4eF8q2AOUQNmR2i_xo24fpVlC_Vy-ny_LFwPj8AH1OhwnEWiY4kDJsVP4Vv4mH6FPfAtwAIPirBxbIL2XrbR6-Dupf-Q1D0RkiklbJVoaimxngQ7x5sslVYCV144amgahGwz5ZgCRW0GnvDCmXATKYRn7UQMsCl2jJqz-QxOECaWAZfMMKsZDxayUdZopgLcLkTGjT9FrSiKyWJT9mJSS-Hs7-VztBdlvcmiukDN1XINl2jXfq6m5fKquqtvXkeTZA
linkProvider IEEE
linkToHtml http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV1LTwIxEG6ImugJFYxve_DoQrft7rZcUcSIuEE03Mi2nSYkCoQFD_5622WDMfHire2hTaZp55vHN4PQtXEml7KcBcwB_IDHNgoyIkmgrE147B3_piji2kv6fTEaybSCbjZcGAAoks-g4YdFLN_M9Mq7yppCRizxRN7tiHNK1mytjUeFOVsmlqKs3xMS2Wy_pQOfvSUbRUvaXw1UCv3Rqf7v5H1U_yHi4XSjYg5QBaaHqFoiR1y-y7yGWil9uV9kpoUHYB1uNNjPPNUxxw6V4mf3MXxMvtwe-BZgjjuZ29i3QXvP6-i1czdsd4OyK0IwoYQtA0k1JdoSZ-lYFYWJToALGxuqaOjErCNhmABBdQSW8MwodxchuIdtYh9iE-wIbU1nUzhGmGgGPGGKacm4s5GV0Eoy4QB3Fkdc2RNU86IYz9eFL8alFE7_Xr5Cu93hU2_ce-g_nqE9L_d1TtU52louVnCBdvTncpIvLot7-wY7zZar
openUrl ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Abook&rft.genre=proceeding&rft.title=Proceedings+%28IEEE+Computer+Society+Conference+on+Computer+Vision+and+Pattern+Recognition.+Online%29&rft.atitle=P2SGrad%3A+Refined+Gradients+for+Optimizing+Deep+Face+Models&rft.au=Zhang%2C+Xiao&rft.au=Zhao%2C+Rui&rft.au=Yan%2C+Junjie&rft.au=Gao%2C+Mengya&rft.date=2019-06-01&rft.pub=IEEE&rft.eissn=1063-6919&rft.spage=9898&rft.epage=9906&rft_id=info:doi/10.1109%2FCVPR.2019.01014&rft.externalDocID=8953711