GaitSCM: Causal representation learning for gait recognition

Gait recognition is a promising biometric technology that aims to identify the target subject via walking pattern. Most existing appearance-based methods focus on learning discriminative spatio-temporal representations from gait silhouettes. However, these methods pay less attention to probing the c...

Celý popis

Uloženo v:

Podrobná bibliografie
Vydáno v:	Computer vision and image understanding Ročník 243; s. 103995
Hlavní autoři:	Huo, Wei, Wang, Ke, Tang, Jun, Wang, Nian, Liang, Dong
Médium:	Journal Article
Jazyk:	angličtina
Vydáno:	Elsevier Inc 01.06.2024
Témata:	Causal representation learning Disentangled representation learning Gait recognition Global and local feature extractor Causal representation learning Global and local feature extractor Disentangled representation learning Gait recognition
ISSN:	1077-3142, 1090-235X
On-line přístup:	Získat plný text
Tagy:	Přidat tag Žádné tagy, Buďte první, kdo vytvoří štítek k tomuto záznamu!

Popis
Shrnutí:	Gait recognition is a promising biometric technology that aims to identify the target subject via walking pattern. Most existing appearance-based methods focus on learning discriminative spatio-temporal representations from gait silhouettes. However, these methods pay less attention to probing the causality between identity factors and identity labels, which often mislead the model to learn gait representations that are susceptible to identity-irrelevant factors. In this paper, we attribute the cause that leads to the decline of model generalization under different external conditions to identity-irrelevant factors. We formulate the causalities among the identity factors, identity-irrelevant factors, and identity labels as a structural causal model (SCM). We accordingly propose a novel gait recognition framework named GaitSCM to learn covariate invariant gait representations, which is mainly composed of three components, including feature extraction module, feature disentanglement module, and backdoor adjustment. Specifically, we design a feature extractor with regard to the movement patterns of different body parts to learn fine-grained gait motion features, and then present a two-branch feature decoupling module to disentangle identity features and identity-irrelevant features with the aid of the classification confusion loss. To relieve the negative effect of identity-irrelevant factors, we develop a backdoor adjustment strategy to eliminate spurious associations between identity and identity-irrelevant features, which further facilitates the proposed framework to generate more powerful identity representations. Extensive experiments conducted on two public datasets validate the effectiveness of our method. The average Rank-1 can reach 93.2% and 90.4% on CASIA-B and OU-MVLP datasets, respectively, which verifies the superiority of GaitSCM. Source code is released at: https://github.com/HuoweiCode/GaitSCM. •A novel causal representation learning framework GaitSCM for gait recognition is proposed.•A causal analysis based on SCM reveals the spurious relation in gait recognition.•Feature decoupling and backdoor adjustment are used to perform causal intervention.•GaitSCM achieves competitive results on the CASIA-B and OU-MVLP datasets.
ISSN:	1077-3142 1090-235X
DOI:	10.1016/j.cviu.2024.103995