FaceDancer: Pose- and Occlusion-Aware High Fidelity Face Swapping

In this work, we present a new single-stage method for subject agnostic face swapping and identity transfer, named FaceDancer. We have two major contributions: Adaptive Feature Fusion Attention (AFFA) and Interpreted Feature Similarity Regularization (IFSR). The AFFA module is embedded in the decode...

Celý popis

Uložené v:
Podrobná bibliografia
Vydané v:Proceedings / IEEE Workshop on Applications of Computer Vision s. 3443 - 3452
Hlavní autori: Rosberg, Felix, Aksoy, Eren Erdal, Alonso-Fernandez, Fernando, Englund, Cristofer
Médium: Konferenčný príspevok..
Jazyk:English
Vydavateľské údaje: IEEE 01.01.2023
Predmet:
Adaptation models
Algorithms: Biometrics
and algorithms (including transfer
and un-supervised learning
body pose
Computational modeling
Computer vision
face
formulations
Fuses
gesture
Image coding
low-shot
Machine learning architectures
self
semi
Shape
Visualization
ISSN:2642-9381
On-line prístup:Získať plný text
Tagy: Pridať tag
Žiadne tagy, Buďte prvý, kto otaguje tento záznam!
Popis
Shrnutí:In this work, we present a new single-stage method for subject agnostic face swapping and identity transfer, named FaceDancer. We have two major contributions: Adaptive Feature Fusion Attention (AFFA) and Interpreted Feature Similarity Regularization (IFSR). The AFFA module is embedded in the decoder and adaptively learns to fuse attribute features and features conditioned on identity information without requiring any additional facial segmentation process. In IFSR, we leverage the intermediate features in an identity encoder to preserve important attributes such as head pose, facial expression, lighting, and occlusion in the target face, while still transferring the identity of the source face with high fidelity. We conduct extensive quantitative and qualitative experiments on various datasets and show that the proposed FaceDancer outperforms other state-of-the-art networks in terms of identity transfer, while having significantly better pose preservation than most of the previous methods. Code available at https://github.com/felixrosberg/FaceDance.
ISSN:2642-9381
DOI:10.1109/WACV56688.2023.00345