FaceDancer: Pose- and Occlusion-Aware High Fidelity Face Swapping

In this work, we present a new single-stage method for subject agnostic face swapping and identity transfer, named FaceDancer. We have two major contributions: Adaptive Feature Fusion Attention (AFFA) and Interpreted Feature Similarity Regularization (IFSR). The AFFA module is embedded in the decode...

Celý popis

Uloženo v:
Podrobná bibliografie
Vydáno v:Proceedings / IEEE Workshop on Applications of Computer Vision s. 3443 - 3452
Hlavní autoři: Rosberg, Felix, Aksoy, Eren Erdal, Alonso-Fernandez, Fernando, Englund, Cristofer
Médium: Konferenční příspěvek
Jazyk:angličtina
Vydáno: IEEE 01.01.2023
Témata:
Adaptation models
Algorithms: Biometrics
and algorithms (including transfer
and un-supervised learning
body pose
Computational modeling
Computer vision
face
formulations
Fuses
gesture
Image coding
low-shot
Machine learning architectures
self
semi
Shape
Visualization
ISSN:2642-9381
On-line přístup:Získat plný text
Tagy: Přidat tag
Žádné tagy, Buďte první, kdo vytvoří štítek k tomuto záznamu!
Popis
Shrnutí:In this work, we present a new single-stage method for subject agnostic face swapping and identity transfer, named FaceDancer. We have two major contributions: Adaptive Feature Fusion Attention (AFFA) and Interpreted Feature Similarity Regularization (IFSR). The AFFA module is embedded in the decoder and adaptively learns to fuse attribute features and features conditioned on identity information without requiring any additional facial segmentation process. In IFSR, we leverage the intermediate features in an identity encoder to preserve important attributes such as head pose, facial expression, lighting, and occlusion in the target face, while still transferring the identity of the source face with high fidelity. We conduct extensive quantitative and qualitative experiments on various datasets and show that the proposed FaceDancer outperforms other state-of-the-art networks in terms of identity transfer, while having significantly better pose preservation than most of the previous methods. Code available at https://github.com/felixrosberg/FaceDance.
ISSN:2642-9381
DOI:10.1109/WACV56688.2023.00345