FaceDancer: Pose- and Occlusion-Aware High Fidelity Face Swapping

In this work, we present a new single-stage method for subject agnostic face swapping and identity transfer, named FaceDancer. We have two major contributions: Adaptive Feature Fusion Attention (AFFA) and Interpreted Feature Similarity Regularization (IFSR). The AFFA module is embedded in the decode...

Full description

Saved in:
Bibliographic Details
Published in:Proceedings / IEEE Workshop on Applications of Computer Vision pp. 3443 - 3452
Main Authors: Rosberg, Felix, Aksoy, Eren Erdal, Alonso-Fernandez, Fernando, Englund, Cristofer
Format: Conference Proceeding
Language:English
Published: IEEE 01.01.2023
Subjects:
Adaptation models
Algorithms: Biometrics
and algorithms (including transfer
and un-supervised learning
body pose
Computational modeling
Computer vision
face
formulations
Fuses
gesture
Image coding
low-shot
Machine learning architectures
self
semi
Shape
Visualization
ISSN:2642-9381
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:In this work, we present a new single-stage method for subject agnostic face swapping and identity transfer, named FaceDancer. We have two major contributions: Adaptive Feature Fusion Attention (AFFA) and Interpreted Feature Similarity Regularization (IFSR). The AFFA module is embedded in the decoder and adaptively learns to fuse attribute features and features conditioned on identity information without requiring any additional facial segmentation process. In IFSR, we leverage the intermediate features in an identity encoder to preserve important attributes such as head pose, facial expression, lighting, and occlusion in the target face, while still transferring the identity of the source face with high fidelity. We conduct extensive quantitative and qualitative experiments on various datasets and show that the proposed FaceDancer outperforms other state-of-the-art networks in terms of identity transfer, while having significantly better pose preservation than most of the previous methods. Code available at https://github.com/felixrosberg/FaceDance.
ISSN:2642-9381
DOI:10.1109/WACV56688.2023.00345