FaceDancer: Pose- and Occlusion-Aware High Fidelity Face Swapping

In this work, we present a new single-stage method for subject agnostic face swapping and identity transfer, named FaceDancer. We have two major contributions: Adaptive Feature Fusion Attention (AFFA) and Interpreted Feature Similarity Regularization (IFSR). The AFFA module is embedded in the decode...

Full description

Saved in:
Bibliographic Details
Published in:Proceedings / IEEE Workshop on Applications of Computer Vision pp. 3443 - 3452
Main Authors: Rosberg, Felix, Aksoy, Eren Erdal, Alonso-Fernandez, Fernando, Englund, Cristofer
Format: Conference Proceeding
Language:English
Published: IEEE 01.01.2023
Subjects:
Adaptation models
Algorithms: Biometrics
and algorithms (including transfer
and un-supervised learning
body pose
Computational modeling
Computer vision
face
formulations
Fuses
gesture
Image coding
low-shot
Machine learning architectures
self
semi
Shape
Visualization
ISSN:2642-9381
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Abstract In this work, we present a new single-stage method for subject agnostic face swapping and identity transfer, named FaceDancer. We have two major contributions: Adaptive Feature Fusion Attention (AFFA) and Interpreted Feature Similarity Regularization (IFSR). The AFFA module is embedded in the decoder and adaptively learns to fuse attribute features and features conditioned on identity information without requiring any additional facial segmentation process. In IFSR, we leverage the intermediate features in an identity encoder to preserve important attributes such as head pose, facial expression, lighting, and occlusion in the target face, while still transferring the identity of the source face with high fidelity. We conduct extensive quantitative and qualitative experiments on various datasets and show that the proposed FaceDancer outperforms other state-of-the-art networks in terms of identity transfer, while having significantly better pose preservation than most of the previous methods. Code available at https://github.com/felixrosberg/FaceDance.
AbstractList In this work, we present a new single-stage method for subject agnostic face swapping and identity transfer, named FaceDancer. We have two major contributions: Adaptive Feature Fusion Attention (AFFA) and Interpreted Feature Similarity Regularization (IFSR). The AFFA module is embedded in the decoder and adaptively learns to fuse attribute features and features conditioned on identity information without requiring any additional facial segmentation process. In IFSR, we leverage the intermediate features in an identity encoder to preserve important attributes such as head pose, facial expression, lighting, and occlusion in the target face, while still transferring the identity of the source face with high fidelity. We conduct extensive quantitative and qualitative experiments on various datasets and show that the proposed FaceDancer outperforms other state-of-the-art networks in terms of identity transfer, while having significantly better pose preservation than most of the previous methods. Code available at https://github.com/felixrosberg/FaceDance.
Author Aksoy, Eren Erdal
Englund, Cristofer
Alonso-Fernandez, Fernando
Rosberg, Felix
Author_xml – sequence: 1
  givenname: Felix
  surname: Rosberg
  fullname: Rosberg, Felix
  email: felix.rosberg@berge.io
  organization: Berge Consulting,Gothenburg,Sweden
– sequence: 2
  givenname: Eren Erdal
  surname: Aksoy
  fullname: Aksoy, Eren Erdal
  email: eren.aksoy@hh.se
  organization: Halmstad University,Halmstad,Sweden
– sequence: 3
  givenname: Fernando
  surname: Alonso-Fernandez
  fullname: Alonso-Fernandez, Fernando
  email: fernando.alonso-fernandez@hh.se
  organization: Halmstad University,Halmstad,Sweden
– sequence: 4
  givenname: Cristofer
  surname: Englund
  fullname: Englund, Cristofer
  email: cristofer.englund@hh.se
  organization: Halmstad University,Halmstad,Sweden
BookMark eNotj81Kw0AURkdRsK2-QRfzAqlzZ24mM-5CtK1QqODfstxJbupITENSKX17K7r6Nucc-Mbiot21LMQU1AxA-dv3vHhLrXVuppU2M6UMpmdiDNam6A1aOBcjbVEn3ji4EuNh-DwxHrwZiXxOJd9TW3J_J592AyeS2kquy7L5HuKuTfID9SyXcfsh57HiJu6P8teRzwfquthur8VlTc3AN_87Ea_zh5dimazWi8ciXyVRO7tPauuYHKAGcOh8huwQs4BYswsM5FAFA8GmlNZkIWQlKsoCUVWrUJXGTMT0rxuZedP18Yv64wZOT5RGND90rErA
CODEN IEEPAD
ContentType Conference Proceeding
DBID 6IE
6IL
CBEJK
RIE
RIL
DOI 10.1109/WACV56688.2023.00345
DatabaseName IEEE Electronic Library (IEL) Conference Proceedings
IEEE Xplore POP ALL
IEEE Xplore All Conference Proceedings
IEEE Electronic Library (IEL)
IEEE Proceedings Order Plans (POP All) 1998-Present
DatabaseTitleList
Database_xml – sequence: 1
  dbid: RIE
  name: IEEE Electronic Library (IEL)
  url: https://ieeexplore.ieee.org/
  sourceTypes: Publisher
DeliveryMethod fulltext_linktorsrc
Discipline Applied Sciences
EISBN 1665493461
9781665493468
EISSN 2642-9381
EndPage 3452
ExternalDocumentID 10030244
Genre orig-research
GroupedDBID 29G
29O
6IE
6IF
6IK
6IL
6IM
6IN
AAJGR
AAWTH
ABLEC
ADZIZ
ALMA_UNASSIGNED_HOLDINGS
BEFXN
BFFAM
BGNUA
BKEBE
BPEOZ
CBEJK
CHZPO
IEGSK
IPLJI
M43
OCL
RIE
RIL
RNS
ID FETCH-LOGICAL-i286t-f68ea814211848974e8447b44fe8be1a840b31b65a5fa61b7c40a7baadf0bdc33
IEDL.DBID RIE
ISICitedReferencesCount 33
ISICitedReferencesURI http://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=Summon&SrcAuth=ProQuest&DestLinkType=CitingArticles&DestApp=WOS_CPL&KeyUT=000971500203054&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D
IngestDate Wed Aug 27 02:48:56 EDT 2025
IsDoiOpenAccess false
IsOpenAccess true
IsPeerReviewed false
IsScholarly true
Language English
LinkModel DirectLink
MergedId FETCHMERGED-LOGICAL-i286t-f68ea814211848974e8447b44fe8be1a840b31b65a5fa61b7c40a7baadf0bdc33
OpenAccessLink https://urn.kb.se/resolve?urn=urn:nbn:se:hh:diva-48618
PageCount 10
ParticipantIDs ieee_primary_10030244
PublicationCentury 2000
PublicationDate 2023-Jan.
PublicationDateYYYYMMDD 2023-01-01
PublicationDate_xml – month: 01
  year: 2023
  text: 2023-Jan.
PublicationDecade 2020
PublicationTitle Proceedings / IEEE Workshop on Applications of Computer Vision
PublicationTitleAbbrev WACV
PublicationYear 2023
Publisher IEEE
Publisher_xml – name: IEEE
SSID ssj0039193
Score 2.4472876
Snippet In this work, we present a new single-stage method for subject agnostic face swapping and identity transfer, named FaceDancer. We have two major contributions:...
SourceID ieee
SourceType Publisher
StartPage 3443
SubjectTerms Adaptation models
Algorithms: Biometrics
and algorithms (including transfer
and un-supervised learning
body pose
Computational modeling
Computer vision
face
formulations
Fuses
gesture
Image coding
low-shot
Machine learning architectures
self
semi
Shape
Visualization
Title FaceDancer: Pose- and Occlusion-Aware High Fidelity Face Swapping
URI https://ieeexplore.ieee.org/document/10030244
WOSCitedRecordID wos000971500203054&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D
hasFullText 1
inHoldings 1
isFullTextHit
isPrint
link http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV1LTwMhECa28eCpPmp8h4NXdFnYhfXWNG48mNrEV28NzA6miWnNtrV_X6Bb9eLBGyEBkoFhHsz3QchlliIIhY5xsBmTgJoZAwmzGMnEVZZBBArfq8FAj0bFsAGrRywMIsbiM7wKzfiWX81gGVJlXsP9kfT2qEVaSuVrsNbm2hWFd0UabBxPiuvXXv_Fuyo6lG-lgcVUBMTSrx9UogEpO_9cepd0f6B4dPhtZPbIFk73SafxHWmjmfMD0isNYNzC-oYOZ3Nk1Ewr-gDwvgwJMdZbmRppKOugZaC28t43DWPo48oEjoa3Lnkub5_6d6z5HoFNUp0vmMs1Gs2lD-G01D4uQC2lslI61Ba58aGbFdzmmcmcyblVIBOjrDGVS2wFQhyS9nQ2xSNCBU9BOADlXCH9hMYUAiWHzIcTTkNxTLpBJOOPNQPGeCONkz_6T8lOkPo6VXFG2ot6iedkGz4Xk3l9EfftC1GXmJ4
linkProvider IEEE
linkToHtml http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV1LSwMxEA5aBT3VR8W3OXiNbjbZ3ay3Ulwq1lqwam8lmU6kIFvZtvbvm2y36sWDtxAIgZlM5pH5vhByGYUIIkHLOJiISUDFtIaAGSzJxJMoghIo3Em6XTUYpL0KrF5iYRCxbD7DKz8s3_JHE5j7UpmzcHcknT9aJxuRlGGwhGutLl6RumCkQsfxIL1-bbZeXLCifANX6HlMhccs_fpDpXQhWf2fm--Qxg8Yj_a-3cwuWcN8j9Sr6JFWtjndJ81MA5ZKLG5obzJFRnU-oo8A73NfEmPNhS6Q-sYOmnlyKxd_U7-GPi20Z2l4a5Dn7LbfarPqgwQ2DlU8YzZWqBWXLolTUrnMAJWUiZHSojLItUvejOAmjnRkdcxNAjLQidF6ZAMzAiEOSC2f5HhIqOAhCAuQWJs60XKtU4GSQ-QSCqsgPSINL5Lhx5IDY7iSxvEf8xdkq91_6Aw7d937E7LtNbAsXJyS2qyY4xnZhM_ZeFqclzr8AnUPm-U
openUrl ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=proceeding&rft.title=Proceedings+%2F+IEEE+Workshop+on+Applications+of+Computer+Vision&rft.atitle=FaceDancer%3A+Pose-+and+Occlusion-Aware+High+Fidelity+Face+Swapping&rft.au=Rosberg%2C+Felix&rft.au=Aksoy%2C+Eren+Erdal&rft.au=Alonso-Fernandez%2C+Fernando&rft.au=Englund%2C+Cristofer&rft.date=2023-01-01&rft.pub=IEEE&rft.eissn=2642-9381&rft.spage=3443&rft.epage=3452&rft_id=info:doi/10.1109%2FWACV56688.2023.00345&rft.externalDocID=10030244