An Attention Fusion-Based Multimodal Autoencoder Deepfake Detection Framework

Deepfake detection is a critical task to mitigate the risks posed by AI-generated fake media content across societal, political, and security domains. As the quality of generated content improves, traditional single-modal detection methods increasingly struggle to address the complexity and diversit...

Full description

Saved in:
Bibliographic Details
Published in:2025 5th International Conference on Neural Networks, Information and Communication Engineering (NNICE) pp. 1030 - 1034
Main Authors: Hu, Tong, Liu, Wei, Fan, Yaqi, Yang, Jie, Qiao, Yuanyuan
Format: Conference Proceeding
Language:English
Published: IEEE 10.01.2025
Subjects:
Attention mechanisms
Autoencoders
Costs
Deep learning
Deepfake detection
Deepfakes
Feature extraction
Feature fusion
Multimodal
Robustness
Scalability
Security
Training
Unsupervised learning
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Deepfake detection is a critical task to mitigate the risks posed by AI-generated fake media content across societal, political, and security domains. As the quality of generated content improves, traditional single-modal detection methods increasingly struggle to address the complexity and diversity of deepfake content. To overcome the limitations of single-modal analysis and tackle challenges such as large-scale datasets and high annotation costs, we propose an Attention Fusion-based Multimodal AutoEncoder (AFM-AE) Framework. This paper designs a dual-branch autoencoder detection network that integrates both video and audio modalities, innovatively incorporating multi-level attention mechanisms to enable efficient feature fusion. Additionally, we propose a tailored loss function and training enhancement strategies to optimize initially weakly-correlated multimodal features. Experimental results demonstrate that the method significantly improves detection accuracy within an unsupervised learning framework, achieving an AUC of 80.13% on the large public DFDC dataset, comparable to supervised methods on the same dataset. This approach provides a solution with low data-labeling dependency, high efficiency, and strong practical applicability.
DOI:10.1109/NNICE64954.2025.11064639