Towards Zero-Shot Differential Morphing Attack Detection with Multimodal Large Language Models

Leveraging the power of multimodal large language models (LLMs) offers a promising approach to enhancing the accuracy and interpretability of morphing attack detection (MAD), especially in real-world biometric applications. This work introduces the use of LLMs for differential morphing attack detect...

Full description

Saved in:
Bibliographic Details
Published in:IEEE International Conference and Workshops on Automatic Face and Gesture Recognition : FG pp. 1 - 10
Main Authors: Shekhawat, Ria, Li, Hailin, Ramachandra, Raghavendra, Venkatesh, Sushma
Format: Conference Proceeding
Language:English
Published: IEEE 26.05.2025
Subjects:
Accuracy
Benchmark testing
Biological system modeling
Cognition
Face recognition
Gesture recognition
Large language models
Prompt engineering
Reliability engineering
ISSN:2770-8330
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Leveraging the power of multimodal large language models (LLMs) offers a promising approach to enhancing the accuracy and interpretability of morphing attack detection (MAD), especially in real-world biometric applications. This work introduces the use of LLMs for differential morphing attack detection (D-MAD). To the best of our knowledge, this is the first study to employ multimodal LLMs to D-MAD using real biometric data. To effectively utilize these models, we design Chain-of-Thought (CoT)-based prompts to reduce failure-to-answer rates and enhance the reasoning behind decisions. Our contributions include: (1) the first application of multimodal LLMs for D-MAD using real data subjects, (2) CoT-based prompt engineering to improve response reliability and explainability, (3) comprehensive qualitative and quantitative benchmarking of LLM performance using data from 54 individuals captured in passport enrollment scenarios, and (4) comparative analysis of two multimodal LLMs: ChatGPT-4o and Gemini providing insights into their morphing attack detection accuracy and decision transparency. Experimental results show that ChatGPT-4o outperforms Gemini in detection accuracy, especially against GAN-based morphs, though both models struggle under challenging conditions. While Gemini offers more consistent explanations, ChatGPT-4o is more resilient but prone to a higher failure-to-answer rate.
ISSN:2770-8330
DOI:10.1109/FG61629.2025.11099318