Towards Zero-Shot Differential Morphing Attack Detection with Multimodal Large Language Models

Leveraging the power of multimodal large language models (LLMs) offers a promising approach to enhancing the accuracy and interpretability of morphing attack detection (MAD), especially in real-world biometric applications. This work introduces the use of LLMs for differential morphing attack detect...

Celý popis

Uloženo v:
Podrobná bibliografie
Vydáno v:IEEE International Conference and Workshops on Automatic Face and Gesture Recognition : FG s. 1 - 10
Hlavní autoři: Shekhawat, Ria, Li, Hailin, Ramachandra, Raghavendra, Venkatesh, Sushma
Médium: Konferenční příspěvek
Jazyk:angličtina
Vydáno: IEEE 26.05.2025
Témata:
Accuracy
Benchmark testing
Biological system modeling
Cognition
Face recognition
Gesture recognition
Large language models
Prompt engineering
Reliability engineering
ISSN:2770-8330
On-line přístup:Získat plný text
Tagy: Přidat tag
Žádné tagy, Buďte první, kdo vytvoří štítek k tomuto záznamu!
Popis
Shrnutí:Leveraging the power of multimodal large language models (LLMs) offers a promising approach to enhancing the accuracy and interpretability of morphing attack detection (MAD), especially in real-world biometric applications. This work introduces the use of LLMs for differential morphing attack detection (D-MAD). To the best of our knowledge, this is the first study to employ multimodal LLMs to D-MAD using real biometric data. To effectively utilize these models, we design Chain-of-Thought (CoT)-based prompts to reduce failure-to-answer rates and enhance the reasoning behind decisions. Our contributions include: (1) the first application of multimodal LLMs for D-MAD using real data subjects, (2) CoT-based prompt engineering to improve response reliability and explainability, (3) comprehensive qualitative and quantitative benchmarking of LLM performance using data from 54 individuals captured in passport enrollment scenarios, and (4) comparative analysis of two multimodal LLMs: ChatGPT-4o and Gemini providing insights into their morphing attack detection accuracy and decision transparency. Experimental results show that ChatGPT-4o outperforms Gemini in detection accuracy, especially against GAN-based morphs, though both models struggle under challenging conditions. While Gemini offers more consistent explanations, ChatGPT-4o is more resilient but prone to a higher failure-to-answer rate.
ISSN:2770-8330
DOI:10.1109/FG61629.2025.11099318