A Dynamic Texture-Based Approach to Recognition of Facial Actions and Their Temporal Models

In this work, we propose a dynamic texture-based approach to the recognition of facial Action Units (AUs, atomic facial gestures) and their temporal models (i.e., sequences of temporal segments: neutral, onset, apex, and offset) in near-frontal-view face videos. Two approaches to modeling the dynami...

Ausführliche Beschreibung

Gespeichert in:
Bibliographische Detailangaben
Veröffentlicht in:IEEE transactions on pattern analysis and machine intelligence Jg. 32; H. 11; S. 1940 - 1954
Hauptverfasser: Koelstra, Sander, Pantic, Maja, Patras, Ioannis
Format: Journal Article
Sprache:Englisch
Veröffentlicht: Los Alamitos, CA IEEE 01.11.2010
IEEE Computer Society
The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Schlagworte:
Algorithms
Applied sciences
Artificial intelligence
Computer science; control theory; systems
Computing Methodologies
Deformable models
dynamic texture
Dynamics
Exact sciences and technology
Face
Face recognition
Facial
Facial Expression
Facial image analysis
Gestures
Gold
Hidden Markov models
Histograms
History
Humans
Image segmentation
Image sequences
Information Storage and Retrieval - methods
Mathematical models
Models, Biological
motion
Pattern Recognition, Automated - methods
Pattern recognition. Digital image processing. Computational geometry
Recognition
Recognition (Psychology)
Segments
Studies
Temporal logic
Testing
Time Factors
Video Recording - methods
Videos
Action
Histogram
Image processing
Free form
Video signal
Facial image analysis
Modeling
Texture
Aggregate model
Atomic operation
Behavioral analysis
Image sequence
Facies
Gesture
Database
Dynamic model
Pattern analysis
Time analysis
Generative model
Image registration
Motion estimation
motion
Orientation
Image analysis
Frontal
Hidden Markov model
dynamic texture
Artificial intelligence
Facial expression
ISSN:0162-8828, 1939-3539, 1939-3539
Online-Zugang:Volltext
Tags: Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
Beschreibung
Zusammenfassung:In this work, we propose a dynamic texture-based approach to the recognition of facial Action Units (AUs, atomic facial gestures) and their temporal models (i.e., sequences of temporal segments: neutral, onset, apex, and offset) in near-frontal-view face videos. Two approaches to modeling the dynamics and the appearance in the face region of an input video are compared: an extended version of Motion History Images and a novel method based on Nonrigid Registration using Free-Form Deformations (FFDs). The extracted motion representation is used to derive motion orientation histogram descriptors in both the spatial and temporal domain. Per AU, a combination of discriminative, frame-based GentleBoost ensemble learners and dynamic, generative Hidden Markov Models detects the presence of the AU in question and its temporal segments in an input image sequence. When tested for recognition of all 27 lower and upper face AUs, occurring alone or in combination in 264 sequences from the MMI facial expression database, the proposed method achieved an average event recognition accuracy of 89.2 percent for the MHI method and 94.3 percent for the FFD method. The generalization performance of the FFD method has been tested using the Cohn-Kanade database. Finally, we also explored the performance on spontaneous expressions in the Sensitive Artificial Listener data set.
Bibliographie:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
content type line 23
ObjectType-Article-2
ObjectType-Feature-1
ISSN:0162-8828
1939-3539
1939-3539
DOI:10.1109/TPAMI.2010.50