Frontalization and adaptive exponential ensemble rule for deep-learning-based facial expression recognition system

Automatic facial expression recognition (FER) is an important technique in human–computer interfaces and surveillance systems. It classifies the input facial image into one of the basic expressions (anger, sadness, surprise, happiness, disgust, fear, and neutral). There are two types of FER algorith...

Full description

Saved in:
Bibliographic Details
Published in:Signal processing. Image communication Vol. 96; p. 116321
Main Authors: Tsai, Kai-Yuan, Tsai, Yi-Wei, Lee, Yih-Cherng, Ding, Jian-Jiun, Chang, Ronald Y.
Format: Journal Article
Language:English
Published: Amsterdam Elsevier B.V 01.08.2021
Elsevier BV
Subjects:
Algorithms
Artificial neural networks
Background noise
Classifiers
Computer vision
Convolutional neural network
Deep learning
Face frontalization
Face recognition
Facial expression
Hierarchical structure
Machine learning
Recognition
Surveillance systems
Computer vision
Face frontalization
Face recognition
Convolutional neural network
Facial expression
Hierarchical structure
ISSN:0923-5965, 1879-2677
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Automatic facial expression recognition (FER) is an important technique in human–computer interfaces and surveillance systems. It classifies the input facial image into one of the basic expressions (anger, sadness, surprise, happiness, disgust, fear, and neutral). There are two types of FER algorithms: feature-based and convolutional neural network (CNN)-based algorithms. The CNN is a powerful classifier, however, without proper auxiliary techniques, its performance may be limited. In this study, we improve the CNN-based FER system by utilizing face frontalization and the hierarchical architecture. The frontalization algorithm aligns the face by in-plane or out-of-plane, rotation, landmark point matching, and removing background noise. The proposed adaptive exponentially weighted average ensemble rule can determine the optimal weight according to the accuracy of classifiers to improve robustness. Experiments on several popular databases are performed and the results show that the proposed system has a very high accuracy and outperforms state-of-the-art FER systems. [Display omitted] •An advanced CNN based facial expression recognition (FER) method is proposed.•Its accuracy is higher than that of other stated-of-the-art CNN-based methods.•Advanced frontalization method is used to make the input of the CNN more meaningful.•A hierarchical AEWEA system is applied to integrate the advantages of each model.•The shortcut CNN, which considers block relations and is easier to train, is adopted.
Bibliography:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
ISSN:0923-5965
1879-2677
DOI:10.1016/j.image.2021.116321