Facial Expression Recognition with Identity and Emotion Joint Learning

Citations Over TimeTop 10% of 2018 papers

Abstract

Different subjects may express a specific expression in different ways due to inter-subject variabilities. In this work, besides training deep-learned facial expression feature (emotional feature), we also consider the influence of latent face identity feature such as the shape or appearance of face. We propose an identity and emotion joint learning approach with deep convolutional neural networks (CNNs) to enhance the performance of facial expression recognition (FER) tasks. First, we learn the emotion and identity features separately using two different CNNs with their corresponding training data. Second, we concatenate these two features together as a deep-learned Tandem Facial Expression (TFE) Feature and feed it to the subsequent fully connected layers to form a new model. Finally, we perform joint learning on the newly merged network using only the facial expression training data. Experimental results show that our proposed approach achieves 99.31 and 84.29 percent accuracy on the CK+ and the FER+ database, respectively, which outperforms the residual network baseline as well as many other state-of-the-art methods.

Related Papers

• → Real-time facial expression recognitions: A review(2009)12 cited
• → A Survey on Factors Affecting Facial Expression Recognition based on Convolutional Neural Networks(2020)10 cited
• → Facial Expression Recognition Using Thermal Image Processing and Efficient Preparation of Training-data(2015)6 cited
• A Survey of Facial Expression Recognition(2012)
• → Development of a Facial Expression Recognition and Expression Skill Training System for ASD Children(2021)2 cited