University of Oulu

Q. N. Vo, K. Tran and G. Zhao, "3D Facial Expression Recognition Based on Multi-View and Prior Knowledge Fusion," 2019 IEEE 21st International Workshop on Multimedia Signal Processing (MMSP), Kuala Lumpur, Malaysia, 2019, pp. 1-6. doi: 10.1109/MMSP.2019.8901797

3D facial expression recognition based on multi-view and prior knowledge fusion

Saved in:
Author: Vo, Quang Nhat1; Tran, Khanh1; Zhao, Guoying1
Organizations: 1Center for Machine Vision and Signal Analysis, University of Oulu, Oulu, Finland
Format: article
Version: accepted version
Access: open
Online Access: PDF Full Text (PDF, 0.8 MB)
Persistent link:
Language: English
Published: Institute of Electrical and Electronics Engineers, 2019
Publish Date: 2019-12-02


This paper presents a novel multi-view convolutional neural network (CNN) model for 3D facial expression recognition (FER). In contrast to existing deep learning-based 3D FER approaches that mainly learn the expressions from frontal facial attribute images, the proposed model incorporates multi-view and facial prior information of the observed 3D face into the learning process. This information is jointly trained in an end-to-end manner to predict the emotion of the input 3D face model. The experiments on public 3D facial expression datasets show that training the CNN with additional information from different views and facial prior knowledge would result in learning more discriminative features as against from a single view. Our model outperforms the state-of-the-art 3D FER methods in term of recognition accuracy indicating its effectiveness. Moreover, the improvement of the proposed model is displayed more clearly in the discrimination of low-intensity facial expressions.

see all

Series: IEEE International Workshop on Multimedia Signal Processing
ISSN: 2163-3517
ISSN-E: 2473-3628
ISSN-L: 2163-3517
ISBN: 978-1-7281-1817-8
ISBN Print: 978-1-7281-1818-5
Pages: 1 - 6
DOI: 10.1109/MMSP.2019.8901797
Host publication: 2019 IEEE 21st International Workshop on Multimedia Signal Processing (MMSP) 27-29 Sept. 2019 Kuala Lumpur, Malaysia
Conference: IEEE International Workshop on Multimedia Signal Processing
Type of Publication: A4 Article in conference proceedings
Field of Science: 113 Computer and information sciences
Funding: This work was supported by Infotech Oulu, the National Natural Science Foundation of China (No. 61772419), Tekes Fidipro Program (No. 1849/31/2015), Business Finland Project (No. 3116/31/2017), and Academy of Finland ICT 2023 project (313600). Moreover, the authors wish to acknowledge CSC-IT Center for Science, Finland, for computational resources. Furthermore, we gratefully acknowledge the support of NVIDIA Corporation with the donation of the GPU used for this research.
Academy of Finland Grant Number: 313600
Detailed Information: 313600 (Academy of Finland Funding decision)
Copyright information: ©2019 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.