University of Oulu

X. Huang, S. Wang, X. Liu, G. Zhao, X. Feng and M. Pietikäinen, "Discriminative Spatiotemporal Local Binary Pattern with Revisited Integral Projection for Spontaneous Facial Micro-Expression Recognition," in IEEE Transactions on Affective Computing, vol. 10, no. 1, pp. 32-47, 1 Jan.-March 2019. doi: 10.1109/TAFFC.2017.2713359

Discriminative spatiotemporal local binary pattern with revisited integral projection for spontaneous facial micro-expression recognition

Saved in:
Author: Huang, Xiaohua1,2; Wang, Su-Jing3,4; Liu, Xin5;
Organizations: 1School of Computer Engineering, Nanjing Institute of Technology, Nanjing 21167, China
2University of Oulu, FI-90014, Finland
3CAS Key Laboratory of Behavioral Science, Institute of Psychology, Beijing 100101, China
4Department of Psychology, University of Chinese Academy of Sciences, Huaibeizhen, Huairou, Beijing 100101, China
5Center for Machine Vision and Signal Analysis, University of Oulu, Oulu 90014, Finland
6School of Information and Technology, Northwest University, Xi’an, Shaanxi Sheng 710065
7Center for Machine Vision and Signal Analysis, University of Oulu, FI-90014, Finland
8School of Electronic and Information, Northwestern Polytechnic University, Xi’an, Shaanxi Sheng 710065, China
Format: article
Version: accepted version
Access: open
Online Access: PDF Full Text (PDF, 6 MB)
Persistent link:
Language: English
Published: Institute of Electrical and Electronics Engineers, 2019
Publish Date: 2019-05-29


Recently, there have been increasing interests in inferring mirco-expression from facial image sequences. Due to subtle facial movement of micro-expressions, feature extraction has become an important and critical issue for spontaneous facial micro-expression recognition. Recent works used spatiotemporal local binary pattern (STLBP) for micro-expression recognition and considered dynamic texture information to represent face images. However, they miss the shape attribute of face images. On the other hand, they extract the spatiotemporal features from the global face regions while ignore the discriminative information between two micro-expression classes. The above-mentioned problems seriously limit the application of STLBP to micro-expression recognition. In this paper, we propose a discriminative spatiotemporal local binary pattern based on an integral projection to resolve the problems of STLBP for micro-expression recognition. First, we revisit an integral projection for preserving the shape attribute of micro-expressions by using robust principal component analysis. Furthermore, a revisited integral projection is incorporated with local binary pattern across spatial and temporal domains. Specifically, we extract the novel spatiotemporal features incorporating shape attributes into spatiotemporal texture features. For increasing the discrimination of micro-expressions, we propose a new feature selection based on Laplacian method to extract the discriminative information for facial micro-expression recognition. Intensive experiments are conducted on three availably published micro-expression databases including CASME, CASME2 and SMIC databases. We compare our method with the state-of-the-art algorithms. Experimental results demonstrate that our proposed method achieves promising performance for micro-expression recognition.

see all

Series: IEEE transactions on affective computing
ISSN: 2371-9850
ISSN-E: 1949-3045
ISSN-L: 2371-9850
Volume: 10
Issue: 1
Pages: 32 - 47
DOI: 10.1109/TAFFC.2017.2713359
Type of Publication: A1 Journal article – refereed
Field of Science: 113 Computer and information sciences
Funding: This work was supported by Central fund of Finnish Cultural Foundation, Kaute Foundation, Nokia Foundation, the Academy of Finland, Tekes Fidipro Program, the strategic Funds of the University of Oulu, Finland, the Infotech Oulu. This work was supported in part by grants from the National Natural Science Foundation of China (61379095), the Beijing Natural Science Foundation (4152055). We gratefully acknowledge the support of NVIDIA Corporation with the donation of the Titan X Pascal GPU used for this research.
Copyright information: © 2017 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.