C. Guo, J. Liang, G. Zhan, Z. Liu, M. Pietikäinen and L. Liu, "Extended Local Binary Patterns for Efficient and Robust Spontaneous Facial Micro-Expression Recognition," in IEEE Access, vol. 7, pp. 174517-174530, 2019. doi: 10.1109/ACCESS.2019.2942358
Extended local binary patterns for efficient and robust spontaneous facial micro-expression recognition
|Author:||Guo, Chengyu1; Liang, Jingyun1; Zhan, Geng2;|
1College of Systems Engineering, National University of Defense Technology, Changsha 410073, China
2School of Electrical and Information Engineering, The University of Sydney, Camperdown, NSW 2006, Australia
3Center for Machine Vision and Signal Analysis, University of Oulu, 90570 Oulu, Finland
|Online Access:||PDF Full Text (PDF, 2.6 MB)|
|Persistent link:|| http://urn.fi/urn:nbn:fi-fe202001131852
Institute of Electrical and Electronics Engineers,
|Publish Date:|| 2020-01-13
Facial Micro-Expressions (MEs) are spontaneous, involuntary facial movements when a person experiences an emotion but deliberately or unconsciously attempts to conceal his or her genuine emotions. Recently, ME recognition has attracted increasing attention due to its potential applications such as clinical diagnosis, business negotiation, interrogations, and security. However, it is expensive to build large scale ME datasets, mainly due to the difficulty of inducing spontaneous MEs. This limits the application of deep learning techniques which require lots of training data. In this paper, we propose a simple, efficient yet robust descriptor called Extended Local Binary Patterns on Three Orthogonal Planes (ELBPTOP) for ME recognition. ELBPTOP consists of three complementary binary descriptors: LBPTOP and two novel ones Radial Difference LBPTOP (RDLBPTOP) and Angular Difference LBPTOP (ADLBPTOP), which explore the local second order information along the radial and angular directions contained in ME video sequences. ELBPTOP is a novel ME descriptor inspired by unique and subtle facial movements. It is computationally efficient and only marginally increases the cost of computing LBPTOP, yet is extremely effective for ME recognition. In addition, by firstly introducing Whitened Principal Component Analysis (WPCA) to ME recognition, we can further obtain more compact and discriminative feature representations, then achieve significantly computational savings. Extensive experimental evaluation on three popular spontaneous ME datasets SMIC, CASME II and SAMM show that our proposed ELBPTOP approach significantly outperforms the previous state-of-the-art on all three single evaluated datasets and achieves promising results on cross-database recognition. Our code will be made available.
|Pages:||174517 - 174530|
|Type of Publication:||
A1 Journal article – refereed
|Field of Science:||
113 Computer and information sciences
This work was supported by the National Natural Science Foundation of China under Grant 61872379.
© The Authors 2019. This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see http://creativecommons.org/licenses/by/4.0/.