University of Oulu

Y. Zong, X. Huang, W. Zheng, Z. Cui and G. Zhao, "Learning From Hierarchical Spatiotemporal Descriptors for Micro-Expression Recognition," in IEEE Transactions on Multimedia, vol. 20, no. 11, pp. 3160-3172, Nov. 2018. doi: 10.1109/TMM.2018.2820321

Learning from hierarchical spatiotemporal descriptors for micro-expression recognition

Saved in:
Author: Zong, Yuan1; Huang, Xiaohua2; Zheng, Wenming1;
Organizations: 1Key Laboratory of Child Development and Learning Science of Ministry of Education, School of Biological Sciences and Medical Engineering, Southeast University, Nanjing, China
2Center for Machine Vision and Signal Analysis, Faculty of Information Technology and Electrical Engineering, University of Oulu
3School of Computer Science and Engineering, Nanjing University of Science and Technology, Nanjing, China
Format: article
Version: accepted version
Access: open
Online Access: PDF Full Text (PDF, 3.3 MB)
Persistent link:
Language: English
Published: Institute of Electrical and Electronics Engineers, 2018
Publish Date: 2018-11-23


Micro-expression recognition aims to infer genuine emotions that people try to conceal from facial video clips. It is a very challenging task because micro-expressions have a very low intensity and short duration, which makes micro-expressions difficult to observe. Recently, researchers have designed various spatiotemporal descriptors to describe micro-expressions. It is notable that for better capturing the low-intensity facial muscle movement, a fixed spatial division grid, 8× 8 for example, is commonly used to partition the facial images into a few facial blocks before extracting descriptors. However, it is hard to choose an ideal division grid for different micro-expression samples because the division grids affect the discriminative ability of spatiotemporal descriptors to distinguish micro-expressions. To address this problem, in this paper, we design a hierarchical spatial division scheme for spatiotemporal descriptor extraction. By using the proposed scheme, it would not be a problem to determine which division grid is most suitable regarding different micro-expression samples. Furthermore, we propose a kernelized group sparse learning (KGSL) model to process hierarchical scheme based spatiotemporal descriptors such that they are more effective for micro-expression recognition tasks. To evaluate the performance of the proposed micro-expression recognition method consisting of the hierarchical scheme based spatiotemporal descriptors and KGSL, extensive experiments are conducted on two public micro-expression databases: CASME II and SMIC. Compared with many recent state-of-the-art approaches, our method achieves more promising recognition results.

see all

Series: IEEE transactions on multimedia
ISSN: 1520-9210
ISSN-E: 1941-0077
ISSN-L: 1520-9210
Volume: 20
Issue: 11
Pages: 3160 - 3172
DOI: 10.1109/TMM.2018.2820321
Type of Publication: A1 Journal article – refereed
Field of Science: 213 Electronic, automation and communications engineering, electronics
Funding: This work was supported in part by the National Basic Research Program of China under Grant 2015CB351704, in part by the National Natural Science Foundation of China under Grant 61572009, Grant 61772276, and Grant 61602244, in part by the Jiangsu Provincial Key Research and Development Program under Grant BE2016616, in part by China Scholarship Council, in part by the Scientific Research Foundation of Graduate School of Southeast University under Grant YBJJ1774, in part by Academy of Finland, in part by Tekes Fidipro Program, in part by Infotech Oulu, and in part by Jorma Ollila Grant of Nokia Foundation and Central Fund of Finnish Cultural Foundation.
Copyright information: © 2018 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.