University of Oulu

Su-Jing Wang, Bing-Jun Li, Yong-Jin Liu, Wen-Jing Yan, Xinyu Ou, Xiaohua Huang, Feng Xu, Xiaolan Fu, Micro-expression recognition with small sample size by transferring long-term convolutional neural network, Neurocomputing, Volume 312, 2018, Pages 251-262, ISSN 0925-2312,

Micro-expression recognition with small sample size by transferring long-term convolutional neural network

Saved in:
Author: Wang, Su-Jing1,2; Li, Bing-Jun3; Liu, Yong-Jin3;
Organizations: 1CAS Key Laboratory of Behavioral Science, Institute of Psychology, Beijing, 100101, China
2Department of Psychology, University of the Chinese Academy of Sciences, Beijing, 100049, China
3Tsinghua National Laboratory for Information Science and Technology, Department of Computer Science and Technology, Tsinghua University, Beijing, P. R. China
4College of Information Science and Engineering, Northeastern University, Shenyang, China
5Cadres Online Learning Institute of Yunnan Province, Yunnan Open University, Kunming, 650223, China
6Center for Machine Vision and Signal Analysis, Faulty of Information Technology and Electrical Engineering, University of Oulu, P. O. Box 4500, FI-90014, Finland
7Shanghai Key Laboratory of Intelligent Information Processing, Key Laboratory for Information Science of Electromagnetic Waves (MoE), and the School of Computer Science, Fudan University, Shanghai, 200433, China
8State Key Laboratory of Brain and Cognitive Science, Institute of Psychology, Chinese Academy of Sciences, Beijing, 100101, China
Format: article
Version: accepted version
Access: embargoed
Persistent link:
Language: English
Published: Elsevier, 2018
Publish Date: 2020-10-27


Micro-expression is one of important clues for detecting lies. Its most outstanding characteristics include short duration and low intensity of movement. Therefore, video clips of high spatial-temporal resolution are much more desired than still images to provide sufficient details. On the other hand, owing to the difficulties to collect and encode micro-expression data, it is small sample size. In this paper, we use only 560 micro-expression video clips to evaluate the proposed network model: Transferring Long-term Convolutional Neural Network (TLCNN). TLCNN uses Deep CNN to extract features from each frame of micro-expression video clips, then feeds them to Long Short Term Memory (LSTM) which learn the temporal sequence information of micro-expression. Due to the small sample size of micro-expression data, TLCNN uses two steps of transfer learning: (1) transferring from expression data and (2) transferring from single frame of micro-expression video clips, which can be regarded as “big data”. Evaluation on 560 micro-expression video clips collected from three spontaneous databases is performed. The results show that the proposed TLCNN is better than some state-of-the-art algorithms.

see all

Series: Neurocomputing
ISSN: 0925-2312
ISSN-E: 1872-8286
ISSN-L: 0925-2312
Volume: 312
Pages: 251 - 262
DOI: 10.1016/j.neucom.2018.05.107
Type of Publication: A1 Journal article – refereed
Field of Science: 113 Computer and information sciences
Funding: This paper is supported in part by grants from the National Natural Science Foundation of China (61772511, 61379095, U1736220).
Copyright information: © 2018 Elsevier B.V. All rights reserved. This manuscript version is made available under the CC-BY-NC-ND 4.0 license