Su-Jing Wang, Bing-Jun Li, Yong-Jin Liu, Wen-Jing Yan, Xinyu Ou, Xiaohua Huang, Feng Xu, Xiaolan Fu, Micro-expression recognition with small sample size by transferring long-term convolutional neural network, Neurocomputing, Volume 312, 2018, Pages 251-262, ISSN 0925-2312, https://doi.org/10.1016/j.neucom.2018.05.107.
Micro-expression recognition with small sample size by transferring long-term convolutional neural network
|Author:||Wang, Su-Jing1,2; Li, Bing-Jun3; Liu, Yong-Jin3;|
1CAS Key Laboratory of Behavioral Science, Institute of Psychology, Beijing, 100101, China
2Department of Psychology, University of the Chinese Academy of Sciences, Beijing, 100049, China
3Tsinghua National Laboratory for Information Science and Technology, Department of Computer Science and Technology, Tsinghua University, Beijing, P. R. China
4College of Information Science and Engineering, Northeastern University, Shenyang, China
5Cadres Online Learning Institute of Yunnan Province, Yunnan Open University, Kunming, 650223, China
6Center for Machine Vision and Signal Analysis, Faulty of Information Technology and Electrical Engineering, University of Oulu, P. O. Box 4500, FI-90014, Finland
7Shanghai Key Laboratory of Intelligent Information Processing, Key Laboratory for Information Science of Electromagnetic Waves (MoE), and the School of Computer Science, Fudan University, Shanghai, 200433, China
8State Key Laboratory of Brain and Cognitive Science, Institute of Psychology, Chinese Academy of Sciences, Beijing, 100101, China
|Persistent link:|| http://urn.fi/urn:nbn:fi-fe2019042913501
|Publish Date:|| 2020-10-27
Micro-expression is one of important clues for detecting lies. Its most outstanding characteristics include short duration and low intensity of movement. Therefore, video clips of high spatial-temporal resolution are much more desired than still images to provide sufficient details. On the other hand, owing to the difficulties to collect and encode micro-expression data, it is small sample size. In this paper, we use only 560 micro-expression video clips to evaluate the proposed network model: Transferring Long-term Convolutional Neural Network (TLCNN). TLCNN uses Deep CNN to extract features from each frame of micro-expression video clips, then feeds them to Long Short Term Memory (LSTM) which learn the temporal sequence information of micro-expression. Due to the small sample size of micro-expression data, TLCNN uses two steps of transfer learning: (1) transferring from expression data and (2) transferring from single frame of micro-expression video clips, which can be regarded as “big data”. Evaluation on 560 micro-expression video clips collected from three spontaneous databases is performed. The results show that the proposed TLCNN is better than some state-of-the-art algorithms.
|Pages:||251 - 262|
|Type of Publication:||
A1 Journal article – refereed
|Field of Science:||
113 Computer and information sciences
This paper is supported in part by grants from the National Natural Science Foundation of China (61772511, 61379095, U1736220).
© 2018 Elsevier B.V. All rights reserved. This manuscript version is made available under the CC-BY-NC-ND 4.0 license http://creativecommons.org/licenses/by-nc-nd/4.0/