University of Oulu
Oulu University Library
OuluCRIS
Laturi
Muuntaja

Similar Items

Cover Image

B. Song et al., "Recognizing Spontaneous Micro-Expression Using a Three-Stream Convolutional Neural Network," in IEEE Access, vol. 7, pp. 184537-184551, 2019. doi: 10.1109/ACCESS.2019.2960629

Recognizing spontaneous micro-expression using a three-stream convolutional neural network

Saved in:
Author: Song, Baolin1; Li, Ke1; Zong, Yuan2;
Organizations: 1Key Laboratory of Underwater Acoustic Signal Processing of Ministry of Education, School of Information Science and Engineering, Southeast University, Nanjing 210096, China
2Key Laboratory of Child Development and Learning Science of Ministry of Education, School of Biological Science and Medical Engineering, Southeast University, Nanjing 210096, China
3Center for Machine Vision and Signal Analysis, University of Oulu, Oulu 90014, Finland
Format: article
Version: published version
Access: open
Online Access: PDF Full Text (PDF, 6 MB)
Persistent link: http://urn.fi/urn:nbn:fi-fe202002195810
Language: English
Published: Institute of Electrical and Electronics Engineers, 2019
Publish Date: 2020-02-19
Description:

Abstract

Micro-expression recognition (MER) has attracted much attention with various practical applications, particularly in clinical diagnosis and interrogations. In this paper, we propose a three-stream convolutional neural network (TSCNN) to recognize MEs by learning ME-discriminative features in three key frames of ME videos. We design a dynamic-temporal stream, static-spatial stream, and local-spatial stream module for the TSCNN that respectively attempt to learn and integrate temporal, entire facial region, and facial local region cues in ME videos with the goal of recognizing MEs. In addition, to allow the TSCNN to recognize MEs without using the index values of apex frames, we design a reliable apex frame detection algorithm. Extensive experiments are conducted with five public ME databases: CASME II, SMIC-HS, SAMM, CAS(ME) 2, and CASME. Our proposed TSCNN is shown to achieve more promising recognition results when compared with many other methods.

see all
Series: IEEE access
ISSN: 2169-3536
ISSN-E: 2169-3536
ISSN-L: 2169-3536
Volume: 7
Pages: 184537 - 184551
DOI: 10.1109/ACCESS.2019.2960629
OADOI: https://oadoi.org/10.1109/ACCESS.2019.2960629
Type of Publication: A1 Journal article – refereed
Field of Science: 113 Computer and information sciences
Subjects:
Apex frame location
Convolutional Neural Networks
Micro-expression recognition
Spatiotemporal information
Article
Funding: This work was supported in part by the National Key Research & Development Program of China under Grant No. 2018YFB1305200, in part by the National Natural Science Foundation of China under Grant 61921004, Grant 61902064, Grant 61572009, Grant 61906094, Grant 61673108, Grant 61571106, and Grant 61703201, in part by the Jiangsu Provincial Key Research and Development Program under Grant BE2016616, in part by the Natural Science Foundation of Jiangsu Province under Grant No. BK20170765, and in part by the Fundamental Research Funds for the Central Universities under Grant 2242018K3DN01, and Grant 2242019K40047.
Copyright information: © 2019 The Authors. This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/.
  https://creativecommons.org/licenses/by/4.0/