University of Oulu

Y. Li, J. Wei, Y. Liu, J. Kauttonen and G. Zhao, "Deep Learning for Micro-Expression Recognition: A Survey," in IEEE Transactions on Affective Computing, vol. 13, no. 4, pp. 2028-2046, 1 Oct.-Dec. 2022, doi: 10.1109/TAFFC.2022.3205170

Deep learning for micro-expression recognition : a survey

Saved in:
Author: Li, Yante1; Wei, Jinsheng1,2; Liu, Yang1;
Organizations: 1Center for Machine Vision and Signal Analysis, University of Oulu, Oulu, Finland
2School of Telecommunications and Information Engineering, Nanjing University of Posts and Telecommunications, Nanjing, China
3School of Digital Business, Haaga-Helia University of Applied Sciences, Helsinki, Finland
Format: article
Version: published version
Access: open
Online Access: PDF Full Text (PDF, 2 MB)
Persistent link:
Language: English
Published: Institute of Electrical and Electronics Engineers, 2022
Publish Date: 2022-12-16


Micro-expressions (MEs) are involuntary facial movements revealing people’s hidden feelings in high-stake situations and have practical importance in various fields. Early methods for Micro-expression Recognition (MER) are mainly based on traditional features. Recently, with the success of Deep Learning (DL) in various tasks, neural networks have received increasing interest in MER. Different from macro-expressions, MEs are spontaneous, subtle, and rapid facial movements, leading to difficult data collection and annotation, thus publicly available datasets are usually small-scale. Currently, various DL approaches have been proposed to solve the ME issues and improve MER performance. In this survey, we provide a comprehensive review of deep MER and define a new taxonomy for the field encompassing all aspects of MER based on DL, including datasets, each step of the deep MER pipeline, and performance comparisons of the most influential methods. The basic approaches and advanced developments are summarized and discussed for each aspect. Additionally, we conclude the remaining challenges and potential directions for the design of robust MER systems. Finally, ethical considerations in MER are discussed. To the best of our knowledge, this is the first survey of deep MER methods, and this survey can serve as a reference point for future MER research.

see all

Series: IEEE transactions on affective computing
ISSN: 2371-9850
ISSN-E: 1949-3045
ISSN-L: 2371-9850
Volume: 19
Issue: 4
Pages: 2028 - 2046
DOI: 10.1109/taffc.2022.3205170
Type of Publication: A1 Journal article – refereed
Field of Science: 113 Computer and information sciences
Funding: This work was supported in part by the Academy of Finland for Academy Professor project EmotionAI under Grants 336116 and 345122 and in part by Ministry of Education and Culture of Finland for AI forum project and Infotech Oulu.
Academy of Finland Grant Number: 336116
Detailed Information: 336116 (Academy of Finland Funding decision)
345122 (Academy of Finland Funding decision)
Copyright information: © The Author(s) 2022. This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see