University of Oulu

X. Liu, H. Shi, H. Chen, Z. Yu, X. Li and G. Zhao, "iMiGUE: An Identity-free Video Dataset for Micro-Gesture Understanding and Emotion Analysis," 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2021, pp. 10626-10637, doi: 10.1109/CVPR46437.2021.01049.

iMiGUE : an identity-free video dataset for micro-gesture understanding and emotion analysis

Saved in:
Author: Liu, Xin1; Shi, Henglin2; Chen, Haoyu2;
Organizations: 1Center for Machine Vision and Signal Analysis, University of Oulu, Finland
2School of Electrical and Information Engineering, Tianjin University, China
Format: article
Version: accepted version
Access: open
Online Access: PDF Full Text (PDF, 2.4 MB)
Persistent link:
Language: English
Published: Institute of Electrical and Electronics Engineers, 2021
Publish Date: 2022-01-03


We introduce a new dataset for the emotional artificial intelligence research: identity-free video dataset for Micro-Gesture Understanding and Emotion analysis (iMiGUE). Different from existing public datasets, iMiGUE focuses on nonverbal body gestures without using any identity information, while the predominant researches of emotion analysis concern sensitive biometric data, like face and speech. Most importantly, iMiGUE focuses on micro-gestures, i.e., unintentional behaviors driven by inner feelings, which are different from ordinary scope of gestures from other gesture datasets which are mostly intentionally performed for illustrative purposes. Furthermore, iMiGUE is designed to evaluate the ability of models to analyze the emotional states by integrating information of recognized micro-gesture, rather than just recognizing prototypes in the sequences separately (or isolatedly). This is because the real need for emotion AI is to understand the emotional states behind gestures in a holistic way. Moreover, to counter for the challenge of imbalanced sample distribution of this dataset, an unsupervised learning method is proposed to capture latent representations from the micro-gesture sequences themselves. We systematically investigate representative methods on this dataset, and comprehensive experimental results reveal several interesting insights from the iMiGUE, e.g., micro-gesture-based analysis can promote emotion understanding. We confirm that the new iMiGUE dataset could advance studies of micro-gesture and emotion AI.

see all

Series: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition
ISSN: 1063-6919
ISSN-E: 2575-7075
ISSN-L: 1063-6919
ISBN: 978-1-6654-4509-2
ISBN Print: 978-1-6654-4510-8
Pages: 10626 - 10637
DOI: 10.1109/CVPR46437.2021.01049
Host publication: 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition : proceedings
Conference: 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR 2021)
Type of Publication: A4 Article in conference proceedings
Field of Science: 113 Computer and information sciences
Funding: This paper is supported by Academy of Finland, KAUTE foundation, and National Natural Science Foundation of China.
Copyright information: © 2021 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.