iMiGUE : an identity-free video dataset for micro-gesture understanding and emotion analysis |
|
Author: | Liu, Xin1; Shi, Henglin2; Chen, Haoyu2; |
Organizations: |
1Center for Machine Vision and Signal Analysis, University of Oulu, Finland 2School of Electrical and Information Engineering, Tianjin University, China |
Format: | article |
Version: | accepted version |
Access: | open |
Online Access: | PDF Full Text (PDF, 2.4 MB) |
Persistent link: | http://urn.fi/urn:nbn:fi-fe202201031075 |
Language: | English |
Published: |
Institute of Electrical and Electronics Engineers,
2021
|
Publish Date: | 2022-01-03 |
Description: |
AbstractWe introduce a new dataset for the emotional artificial intelligence research: identity-free video dataset for Micro-Gesture Understanding and Emotion analysis (iMiGUE). Different from existing public datasets, iMiGUE focuses on nonverbal body gestures without using any identity information, while the predominant researches of emotion analysis concern sensitive biometric data, like face and speech. Most importantly, iMiGUE focuses on micro-gestures, i.e., unintentional behaviors driven by inner feelings, which are different from ordinary scope of gestures from other gesture datasets which are mostly intentionally performed for illustrative purposes. Furthermore, iMiGUE is designed to evaluate the ability of models to analyze the emotional states by integrating information of recognized micro-gesture, rather than just recognizing prototypes in the sequences separately (or isolatedly). This is because the real need for emotion AI is to understand the emotional states behind gestures in a holistic way. Moreover, to counter for the challenge of imbalanced sample distribution of this dataset, an unsupervised learning method is proposed to capture latent representations from the micro-gesture sequences themselves. We systematically investigate representative methods on this dataset, and comprehensive experimental results reveal several interesting insights from the iMiGUE, e.g., micro-gesture-based analysis can promote emotion understanding. We confirm that the new iMiGUE dataset could advance studies of micro-gesture and emotion AI. see all
|
Series: |
Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition |
ISSN: | 1063-6919 |
ISSN-E: | 2575-7075 |
ISSN-L: | 1063-6919 |
ISBN: | 978-1-6654-4509-2 |
ISBN Print: | 978-1-6654-4510-8 |
Pages: | 10626 - 10637 |
DOI: | 10.1109/CVPR46437.2021.01049 |
OADOI: | https://oadoi.org/10.1109/CVPR46437.2021.01049 |
Host publication: |
2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition : proceedings |
Conference: |
2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR 2021) |
Type of Publication: |
A4 Article in conference proceedings |
Field of Science: |
113 Computer and information sciences |
Subjects: | |
Funding: |
This paper is supported by Academy of Finland, KAUTE foundation, and National Natural Science Foundation of China. |
Copyright information: |
© 2021 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works. |