University of Oulu

Yante Li and Guoying Zhao. 2021. Intra- and Inter-Contrastive Learning for Micro-expression Action Unit Detection. Proceedings of the 2021 International Conference on Multimodal Interaction. Association for Computing Machinery, New York, NY, USA, 702–706.

Intra- and inter-contrastive learning for micro-expression action unit detection

Saved in:
Author: Li, Yante1; Zhao, Guoying1
Organizations: 1University of Oulu
Format: article
Version: accepted version
Access: open
Online Access: PDF Full Text (PDF, 1.6 MB)
Persistent link:
Language: English
Published: Association for Computing Machinery, 2021
Publish Date: 2021-12-10


Encoding facial expressions via Action Units (AUs) has been found effective for resolving the ambiguity issue among different expressions. In the literature, AU detection has extensive researches in macro-expressions. However, there is limited research about AU analysis for micro-expressions (MEs). Micro-expression Action Unit (MEAU) detection becomes a challenging problem because of the subtle facial motion. To alleviate this problem, in this paper, we study the contrastive learning for modeling subtle AUs and propose a novel MEAU detection method by learning the intra- and inter-contrastive information among MEs. Through the intra-contrastive learning module, the difference between the onset and apex frames is enlarged and utilized to obtain the discriminative representation for low-intensity AU detection. In addition, considering the subtle difference between MEAUs, the inter-contrastive learning is designed to automatically explore and enlarge the difference between different AUs to enhance the MEAU detection robustness. Intensive experiments on two widely used ME databases have demonstrated the effectiveness and generalization ability of our proposed method.

see all

ISBN: 978-1-4503-8481-0
Pages: 702 - 706
DOI: 10.1145/3462244.3479956
Host publication: ICMI '21: Proceedings of the 2021 International Conference on Multimodal Interaction
Conference: ACM International Conference on Multimodal Interaction
Type of Publication: A4 Article in conference proceedings
Field of Science: 113 Computer and information sciences
Funding: This work was supported by Infotech Oulu, National Natural Science Foundation of China (Grant No: 61772419), Ministry of Education and Culture of Finland for AI forum project, and Academy of Finland for ICT 2023 project (grant 328115).
Academy of Finland Grant Number: 328115
Detailed Information: 328115 (Academy of Finland Funding decision)
Copyright information: © 2021 Association for Computing Machinery. The final authenticated version is available online at