University of Oulu

Y. Liu, X. Zhang, J. Kauttonen and G. Zhao, "Uncertain Label Correction via Auxiliary Action Unit Graphs for Facial Expression Recognition," 2022 26th International Conference on Pattern Recognition (ICPR), Montreal, QC, Canada, 2022, pp. 777-783, doi: 10.1109/ICPR56361.2022.9956650

Uncertain label correction via auxiliary action unit graphs for facial expression recognition

Saved in:
Author: Liu, Yang1,2; Zhang, Xingming1; Kauttonen, Janne3;
Organizations: 1School of Computer Science and Engineering, South China University of Technology, Guangzhou, China, 510006
2Center for Machine Vision and Signal Analysis, University of Oulu, Oulu, Finland, FI-90014
3Haaga-Helia University of Applied Sciences, Helsinki, Finland, FI-00520
Format: article
Version: accepted version
Access: open
Online Access: PDF Full Text (PDF, 0.9 MB)
Persistent link:
Language: English
Published: IEEE Computer Society, 2022
Publish Date: 2023-04-06


High-quality annotated images are significant to deep facial expression recognition (FER) methods. However, uncertain labels, mostly existing in large-scale public datasets, often mislead the training process. In this paper, we achieve uncertain label correction of facial expressions using auxiliary action unit (AU) graphs, called ULC-AG. Specifically, a weighted regularization module is introduced to highlight valid samples and suppress category imbalance in every batch. Based on the latent dependency between emotions and AUs, an auxiliary branch using graph convolutional layers is added to extract the semantic information from graph topologies. Finally, a re-labeling strategy corrects the ambiguous annotations by comparing their feature similarities with semantic templates. Experiments show that our ULC-AG achieves 89.31% and 61.57% accuracy on RAF-DB and AffectNet datasets, respectively, outperform the baseline and state-of-the-art methods.

see all

Series: International Conference on Pattern Recognition
ISSN: 1051-4651
ISSN-L: 1051-4651
ISBN: 978-1-6654-9062-7
ISBN Print: 978-1-6654-9063-4
Pages: 777 - 783
DOI: 10.1109/icpr56361.2022.9956650
Host publication: 2022 26th International Conference on Pattern Recognition (ICPR)
Conference: International Conference on Pattern Recognition
Type of Publication: A4 Article in conference proceedings
Field of Science: 113 Computer and information sciences
Funding: This work was supported by the China Scholarship Council under Grant 202006150091, and the Academy of Finland for Academy Professor project EmotionAI (grants 336116, 345122), and the Ministry of Education and Culture of Finland for AI forum project.
Academy of Finland Grant Number: 336116
Detailed Information: 336116 (Academy of Finland Funding decision)
345122 (Academy of Finland Funding decision)
Copyright information: © 2022 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.