University of Oulu

Y. Liu, X. Zhang, Y. Li, J. Zhou, X. Li and G. Zhao, "Graph-based Facial Affect Analysis: A Review," in IEEE Transactions on Affective Computing, 2022, doi: 10.1109/TAFFC.2022.3215918

Graph-based facial affect analysis : a review

Saved in:
Author: Liu, Yang1,2; Zhang, Xingming1; Li, Yante2;
Organizations: 1School of Computer Science and Engineering, South China University of Technology, Guangzhou, China
2Center for Machine Vision and Signal Analysis, University of Oulu, Oulu, Finland
3Department of Electrical and Computer Engineering, Rutgers University, Piscataway, USA
Format: article
Version: published version
Access: open
Online Access: PDF Full Text (PDF, 3.3 MB)
Persistent link:
Language: English
Published: Institute of Electrical and Electronics Engineers, 2022
Publish Date: 2022-12-20


As one of the most important affective signals, facial affect analysis (FAA) is essential for developing human-computer interaction systems. Early methods focus on extracting appearance and geometry features associated with human affects while ignoring the latent semantic information among individual facial changes, leading to limited performance and generalization. Recent work attempts to establish a graph-based representation to model these semantic relationships and develop frameworks to leverage them for various FAA tasks. This paper provides a comprehensive review of graph-based FAA, including the evolution of algorithms and their applications. First, the FAA background knowledge is introduced, especially on the role of the graph. We then discuss approaches widely used for graph-based affective representation in literature and show a trend towards graph construction. For the relational reasoning in graph-based FAA, existing studies are categorized according to their non-deep or deep learning methods, emphasizing the latest graph neural networks. Performance comparisons of the state-of-the-art graph-based FAA methods are also summarized. Finally, we discuss the challenges and potential directions. As far as we know, this is the first survey of graph-based FAA methods. Our findings can serve as a reference for future research in this field.

see all

Series: IEEE transactions on affective computing
ISSN: 2371-9850
ISSN-E: 1949-3045
ISSN-L: 2371-9850
Issue: Online first
DOI: 10.1109/taffc.2022.3215918
Type of Publication: A1 Journal article – refereed
Field of Science: 113 Computer and information sciences
Funding: This work was supported by the China Scholarship Council [CSC, No.202006150091], and the Ministry of Education and Culture of Finland for AI forum project.
Copyright information: © The Author(s) 2022. This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see