M. Behzad, X. Li and G. Zhao, "Disentangling 3D/4D Facial Affect Recognition With Faster Multi-View Transformer," in IEEE Signal Processing Letters, vol. 28, pp. 1913-1917, 2021, doi: 10.1109/LSP.2021.3111576
Disentangling 3D/4D facial affect recognition with faster multi-view transformer
|Author:||Behzad, Muzammil1; Li, Xiaobai1; Zhao, Guoying1|
1Center for Machine Vision and Signal Analysis, University of Oulu, Oulu 90570, Finland
|Online Access:||PDF Full Text (PDF, 1.3 MB)|
|Persistent link:|| http://urn.fi/urn:nbn:fi-fe2021100649450
Institute of Electrical and Electronics Engineers,
|Publish Date:|| 2021-10-06
In this paper, we propose MiT: a novel multi-view transformer model 1 for 3D/4D facial affect recognition. MiT incorporates patch and position embeddings from various patches of multi-views and uses them for learning various facial muscle movements to showcase an effective recognition performance. We also propose a multi-view loss function that is not only gradient-friendly, and hence speeds up the gradient computation during back-propagation, but it also leverages the correlation associated with the underlying facial patterns among multi-views. Additionally, we offer multi-view weights that are trainable and learnable, and help substantially in training. Finally, we equip our model with distributed performance for faster learning and computational convenience. With the help of extensive experiments, we show that our model outperform the existing methods on widely-used datasets for 3D/4D FER.
IEEE signal processing letters
|Pages:||1913 - 1917|
|Type of Publication:||
A1 Journal article – refereed
|Field of Science:||
113 Computer and information sciences
This work was supported in part by Infotech Oulu and the Academy of Finland through project MiGA under Grant 316765, project 6 + E under Grant 323287, and ICT 2023 project under Grant 328115; in part by the Riitta ja Jorma J. Takanen Foundation; and in part by the Tauno Tönning Foundation.
|Academy of Finland Grant Number:||
316765 (Academy of Finland Funding decision)
323287 (Academy of Finland Funding decision)
328115 (Academy of Finland Funding decision)
© 2021 The Authors. This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/.