University of Oulu

M. Behzad and G. Zhao, "Self-Supervised Learning via Multi-view Facial Rendezvous for 3D/4D Affect Recognition," 2021 16th IEEE International Conference on Automatic Face and Gesture Recognition (FG 2021), 2021, pp. 1-5, doi: 10.1109/FG52635.2021.9666942

Self-supervised learning via multi-view facial rendezvous for 3D/4D affect recognition

Saved in:
Author: Behzad, Muzammil1; Zhao, Guoying1
Organizations: 1Center for Machine Vision and Signal Analysis, University of Oulu, Finland
Format: article
Version: accepted version
Access: open
Online Access: PDF Full Text (PDF, 2.6 MB)
Persistent link:
Language: English
Published: Institute of Electrical and Electronics Engineers, 2021
Publish Date: 2022-01-14


In this paper, we present Multi-view Facial Rendezvous (MiFaR): a novel multi-view self-supervised learning model for 3D/4D facial affect recognition. Our self-supervised learning architecture has the capability to learn collaboratively via multi-views. For each view, our model learns to compute the embeddings via different encoders and robustly aims to correlate two distorted versions of the input batch. We additionally present a novel loss function that not only leverages the correlation associated with the underlying facial patterns among multi-views but it is also robust and consistent towards different batch sizes. Finally, our model is equipped with distributed training to ensure better learning along with computational convenience. We conduct extensive experiments and report ablations to validate the competence of our model on widely-used datasets for 3D/4D FER.

see all

ISBN: 978-1-6654-3176-7
ISBN Print: 978-1-6654-3177-4
Pages: 1 - 5
DOI: 10.1109/FG52635.2021.9666942
Host publication: Proceedings of the IEEE International Conference on Automatic Face and Gesture Recognition, Jodhpur, India (virtual event), December 15-18, 2021
Conference: IEEE International Conference on Automatic Face and Gesture Recognition
Type of Publication: A4 Article in conference proceedings
Field of Science: 113 Computer and information sciences
Funding: This work was supported by Infotech Oulu, and the Academy of Finland. As well, the financial supports from Riitta ja Jorma J. Takanen Foundation and Tauno Tönning Foundation are acknowledged.
Copyright information: © 2021 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.