University of Oulu

S. Zafeiriou, A. Papaioannou, I. Kotsia, M. Nicolaou and G. Zhao, "Facial Affect “In-the-Wild”: A Survey and a New Database," 2016 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), Las Vegas, NV, 2016, pp. 1487-1498. doi: 10.1109/CVPRW.2016.186

Facial affect “in-the-wild” : a survey and a new database

Saved in:
Author: Zafeiriou, Stefanos1,2; Papaioannou, Athanasios; Kotsia, Irene3,4;
Organizations: 1Imperial College London, UK
2Center for Machine Vision and Signal Analysis, University of Oulu, Finland
3Middlesex University London, UK
4International Hellenic University, Greece
5Goldsmiths, University of London, UK
Format: article
Version: accepted version
Access: open
Online Access: PDF Full Text (PDF, 3.2 MB)
Persistent link:
Language: English
Published: Institute of Electrical and Electronics Engineers, 2016
Publish Date: 2019-02-27


Well-established databases and benchmarks have been developed in the past 20 years for automatic facial behaviour analysis. Nevertheless, for some important problems regarding analysis of facial behaviour, such as (a) estimation of affect in a continuous dimensional space (e.g., valence and arousal) in videos displaying spontaneous facial behaviour and (b) detection of the activated facial muscles (i.e., facial action unit detection), to the best of our knowledge, well-established in-the-wild databases and benchmarks do not exist. That is, the majority of the publicly available corpora for the above tasks contain samples that have been captured in controlled recording conditions and/or captured under a very specific milieu. Arguably, in order to make further progress in automatic understanding of facial behaviour, datasets that have been captured in in the-wild and in various milieus have to be developed. In this paper, we survey the progress that has been recently made on understanding facial behaviour in-the-wild, the datasets that have been developed so far and the methodologies that have been developed, paying particular attention to deep learning techniques for the task. Finally, we make a significant step further and propose a new comprehensive benchmark for training methodologies, as well as assessing the performance of facial affect/behaviour analysis/ understanding in-the-wild. To the best of our knowledge, this is the first time that such a benchmark for valence and arousal "in-the-wild" is presented.

see all

Series: IEEE Computer Society Conference on Computer Vision and Pattern Recognition workshops
ISSN: 2160-7508
ISSN-E: 2160-7516
ISSN-L: 2160-7508
ISBN Print: 978-1-5090-1437-8
Pages: 1487 - 1498
DOI: 10.1109/CVPRW.2016.186
Host publication: 2016 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW)
Conference: IEEE Conference on Computer Vision and Pattern Recognition
Type of Publication: A4 Article in conference proceedings
Field of Science: 113 Computer and information sciences
213 Electronic, automation and communications engineering, electronics
Funding: The work of S. Zafeiriou was funded by the FiDiProprogram of Tekes (project number: 1849/31/2015).
Copyright information: © 2016 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.