University of Oulu

M. Tavakolian, C. G. Bermudez Cruces and A. Hadid, "Learning to Detect Genuine versus Posed Pain from Facial Expressions using Residual Generative Adversarial Networks," 2019 14th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2019), Lille, France, 2019, pp. 1-8. doi: 10.1109/FG.2019.8756540

Learning to detect genuine versus posed pain from facial expressions using residual generative adversarial networks

Saved in:
Author: Tavakolian, Mohammad1; Cruces, Carlos Guillermo Bermudez1,2; Hadid, Abdenour1
Organizations: 1Center for Machine Vision and Signal Analysis (CMVS), University of Oulu, Finland
2School of Telecommunications Engineering, Technical University of Madrid, Spain
Format: article
Version: accepted version
Access: open
Online Access: PDF Full Text (PDF, 3.4 MB)
Persistent link:
Language: English
Published: Institute of Electrical and Electronics Engineers, 2019
Publish Date: 2019-12-18


We present a novel approach based on Residual Generative Adversarial Network (R-GAN) to discriminate genuine pain expression from posed pain expression by magnifying the subtle changes in the face. In addition to the adversarial task, the discriminator network in R-GAN estimates the intensity level of the pain. Moreover, we propose a novel Weighted Spatiotemporal Pooling (WSP) to capture and encode the appearance and dynamic of a given video sequence into an image map. In this way, we are able to transform any video into an image map embedding subtle variations in the facial appearance and dynamics. This allows using any pre-trained model on still images for video analysis. Our extensive experiments show that our proposed framework achieves promising results compared to state-of-the-art approaches on three benchmark databases, i.e., UNBC-McMaster Shoulder Pain, BioVid Head Pain, and STOIC.

see all

ISBN: 978-1-7281-0089-0
ISBN Print: 978-1-7281-0090-6
Pages: 1 - 8
DOI: 10.1109/FG.2019.8756540
Host publication: 14th IEEE International Conference on Automatic Face and Gesture Recognition, FG 2019, 14-18 May 2019, Lille, France
Conference: IEEE International Conference on Automatic Face and Gesture Recognition
Type of Publication: A4 Article in conference proceedings
Field of Science: 113 Computer and information sciences
Funding: The financial support of the Academy of Finland, Infotech Oulu, Nokia Foundation, and Tauno Tönning Foundation is acknowledged.
Copyright information: © 2019 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.