University of Oulu

C. Á. Casado, M. L. Cañellas and M. B. López, "Depression Recognition using Remote Photoplethysmography from Facial Videos," in IEEE Transactions on Affective Computing, doi: 10.1109/TAFFC.2023.3238641

Depression recognition using remote photoplethysmography from facial videos

Saved in:
Author: Álvarez Casado, Constantino1; Lage Cañellas, Manuel1; Bordallo López, Miguel1,2
Organizations: 1Center for Machine Vision and Signal Analysis (CMVS), University of Oulu, 90570 Oulu, Finland
2VTT Technical Research Centre of Finland Ltd, 90571 Oulu, Finland
Format: article
Version: published version
Access: open
Online Access: PDF Full Text (PDF, 8.3 MB)
Persistent link:
Language: English
Published: Institute of Electrical and Electronics Engineers, 2023
Publish Date: 2023-03-27


Depression is a mental illness that may be harmful to an individual’s health. The detection of mental health disorders in the early stages and a precise diagnosis are critical to avoid social, physiological, or psychological side effects. This work analyzes physiological signals to observe if different depressive states have a noticeable impact on the blood volume pulse (BVP) and the heart rate variability (HRV) response. Although typically, HRV features are calculated from biosignals obtained with contact-based sensors such as wearables, we propose instead a novel scheme that directly extracts them from facial videos, just based on visual information, removing the need for any contact-based device. Our solution is based on a pipeline that is able to extract complete remote photoplethysmography signals (rPPG) in a fully unsupervised manner. We use these rPPG signals to calculate over 60 statistical, geometrical, and physiological features that are further used to train several machine learning regressors to recognize different levels of depression. Experiments on two benchmark datasets indicate that this approach offers comparable results to other audiovisual modalities based on voice or facial expression, potentially complementing them. In addition, the results achieved for the proposed method show promising and solid performance that outperforms hand-engineered methods and is comparable to deep learning-based approaches.

see all

Series: IEEE transactions on affective computing
ISSN: 2371-9850
ISSN-E: 1949-3045
ISSN-L: 2371-9850
Issue: Online first
DOI: 10.1109/taffc.2023.3238641
Type of Publication: A1 Journal article – refereed
Field of Science: 113 Computer and information sciences
217 Medical engineering
3124 Neurology and psychiatry
Funding: This research has been supported by the Academy of Finland 6G Flagship program under Grant 346208 and PROFI5 HiDyn program under Grant 326291.
Academy of Finland Grant Number: 346208
Detailed Information: 346208 (Academy of Finland Funding decision)
Copyright information: © The Author(s) 2023. This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see