University of Oulu

Z. Yu, X. Li and G. Zhao, "Facial-Video-Based Physiological Signal Measurement: Recent advances and affective applications," in IEEE Signal Processing Magazine, vol. 38, no. 6, pp. 50-58, Nov. 2021, doi: 10.1109/MSP.2021.3106285

Facial-video-based physiological signal measurement : recent advances and affective applications

Saved in:
Author: Yu, Zitong1; Li, Xiaobai1; Zhao, Guoying2,1
Organizations: 1Center for Machine Vision and Signal Analysis, University of Oulu, Oulu, 90014, Finland
2School of Information and Technology, Northwest University, PRC
Format: article
Version: accepted version
Access: open
Online Access: PDF Full Text (PDF, 1.2 MB)
Persistent link:
Language: English
Published: Institute of Electrical and Electronics Engineers, 2021
Publish Date: 2021-12-09


Monitoring physiological changes [e.g., heart rate (HR), respiration, and HR variability (HRV)] is important for measuring human emotions. Physiological responses are more reliable and harder to alter compared to explicit behaviors (such as facial expressions and speech), but they require special contact sensors to obtain. Research in the last decade has shown that photoplethysmography (PPG) signals can be remotely measured (rPPG) from facial videos under ambient light, from which physiological changes can be extracted. This promising finding has attracted much interest from researchers, and the field of rPPG measurement has been growing fast. In this article, we review current progress on intelligent signal processing approaches for rPPG measurement, including earlier works on unsupervised approaches and recently proposed supervised models, benchmark data sets, and performance evaluation. We also review studies on rPPG-based affective applications and compare them with other affective computing modalities. We conclude this article by emphasizing the current main challenges and highlighting future directions.

see all

Series: IEEE signal processing magazine
ISSN: 1053-5888
ISSN-E: 1558-0792
ISSN-L: 1053-5888
Volume: 38
Issue: 6
Pages: 50 - 58
DOI: 10.1109/MSP.2021.3106285
Type of Publication: A1 Journal article – refereed
Field of Science: 113 Computer and information sciences
Funding: This work was supported by the National Natural Science Foundation of China (grant 61772419) and the Academy of Finland (grants 316765 and 323287).
Academy of Finland Grant Number: 316765
Detailed Information: 316765 (Academy of Finland Funding decision)
323287 (Academy of Finland Funding decision)
Copyright information: © 2021 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.