University of Oulu

Sabokrou, M., Pourreza, M., Li, X., Fathy, M., & Zhao, G. (2021). Deep-HR: Fast heart rate estimation from face video under realistic conditions. Expert Systems with Applications, 186, 115596.

Deep-HR : fast heart rate estimation from face video under realistic conditions

Saved in:
Author: Sabokrou, Mohammad1,2; Pourreza, Masoud1; Li, Xiaobai2;
Organizations: 1School of Computer Science, Institute for Research in Fundamental Sciences (IPM), Tehran, IRAN
2CMVS, University of Oulu, Finland
Format: article
Version: accepted version
Access: open
Online Access: PDF Full Text (PDF, 5.7 MB)
Persistent link:
Language: English
Published: Elsevier, 2021
Publish Date: 2023-07-31


This paper presents a novel method for remote heart rate (HR) estimation. Recent studies have proved that blood pumping by the heart is highly correlated to the intense color of face pixels, and surprisingly can be utilized for remote HR estimation. Researchers successfully proposed several methods for this task, but making it work in realistic situations is still a challenging problem in computer vision community. Furthermore, learning to solve such a complex task on a dataset with very limited annotated samples is not reasonable. Consequently, researchers do not prefer to use the deep learning approaches for this problem. In this paper, we propose a simple yet efficient approach to benefit the advantages of the Deep Neural Network (DNN) by simplifying HR estimation from a complex task to learning from very correlated representation to HR. Inspired by previous work, we learn a component called Front-End (FE) to provide a discriminative representation of face videos, afterward a light deep regression auto-encoder as Back-End (BE) is learned to map the FE representation to HR. Regression task on the informative representation is simple and could be learned efficiently on limited training samples. Beside of this, to be more accurate and work well on low-quality videos, two deep encoder–decoder networks are trained to refine the output of FE. We also introduce a challenging dataset (HR-D) to show that our method can efficiently work in realistic conditions. Experimental results on HR-D and MAHNOB datasets confirm that our method could run as a real-time method and estimate the average HR better than state-of-the-art ones.

see all

Series: Expert systems with applications
ISSN: 0957-4174
ISSN-E: 1873-6793
ISSN-L: 0957-4174
Volume: 186
Article number: 115596
DOI: 10.1016/j.eswa.2021.115596
Type of Publication: A1 Journal article – refereed
Field of Science: 113 Computer and information sciences
Copyright information: © 2021 Published by Elsevier Ltd. This manuscript version is made available under the CC-BY-NC-ND 4.0 license