University of Oulu

Z. Yu, W. Peng, X. Li, X. Hong and G. Zhao, "Remote Heart Rate Measurement From Highly Compressed Facial Videos: An End-to-End Deep Learning Solution With Video Enhancement," 2019 IEEE/CVF International Conference on Computer Vision (ICCV), Seoul, Korea (South), 2019, pp. 151-160.

Remote heart rate measurement from highly compressed facial videos : an end-to-end deep learning solution with video enhancement

Saved in:
Author: Yu, Zitong1; Peng, Wei1; Li, Xiaobai1;
Organizations: 1Center for Machine Vision and Signal Analysis, University of Oulu, Finland
2MOE Key Lab. for Intelligent Networks and Network Security Faculty of Electronic and Information Engineering, Xi’an Jiaotong University, PRC
3School of Information and Technology , Northwest University, PRC
Format: article
Version: accepted version
Access: open
Online Access: PDF Full Text (PDF, 1.2 MB)
Persistent link:
Language: English
Published: Springer Nature, 2020
Publish Date: 2020-03-25


Remote photoplethysmography (rPPG), which aims at measuring heart activities without any contact, has great potential in many applications (e.g., remote healthcare). Existing rPPG approaches rely on analyzing very fine details of facial videos, which are prone to be affected by video compression. Here we propose a two-stage, end-to-end method using hidden rPPG information enhancement and attention networks, which is the first attempt to counter video compression loss and recover rPPG signals from highly compressed videos. The method includes two parts: 1) a Spatio-Temporal Video Enhancement Network (STVEN) for video enhancement, and 2) an rPPG network (rPPGNet) for rPPG signal recovery. The rPPGNet can work on its own for robust rPPG measurement, and the STVEN network can be added and jointly trained to further boost the performance especially on highly compressed videos. Comprehensive experiments are performed on two benchmark datasets to show that, 1) the proposed method not only achieves superior performance on compressed videos with high-quality videos pair, 2) it also generalizes well on novel data with only compressed videos available, which implies the promising potential for real-world applications.

see all

Series: Advances in computer vision and pattern recognition
ISSN: 2191-6586
ISSN-E: 2191-6594
ISSN-L: 2191-6586
ISBN: 978-1-7281-4803-8
ISBN Print: 978-1-7281-4804-5
Pages: 151 - 160
DOI: 10.1109/ICCV.2019.00024
Host publication: 2019 IEEE International Conference on Computer Vision (ICCV) : 27th October- 2nd Novenber 2019, Seoul, Korea
Conference: IEEE International Conference on Computer Vision
Type of Publication: A4 Article in conference proceedings
Field of Science: 113 Computer and information sciences
Copyright information: © 2020 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.