University of Oulu

Niu X., Yu Z., Han H., Li X., Shan S., Zhao G. (2020) Video-Based Remote Physiological Measurement via Cross-Verified Feature Disentangling. In: Vedaldi A., Bischof H., Brox T., Frahm JM. (eds) Computer Vision – ECCV 2020. ECCV 2020. Lecture Notes in Computer Science, vol 12347. Springer, Cham.

Video-based remote physiological measurement via cross-verified feature disentangling

Saved in:
Author: Niu, Xuesong1,2; Yu, Zitong3; Han, Hu1,4;
Organizations: 1Key Laboratory of Intelligent Information Processing of Chinese Academy of Sciences (CAS), Institute of Computing Technology, CAS, Beijing 100190, China
2University of Chinese Academy of Sciences, Beijing, 100049, China
3Center for Machine Vision and Signal Analysis, University of Oulu, Finland
4Peng Cheng Laboratory, Shenzhen, 518055, China
Format: article
Version: accepted version
Access: open
Online Access: PDF Full Text (PDF, 1.5 MB)
Persistent link:
Language: English
Published: Springer Nature, 2020
Publish Date: 2021-02-19


Remote physiological measurements, e.g., remote photoplethysmography (rPPG) based heart rate (HR), heart rate variability (HRV) and respiration frequency (RF) measuring, are playing more and more important roles under the application scenarios where contact measurement is inconvenient or impossible. Since the amplitude of the physiological signals is very small, they can be easily affected by head movements, lighting conditions, and sensor diversities. To address these challenges, we propose a cross-verified feature disentangling strategy to disentangle the physiological features with non-physiological representations, and then use the distilled physiological features for robust multi-task physiological measurements. We first transform the input face videos into a multi-scale spatial-temporal map (MSTmap), which can suppress the irrelevant background and noise features while retaining most of the temporal characteristics of the periodic physiological signals. Then we take pairwise MSTmaps as inputs to an autoencoder architecture with two encoders (one for physiological signals and the other for non-physiological information) and use a cross-verified scheme to obtain physiological features disentangled with the non-physiological features. The disentangled features are finally used for the joint prediction of multiple physiological signals like average HR values and rPPG signals. Comprehensive experiments on different large-scale public datasets of multiple physiological measurement tasks as well as the cross-database testing demonstrate the robustness of our approach.

see all

Series: Lecture notes in computer science
ISSN: 0302-9743
ISSN-E: 1611-3349
ISSN-L: 0302-9743
ISBN: 978-3-030-58536-5
ISBN Print: 978-3-030-58535-8
Pages: 295 - 310
DOI: 10.1007/978-3-030-58536-5_18
Host publication: Computer Vision – ECCV 2020
Host publication editor: Vedaldi, Andrea
Bischof, Horst
Brox, Thomas
Frahm, Jan-Michael
Conference: European Conference on Computer Vision
Type of Publication: A4 Article in conference proceedings
Field of Science: 113 Computer and information sciences
Funding: This work is partially supported by National Key R&D Program of China (grant 2018AAA0102501), Natural Science Foundation of China (grant 61672496), the Academy of Finland for project MiGA (grant 316765), project 6+E (grant 323287), ICT 2023 project (grant 328115), and Infotech Oulu.
Academy of Finland Grant Number: 316765
Detailed Information: 316765 (Academy of Finland Funding decision)
323287 (Academy of Finland Funding decision)
328115 (Academy of Finland Funding decision)
Copyright information: © Springer Nature Switzerland AG 2020. This is a post-peer-review, pre-copyedit version of an article published in Computer Vision – ECCV 2020. The final authenticated version is available online at: