Camera-based Recovery of Cardiovascular Signals from Unconstrained Face Videos using an Attention Network

Yogesh Deshpande, Surendrabikram Thapa, Abhijit Sarkar

Abstract

This paper addresses the problem of recovering the shape morphology of blood volume pulse (BVP) information from a video of a person’s face. Video-based remote plethysmography methods have shown promising results in estimating vital signs such as heart rate and breathing rate. However, recovering the instantaneous pulse rate signals is still a challenge for the community. This is due to the fact that most of the previous methods concentrate on capturing the temporal average of the cardiovascular signals. In contrast, we present an approach in which BVP signals are extracted with a focus on the recovery of the signal shape morphology as a generalized form for the computation of physiological metrics. We also place emphasis on allowing natural movements by the subject. Furthermore, our system is capable of extracting individual BVP instances with sufficient signal detail to facilitate candidate re-identification. These improvements have resulted in part from the incorporation of a robust skin-detection module into the overall imaging-based photoplethysmography (iPPG) framework. We present extensive experimental results using the challenging UBFC-Phys dataset and the well-known COHFACE dataset.

Yogesh Deshpande, Surendrabikram Thapa, Abhijit Sarkar, A. Lynn Abbott: Camera-based Recovery of Cardiovascular Signals from Unconstrained Face Videos using an Attention Network. CVPR Workshops 2023: 5975-5984