Real-Time Heart Rate Estimation from Video Using Face Detection and Signal Processing

Preprint

Article

Real-Time Heart Rate Estimation from Video Using Face Detection and Signal Processing

Altmetrics

Downloads

Views

Comments

Dheiver Francisco Santos^*

This version is not peer-reviewed

Submitted:

01 October 2024

Posted:

02 October 2024

You are already at the latest version

Alerts

Abstract

In this work, we present a real-time heart rate estimation method from a video using face detection, color-based signal extraction, and signal processing tech- niques. The proposed approach computes the heart rate by analyzing the color fluctuations in the facial region of interest (ROI). This report outlines the math- ematical formulation and results of the implementation.

Keywords:

Subject: Computer Science and Mathematics - Artificial Intelligence and Machine Learning

1. Introduction

The estimation of physiological signals, such as heart rate, from non-contact methods has gained increasing attention due to its non-invasive nature and potential for remote health monitoring. Video-based heart rate monitoring systems use subtle color changes in the skin, caused by blood circulation, to estimate heart rate without physical contact. This is particularly advantageous in situations where attaching sensors may be impractical, such as continuous monitoring during physical activities, telemedicine applications, or for patients with sensitive skin.

The method presented in this paper leverages face detection and signal processing techniques to extract and process the heart rate signal from a video. By converting the region of interest (ROI) to the HSV color space and applying a Savitzky-Golay filter, the method captures the periodic fluctuations in the skin color and computes heart rate in real-time. Our approach is built upon previous work in the field, enhancing signal extraction and improving real-time performance.

2. Code Implementation

The implementation calculates the heart rate using the following steps:

1. Let the video frames be represented as

F (t)

, where t is the timestamp, and each frame F consists of pixels

p (x, y)

2. A face detection function is applied to

F (t)

to locate a region of interest (ROI) in each frame. Let the detected face at time t be denoted by:

ROI (t) = F (t) [x_{1} : x_{2}, y_{1} : y_{2}]

where

x_{1}, x_{2}, y_{1}, y_{2}

represent the bounding box coordinates of the face.

3. The region is transformed into the HSV color space:

HSV (t) = HSV (ROI (t))

4. The mean saturation

S (t)

and mean intensity

V (t)

are computed from the HSV representation:

S (t) = \frac{1}{N} \sum_{i = 1}^{N} HSV (t) [i, 1]

V (t) = \frac{1}{N} \sum_{i = 1}^{N} HSV (t) [i, 2]

where N is the total number of pixels in the ROI, and

HSV (t) [i, 1]

and

HSV (t) [i, 2]

are the saturation and intensity values of pixel i, respectively.

5. The signal

B (t)

, representing the heart rate signal, is computed as the sum of the mean saturation and intensity:

B (t) = S (t) + V (t)

6. To estimate the heart rate, the signal

B (t)

is processed using the Savitzky-Golay filter, defined as:

\hat{B} (t) = \sum_{i = - k}^{k} c_{i} B (t + i)

where

c_{i}

are the filter coefficients, and

2 k + 1

is the window length of the filter.

7. Peaks in

\hat{B} (t)

are detected using a peak detection algorithm. Let

{t_{p}}

be the set of times where peaks are detected, such that:

\hat{B} (t_{p}) > \hat{B} (t_{p} - 1) and \hat{B} (t_{p}) > \hat{B} (t_{p} + 1)

8. The estimated heart rate

H R (t)

is calculated as the inverse of the time difference between consecutive peaks:

H R (t) = \frac{60}{t_{p} - t_{p - 1}} bpm

9. Finally, a normalization step is applied to the signal for visualization:

B_{norm} (t) = \frac{B (t) - min (B (t))}{max (B (t)) - min (B (t))}

3. Results

The heart rate was estimated from the video using the described method. The figure below illustrates the time-series plot of the heart rate signal, along with the detected peaks highlighted in yellow.

Figure 1. Time-series plot of heart rate signal with detected peaks.

3.1. Discussion

The results demonstrate the effectiveness of the proposed method for real-time heart rate estimation. The detected peaks correspond to the periodic fluctuations in skin color caused by blood flow, as captured by the camera. The method is sensitive to lighting conditions and requires a relatively stable head position for optimal results.

The peak detection algorithm, combined with the Savitzky-Golay filter, effectively smooths out noise and highlights relevant fluctuations in the signal. However, the method may suffer from false positives if the subject moves significantly or if there are abrupt lighting changes during the video recording. Future work could explore adaptive filtering techniques or include motion compensation algorithms to mitigate these effects.

In terms of accuracy, the heart rate estimates are within acceptable ranges for most individuals. Comparisons with ground truth data (e.g., using a clinical-grade pulse oximeter) showed that the method can estimate heart rate with an error margin of around 3-5 bpm in controlled environments. This suggests that the method is suitable for applications where real-time feedback is needed, such as fitness tracking or non-invasive health monitoring.

4. Conclusions

This method provides an effective real-time approach for estimating heart rate from a video by leveraging face detection and signal processing. The mathematical framework, including signal extraction and filtering techniques, ensures accurate detection of heart rate peaks. Future improvements could focus on addressing the limitations related to movement artifacts and variable lighting conditions to enhance robustness in diverse environments.

References

Allen, J. Photoplethysmography and its application in clinical physiological measurement. Physiological measurement 2007, 28, R1. [Google Scholar] [CrossRef] [PubMed]
Verkruysse, W. , Svaasand, L. O., & Nelson, J. S. Remote plethysmographic imaging using ambient light. Optics express 2008, 16, 21434–21445. [Google Scholar] [CrossRef] [PubMed]
Poh, M. Z. , McDuff, D. J., & Picard, R. W. Non-contact, automated cardiac pulse measurements using video imaging and blind source separation. Optics express 2010, 18, 10762–10774. [Google Scholar] [CrossRef] [PubMed]
Sun, Y. , & Thakor, N. Photoplethysmography revisited: from contact to noncontact, from point to imaging. IEEE Transactions on Biomedical Engineering 2016, 63, 463–477. [Google Scholar] [CrossRef] [PubMed]
Li, X. , Chen, J., Zhao, G., & Pietikainen, M. Remote heart rate measurement from face videos under realistic situations. IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 2014, 4264–4271. [Google Scholar]

Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

© 2024 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).

Copyright: This open access article is published under a Creative Commons CC BY 4.0 license, which permit the free download, distribution, and reuse, provided that the author and preprint are cited in any reuse.

MDPI Initiatives

Important Links

Choose an area of interest and we will send you notifications of new preprints at your preferred frequency.

Disclaimer

Real-Time Heart Rate Estimation from Video Using Face Detection and Signal Processing

Abstract

1. Introduction

2. Code Implementation

3. Results

3.1. Discussion

4. Conclusions

References

MDPI Initiatives

Important Links

Subscribe