46
pHealth 2016 N. Maglaveras and E. Gizeli (Eds.) IOS Press, 2016 © 2016 The authors and IOS Press. All rights reserved. doi:10.3233/978-1-61499-653-8-46
Non-Contact Heart Rate Monitoring Using Lab Color Space Hamidur RAHMANa,1, Mobyen Uddin AHMEDa, and Shahina BEGUM a a School of Innovation, Design and Engineering Mälardalen University, SE-72123 Västerås, Sweden
Abstract. Research progressing during the last decade focuses more on non-contact based systems to monitor Heart Rate (HR) which are simple, low-cost and comfortable to use. Most of the non-contact based systems are using RGB videos which is suitable for lab environment. However, it needs to progress considerably before they can be applied in real life applications. As luminance (light) has significance contribution on RGB videos HR monitoring using RGB videos are not efficient enough in real life applications in outdoor environment. This paper presents a HR monitoring method using Lab color facial video captured by a webcam of a laptop computer. Lab color space is device independent and HR can be extracted through facial skin color variation caused by blood circulation considering variable environmental light. Here, three different signal processing methods i.e., Fast Fourier Transform (FFT), Independent Component Analysis (ICA) and Principal Component Analysis (PCA) have been applied on the color channels in video recordings and blood volume pulse (BVP) has been extracted from the facial regions. In this study, HR is subsequently quantified and compare with a reference measurement. The result shows that high degrees of accuracy have been achieved compared to the reference measurements. Thus, this technology has significant potential for advancing personal health care, telemedicine and many real life applications such as driver monitoring. Keywords. Heart Rate, Signal Processing, Lab Color Space
1. Introduction Non-contact HR monitoring systems using camera needs to measure color variation in skin caused by cardiac pulse. The two factors that affect the color values are: the first one is the blood volume variations caused by cardiac pulse and the second one is the (temporal) environmental illumination variations during the video recording [1]. Almost all the non-contact methods are applying RGB color facial video which is suitable for laboratory environment or where the illumination of source light is constant [2-4]. In 2011, Poh et al. have proposed an algorithm to extract underlying source signals from R, G, and B bands [5]. The experiments were conducted using built-in webcam (iSight camera) in indoor environment however their method is not real time and unable to extract HR where the ambient light is not constant. Similar experiments were taken place using different cameras for physiological parameters extraction which are referred in [6]. One of the main drawback of these systems are that they are not applicable for outdoor 1
Hamidur Rahman: Mälardalen University, SE-72123 Västerås, Sweden,
[email protected]
H. Rahman et al. / Non-Contact Heart Rate Monitoring Using Lab Color Space
47
environment where light source is not constant. A novel motivation has been observed in 2014 by Xiaobi et al. to reduce environmental illumination from RGB facial images using Normalized Mean Least Square (NLMS) adaptive filter [1]. NMLS filter is very sensitive to the scaling of the input light source and the calculation is also very complex. Another method has been proposed by Zhang et al. to eliminate environmental illuminance during real time HR monitoring using YCbCr images of the facial skin [7]. This paper presents a non-contact HR monitoring system based on Lab color space using a web camera. The environmental illuminance has a significant contribution in the color variation of the body blood. The proposed method could eliminate the illuminance effect during HR extraction from facial images. In this paper, first the RGB color space is converted into Lab color space consisting of three independent signals such as L, a and b where L represents the lightness of the images and a (red/green) and b (yellow/blue) represents the combination of other color channels. These two color channels have been separated from the Lab color space and using those color channels HR has been extracted. Three signal processing algorithm (FFT, ICA and PCA) have been applied on the color channels to extract HR and a high degree of accuracy is attained while comparing with a reference signal. Though this experiment has been conducted in lab environment but it is useful for the applications like [8, 9]. The rest of the paper is organized as follows: chapter 2 describes materials and methods including Data Collection procedures, Feature Extraction, Approach and transformation of RGB color space into Lab color space, chapter 3 presents experimental works. Finally, chapter 3 summarizes the work.
2. Materials and Methods Data acquisition was conducted on 10 participants (8 Male, 2 female) of different ages (25 to 50 years) and skin colors. The experiments were carried out in Lab environment and with a varying amount of ambient sunlight and artificial light. The participants were informed the aim of the study and they seated in front of a laptop computer at a distance of approximately 0.5 m from the built-in webcam (HP HD webcam). During the experiment, participants were asked to keep still, breathe spontaneously, and face the webcam while their video were recorded for 5 minutes at 30 frames per second (fps) with pixel resolution of 640 x 480 and saved in PNG (Portable Network Graphics) format in the laptop. Here, the RGB videos are converted into Lab videos as described in chapter 3 and finally L (luminance) were subtracted from Lab images to build ab pure color images without any environmental illumination. HR was extracted in offline using the converted Lab images and saved in an excel file. Simultaneously, HR was also recorded using ECG sensors and cStress system2 and saved in another excel file. The main features of the Lab image is L, a and b. The system reads image frames one by one and for each image frame Lab features are extracted from the cropped facial image and saved in a temporary database which are used for further processing. Feature extraction for HR monitoring using RGB videos are described in [10, 11]. For the Lab features that have been extracted using RGB videos is shown in Fig. 1.
2
http://stressmedicin.se/neuro-psykofysilogiska-matsystem/cstress-matsystem/.
48
H. Rahman et al. / Non-Contact Heart Rate Monitoring Using Lab Color Space
Figure 1. Step diagram for Feature Extraction and HR monitoring
Three algorithms such as FFT, ICA and PCA have been applied to extract HR. The average of ‘a’ and ‘b’ signals were calculated using the FFT method [12]. For ICA [13], normalized raw traces were decomposed into two independent source signals (a and b) using the joint approximate diagonalization of Eigen matrices (JADE) algorithm. Similarly, normalized raw traces were also decomposed by PCA to find the principal components [14]. Finally, FFT is applied on the selected source signal to obtain the power spectrum. The pulse frequency was designated as the frequency that corresponds to the highest power of the spectrum within an operational frequency band. To extract HR, the number of peaks in frequency domain was calculated for first 50 image frames in the beginning and the required time is recorded. Therefore, HR is calculated as HR = 60*fh bpm (beat per minute) where fh is the extracted frequency of the HR. In the proposed method, HR extraction begins after the first 50 image frames and after that each image frame has been added to the database and the method provides a new HR. The Graphical User Interface (GUI) has been developed using Matlab to monitor the extracted HR. Fig. 2 shows the GUI which displays the Lab Color image, the detected face, pulse peak and current HR using the three methods.
Figure 2. HR Monitoring GUI.
H. Rahman et al. / Non-Contact Heart Rate Monitoring Using Lab Color Space
49
2.1. RGB to Lab Transformation RGB image of an object may look different when they are captured by different camera and different environmental light [8]. On the other hand, Lab color space is device independent where L represents the lightness of the image and the color channels ‘a’ and ‘b’ represents the other color combinations. ‘a’ axis represents red/green opponent color with green at negative a values and red at positive a values. ‘b’ axis represents yellow/blue opponent colors with blue at negative b values and yellow at positive b values. A true neutral gray values at a = 0 and b = 0 and L = 0 represents completely black and L = 100 represents the brightest white [15]. Let, r, g and b are the red, blue and green color values in RGB color space. It’s chromaticity coordinates (xr,yr), ( xg,yg) and ( xb,yb) and it’s reference white coordinate is ( Xw,Yw,Zw). RGB color space is transformed into Lab color space by two steps calculations. First, RGB color space is transformed into XYZ color space [16] and later XYZ color space is converted into Lab color space [17]. These equations have been modified to fit for our expectation and later the algorithm has been implemented using MATLAB. $ 8 ;< Where, M is the transformation matrix and (0 / . ) is the source color values related as: 0 0 8 0 0 0 0
/ / / / / /
. . . . ;< . .
Here, ; 7 (0 7 )0 < ;< )0
0 8
(0 8 )0 0
/ 8
(/ ; 7 (/ 7 )/ < 8 / 8 ;< )/ )/ /
. 8
(. ; 7 (. 7 ). < 8 . 8 ; < ). . ).
0 8
Using the equation (2)-(5), equation (1) implies: 0 0 / 8 0 0 .
/ / /
. . .
-,
1 1 ; < 1
Now Lab color space can be calculated as: 8 3 7 8 ; 2 7 3
