Diagonalization of Eigenmatrices (JADE) [14], the FastICA. [15] and the RobustICA [16]. The objective is to perform a comparative evaluation of their accuracy ...
Comparison of Blind Source Separation Algorithms for Optical Heart Rate Monitoring Eirini Christinaki1, Giorgos Giannakakis1, Franco Chiarugi1, Matthew Pediaditis1, Galateia Iatraki1, Dimitris Manousos1, Kostas Marias1, Manolis Tsiknakis1,2 1
Computational Medicine Laboratory, Institute of Computer Science, Foundation for Research and Technology - Hellas, Heraklion, Crete, Greece 2 Department of Informatics Engineering, Technological Educational Institute of Crete, Heraklion, Crete, Greece {echrist, ggian, chiarugi, mped, giatraki, mandim, kmarias, tsiknaki}@ics.forth.gr
Abstract— Monitoring of physiological signals of an individual via remote and contactless means is an important scientific challenge, whose resolution will enable the development of novel, nonintrusive mHealth and wellness-management systems and services. In this paper, the performance of three blind source separation algorithms for the optical estimation of the heart rate have been studied. The objective is to perform a comparative evaluation of their accuracy and convergence capability, for the optical estimation of the heart rate. Keywords- blood volume pulse (BVP); heart rate (HR); photoplethysmography (PPG); independent component analysis (ICA); blind source separation (BSS); remote sensing.
I.
INTRODUCTION
The human face is a precious revealer of key information about the health status of individuals, in form of a combination of physical signs and expressive features. The quantification of facial signs into meaningful measures and computational descriptors can be exploited for the automatic assessment of cardiovascular disease risk and the evaluation of a wellness index [1]. In this context, a patient’s physiological signal monitoring via remote and noncontact means is an important objective. Currently, proposed solutions for contactless measurement of vital signs, such as heart rate (HR) and respiratory rate (RR), include laser Doppler [2], microwave Doppler radar [3], thermal imaging [4] or even the use of the camera of a mobile phone [5]. Noncontact assessment of heart rate variability (HRV) represents a greater challenge and few attempts have been made so far [6]. Photoplethysmography (PPG) is a noninvasive optical technique for measuring blood flow in tissue. This method has widespread clinical application [7], since it can be used to monitor vital body signs such as HR and to provide valuable information about the cardiovascular system. Traditionally, PPG uses dedicated light sources (e.g. red or infrared light) and a photodiode to measure the amount of light either transmitted or reflected. Imaging PPG (iPPG), which replaces the photodiode with a digital camera, is also capable of remotely monitoring blood perfusion based on transmission and reflection modes, respectively [8], [9]. Considering the light sources, Verkruysse et al. [10] reported that remote measurement of the cardiac pulse could be obtained with the use of ambient light as the only illumination source. Furthermore, Sun et al. [11] investigated the
camera-based technique for remote acquisition of PPG signals and the influence of the ambient light intensity on the physiological information. They revealed that, under variable external weather conditions, information contained in the PPG signal, such as HR, is not corrupted by changes in ambient light intensity. However, PPG systems using ambient light and a digital camera are susceptible to motion artifacts [12]. Motion can affect the accuracy of measurements and Blind Source Separation (BSS) techniques have been considered as a solution to overcome this limitation [12], [13]. Nevertheless, the reliability of these methods has not been systematically explored yet, with respect to optical estimation of HR. In this paper, the performances of three BSS algorithms, based on Independent Component Analysis (ICA), have been studied. The selected algorithms are the Joint Approximate Diagonalization of Eigenmatrices (JADE) [14], the FastICA [15] and the RobustICA [16]. The objective is to perform a comparative evaluation of their accuracy and convergence capability, for the optical estimation of HR. II.
RELATED WORK
Typically, physiological signals are recorded from a set of sensors, where each sensor collects information of both the signal we wish to analyze and an unwanted noise component. BSS is one of several methods used for noise or artifact removal from physiological signals [17]. A. Blind source separation for artifact or noise reduction The BSS refers to the recovery of source signals from a set of observed mixtures, with superimposed noise and with no prior information about the mixing process. Based on the assumptions on the source signals and the mixture matrices, different approaches of BSS can be identified such as Independent Component Analysis (ICA), Principal Component Analysis (PCA), Sparse Component Analysis (SCA) and Non-negative Matrix Factorization (NMF). The BSS techniques are used to generate an un-mixing matrix for the estimation of the original sources and consequently the removal of the sources representing the artifacts. The most commonly employed methods are ICA and PCA. ICA decomposes mixtures of source signals into statistically independent, or as independent as possible, components. On the
other hand, PCA uses an orthogonal matrix composed by the eigenvectors of the covariance matrix of the original variables in order to identify the principal components, a small subset of which, when some of the original variables are correlated, is able to describe a very large variation of the original data. Both of these methods have been successfully applied for noise or artifact removal to electroencephalographic (EEG) [18], [19] electrocardiographic (ECG) [20], [21] and functional magnetic resonance imaging (fMRI) data [22]. B. Blind source separation in noncontact methods for heart rate measurement In recent years, simple and low cost methods have been employed for noncontact and remote measurement of physiological parameters. Researchers have begun to explore the use of video imaging and PPG for HR estimation. While the initial results appear encouraging, there are drawbacks including susceptibility to motion. The reflected plethysmographic signal acquired from the camera is a mixture of the physiological pulse along with other sources of fluctuation in light due to motion. The main reported attempts to reduce the motion artifacts from iPPG signals include different BSS techniques.
The experiments were conducted indoors, at different times of the day, with a varying amount of indirect sunlight and fluorescent lights as the only sources of illumination. The participants were seated at a distance of about 0.5m from the cameras. Seven subjects (3 males and 4 females) participated at the recording, out of which two wore glasses and had light beard. The participants were asked to sit naturally and stare at the cameras during video capturing. They were free to make facial expressions and to move their head or body slightly while remaining seated. Two facial videos, each lasting one minute, were simultaneously recorded for all participants from the cameras. One participant was also recorded after a running session. In total, 16 separate videos were stored for analysis (6 subjects x 2 videos + 1 subject x 4 videos). B. Processing pipeline for heart rate estimation Heart rate estimation from the recorded videos was performed implementing the method described by Poh et al. [13] in MATLAB environment. The processing pipeline is graphically depicted in Figure 1.
Poh et al. [13] developed a method for HR measurement from color video recordings of the human face. This study reported primary attempts to remove/attenuate motion artifacts from the PPG signal. Their method was based on BSS applied to the three color channels (RGB), in order to produce independent components with the use of the JADE algorithm. They also extended this methodology and obtained additional physiological parameters [23]. Specifically, they extracted the blood volume pulse (BVP) waveform and determined apart from HR, the HRV and the RR. The JADE implementation of ICA has also been used in [24], [25] and [26] to successfully recover source signals from the observations. In addition, Lewandowska et al. [27] managed to extract the pulse component and obtain the HR from PCA and FastICA with similar accuracy. III.
METHODS
The algorithms compared in this study are JADE, FastICA and RobustICA. These algorithms are variations of ICA regarding the criteria employed to identify the unique character of the individual unmixed components. The JADE algorithm uses fourth order moments of cumulant matrices in order to diagonalize eigenmatrices, FastICA uses a fixed-point iterative algorithm utilizing negentropy in order to maximize the nonGaussianity of the sources and RobustICA is a modification of FastICA, which optimizes the kurtosis contrast by using an optimal step size, leading to a higher convergence speed. A. Experimental setup The experimental setup consisted of a built-in laptop camera (iSight on an Apple Macbook Pro) and a basic webcam (Logitech 720p HD). All videos were recorded at 30 frames per second (fps), in 24-bit RGB color with a resolution of 640×480 pixels, and saved in uncompressed Audio Video Interleave (AVI) format. In addition, the experimental setup included a commercially available fingertip pulse oximeter (Heal Force Prince-100A), which was used to obtain the participant’s reference cardiac pulse in parallel with video recordings.
Figure 1. The processing pipeline leading to the estimation of the Independent Components by applying the JADE, FastICA and RobustICA algorithms (written consent has been obtained from the subject whose photo is in this figure).
Initially, a region of interest (ROI) was detected for each frame on the right cheek of the participant. The video in this ROI
TABLE I. EXPERIMENTAL RESULTS OBTAINED FROM THE ANALYSIS OF VIDEOS RECORDED WITH THE ISIGHT AND THE LOGITECH CAMERAS
FastICA
RobustICA
0.62 53.47 1.84 1.83 2.54 4.45 3.74 0.54
0.62 53.47 1.84 1.83 2.54 4.45 0.58 0.54
32.49 53.47 1.84 1.83 2.54 4.45 0.58 0.54
3.19 0.82 1.84 1.84 2.54 4.45 0.59 0.54
3.19 0.82 1.84 0.45 2.54 0.59 0.54
3.19 50.88 1.84 1.84 2.54 4.45 0.59 0.54
8.63
8.24
12.22
1.98
1.42
8.23
JADE
RobustICA
69.00 102.00 63.00 77.00 78.00 69.00 61.00 76.00 Average percent error
PERCENT ERROR (RGB) (CAMERA B -LOGITECH)
FastICA
PERCENT ERROR (RGB) (CAMERA A -ISIGHT) JADE
Figure 2 shows an example of the power spectrum obtained through the analysis of the recorded signal for one subject. The frequency related to the maximum power was 1.069 Hz, which corresponded to HR=64.14 bpm. The result was in very good agreement with the measured reference cardiac pulse.
increment in the errors, observed in iSight compared to the Logitech, reveals that results are camera dependent.
Measured HR
was then divided into the three color channels (RGB). The pixel values for each color channel in each frame were spatially averaged and the resulting frame-to-frame time series of the RGB signals formed the three raw traces, which were subsequently de-trended and normalized (zero mean, unit variance). The resulting raw signals contained pulse and motion noise. Thus, subsequent processing was performed for motion artifact removal by applying the three different aforementioned BSS-ICA methods to decompose the normalized raw traces into independent components. Then, Fast Fourier Transform (FFT) was computed for each component to obtain their power spectrums. The component whose power spectrum contained the highest peak was exploited for the HR measurement. The frequency with the highest power in the band 0.75-4 Hz (corresponding to 45-240 beats per minute (bpm)), was finally used as the estimated HR.
According to previous studies [10], [11], the green channel features the strongest plethysmographic signal. Thus, a further analysis was performed on the raw traces recorded with both cameras bypassing the BSS stage. The results, reported in Table II, are in agreement with these studies, as the outcomes show that the green channel was largely more selected. Only in very few cases, better results were obtained through the selection of another channel meaning that a reliable HR can be measured by exploiting only the green channel. Finally, it is worthy to note that in few cases there is a very light improvement when using separation techniques, while in some others the percent error remains unchanged.
However, FastICA, in one of our cases, failed to initialize properly and was not able to produce results. It is also important to mention that the worst results obtained with RobustICA are mainly due to the bad result yielded in just one case. This specific video presented the subject under investigation with heavy head motion, which constitutes a common problem for these algorithms. If we exclude this case, the outcomes are comparable with the previous ones. Finally, the discernible
V.
Estimated HR
Percent error
Selected channel
𝑜𝑏𝑠𝑒𝑟𝑣𝑒𝑑 𝑣𝑎𝑙𝑢𝑒 − 𝑡𝑟𝑢𝑒 𝑣𝑎𝑙𝑢𝑒 | ∗ 100 𝑡𝑟𝑢𝑒 𝑣𝑎𝑙𝑢𝑒 The results from the two cameras presented in Table I indicate that both JADE and FastICA performed well with the FastICA algorithm performing slightly better. 𝑝𝑒𝑟𝑐𝑒𝑛𝑡 𝑒𝑟𝑟𝑜𝑟 = |
69.00 68.55 102.00 100.19 63.00 64.16 77.00 76.46 78.00 79.98 69.00 72.07 61.00 60.64 76.00 75.59 Average percent error
Selected channel
An analysis was performed on the videos obtained from cameras. The HR was computed by following the procedure described in Section III and the results were compared with the fingertip pulse oximeter’s measurements. For our analysis, we have decided to adopt the percent error defined as:
Logitech camera
Percent error
RESULTS
iSight camera
Estimated HR
IV.
TABLE II. EXPERIMENTAL RESULTS OBTAINED WHEN USING THE RAW TRACES BYPASSING THE BSS STAGE ON THE VIDEOS RECORDED WITH BOTH CAMERAS
Measured HR
Figure 2. Result of applying FFT to one of the components produced by the BSS stage.
0.65 1.77 1.84 0.70 2.54 4.45 0.59 1.70 1.78
Green Green Green Blue Green Green Green Green
66.80 71.20 64.16 77.34 79.98 72.07 60.64 75.59
1.90 20.64 1.84 0.44 2.54 4.45 0.59 0.54 4.12
Green Red Green Green Green Green Green Green
CONCLUSIONS
Based on the results obtained from the present study, we have demonstrated the feasibility of using a simple webcam to estimate HR. It is evident from the experimental data that the type of camera used does have an effect on the observed accuracy of estimation, although the changes are not relevant in the large majority of cases.
The fact that the choice of algorithm does not significantly influence the estimates is interesting. Percentage errors range between 5 and 0.5 % when excluding the outlier value of 102 bpm from analysis. It is remarkable that all algorithms, at least in one case, present inadequate response in videos of high HR with notable head motion (related to the recording after physical exercise). This needs to be studied more extensively and its physical origins required explanation. An assumption may be that the algorithm for the channel selection has to be improved specially in cases with low signal to noise ratio (SNR), which in our experiment seems to be the case with the high HR (high head motion), and techniques for enhancing SNR must be applied in the process pipeline before the application of BSS. These results must be considered in the light of several limitations. Such limitations include: a) the inaccuracy of the fingertip pulse oximeter, b) the limited amount of subjects and measures on the available subjects, c) the uncontrolled head motion which may contaminate with high noise the methods (investigations using controlled movements at different velocities should be performed in order to evaluate the capabilities of the various algorithms of filtering the motion artifacts and to understand the amount of movements that are permitted in order not to skew results) and finally d) the distance from camera and camera resolutions may be further studied since they are parameters that potentially could alter the results. Our findings should motivate extensive validation and continued systematic exploration of these variables.
[8]
[9]
[10]
[11]
[12]
[13]
[14] [15]
[16]
[17]
[18]
ACKNOWLEDGEMENT This work was performed in the framework of the FP7 Specific Targeted REsearch Project SEMEOTICONS (SEMEiotic Oriented Technology for Individual’s CardiOmetabolic risk self-assessmeNt and Self-monitoring) partially funded by the European Commission under Grant Agreement 611516.
[19]
[20]
REFERENCES [1]
[2]
[3]
[4]
[5]
[6]
[7]
F. Chiarugi, et al., "Facial signs and psycho-physical status estimation for well-being assessment," in Proceedings of the 7th International Conference on Health Informatics (HEALTHINF), pp. 555-562, 2014. S.S. Ulyanov, and V.V. Tuchin, "Pulse-wave monitoring by means of focused laser beams scattered by skin surface and membranes," in Proceedings of the International Society of Optics and Photonics (SPIE), vol. 1884, pp. 160-167, 1993. E.F. Greneker, "Radar sensing of heartbeat and respiration at a distance with applications of the technology," in IEE Radar 449, pp. 150-154, 1997. M. Garbey, N. Sun, A. Merla, and I. Pavlidis, "Contact-free measurement of cardiac pulse based on the analysis of thermal imagery," IEEE Trans Biomed Eng., vol. 54, issue 8, pp. 1418-1426, 2007. P. Pelegris, K. Banitsas, T. Orbach, and K. Marias, "A novel method to detect heart beat rate using a mobile phone," in Proceedings of the 32nd Annual International Conference of the IEEE EMBS, pp. 5488-5491, 2010. U. Morbiducci, L. Scalise, M. De Melis, and M. Grigioni, "Optical vibrocardiography: a novel tool for the optical monitoring of cardiac activity," Ann Biomed Eng., vol. 35, issue 1, pp. 45-58, 2007. J. Allen, "Photoplethysmography and its application in clinical physiological measurement," Physiol Meas., vol. 28, no. 3, pp. R1-R39, 2007.
[21]
[22]
[23]
[24]
[25]
[26]
[27]
J. Zheng, and S. Hu, "The preliminary investigation of imaging photoplethysmographic system," J Phys: Conf S., vol. 85, issue 1, 012031, 2007. S. Hu, J. Zheng, V. Chouliaras, and R. Summers, "Feasibility of imaging photoplethysmography," in Proceedings of the International Conference on BioMedical Engineering and Informatics (BMEI), vol. 2, pp. 72-75, 2008. W. Verkruysse, L.O. Svaasand, and J.S. Nelson, "Remote plethysmographic imaging using ambient light," Opt Express, vol. 16, issue 26, pp. 21434-21445, 2008. Y. Sun, et al., "Use of ambient light in remote photoplethysmographic systems: comparison between a high-performance camera and a low-cost webcam," J Biomed Opt., vol. 17, issue 3, 037005, 2012. Y. Sun, et al., "Motion-compensated noncontact imaging photoplethysmography to monitor cardiorespiratory status during exercise," J Biomed Opt., vol. 16, issue 7, 077010, 2011. M.-Z. Poh, D.J. McDuff, and R.W. Picard, "Non-contact, automated cardiac pulse measurements using video imaging and blind source separation," Opt Express, vol. 18, issue 10, pp. 10762-10774, 2010. J.-F. Cardoso, and A. Souloumiac, "Blind beamforming for non-Gaussian signals," IEE Proceedings-F, vol. 140, issue 6, pp. 362-370, 1993. A. Hyvärinen, and E. Oja, "A fast fixed-point algorithm for independent component analysis," Neural Comput., vol. 9, issue 7, pp. 1483-1492, 1997. V. Zarzoso, and P. Comon, "Robust independent component analysis by iterative maximization of the kurtosis contrast with algebraic optimal step size," IEEE Trans Neural Netw., vol. 21, issue 2, pp. 248-261, 2010. K.T. Sweeney, T.E. Ward, and S.F. McLoone, "Artifact removal in physiological signals—practices and possibilities," IEEE Trans Inf Technol Biomed., vol. 16, issue 3, pp. 488-500, 2012. T.D. Lagerlund, F.W. Sharbrough, and N.E. Busacker, "Spatial filtering of multichannel electroencephalographic recordings through principal component analysis by singular value decomposition," J Clin Neurophysiol., vol. 14, issue 1, pp. 73-82, 1997. M. Antonakakis, G. Giannakakis, M. Tsiknakis, S. Micheloyannis, and M. Zervakis, "Synchronization coupling investigation using ICA cluster analysis in resting MEG signals in reading difficulties," in Proceedings of the IEEE 13th International Conference on Bioinformatics and Bioengineering (BIBE), pp. 1-5, 2013. T. He, G. Clifford, and L. Tarassenko, "Application of independent component analysis in removing artefacts from the electrocardiogram," Neural Comput Appl., vol. 15, issue 2, pp. 105-116, 2006. H. Zhang, and L.-Q. Zhang, "ECG analysis based on PCA and support vector machines," in Proceedings of the International Conference on Neural Networks and Brain (ICNN&B), vol. 2, pp. 743-747, 2005. N. Correa, T. Adali, and V.D. Calhoun, "Performance of blind source separation algorithms for fMRI analysis using a group ICA method," Magn Reson Imaging, vol. 25, issue 5, pp. 684-694, 2007. M.-Z. Poh, D.J. McDuff, and R.W. Picard, "Advancements in noncontact, multiparameter physiological measurements using a webcam," IEEE Trans Biomed Eng., vol. 58, issue 1, pp. 7-11, 2011. S. Kwon, H. Kim, and K.S. Park, "Validation of heart rate extraction using video imaging on a built-in camera system of a smartphone," in Proceedings of the 34th Annual International Conference of the IEEE EMBS, pp. 2174-2177, 2012. D. Datcu, M. Cidota, S. Lukosch, and L. Rothkrantz, "Noncontact automatic heart rate analysis in visible spectrum by specific face regions," in Proceedings of the 14th International Conference on Computer Systems and Technologies (CompSysTech), pp. 120-127, 2013. D. McDuff, S. Gontarek, and R.W. Picard, "Improvements in remote cardio-pulmonary measurement using a five band digital camera," IEEE Trans Biomed Eng., vol. PP, issue 99, 2014. M. Lewandowska, J. Ruminski, T. Kocejko, and J. Nowak, "Measuring pulse rate with a webcam—a non-contact method for evaluating cardiac activity," in Proceedings of the Federated Conference on Computer Science and Information Systems (FedCSIS), pp. 405-410, 2011.
