Event Recognition for Contactless Activity Monitoring ... - IEEE Xplore

IEEE TRANSACTIONS ON BIOMEDICAL ENGINEERING, VOL. 64, NO. 2, FEBRUARY 2017

479

Event Recognition for Contactless Activity Monitoring Using Phase-Modulated Continuous Wave Radar Mohamad Forouzanfar∗ , Member, IEEE, Mohamed Mabrouk, Member, IEEE, Sreeraman Rajan, Senior Member, IEEE, Miodrag Bolic, Senior Member, IEEE, Hilmi R. Dajani, Senior Member, IEEE, and Voicu Z. Groza, Fellow, IEEE

Abstract—Objectives: The use of remote sensing technologies such as radar is gaining popularity as a technique for contactless detection of physiological signals and analysis of human motion. This paper presents a methodology for classifying different events in a collection of phase modulated continuous wave radar returns. The primary application of interest is to monitor inmates where the presence of human vital signs amidst different, interferences needs to be identified. Methods: A comprehensive set of features is derived through time and frequency domain analyses of the radar returns. The Bhattacharyya distance is used to preselect the features with highest class separability as the possible candidate features for use in the classification process. The uncorrelated linear discriminant analysis is performed to decorrelate, denoise, and reduce the dimension of the candidate feature set. Linear and quadratic Bayesian classifiers are designed to distinguish breathing, different human motions, and nonhuman motions. The performance of these classifiers is evaluated on a pilot dataset of radar returns that contained different events including breathing, stopped breathing, simple human motions, and movement of fan and water. Results: Our proposed pattern classification system achieved accuracies of up to 93% in stationary subject detection, 90% in stop-breathing detection, and 86% in interference detection. Conclusion: Our proposed radar pattern recognition system was able to accurately distinguish the predefined events amidst interferences. Significance: Besides inmate monitoring and suicide attempt detection, this paper can be extended to other radar applications such as home-based monitoring of elderly people, apnea detection, and home occupancy detection. Index Terms—Classification, contactless sensing, interference, phase-modulated radar, physiological monitoring.

Manuscript received July 28, 2015; revised April 19, 2016; accepted May 5, 2016. Date of publication May 11, 2016; date of current version January 18, 2017. This work was supported in part by Mitacs Canada, National Research Council’s Industrial Research Assistance Program, Correctional Service Canada, and K&G Spectrum, Inc. Asterisk indicates corresponding author. * M. Forouzanfar was with the School of Electrical Engineering and Computer Science, University of Ottawa, Ottawa, ON K1N 6N5, Canada. He is now with the Department of Electrical Engineering, Stanford University, Stanford, CA 94305 USA (e-mail: [email protected]). M. Mabrouk, M. Bolic, H. R. Dajani, and V. Z. Groza are with the School of Electrical Engineering and Computer Science, University of Ottawa. S. Rajan is with the Department of Systems and Computer Engineering, Carleton University. Digital Object Identifier 10.1109/TBME.2016.2566619

I. INTRODUCTION ONINVASIVE sensing has evolved in the last decade as a preferred sensing system for applications such as institutional and private home-based monitoring of elderly people, detection of attempted suicide events in prisons, and monitoring of respiration and heart activity. Technologies such as imaging and video surveillance cameras [1] and electronic tags equipped with RF transmitters [2], [3] have been employed for surveillance but are intrusive or violate privacy. Electronic tags such as bracelet and ankle monitors are traditionally used to monitor and locate elderly persons, patients, and inmates [2], [3]. Recently, sensors have been used on bracelets to measure the physiological signals and detect events like suicide attempts [4]. However, bracelets are prone to removal and manipulation; hence may not be acceptable for long-term use. Therefore, alternate contactless technologies that can provide nonintrusive monitoring are needed. Recently, remote human detection and monitoring using radar systems have found important applications in search and rescue for earthquake or fire victims [5], [6], border patrol and entrance security [7], gait analysis [8], [9], fall detection [10], [11], and other human activities [12]–[15]. Moreover, radars can also provide valuable information about a person’s state of health by monitoring cardiopulmonary and other vital signs [16]–[28]. Depending on the application, different radar technologies can be used [29]. A pure continuous wave (CW) radar can readily detect moving objects via the Doppler shift in the radar signal returns. However, it cannot detect the range of the moving object. A frequency or phase modulated continuous wave (PMCW) radar can detect a moving object and also estimate its range. CW radars are not sensitive to stationary or slow-moving clutter and, therefore, are preferred in health applications for detecting motion due to heartbeat and respiration [30]. In [18]–[21], different architectures of CW radars were designed for remote monitoring of human vital signs. Spectral analysis using the Fourier transform was applied to detect the respiration and heart fundamental frequencies [20], [21]. In [23], an infant monitoring system was designed using CW radar which was able to monitor the infant breathing rate. Similar to [20] and [21], spectral analysis was performed for breathing rate estimation in [23]. In [24] and [25], CW radar was used for

N

0018-9294 © 2016 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications standards/publications/rights/index.html for more information.

480


assessment of heart rate variability (HRV) and respiratory sinus arrhythmia. The heart rate was estimated by searching for the local peaks of radar returns using the autocorrelation method. In [13], support vector machines (SVM) were used to classify the CW radar returns belonging to seven human activities including running, walking, walking while holding a stick, crawling, boxing while moving forward, boxing while standing in place, and sitting still. In [9], [12], and [8], time–frequency analysis was performed to characterize the radar returns belonging to different human motions. A hierarchical image classification method based on SVMs was developed in [8], to classify the spectrograms of the radar returns. In [31], a frequencymodulated CW radar was used to scan a subject. Principal component analysis and linear discriminant analysis (LDA) were used to extract features from the spectrogram of the radar returns, and SVMs were applied to classify different human activities. In [14], systems and methods were proposed for monitoring of a subject in resting state using pulsed radars. The radar returns were filtered into different frequency bands representing the respiration, heart activity, and motion. Features such as minimum, maximum, average, and root-mean-square of the filtered radar signals were extracted. Using the extracted features, motion parameters including levels of activity and physiological parameters including respiration and heart rate were determined. In [15] and [32], a physiological monitoring and alerting system using pulsed radars was developed. Similar to [14], the radar returns were first filtered into several frequency bands and features were extracted. Learning algorithms were then used to detect empty room, motion, heart activity, and respiration. Alerts were generated when the heart or respiration rate were out of an accepted range. Other feature extraction techniques from radar returns are based on entropy [33], wavelet packets [34], and time–frequency analysis using the S-method [35]. While there have been several studies on feature extraction from radar returns and pattern classification, a detailed study to derive a comprehensive feature set that can distinguish between different human vital signs and activities in the presence of interferences is still lacking. Furthermore, a general framework for development of a radar pattern classification system that appropriately processes the radar returns, extracts and selects the features, and determines the correct classes for the radar patterns needs to be developed. Although several patents [10], [14], [15] exist for feature extraction and learning algorithms in monitoring of subjects, details, and evaluation are never presented; hence, it is impossible to compare the merits and demerits of these algorithms or analyze, evaluate, and validate these algorithms. Radar recognition systems have to perform well in the presence of different interferences that may affect the radar returns. Often additional sensor systems are employed for dealing with interferences such as motion of the targets. Cameras [36], accelerometers [37], RF tags [38], or additional radar systems [39] have been used for this purpose. However, the use of an additional sensor is not always feasible. In [40], a dual-frequency ultrawideband radar was designed to cancel the respiration-like clutter. Other studies have used signal processing techniques to remove the interferences from the radar returns [41]. In real

situations, different sources of noise and interference such as water movement due to flushing, water flowing in the pipes, and fan movement can affect the radar returns in a closed room. Due to the periodic pattern of such interferences and their frequency contents overlapping with those of respiration and heart activities, they may be misidentified as human vital signs. The presence of such interferences that overlap in time and frequency makes further processing and classification of radar returns very challenging. The contributions of this paper are a step forward toward developing a remote event recognition system that can operate in the presence of various interferences. In this paper, a PMCW radar is used for monitoring of different events in various facilities such as elderly care and correctional facilities. Based on the radar returns, a novel classification system is developed for identifying different events including breathing and stop breathing amidst common interferences in a typical monitoring environment such as human motion and water and fan movement. The initial intention was to limit the number of possible events of interest to the common cases that we observed in the prison environment. These interferences may have spectral and time overlap with the events associated with human vital signs. The radar returns are first filtered and decimated to reduce the noise and clutter effect in the proposed method. In order to characterize the different events of interest and distinguish between the interferences and human vital signs and activities, a comprehensive set of features are extracted from the radar returns both in time and frequency domains. The Bhattacharyya distance is used as a measure of class separability to analyze and select the most effective features. In order to improve the classification performance and reduce the noise, the dimension of the selected feature set is reduced by decorrelating the features, while maximizing the class separability using the uncorrelated linear discriminant analysis (ULDA). Bayesian algorithms based on the normal prior for the distribution of the input features are then designed to classify different events. The performance of our proposed radar pattern recognition system is evaluated using the data obtained from a pilot study with nine different activities. The rest of this paper is organized as follows. In Section II, the PMCW radar used in our proposed pattern recognition system is introduced, the data collection protocol is described, and a feature-based Bayesian algorithm for radar pattern classification is proposed. In Section III, the performance of proposed radar pattern recognition system is evaluated. In Section IV, the paper is concluded and in Section V, the limitations of our study are discussed and possible solutions are provided. II. METHODOLOGY A. Radar System In our experiments, we used the SR4505 PMCW radar produced by K&G Spectrum Inc., Gatineau, Quebec, Canada. The SR4505 PMCW radar is a monostatic millimetric wave radar in which the transmitter and the receiver antennas are at the same location from the perspective of the target. The transmitter sends out a direct-sequence spread spectrum waveform [42], which is a biphase shift keying modulated

FOROUZANFAR et al.: EVENT RECOGNITION FOR CONTACTLESS ACTIVITY MONITORING PMCW RADAR

TABLE I SR4505 PMCW RADAR PARAMETERS Parameter Antenna length 3-dB beamwidth Antenna gain Sidelobes Center frequency Bandwidth Transmit power Range bin resolution Maximum detection range PN code clock frequency PN code length Dynamic range

Value 35.5 cm 5◦ × 120◦ 24 dB 28 dB 24.125 GHz 500 MHz 17 mW 3m 50 m 50 MHz 1023 120 dB

481

return signals. In other words, since the time delay of each return signal depends on the distance of the target from the antenna, the correlation process only enhances the return signals originated from the targets within the range bin (zone) of interest. The output of the correlator is a sum of waveforms each resulting from the reflection of a target within the range bin of interest. The amplitude of each waveform is proportional to the radar cross section of the corresponding target, while its frequency is proportional to the speed of the target. The output of the correlator (sum of the waveforms originated from the range bin of interest) is then digitized and after appropriate preprocessing described in Section II-C, the digitized output is available for classification. It should be noted that in a pure CW radar, a CW signal is transmitted and a baseband signal is obtained at the receiver by mixing the received signal with the local oscillator signal. While a pure CW radar does not provide the target range information, it can be shown that the baseband signal at the receiver is proportional to the target displacements when the small-angle approximation is valid [17], [30], [44]. In our PMCW radar, the CW signal is modulated with a PN code and it is then transmitted. At the receiver, the target return signal is mixed with the local oscillator signal to output a time-delayed Doppler shifted code. The received code is then correlated with a delayed version of the transmitted code. The PN sequences have an autocorrelation function which is a periodic triangle of height equal to the code length and base of twice the code clock period [45]. Utilizing the autocorrelation characteristics of PN sequences, the PMCW radar can measure range from the phase difference of PN sequences. The phase can be measured as the delay at which the peak of the correlation occurs [46]. B. Measurement Setup

Fig. 1.

General block diagram of SR4505 PMCW radar.

signal with a 24-GHz center frequency. Pseudorandom noise (PN) codes are used for phase modulation. Table I lists the main parameters of our radar system. Fig. 1 shows the general block diagram of the SR4505 radar. PN codes are used to phase modulate the carrier signal generated by a local oscillator. The phase-modulated signal is then transmitted and the reflected signals are collected at the receiver. At the receiver of our PMCW radar, the reflected signals are first demodulated using the local oscillator which operates at the same frequency as the transmitter. In order to cancel the leakage of the transmitted signal into the receiver, an attenuated version of the transmitted code is subtracted from the baseband signal as described in [43]. The demodulated output is then correlated with a delayed version of the transmitted PN sequence and the output of the correlator is integrated over the entire sequence. Since the code clock period of our radar is 20 ns (clock frequency of 50 MHz), the PN sequence can be delayed by multiples of 20 ns, resulting in a range resolution of 3 m. The correlation process enhances the return signals that have the same delay as the PN sequence and suppresses the rest of the

The data collection was performed in a laboratory setting at the University of Ottawa, as shown in Fig. 2. The laboratory is about 9 m × 6 m with desks, chairs, and book shelves. A 6 m × 2 m area was cleared of any desk, chair, and book shelf and was used as the area for data collection. The radar was placed at one end on a desk of approximately 70 cm in height. Within the 6 m × 2 m space, subjects were seated on a chair about 4.5 m from the radar. For this pilot study, nine different events (six events that contain activities and three events with no activities) were considered. Three volunteers (two males, age 33 and 37, and one female, age 22) were asked to perform the following activities each representing an event: 1) Sitting on a chair as still as possible while breathing normally (3 min). 2) Sitting on the chair and randomly moving the head (90 s). This event represents low-level body movements. 3) Sitting on the chair and randomly moving the torso (90 s). This event represents high-level body movements. 4) Sitting on the chair as still as possible while stopping breathing for 20 s in every minute (over 6 min). This activity represents the event when the subject is breathing normally and suddenly stops breathing such as the case of attempted suicide via asphyxiation.

482


Fig. 2. Experimental setup. Arrows show the (a) radar system, (b) computer connected to the radar, (c) subject sitting on a chair, and (d) fan/bottle of water location.

5) Sitting on the chair as still as possible while breathing normally with water movement (1 min). The water movement was simulated by hitting a half-full 4 L bottle of water placed in front of the subject on the ground. The subject hit the bottle of water with his/her left leg and the radar started recording as soon as the subject’s leg was still and until the water movement stopped. 6) Sitting on the chair as still as possible while breathing normally with fan on (3 min). A fan was placed in front of the subject and was set to medium speed. In addition, the following events were also recorded: 7) No subject in the room with water movement (90 s). Water movement due to flushing or the water flowing in the pipes can interfere with breathing signals as their frequency spectra overlap with those of breathing signals. The water movement was simulated as in the Event #5. This event is recorded to investigate if the radar can distinguish between human breathing and water movement. 8) No subject in the room with fan on (3 min). The fan was set as in the Event #6. Fan movement is also a major source of interference in any facility such as home care and correctional facilities. 9) No subject in the room (3 min). The BioRadio 150 wireless data acquisition system (Cleveland Medical Devices Inc., Cleveland, Ohio) was used to record and display the reference respiration and ECG signals from the subjects. The BioRadio 150 consists of the BioRadio 150 user unit, USB receiver, and BioCapture software. The user unit is worn by the subject to acquire the physiological signals from

Fig. 3. Block diagram representation of our proposed radar pattern recognition system.

sensors or surface electrodes attached on the body. Chest movements due to respiration were acquired by placing the respiratory effort belt around the subject’s chest. The standard lead I ECG signal was acquired as part of our protocol by placing three electrodes, one each on the right wrist, left wrist, and right leg of the subject, but it was not used in this study. The user unit amplifies, samples, and digitizes the physiological signals and wirelessly transmits them to the USB receiver. The USB receiver forwards the data into the PC, where the data are displayed and analyzed using the BioCapture software. The study protocol was approved by the institutional research ethics board and written consent was obtained from the subjects. C. Preprocessing Fig. 3 illustrates a high-level block diagram representation of our developed radar pattern recognition system. In the first stage, the recorded radar returns were labeled into nine classes each representing one of the nine events described


in Section II-B. For Class #4, only the 20-s breath holds were considered and the rest of the radar signal was discarded. By visual inspection and based on the reference respiration signal, the segments of the radar returns that appeared to be inconsistent with the class labels were discarded from our dataset. These inconsistencies usually occurred when the subjects did not follow the measurement protocol, for example, moving when needed to be still or breathing when needed to hold the breath. The radar returns of the nine different classes were then segmented into 10-s nonoverlapping segments forming a dataset of 245 radar return segments. The lengths of the segments were chosen to be as short as possible while containing all the events of interests including breathing and low-frequency movements. As our dataset of radar segments is randomly divided into training and test sets for performance evaluation (see Section III), to fairly evaluate the performance of our classifier on unseen data, the radar returns were segmented into nonoverlapping segments instead of overlapping segments. While our radar system has a high sampling rate of 1.75 kHz, most of the human vital signs and activities exhibit lower frequency content (< 20 Hz). In order to reduce the computational costs in the preprocessing and feature extraction stages, the radar returns were decimated by a factor of 30 which resulted in radar return segments with 583 time samples. In order to avoid aliasing, prior to decimation, the radar returns were filtered using a low-pass second-order butterworth IIR filter with cutoff frequency of 20 Hz which meets the criteria fc < (0.8/r)(fs /2) [47], where fc is the low-pass filter cutoff frequency, fs is the sampling rate, and r is the decimation rate. The choice of the low-pass filter cutoff frequency is based on our prior information that neither the human vital signs, body movements, water movement, or fan posses frequencies higher than 20 Hz. Next, in order to reduce the effect of clutter, the radar returns were filtered using a high-pass second-order butterworth IIR filter with a cutoff frequency of 0.08 Hz. The choice of the high-pass filter cutoff frequency is based on our prior information that the lowest frequency content in the radar returns belongs to the human respiration which could be as low as 0.08 Hz. Next, the periodogram estimate of the power spectral density (PSD) was calculated [48] Sxx =

1 |X(k)|2 NFFT

(1)

where X(k) is the discrete Fourier transform (DFT) of radar return segment x(n) implemented using the fast Fourier transform (FFT) algorithm, and NFFT is the number of FFT points. The number of FFT points was set to the next power of two greater than the signal length, i.e., NFFT = 210 . In order to reduce the spectral leakage and obtain a smooth periodogram, the periodogram was convolved with a Hanning window of the same size. D. Feature Extraction and Selection After preprocessing, the radar return of activity patterns are transformed into a set of representative features that distinguishes one class from another.

483

Time pattern and frequency content of radar returns change according to the type of the movement performed. Therefore, accurate classification of radar returns will require the use of appropriate features characterizing different time and frequency properties of each class. Time Features: Inspired from the literature on biomedical and speech signal analysis and classification, several features representing different signal characteristics were extracted from the time-domain radar returns. The time features extracted from the radar returns include the following. 1) Root-Mean-Squared (RMS) Value: The RMS value is as an indicator of the average power of the radar return and can be used to distinguish different levels of movement [49]. The RMS value of radar return x(n) over its duration of N samples, is given by 12 N −1 1 2 x (n) . RMS = N n =0

(2)

In order to evaluate the class separability achieved using the RMS feature, the Bhattacharyya distance was calculated. Under normal distribution assumption, the Bhattacharyya distance is given by [50] DB (wi , wj ) =

1 (μi − μj )T 8

σi + σj 2

1 |(σi + σj )/2| + ln 2 |σi ||σj |

−1

(μi − μj ) (3)

where DB (wi , wj ) represents the Bhattacharyya distance between classes wi and wj , μi and μj represent the class means, and σi and σj represent the class standard deviations. The Bhattacharyya distance between two classes is 0 when the class distributions are exactly the same and increases as the class distributions become more different. In [51], the relationship between the Bhattacharyya distance and the Bayes error was studied and a Bhattacharyya distance of > 0.8 was found to achieve a classification error of < 10%. In this paper, we adopt the 0.8 threshold to select the features with higher discrimination capability for radar pattern classification. Using the RMS as the discriminating feature, it was found that the average Bhattacharyya distance between all the classes is DB = 3.72, which is reasonably higher than the threshold 0.8. 2) Zero-Crossing Rate (ZCR): The ZCR is defined as the number of times the signal crosses the zero reference level in a specific time interval, and can be used as an indicator of the “busyness” of the signal [48]. ZCR can be used to distinguish between activities with different movement speeds. In this paper, the ZCR was computed as the number of zero crossings of each 10-s segment divided by 10. The average Bhattacharyya distance between all the classes of our radar pattern dataset using ZCR as the discriminating feature was DB = 6.52, which is substantially higher than the threshold 0.8. 3) Turns Count: Turns count is equal to the number of zero crossings of the derivative of the signal. Unlike ZCR, the

484


turns count is not affected by dc bias and low-frequency movements and artifacts [52]. Similar to the ZCR, the turns count can also be used for radar return analysis and classification. The average Bhattacharyya distance between all the classes of our radar pattern dataset using turns count as the discriminating feature was DB = 2.06, which is reasonably higher than 0.8. 4) Central Moments: The central moments of a signal can be useful in characterizing the general trends and in the distribution of signal amplitude [53]. The kth central moment of the signal x(n) is given by mk = E(x − μ)k

where x represents the derivative of the signal. The average Bhattacharyya distance between all the classes using mobility as the discriminating feature was DB = 2.07, which is reasonably higher than 0.8. 6) Form Factor (FF): FF is another measure of signal activity and is computed as the square root of the ratio of the mobility of the first derivative of the signal to the mobility of the signal, as follows [54]: Mx σx /σx = Mx σx /σx

f¯ = fs

(6)

where x represents the second derivative of the signal. The average Bhattacharyya distance between all the classes using FF as the discriminating feature was DB = 1.17. Frequency Features: Since different movements contain different frequency components, deriving frequency-domain features will aid in classifying events. 1) Spectral moments: The power spectral density (PSD) provides a density function of signal power versus frequency. In

N F F T /2−1

2 NFFT Ex

kSxx (k)

(7)

k =0

where fs is the sampling frequency and Ex is the total power of the signal used for normalization. Ex can be obtained as follows:

(4)

where E(x) is the expected value of x and μ is the mean value. Note that the central first moment is zero and the second central moment is the variance. The third central moment normalized by square root of variance to the third power is the skewness and the fourth central moment normalized by square of variance is the kurtosis. The higher order moments are sensitive to noise and may not yield useful information for characterization of radar returns. Therefore, in this paper, only the variance, skewness, and kurtosis were computed. The expected value was computed as the time average over the duration of the radar segment. The average Bhattacharyya distances between all the classes of our radar pattern dataset were DB = 2.90 for variance as the discriminating feature, DB = 0.26 for skewness as the discriminating feature, and DB = 0.38 for kurtosis as the discriminating feature. It is observed that while variance provides high separability between different classes, the higher order moments such as skewness and kurtosis are not as effective features as the variance. 5) Mobility: Mobility is a measure of signal activity and is computed as the square root of the ratio of the variance of the first derivative of the signal to the variance of the signal, as follows [54]: 2 12 σ σx = (5) Mx = x2 σx σx

F Fx =

order to characterize the general trends in the PSD, one can compute the moments of the PSD. The first-order moment is the mean frequency computed as follows [48], [55]:

Ex =

N F F T −1

1 NFFT

|X(k)|2 .

(8)

k =0

Ex can also be used as a feature. In addition to the mean frequency, the median frequency can be computed as follows: m fmed = fs (9) NFFT with the largest m such that 2 NFFT Ex

m

Sxx (k)
2.84. The features with Bhattacharyya distance greater than 0.8 were selected as the features to be used for classification purpose. Out of the extracted time features, six features including RMS, ZCR, turns count, variance, mobility, and FF were selected. Out of the extracted frequency features, 37 features including signal total power, mean frequency, median frequency, spectral variance, spectral skewness, spectral kurtosis, six spectral power fraction, 15 spectral power ratios, and the frequencies of the first ten spectral power peaks were selected. E. Feature Reduction The radar feature extraction and selection process proposed in Section II-D results in a high-dimensional feature vector of size 43. A large number of such features would make the development of further pattern classification rules difficult [57]. Therefore, we perform feature reduction to reduce the redundancy of the feature vectors and simplify the classification process. In addition, feature reduction can reduce the noise. In a classification problem, it is desired to perform the feature reduction in a way that the within-class variations are reduced while between-class variations are increased [58]. This makes it easier for the classifier to find the optimum discriminant surfaces which separate different classes. Moreover, it is desired to reduce the correlation between the input patterns before feeding them into the classifier [59]. An uncorrelated set of features enables the classifier to better learn the actual relationship between the input patterns and their corresponding classes. In this paper, in order to obtain an uncorrelated and a concise set of radar pattern features, we employ the ULDA [60]. ULDA decorrelates the features by linearly mapping them into a lower dimensional space of C − 1 features, where C is the number of classes, while increasing the class separability. This reduces the dimension of our selected features set from 43 to 8. The classical LDA is a supervised method which finds a set of projection vectors in such a way that the ratio of the betweenclass distance to the within-class distance is maximized and, therefore, maximum class separability is achieved. A withinclass scatter matrix shows the scatter of samples around their respective class expected vectors. On the other hand, a betweenclass scatter matrix shows the scatter of the expected vectors around the mixture mean. Such an optimal transformation for LDA can be found by simultaneously maximizing the trace of the between-class scatter matrix and minimizing the trace of the within-class scatter matrix. In ULDA, this optimization is performed with a constraint that enforces the transformed features are mutually uncorrelated [59]. F. Bayesian Classifier In this paper, we design a Bayesian algorithm for radar pattern classification. Given a classification task of C classes, w1 , w2 , . . . , wC , an unknown radar return represented by a feature vector z is to be classified. We form the C a posteriori probabilities (conditional probabilities) P (wi |z), which represent the probability that the unknown radar return belongs to

486


class wi , given the feature vector takes the value z. Let L(wy |wi ) indicate the cost of classifying an observation as class wy when its true class is wi , an expected classification cost is defined and minimized to determine the true class of each feature vector as follows [58]: w ˆy = arg

min

wC

P (wi |z)L(wy |wi )

(13)

w y =w 1 ,...,w C w i =1

where w ˆy represents the predicted class. Using the Bayes theorem, the a posteriori probabilities can be written as a function of the a priori probabilities P (wi ) and the class likelihoods P (z|wi ). The a priori probabilities can be easily estimated from the available radar training feature vectors. If N is the total number of available training patterns, and Ni of the belong to class wi , then P (wi ) ≈ Ni /N . The class likelihood P (z|wi ) describes the distribution of the feature vectors in class wi . The class likelihoods can be estimated from the available radar training feature vectors using different techniques such Parzen windows and k-nearest neighbors algorithms [58]. However, such techniques will require the use of numerical methods to solve the Bayesian classification problem in (13) which could be computationally expensive and may lead to inaccurate classification results in the case of a small number of available data. If the class likelihoods P (z|wi ) describing the data distribution in each class are multivariate normal distributions, the Bayesian classification problem in (13) can be further simplified to the following class discriminant functions:

TABLE II DATASET OF RADAR RETURNS Class number

1 2 3 4 5 6 7 8 9

Event

Number of segments (class samples)

Normal breathing Low-level movement High-level movement Stop breathing Normal breathing with water movement Normal breathing with fan movement Water movement with no subject Fan movement with no subject Empty room

52 25 25 34 15 51 8 17 18

The dataset consists of 2450 s recording which has been divided into 245 10-s segments

1 1 T −1 1 T −1 gi (z) = − zT Σ−1 i z + z Σi µi − µi Σi µi 2 2 2 1 T −1 + µi Σi z + ln P (wi ) + ci (14) 2 where gi (z) represents the ith class discriminant function, µi and Σi are the mean vector and the covariance matrix of class wi , ci is a constant equal to −(1/2)(ln 2π + ln |Σi |), and |Σi | denotes the determinant of Σi . In general, the Bayesian discriminant functions have a nonlinear quadratic form. That is, the feature space is partitioned via quadratic decision surfaces. If we assume that the covariance matrix is the same in all classes, we will have a linear classifier which partitions the feature space using linear decision surfaces. In order to estimate the unknown parameters of the multivariate normal distribution including the class mean vector and covariance matrix, we employ the maximum likelihood method [58]. III. EXPERIMENTAL RESULTS The performance of our proposed radar pattern recognition system was evaluated on the dataset of 245 radar returns introduced in Section II-B. The dataset consisted of nine classes as described in Table II. Examples of radar returns belonging to these nine classes after applying the preprocessing steps discussed in Section II-C are shown in Fig. 4. It is important to evaluate the performance of our proposed radar pattern recognition system on a test set because such an

Fig. 4. Examples of radar returns belonging to nine different classes described in Table II.

evaluation provides an unbiased estimate of the classifier generalization error. For this purpose, a fivefold cross validation was performed. That is, the dataset was divided into 80% training data and 20% testing data. The optimum parameters of the classifier were found on the training data using the class labels as the corresponding targets. The performance of the classifier was


487

Fig. 5. Classification confusion matrices for our proposed (a) linear and (b) quadratic Bayesian radar pattern classifiers obtained on the test data. The classifier precision or PPV and the classifier FDR are shown in the last column of the confusion matrices. The sensitivity or recall or TPR and FNR are shown in the last row of the confusion matrices. The last element of the confusion matrices shows the classifier total accuracy (ACC) and total error (E).

then evaluated on the testing data. This procedure was repeated so that all the radar patterns were used once in the test set. The uncorrelated discriminant analysis (ULDA) was performed on the training dataset and the optimum mapping was found. The extracted reduced dimension feature set was then fed to two Bayesian classifiers, the linear Bayesian classifier and the quadratic Bayesian classifier. The optimum parameters of these classifiers were found on the training data. The performance of our proposed radar pattern recognition system on the testing data is illustrated in Fig. 5 by classification confusion matrices. The columns of the matrices represent the instances in actual classes, while the rows represent the instances in predicted classes. Each element ij of the matrices shows the number and percentage of correct predictions of class i if i = j (true positives), wrong assignments from class j into class i if i < j (false positives), and wrong assignments of class i into class j if i > j (false negatives). All correct predictions are located in the diagonal of the table. The last column of the confusion matrix shows the classifier precision or positive predicted value (PPV) and the classifier false discovery rate (FDR) in green and red, respectively. Given the number of true positives (TP) and false positives (FP), the classifier PPV and FDR for each class are defined as [61] TP TP + FP FP FDR = = 1 − PPV. TP + FP PPV =

(15) (16)

The PPV determines the fraction of retrieved (positive) instances that are true. On the other hand, the FDR determines the fraction

of retrieved instances that are false. It is observed that, except for class 5, both the quadratic and linear Bayesian classifiers achieve high PPV (>86%) and low FDR (88%) and low FNR (96%) and NPV (>95%) and low FPR (86%) except for Class 5 where it is 56.02% and 58.82% using the linear and quadratic classifiers, respectively. As discussed earlier, in order to reduce the high number of false negatives of Class 5, the last 20 s of all the measurements belonging to Class 5 were discarded from our dataset. Our proposed radar pattern recognition system was then trained SPC =


TABLE III SPC, FPR, NPV, AND F-SCORE VALUE FOR OUR PROPOSED LINEAR AND QUADRATIC RADAR PATTERN CLASSIFIERS COMPUTED ON THE TEST DATA Class

SPC

FPR

NPV

489

TABLE IV ACCURACY, SENSITIVITY, AND SPECIFICITY OF OUR PROPOSED METHOD IN DETECTION OF STATIONARY SUBJECT, STOP-BREATHING EVENT, AND INTERFERENCE

F-score Stationary detection

Stop-breathing detection

Interference detection

Linear Bayesian Radar Pattern Classifier Linear Bayesian Radar Pattern Classifier 1 2 3 4 5 6 7 8 9

95.90% 99.55% 100% 98.10% 98.70% 96.91% 100% 100% 100%

4.10% 0.45% 0% 1.90% 1.30% 3.09% 0% 0% 0%

96.86% 100% 100% 98.10% 96.60% 97.92% 100% 100% 100%

86.82% 98.06% 100% 88.2% 56.02% 90.42% 100% 100% 100%

Accuracy Sensitivity Specificity

93.06% 94.08% 91.40%

89.47% 88.24% 90.68%

86.30% 85.00% 87.21%

Quadratic Bayesian Radar Pattern Classifier Accuracy Sensitivity Specificity

91.84% 92.11% 91.40%

90.13% 91.18% 90.68%

86.99% 85.00% 88.37%

Quadratic Bayesian Radar Pattern Classifier 1 2 3 4 5 6 7 8 9

97.41% 100% 100% 97.63% 96.09% 98.97% 100% 100% 100%

2.59% 0% 0% 2.37% 3.91% 1.03% 0% 0% 0%

95.92% 100% 100% 98.56% 97.79% 97.46% 100% 100% 100%

87.12% 100% 100% 88.58% 58.82% 92.92% 100% 100% 100%

and tested on the new dataset. It was observed that the Class 5 precision, sensitivity, and F-score are improved to 87.50%, 77.78%, and 82.35%, for the linear Bayesian classifier, and 72.73%, 88.89%, and 80.00%, for the quadratic Bayesian classifier, respectively. It was also observed that the Class 1 precision, sensitivity, and F-score are improved to 92.31%, 92.31%, and 92.31%, for the linear Bayesian classifier, and 98.04%, 96.15%, and 97.09%, for the quadratic Bayesian classifier, respectively. The rest of the performance metrics did not significantly change. The total accuracy of the linear and quadratic Bayesian classifiers were improved to 91.87% and 93.06%, respectively. It should be noted that the measures that should be maximized depend on the application. For example, in the case of suicide attempt prevention, we can tolerate more false positives in detection of stop breathing if this allows us to minimize the false negatives. In order to evaluate the performance of our proposed method in detection of stationary subjects, our dataset was divided into two new classes corresponding to stationary (Classes 1, 4, 5, and 6) and the rest of the events (Classes 2, 3, 7, 8, and 9) and the classification performance was tested using a five-fold cross validation as explained earlier in this section. In a similar fashion, the performance of our proposed method was evaluated in detection of stop breathing while the subject is stationary (Class 4 versus classes 1, 5, and 6), and detection of interference while the subject is stationary (Classes 5 and 6 versus Classes 1 and 4). The classification results, in terms of accuracy, sensitivity (TPR) and specificity (1-FPR), are shown in Table IV. Detection of stationary subjects is important as it is the first step in identifying the stop-breathing event and in respiration and heart rate estimation. Detection of stop-breathing event can help in identifying the inmates’ suicide attempts in prison cells. Also,

detection of interferences is important as their frequency contents can overlap with those of respiration and heart activities. IV. CONCLUSION Noncontact monitoring using a PMCW radar was considered in this paper. The signal received at the radar is a superposition of signals from movement of the torso, movement of the limbs, expansion, and contraction of the chest cavity, in addition to noise and interference. Physiological radar signal processing, therefore, requires the following steps to be performed: noise and clutter suppression, range estimation, classification of activities, and breathing and heartbeat signal extraction. This paper focused on the classification of activities and the detection of the time points at which common interference occurs. The experiments that were done correspond to a realistic environment such as a room or a bathroom or a prison cell. They interfering signals considered in this study originated from human motion and water and fan movement. The proposed method was verified with a pilot study on a dataset of radar patterns recorded from nine different events. It was observed that the overall accuracy of our proposed radar pattern recognition system was up to 93%. The successful validation of our proposed method shows promise toward the development of an automated contactless monitoring system for home-based monitoring of elderly people and prevention of suicide in the presence of interference. Our proposed method can be used as a primary stage in estimation and monitoring of heart rate and respiration rate. The radar returns can be first classified based on the underlying events in progress. Estimation algorithms can be then employed to estimate the heart rate and respiration rate if a subject is detected within the target area, such as a cell. If our proposed method demonstrate sufficiently high accuracy in this regard, it could also be used in the measurement of HRV and the diagnosis of some health conditions such as sleep apnea. V. LIMITATIONS AND FUTURE WORK The results presented in this paper are preliminary. Our proposed method was validated on a dataset of radar patterns where the subjects were sitting on a chair in front of the radar

490


performing predefined activities. Future work will involve further validation of our proposed method on a dataset of radar patterns obtained from subjects in different postures and body positions. A possible limitation of our proposed system is that it cannot distinguish the activities if the nearby subjects are moving at the same frequency and reflect the same amount of RF power back to radar. In this paper, the dataset of radar returns consisted of nonoverlapping segments for each class. Future work can be directed toward the design of radar pattern classification systems that can detect multiple events at the same time, and can detect the exact beginning and end of an event. The comprehensive feature set that was extracted in this paper can be employed for this purpose. A hierarchical classifier may be designed to distinguish between the large number of classes with unequal number of samples by exploiting the fact that classes that are similar according to one feature may be dissimilar according to another, allowing the large number of classes to be grouped and handled separately. If the target movement is large or the interfering reflections are strong, the reflections may saturate the radar receiver. This paper did not consider the radar saturation case. Our classification system can be simply modified to detect the saturation case; however, it will not be able to classify the human activity events because no useful information will be available. Our radar pattern recognition system was designed based on a Bayesian classifier. The main reasons to use a Bayesian classifier as the initial choice over other classifiers were its simplicity, as it requires only a small amount of training data, and because it is less prone to under and over fitting to the training data. In this study, the total duration of our recordings was 2450 s. A longer dataset of radar returns will be collected in future work. With a larger set of training data, it is expected that more sophisticated learning algorithms such as the multilayer perceptron would be the ideal choice. ACKNOWLEDGMENT The authors would like to thank A. Gagnon from K&G Spectrum, Inc., for providing the SR4505 radar prototype, S. Bisson from Correctional Service Canada for his suggestions on creating our dataset of radar patterns based on the common interferences in prison cells, and X. Zhang for her help in data collection. REFERENCES [1] J. Paton and R. Jenkins, “Suicide and suicide attempts in prisons,” in Prevention and Treatment of Suicidal Behaviour: From Science to Practice, K. Hawton, Ed. Oxford, U. K.: Oxford Univ. Press, 2005. [2] J. S. Coffin et al.“Virtual custody systems for improved house arrest and prison security,” in Proc. IEEE Int. Carnahan Conf. Security Technol., Atlanta, GA, USA, Oct. 1992, pp. 81–85. [3] P. B. Patil et al., “Monitoring system for prisoner with GPS using wireless sensor network,” Int. J. Comput. Appl., vol. 91, pp. 1085–1087, 2014. [4] M. Bailey, “Method of preventing an inmate from committing suicide,” U.S. Patent 20 110 260 870 A1, Oct. 27, 2011. [5] Z. Zhang et al. “Human-target detection and surrounding structure estimation under a simulated rubble via UWB radar,” IEEE Geosci. Remote Sens. Lett., vol. 10, no. 2, pp. 328–331, Mar. 2013. [6] K. Chen et al. “Microwave life-detection systems for searching human subjects under earthquake rubble or behind barrier,” IEEE Trans Biomed. Eng., vol. 27, no. 1, pp. 105–114, Jan. 2000.

[7] D. Benitez and Z. Jin, “Method for human only activity detection based on radar signals,” U.S. Patent 7 924 212 B2, Apr. 12, 2011. [8] F. H. C. Tivive et al. “A human gait classification method based on radar Doppler spectrograms,” EURASIP J. Adv. Signal Process., vol. 2010, pp. 1–12, 2010. [9] I. Orovic et al. “A new approach for classification of human gait based on time–frequency feature representations,” Signal Process., vol. 91, pp. 1448–1456, 2010. [10] P. E. Cuddihy et al. “Radar based systems and methods for detecting a fallen person,” U.S. Patent 8 742 935 B2, Jun. 03, 2014. [11] L. Liu et al. “Fall detection using doppler radar and classifier fusion,” in Proc. IEEE-EMBS Int. Conf. Biomed. Health Informat., Hong Kong, Jan. 2012, pp. 180–183. [12] V. Chen, “Doppler signatures of radar backscattering from objects with micro-motions,” IET Signal Process., vol. 2, pp. 291–300, 2008. [13] Y. Kim and L. Hao, “Human activity classification based on micro-doppler signatures using a support vector machine,” IEEE Trans. Geosci. Remote Sens., vol. 47, no. 5, pp. 1328–1337, May 2009. [14] P. E. Cuddihy et al. “Radar based systems and methods for monitoring a subject,” U.S. Patent 8 740 793 B2, Jun. 03, 2014. [15] M. Ganesh et al. “Physiology monitoring and alerting system and process,” U.S. Patent Application 20 120 245 479 A1, Sep. 27, 2012. [16] M. Mabrouk et al. “Human breathing rate estimation from radar returns using harmonically related filters,” J. Sensors, vol. 2016, pp. 1–7, 2016. [17] C. Li et al. “A review on recent advances in doppler radar sensors for noncontact healthcare monitoring,” IEEE Trans. Mircrow. Theory Tech., vol. 60, no. 5, pp. 2046–2060, May 2013. [18] P. S. Girao et al. “Microwave doppler radar in unobtrusive health monitoring,” J. Phys. Conf. Ser., vol. 588, pp. 1–10, 2015. [19] A. D. Droitcour et al. “Signal-to-noise ratio in doppler radar system for heart and respiratory rate measurements,” IEEE Trans. Microw. Theory Tech., vol. 57, no. 10, pp. 2498–2507, Oct. 2009. [20] L. Chioukh et al., “Noise and sensitivity of harmonic radar architecture for remote sensing and detection of vital signs,” IEEE Trans. Microw. Theory Tech., vol. 62, no. 9, pp. 1847–1855, Sep. 2014. [21] D. Girbau et al. “Remote sensing of vital signs using a doppler radar and diversity to overcome null detection,” IEEE Sensors J., vol. 12, no. 3, pp. 512–518, Mar. 2012. [22] W. Hu et al. “Noncontact accurate measurement of cardiopulmonary activity using a compact quadrature doppler radar sensor,” IEEE Trans. Biomed. Eng., vol. 61, no. 3, pp. 725–735, Mar. 2014. [23] C. Li, et al. “Radar remote monitoring of vital signs,” IEEE Microw. Mag., vol. 10, no. 1, pp. 47–56, Feb. 2009. [24] W. Massagram, et al. “Assessment of heart rate variability and respiratory sinus arrhythmia via doppler radar,” IEEE Trans. Microw. Theory Tech., vol. 57, no. 10, pp. 2542–2549, Oct. 2009. [25] J. Kranjec et al. “Novel methods for noncontact heart rate measurement: A feasibility study,” IEEE Trans. Instrum. Meas., vol. 63, no. 4, pp. 838–847, Apr. 2014. [26] Y. Nijsure, et al. “An impulse radio ultrawideband system for contactless noninvasive respiratory monitoring,” IEEE Trans. Biomed. Eng., vol. 60, no. 6, pp. 1509–1517, Jun. 2013. [27] S. Bakhtiari, et al. “A real-time heart rate analysis for a remote millimeter wave I-Q sensor,” IEEE Trans. Biomed. Eng., vol. 58, no. 6, pp. 1839– 1845, Jun. 2011. [28] M. C. Huang et al. “A self-calibrating radar sensor system for measuring vital signs,” IEEE Trans. Biomed. Circuits Syst., vol. 10, no. 2, pp. 1–1, Apr. 2016. [29] O. Boric-Lubecke et al., Doppler Radar Physiological Sensing. Hoboken, NJ, USA: Wiley, 2015. [30] A. D. Droitcour, “Non-contact measurement of heart and respiration rates with a single-chip microwave doppler radar,” Ph.D. dissertation, Stanford, CA, USA: Stanford Univ. Press, 2006. [31] J. Li et al. “Automatic classification of human motions using doppler radar,” in Proc. IEEE Int. Joint Conf. Neural Netw., Brisbane, Australia, Jun. 2012, pp. 1–6. [32] C. Graichen, et al. “Unobtrusive vital signs monitoring with rangecontrolled radar,” in Proc. IEEE Signal Process. Med. Biol. Symp., New York, NY, USA, Dec. 2012, pp. 1–6. [33] S. Wang et al. “Radar emitter signal recognition based on sample entropy and fuzzy entropy,” in Intelligent Science and Intelligent Data Engineering. Berlin, Germany: Springer, 2012, pp. 637–643.


[34] E. Avci et al. “Intelligent target recognition based on wavelet packet neural network,” Expert Syst. Appl., vol. 29, pp. 175–182, 2005. [35] T. Thayaparanet al. “Micro-doppler-based target detection and feature extraction in indoor and outdoor environments,” J. Franklin Inst., vol. 345, no. 6, pp. 700–722, 2008. [36] C. Guet al. “A hybrid radar-camera sensing system with phase compensation for random body movement cancellation in doppler vital sign detection,” IEEE Trans. Microw. Theory Tech., vol. 61, pp. 4678–4688, Dec. 2013. [37] I. Mostafanezhad et al., “Cancellation of unwanted motion in a handheld doppler radar used for non-contact life sign monitoring,” in Proc. IEEE MTT-S Int. Microw. Symp. Dig., Atlanta, GA, USA, Jun. 2008, pp. 1171–1174. [38] A. Singh and V. M. Lubecke, “Respiratory monitoring and clutter rejection using a CW Doppler radar with passive RF tags,” IEEE Sensors J., vol. 12, no. 3, pp. 558–565, Feb. 2012. [39] C. Li and J. Lin, “Random body movement cancellation in doppler radar vital sign detection,” IEEE Trans. Microw. Theory Tech., vol. 56, no. 12, pp. 3143–3152, Dec. 2008. [40] Z. Li et al., “A novel method for respiration-like clutter cancellation in life detection by dual-frequency IR-UWB radar,” IEEE Trans. Microw. Theory Tech., vol. 61, no. 5, pp. 2086–2092, May 2013. [41] I. Mostafanezhad et al. “Cancellation of unwanted doppler radar sensor motion using empirical mode decomposition,” IEEE Sensors J., vol. 13, no. 5, pp. 1897–1904, May 2013. [42] M. A. Richards, Fundamentals of Radar Signal Processing, 2nd ed. New York, NY, USA: McGraw-Hill, 2014. [43] S. Rauch and M. K. C. Kwong, “Spread spectrum radar with leak compensation at baseband,” U.S. Patent 6 801 153 B2, Oct. 05, 2004. [44] Y. Xiao et al. “Frequency-tuning technique for remote detection of heartbeat and respiration using low-power double-sideband transmission in the Ka-band,” IEEE Trans. Mircrow. Theory Tech., vol. 54, no. 5, pp. 2023– 2032, May 2006. [45] H. Z. amd L Li and K. Wu, “24GHz software-defined radar system for automotive applications,” in Proc. IEEE Eur. Conf. Wireless Tech., Munich, Germany, Oct. 2007, pp. 138–141. [46] S. Rauch, “Pseudo-randomly phase modulated radar altimeter,” U.S. Patent 3 774 206 A, Nov. 20, 1973. [47] Digital Signal Processing Committee, IEEE Acoustics, Speech, and Signal Processing Society, Programs for Digital Signal Processing. New York, NY, USA: IEEE Press, 1979. [48] R. M. Rangayyan, Biomedical Signal Analysis: A Case-Study Approach. Piscataway, NJ, USA: IEEE Press, 2002.

491

[49] Y. T. Zhang et al., “A comparative study of simultaneous vibromyography and electromyography with active human quadriceps,” IEEE Trans. Biomed. Eng., vol. 39, no. 10, pp. 1045–1052, Oct. 1992. [50] E. Choi and C. Lee, “Feature extraction based on the Bhattacharyya distance,” Pattern Recog., vol. 36, pp. 1703–1709, 2003. [51] C. Lee and E. Choi, “Bayes error evaluation of the gaussian ML classifier,” IEEE Trans. Geosci. Remote Sens., vol. 38, no. 3, pp. 1471–1475, May 2000. [52] A. Fuglsang-Frederiksen and A. Mansson, “Analysis of electrical activity of normal muscle in man at different degrees of voluntary effort,” J. Neurol. Neurosurg. Psychiatry, vol. 17, pp. 683–694, 1975. [53] A. V. Totsky et al., Bispectral Methods of Signal Processing: Applications in Radar, Telecommunications and Digital Image Restoration. Berlin, Germany: Walter de Gruyter GmbH & Co KG, 2014. [54] B. Hjorth, “EEG analysis based on time domain properties,” Electroencephalogr. Clin. Neurophysiol., vol. 29, pp. 306–310, 1970. [55] R. Merletti et al. “Estimation of shape characteristics of surface muscle signal spectra from time domain data,” IEEE Trans. Biomed. Eng., vol. 42, no. 8, pp. 769–776, Aug. 1995. [56] G. R. Johnson et al. “Estimation of the severity of aortic valve stenosis by frequency analysis of the murmur,” J. Amer. Coll. Cardiol., vol. 1, pp. 1315–1323, 1983. [57] M. Forouzanfar et al. “Feature-based neural network approach for oscillometric blood pressure estimation,” IEEE Trans. Instrum. Meas., vol. 60, no. 8, pp. 2786–2796, Aug. 2011. [58] S. Theodoridis and K. Koutroumbas, Pattern Recognition. Cambridge, U.K.: Academic, 2008. [59] L. Tolosi and T. Lengauer, “Classification with correlated features: unreliability of feature ranking and solutions,” Bioinformatics, vol. 27, pp. 1986–1994, 2011. [60] J. Ye et al. “Feature reduction via generalized uncorrelated linear discriminant analysis,” IEEE Trans. Knowl. Data Eng., vol. 18, no. 10, pp. 1312–1322, Oct. 2006. [61] T. Fawcett, “An introduction to ROC analysis,” Pattern Recogn. Lett., vol. 27, pp. 861–874, 2006.

Authors’ photographs and biographies not available at the time of publication.