SVM-Based Fall Detection Method for Elderly People Using Android ...

SVM-Based Fall Detection Method for Elderly People Using Android Low-Cost Smartphones Paola Pierleoni, Luca Pernini, Alberto Belli, Lorenzo Palma and Simone Valenti

Michele Paniccia

Information Engineering Department Universit´a Politecnica delle Marche Ancona, Italy Email: [email protected]

Istituto di Riabilitazione Santo Stefano Porto Potenza Picena (AN), Italy

Abstract—Nowadays society is moving to a scenery where autonomous elderly live alone in their houses. An automatic remote monitoring system using wearable and ambient sensors is becoming even more important, and is a challenge for the future in WSNs, AAL, and Home Automation areas. Relating to this, one of the most critical events for the safety and the health of the elderly is the fall. Lot of methods, applications, and stand-alone devices have been presented so far. This work proposes a novel method based on the Support Vector Machine technique and addressed to Android low-cost smartphones. Our method starts from data acquired from accelerometer and magnetometer, now available in all the low-end devices, and uses a set of features extracted from a processing of the two signals. After an initial training, the classification of fall events and non-fall events is performed by the Support Vector Machine algorithm. Since we have decided to use the smartphone as monitoring device, the use of other invasive wearable sensors is avoided, and the user have simply to hold the phone on his pocket. Moreover, we can use the cellular network for the eventual sending of notifications and alerts to relatives in case of falls. Actually, our tests show a good performance with a sensitivity of 99.3% and a specificity of 96%.

I.

I NTRODUCTION

People ageing is an even more important trend in our society: for example in Italy the percentage of people aged over 60 will increase from 24% in 2000 to 41% in 2050 [1]. For the elderly one of the most hazardous event is the fall: about 30% of people aged over 65 fall each year (35% over 70 years of age) [2]. Fall-related injuries represent the 6% of all medical expenditures for persons aged over 65 in the U.S., and are a significantly notable cause of death for this category of people [3]. A key issue in falls is the “longlie” after the injury. In fact, often elderly cannot return to standing position after a fall. This situation is very dangerous, and there is a close relationship between the delay of the assistance to the injury and the mortality rate [4]. Therefore, an instantaneous detection of the fall may allow a fast aid and can definitely avoid serious consequences in many cases. Thanks to the latest technological innovations, many solutions ranging in this direction have been presented in the last 10 years. Fall detection systems can be divided in three categories: wearable devices based, ambience sensors based, and vision based [5]. The solution presented in this work belongs to the first category. This kind of approach uses embedded sensors to estimate the motion, the posture and eventually the location of the user. Systems of this type are environment-independent -

can be used everywhere - and are the easiest to use. Moreover, if the device contains a communication module (e.g. GSM, 3G) a remote ubiquitous monitoring is allowed. The current smartphones can be included in this category because contain numerous types of sensors. The strategies for identifying falls using one or more wearable devices are numerous. The totality of the proposed systems uses accelerometers, while just a few also use other sensors such as magnetometers, gyroscopes and barometers. Signals acquired from the sensors are processed, and the most used methods to recognize fall events or unnatural postures are acceleration magnitude threshold based, Pattern Recognition Methods based and Neural Network based. Our fall detection system is based on Android OS over low cost smartphones. In fact, nowadays smartphones are widely used in the world (1.75 Billion in the world [6]) and sensors such as accelerometer and magnetometer are contained in the almost all new smartphones, even the low-end (while sensors like gyroscope and barometer are usually embedded only in much expensive phones). The chosen platform is Android, because of its large diffusion. In fact, Android represents the 84.7% of the market share of smartphone OS in the second quarter of 2014 [7]. Android has also proved very suitable for use in this type of application, as shown in [8]. Other smartphone solutions have been already presented. The first work in this direction has been done by Zhang et al. [9] and consisted in the integration of an accelerometer in a mobile phone. Referring to the most recent works, for example He et al. used a threshold based method on acceleration signal [10]. Abbate et al. used a Neural Network on a certain number of features extracted from acceleration signal [11]. Hsieh et al. used accelerometer signal as input for a Finite State Machine [12]. Yavuz et al. used a Discrete Wavelet Transform method for extract features from acceleration signal [13]. Many of these works have largely used other information and features made available by the device, such as telephony, internet connectivity, and GPS location. In this paper, a multidimensional Support Vector Machine (SVM) method is implemented over a set of features extracted from a data fusion of the two signals acquired by accelerometer and magnetometer. SVM method has been already used for fall detection purposes, but only over accelerometer signal and without any data fusion with other sensors [14]. We performed a long period of simulations of falls and of everyday life situations in order to have a broad set of acquisitions useful for the training of the system. After this, we obtained a set of

arrays useful for the classification of new events. The arrays are then continuously adjusted in order to avoid false positives. Finally, the application allows to carry out an automatic alert to presets friend-numbers in case of injury. II.

UI Activity (On-Off, Settings, Info)

Alert Activity

Signals Acquisition Service

M ATERIALS AND M ETHODS DSP and Data Fusion Class

A. Hardware We choose the Android OS platform, an open source software stack provided by Google for mobile phones. The used device is a Motorola Moto G, mounting 4.4.4 version of the OS. This smartphone embeds the LIS3DH 3-axis accelerometer by ST Microelectronics, an ultra low-power sensor with I2C and SPI interfaces, that has a selectable full scale up to ±16g, and that is capable to provide an output data rate up to 5 KHz. Moreover, Moto G embeds an AK8963 3-axis compass that incorporates magnetic sensor with a measurement range of ±4900µT .

Fall Detection Algorithm Class

Localization Service

Figure 1.

Cadal application components

Figure 2.

Cadal screenshots

B. Application Overview The application, called “CadAl”, consists of multiple components (Fig. 1). The most important component is the AcquisitionService that is continuously running when the detection is activated. The user is advised about its execution by an element in the notification area. This Service is not running in default background mode as common Services, but the foreground feature is requested to prevent the system to kill it when memory is needed by other applications. This Service implements the SensorEventListener class, useful for receiving notification from the SensorManager when new values from sensors are available. This main Service is strictly linked to other two Classes. The first is essential to process and merge the accelerometer and magnetometer signals. The second Class contains the SVM based algorithm for the new events classification (and eventually to update the SVM structure). Another Service runs in background and provides the user’s geographic location based on GPS, Wi-Fi, and cellular network, if any (using Android LocationManager class). This action can be activated or not from the user that can also choose the time interval between the localizations (a continuous localization Service would require too much battery power and would be redundant). The other two components of the application also include graphical user interfaces. The first is the AlertActivity, and it is called from the AcquisitionService when a fall event is detected. This activity allows the user to cancel any alarms until a timer expires. When the timer expires, the Activity launches an Intent to TelephonyManager to start an emergency call using handsfree and/or SMSManager to send a preconfigurated SMS towards one or more contacts decided by the user. Also an Intent to the WhatsApp Messenger application - the most widely used instant messaging application - can be sent, if installed on the phone; in this second case the geographic location of the user can be shared with the contact. The second is the SettingsActivity, and it is the Activity that is launched when the user touches the application icon. In this activity the user can start or terminate the fall detection service, can decide if start it at the phone boot, can set up the friend-numbers and the alert message, can choose the timeout duration in case of fall detection (max 30 s), and can see all the statistics of the application usage.

Some screenshots of the application can be seen in Fig. 2.

C. Algorithm Description When new values of acceleration and geomagnetic field are available, the RMS is calculated over the three raw acceleration components: this is useful to detect possible fall events that will be classified subsequently. This step is only a first discriminant, and we use a low threshold in order to consider all the possible events. This threshold is set to 2g, 1g over the mean value of this RMS that includes gravity acceleration (Fig. 3). During the training phase the threshold was lowered to 1.8g to increase as much as possible the set of ADL events to be included into the classifier. If the RMS is higher than the threshold, the acceleration and geomagnetic potential signals are processed and merged in order to extract the needed features for the SVM. The processing is performed over a 4s window. This window is defined searching the maximum RMS value of the considered event and taking signals samples of the interval [−2s, 2s] around the maximum. If the SVM classifier detects a fall event, the countdown starts. Until it expires, the user can cancel the fall event, and in this case the classifier is updated with this new known event. Otherwise, when the timer expires the alert is launched. The algorithms is well described in Fig. 4.

aFILT(x,y,z)

aRAW(x,y,z)

IIR aFILT(x,y,z) low-pass filter

IIR low-pass mFILT(x,y,z) mRAW(x,y,z) filter

Figure 5.

Figure 3.

Threshold over raw RMS

mi

> th Y DSP and features extraction

Comparison with previous Rot

aEF(x,y,z)

Δy, Δp, Δr

Processing steps

to eliminate noise the two signal are low-pass filtered through a 4th order IIR biquadratic filter (direct form II), with a normalized passband edge frequency of 0.2 (aF ILT and mF ILT );

•

rotation matrix (Rot) is calculated from filtered acceleration signal and fitered magnetometer signal using the getRotationMatrix method provided by SensorManager class;

•

gravity component is removed from high-pass filtered acceleration signal through a FIR high-pass filter with α = 0.8 (ad );

•

acceleration signal referred to the earth frame is computed by multiplying the filtered dynamic acceleration signal by the rotation matrix (aEF );

•

yaw, pitch and roll change angles are extracted using the Android getAngleChange method that compares actual rotation matrix with previous rotation matrix;

i  i+1

N

rotation matrix Rot comput.

⊗

•

ai

RMS calculus

FIR ad(x,y,z) high-pass filter

SVM classification

E. Features Extraction for SVM Classifier Fall?

N

Y Canceled by user? N

Send Alert

The next step is to extract the features for the linear SVM classifier [15]. The features are the values of the following elements in the 4s window: •

(a) Maximum RMS raw value;

Y SVM update

M ax(RM SRAW = •

Figure 4.

Algorithm

When a possible fall event is detected by the threshold based method, the processing steps over acceleration (aRAW ) and geomagnetic field (mRAW ) signals are (Fig 5):

a2xRAW + a2yRAW + a2zRAW ) (1)

(b) Maximum RMS value computed over ad ; M ax(RM S0 =

D. Signals Acquisition and Processing SensorManager class is used to access the device’s sensors and to acquire values from accelerometer and magnetometer. The chosen delay between consecutive samples is 25ms, so the acquisition frequency is 40Hz. This value is the result of a tradeoff and allows the phone to have enough time to elaborate the signals - although the system is executing a multitude of other tasks in multithreading way - maintaining a good precision and efficiency in the detection of fall events.

q

•

q a2x0 + a2y0 + a2z0 )

(c) Maximum absolute value of the three angle changes; M ax({M ax(|∆y|)}, {M ax(|∆p|)}, {M ax(|∆r|)})

•

(3)

(d) Maximum variance over the three angle changes; M ax({M ax(V ar(∆y))}, {M ax(V ar(∆p)}, {M ax(V ar(∆r)})

•

(2)

(e) Maximum value of azEF ;

(4)

Table I. Category

Ending on the knees With arm protection With rotation, ending in lateral position Ending lying

Positive Positive Positive Positive

Lateral

Ending lying

Positive

Lateral

Ending lying

Positive

Syncope

Slipping against a wall, ending sitting Lying on a bed then standing up Walking a few meters Sitting on a chair then standing up Collapsing on a chair Going up and down stairs

Negative Negative Negative Negative Negative

ADL

M ax(azEF ) •

(f) Signal magnitude area of azEF . 1 SM A(azEF (t)) = 4

•

(5)

Z

t+2

azEF (t) dt

(6)

t−2

(g) Variance of azEF .

The features were identified from a larger set after analyzing the results of a long series of tests, looking for those shown notable difference between cases of falls and non-falls. This differences over the features extracted by the training procedure can be observed in a statistical box-plot in Fig. 6. The used SVM classifier is a linear Type 1 classifier (cSVM) that uses Sequential Minimal Optimization to separate the hyperplanes and that is based on 3rd order polynomial. The LIBSVM [16] is used both in Matlab environment (to test the algorithm) and in Android environment. F. Experimental Procedures The training has been performed by 5 people aged between 24 and 29 years. The phone was held in trousers pocket or in shirt pocket. Each trainer user simulated 10 different falls for three times backward falls, forward falls, left lateral falls,

Outcome Positive Positive Positive

Forward Fall

Differences between falls and ADLs features

Activities Ending sitting Ending in lateral position Ending lying

Backward Fall

Figure 6.

E XPERIMENTAL P ROTOCOL

Positive

Bendind down to pick up something on the floor, and getting up

Negative

Using the touch screen of the phone Using the phone to call Putting the smartphone on a table Inserting and removing the phone from a pocket

Negative Negative Negative Negative

right lateral fall, syncope and a variable number (25÷30) of activities of daily living (ADL) - sitting down and standing up, walking, pick something from the floor, lying on a bed, go up and down stairs, coughing or sneezing. Our protocol is based on the one suggested by Noury et al. [17] and Bourke et al. [18]. We choose this protocol because is largely used in literature and in many recent works [19] [20]. We do not include the cases of falls followed by a recovery, because the application allows the user to cancel the alert within a certain time, in those cases. Moreover, we considered the slipping against a wall ending in sitting position as a fall, because can be a dangerous situation for an elderly. In addition we also considered in the ADL group the action of collapsing on a chair - often confused for a fall in many systems - and the most common actions taken with a smartphone: inserting and removing from a pocket, calling, using the touch screen, placing it on a surface. All the features extracted from each known event have been used to train the classifier. In total 150 falls and 140 ADLs have been used to train the classifier. For the method’s performance evaluation other 5 people (aged 23÷27) were involved. Each person performed the complete protocol described in Tab. I three times, for a total of 150 fall events and 150 ADL events. III.

R ESULTS AND D ISCUSSION

The proposed system detected a total of 149 falls, so the sensitivity (true positive rate, TPR) is 99.3%. The only not revealed fall event is a syncope simulation. As regards ADLs only 6 ADL events have been considered as fall events, so the value of the specificity (true negative rate, SPC) is 96%. The total accuracy (ACC) is 97.7% as shown in Tab II. The 6 ADL events wrongly classified as falls were 5 simulations of collapsing on a chair and 1 simulation of lying on a bed. This is a very good result if compared with other similar previous works: Hsieh et al. reached a TPR of 97% and a SPC of 100% [12]; Yavuz et al. reached a TPR of 85% and a PPV of 95%

Table II.

R ESULTS

[2]

P = 150

N = 150

T P = 149 FN = 1

T N = 144 FP = 6

TPR =

TP P

= 99.3%

ACC =

SP C =

T P +T N P +N

TN N

[3] = 96%

[4]

= 97.7%

[5]

[13]; Abbate et al. reached a TPR of 100%, SPC of 100% and an ACC of 100% but used an additional external sensor unit [11]; Aguiar et al. reached a TPR of 96.6%, a SPC of 98.6% and an ACC of 97.5%. In particular, the work by Zhang et al. [9], that used a similar SVM approach, reached a TPR of 84.6%. It is important to note that none of the above methods has considered such a broad set of common actions, carried out also in the normal use of the phone. It can be assumed that the accuracy will improve by increasing the set of acquisitions for training, and by having made the system adaptive (the user can cancel eventual false positives: in this case the system will learn and the SVM vectors will be re-computed).

[6]

[7] [8]

[9]

[10]

IV.

C ONCLUSIONS

In this paper a novel method to detect fall events using a smartphone and intended for the elderly is presented. The fact that we have chosen a mobile phone as hardware of the system implies that the elderly does not need to wear any other device or sensor. This is a big advantage, beacuse often the use of invasive objects like wearable sensors represents a problem for the patient. In this case the elderly has only to hold the smartphone in his pocket, and this can be considered as a quite usual action. Another advantage is that we focused on smartphones at an affordable price because the only condicio sine qua non for the functioning of the system is the presence of accelerometer and magnetometer. The used technique proved to be very effective to discriminate falls from the actions of everyday life, with high values of sensitivity and specificity. Similar methods have already been used in the past, but never on a smartphone using a fusion of data from accelerometer and magnetometer. Moreover the application allows the user to continuously update and improve the SVM classifier in case of false positives. The critical point revealed by the tests is the difficulty to distinguish a collapse on an armchair from a fall event, and the actual state of the work is to search new features from signals useful for this purpose.

[11]

[12]

[13]

[14]

[15]

[16]

[17]

[18]

Actually the application is under debug testing, through an intensive and stressfull usage by the developers, and will be released on Play Store as soon as possible.

[19]

ACKNOWLEDGMENT

[20]

The authors are grateful to WiSense Ltd. (www.wisense.it) for the support during their research activities. R EFERENCES [1]

W. H. Organization et al., “Health and ageing: A discussion paper,” 2001.

W. H. O. Ageing and L. C. Unit, WHO global report on falls prevention in older age. World Health Organization, 2008. L. Z. Rubenstein, “Falls in older people: epidemiology, risk factors and strategies for prevention,” Age and ageing, vol. 35, no. suppl 2, pp. ii37–ii41, 2006. R. J. Gurley, N. Lum, M. Sande, B. Lo, and M. H. Katz, “Persons found in their homes helpless or dead,” New England Journal of Medicine, vol. 334, no. 26, pp. 1710–1716, 1996. M. Mubashir, L. Shao, and L. Seed, “A survey on fall detection: Principles and approaches,” Neurocomputing, vol. 100, pp. 144–152, 2013. Emarketer, “Smartphone users worldwide will total 1.75 billion in 2014,” Jan. 2014. [Online]. Available: http://www.emarketer.com/Article/Smartphone-Users-WorldwideWill-Total-175-Billion-2014/1010536 IDC, “Smartphone os market share, q2 2014,” 2014. [Online]. Available: http://www.idc.com/prodserv/smartphone-os-market-share.jsp Y. Zhuang, J. Baldwin, L. Antunna, Y. O. Yazir, and S. Ganti, “Tradeoffs in cross platform solutions for mobile assistive technology,” in Communications, Computers and Signal Processing (PACRIM), 2013 IEEE Pacific Rim Conference on. IEEE, 2013, pp. 330–335. T. Zhang, J. Wang, P. Liu, and J. Hou, “Fall detection by embedding an accelerometer in cellphone and using kfd algorithm,” International Journal of Computer Science and Network Security, vol. 6, no. 10, pp. 277–284, 2006. Y. He, Y. Li, and S.-D. Bao, “Fall detection by built-in tri-accelerometer of smartphone,” in Biomedical and Health Informatics (BHI), 2012 IEEE-EMBS International Conference on. IEEE, 2012, pp. 184–187. S. Abbate, M. Avvenuti, F. Bonatesta, G. Cola, P. Corsini, and A. Vecchio, “A smartphone-based fall detection system,” Pervasive and Mobile Computing, vol. 8, no. 6, pp. 883–899, 2012. S.-L. Hsieh, M. H. Su, L. F. Liu, and W.-W. Jiang, “A finite state machine-based fall detection mechanism on smartphones,” in Ubiquitous Intelligence & Computing and 9th International Conference on Autonomic & Trusted Computing (UIC/ATC), 2012 9th International Conference on. IEEE, 2012, pp. 735–739. G. Yavuz, M. Kocak, G. Ergun, H. O. Alemdar, H. Yalcin, O. D. Incel, and C. Ersoy, “A smartphone based fall detector with online location support,” in International Workshop on Sensing for App Phones; Zurich, Switzerland, 2010, pp. 31–35. T. Zhang, J. Wang, L. Xu, and P. Liu, “Fall detection by wearable sensor and one-class svm algorithm,” in Intelligent Computing in Signal Processing and Pattern Recognition. Springer, 2006, pp. 858–863. N. Cristianini and J. Shawe-Taylor, An introduction to support vector machines and other kernel-based learning methods. Cambridge university press, 2000. C.-C. Chang and C.-J. Lin, “LIBSVM: A library for support vector machines,” ACM Transactions on Intelligent Systems and Technology, vol. 2, pp. 27:1–27:27, 2011, software available at http://www.csie.ntu.edu.tw/ cjlin/libsvm. N. Noury, A. Fleury, P. Rumeau, A. Bourke, G. Laighin, V. Rialle, and J. Lundy, “Fall detection-principles and methods,” in Engineering in Medicine and Biology Society, 2007. EMBS 2007. 29th Annual International Conference of the IEEE. IEEE, 2007, pp. 1663–1666. A. Bourke, J. Obrien, and G. Lyons, “Evaluation of a threshold-based tri-axial accelerometer fall detection algorithm,” Gait & posture, vol. 26, no. 2, pp. 194–199, 2007. B. Aguiar, T. Rocha, J. Silva, and I. Sousa, “Accelerometer-based fall detection for smartphones,” in Medical Measurements and Applications (MeMeA), 2014 IEEE International Symposium on. IEEE, 2014, pp. 1–6. P. Pierleoni, A. Belli, L. Palma, L. Pernini, and S. Valenti, “A versatile ankle-mounted fall detection device based on attitude heading systems,” in Biomedical Circuits and SYstems Conference, 2014 IEEE Conference on. IEEE, 2014.