MEMS Based Sensing and Algorithm Development for Fall Detection ...

MEMS Based Sensing and Algorithm Development for Fall Detection and Gait Analysis Piyush Gupta, Gabriel Ramirez, Donald Y.C. Lie, Tim Dallas, Ron E. Banister+ and Andrew Dentino+ Dept. of Electrical and Computer Engineering, Texas Tech University, Lubbock, TX 79409, USA + Texas Tech Health Sciences Center (TTUHSC), Lubbock, TX 79409, USA ABSTRACT Falls by the elderly are highly detrimental to health, frequently resulting in injury, high medical costs, and even death. Using a MEMS-based sensing system, algorithms are being developed for detecting falls and monitoring the gait of elderly and disabled persons. In this study, wireless sensors utilize Zigbee protocols were incorporated into planar shoe insoles and a waist mounted device. The insole contains four sensors to measure pressure applied by the foot. A MEMS based tri-axial accelerometer is embedded in the insert and a second one is utilized by the waist mounted device. The primary fall detection algorithm is derived from the waist accelerometer. The differential acceleration is calculated from samples received in 1.5s time intervals. This differential acceleration provides the quantification via an energy index. From this index one may ascertain different gait and identify fall events. Once a pre-determined index threshold is exceeded, the algorithm will classify an event as a fall or a stumble. The secondary algorithm is derived from frequency analysis techniques. The analysis consists of wavelet transforms conducted on the waist accelerometer data. The insole pressure data is then used to underline discrepancies in the transforms, providing more accurate data for classifying gait and/or detecting falls. The range of the transform amplitude in the fourth iteration of a Daubechies-6 transform was found sufficient to detect and classify fall events. Keywords: fall detection, tri-axial accelerometer, MEMS, wireless sensors, gait classification

1. INTRODUCTION Falls are unfortunate events that aggravate the risks for mortality, morbidity, disability, and frailty among the elderly community1. According to the Centers for Disease Control and Prevention (CDC) records for 2006, 16,650 people aged 65 and older died from fall-related injuries. In 2008, more than 2.11 million elderly people (age 65 +) were treated for non-fatal falls with approximately 730,879 resulting in hospitalization2. In past years, average length of such fall-related hospitalization among older adults was found to be approximately 11.6 days1. Falls generally degrade the quality of life of an elderly person and often result in loss of victim’s ability for independent living. Furthermore, falls are also associated with high medical expenses. Approximately $179 million were calculated as direct medical costs for treating fatal falls and $19 billion for non-fatal fall injuries in 20003. Therefore, a lot of time and money has been invested in different parts of the world by different organizations, both public and private, to determine symptoms of falls and their prevention. Before proceeding to the algorithm, it is necessary to understand the meaning of the term fall. A fall can be considered as an unintentional and uncontrolled movement of a human body causing it to lie down on the floor4. Moreover, it may also be known as an unintentional and sudden change in position of an individual, causing him/her to land at a lower level, on an object, the floor or the ground4. Therefore, the individual would not necessarily end up lying on a horizontal floor in order to call it a fall; he/she may come to rest on an inclined floor, stairs or an object after the sudden and unintentional movement. However, if an intervention or a support during the unintentional movement prevents the human body from lying down on the ground, it is usually termed as a “stumble”. Moreover, if a person is found on the floor, the event may still not be a fall unless it was unintentional. Medical monitoring systems have become an important area of research and development due to the possibility of allowing improved quality of life and care while reducing the overall medical costs. A number of systems have been proposed and sometimes tested. A few of these systems are discussed in this section to allow our proposed system to be

put in the proper context. Nyan5-7 proposed a system using MEMSWear Smartshirt to predict a fall using the data from electrocardiography (ECG) and pulse oximeter oxygen saturation (SpO 2) sensors. The system also detects the onset of the fall by using an accelerometer and a gyroscope. The system determines the onset of a fall around 700 ms before the impact on the ground by correlating the orientation of the trunk and thigh with respect to the vertical axis. However, that system mainly targeted patients with cardiovascular disorders who were prone to falls and did not cover patients that may be at the risk of falls due to muscular weakness, effects of medication, suffering from Parkinson’s disease, or gait imbalances. Lan8 proposed the design and implementation of an automatic fall detection system, “SmartFall.” The system was developed to automatically alert the care givers when a fall was detected using sensors mounted in a cane used by the person being monitored. The system consists of a 3-axis accelerometer, three single-axis gyroscopes, and pressure sensors at the handle and tip of the cane. The system classifies an event as a fall after evaluating it in three basic stages: collapse, impact, and inactivity. However, it is necessary for the person to be utilizing the cane prior to the fall, which might not be always possible during the daily routine of activities. There are a few other systems that are proposed for monitoring the gait of an individual to determine the risk of falling 9-15. However, these systems are working towards the analysis of the precursors for falls rather than the detection of falls. In this work, we discuss a system that can successfully detect falls in real-time. The system enables autonomous monitoring and reporting of falls for geriatric patients. The system is designed in such a way that it satisfies a number of requirements ranging from the electronic communication protocols to ergonomic sensors. The system required an algorithm that can evaluate an event based on some pre-determined calculations to classify if it is a fall or not. Therefore, an effective integration of hardware and software tools was also needed that provides robust reporting of falls to doctors and medical staff if used in a medical facility. The system was developed with the help of corporate partners such as Texas Instruments (TI), AT&T, 24eight Inc., and clinicians at Texas Tech University Health Sciences Center (Lubbock, TX).

2. SYSTEM OVERVIEW 2.1 System Components 2.1.1 Insoles A shoe insole can be worn as an insert, inside the socks or even be incorporated into slippers. A geriatric patient under monitor is supposed to wear a pair of insoles almost at all times during daily living activities. The insoles have four pressure sensors each and a tri-axial accelerometer to provide data for pressure distribution across the foot, as well as the acceleration and orientation of the foot during motion at the same time. The insoles are provided by our corporate partner, 24-eight, and are implemented as part of 24-eight’s patented sensor platform. The insoles stream data wirelessly and utilize the Zigbee protocol for communications with the gateway. The insoles transmit data when the pressure threshold (set internally) is exceeded for any of the four pressure sensors, or when the acceleration on any of the axes exceeds its threshold acceleration. Therefore, if an elderly patient sleeps for 7 hours a day on an average, the insoles will be automatically off for that entire duration without burning out battery during the sleep time.

Fig 1(a)

Figure 1(b)

Figure 1 (a) Image of the insole that incorporates four pressure sensors, 3-axis accelerometer, battery, and a wireless radio (by 24eight Inc.); (b)Positions of pressure sensors in the insole. 16, 17

2|P a g e

2.1.2 Belt-clip accelerometer A tri-axial accelerometer is a device which measures acceleration in three orthogonal directions. The accelerometer is attached to a belt-clip that can be easily worn over the waist on a belt. The belt clip is supported by two AAA batteries which will last several weeks before requiring the replacement. The belt-clip accelerometer is a MEMS sensor provided by our corporate partner 24eight and utilizes the Zigbee protocol for wireless communication to the gateway. The beltclip is 11 cm x 4 cm x 4.5 cm in size and can be easily worn. The belt-clip has an ON-OFF switch at the top that will help the elderly person or medical staff to switch it OFF. The accelerometer reports acceleration in the range of + 6.0g.

Figure 2: Picures of belt-clip accelerometers (provided by 24eight Inc.)

2.1.3 AT&T Gateway AT&T is the another major corporate partner in this research project. AT&T is lending their support by providing a preproduction Actuarius gateway which converts a message in the Zigbee protocol to one in TCP/IP. The Zigbee protocol is used by the wireless devices for sending the data through the gateway. Once reached at the gateway, the data packets are sent to the server side processing unit utilizing TCP/IP protocol as shown in Figure 3.

Figure 3: AT&T Actuarius gateway and the conversion of Zigbee packets into TCP/IP

2.2 System Integration A wireless real-time monitoring system built using the devices explained in the above sections can monitor and record data during different daily living activities of an elderly person. The system is comprised of wireless MEMS and pressure sensors transmitting data to the Actuarius gateway. The sensors utilize the Zigbee protocol for over-the-air communication. Once the data is received by the gateway, it is translated into TCP/IP and sent to the main server and database through a hard-wire connection where the calculations can be performed. Moreover, the data is safely recorded into a database in the form of ‘.xml’ file for future analysis. An important point that was carefully taken care of during the system integration was that each message will be time-stamped whenever it enters the gateway and whenever the data packet is received by the server. Therefore, the data will remain useful in the future and moreover, time-stamping allows the debugging of the system to identify the delay in the packets received by the server. During the testing and system integration phase, time delay of 90 ms and less was observed in the data received by the server. Furthermore, while in the research phase, a video camera is integrated in the system that tracks and records the movements of the subject, simulating different daily living activities and different types of falls. This video recording is done concurrently with the data capturing so as to allow better understanding of the data.

3|P a g e

Figure 4: Fall detection system overview

3. Fall Detection Algorithm The main goal of this work was to come up with an algorithm that can be implemented in the real-world so that all the falls can be detected without generating a significant number of false alarms. In simpler words, the system should provide high sensitivity and specificity for fall detection. The algorithm was initially started by simulating different activities and induced falls, and the data for each activity was recorded in an .xml file for analysis and further manipulation. The acceleration of the whole body, or of the center of mass of the body, was analyzed to observe the particular features that can describe a falling body. Therefore, the data was observed in several ways by taking resultant acceleration of the three axes, and examining the change of acceleration on only two axes (X - Z or Y – Z) at a time. The following activities were simulated and recorded for the analysis after placing accelerometer on three subjects: Table 1: Data-set for the fall detection analysis Activities

Data set

Standing still

3 minutes (1 min x 3)

Walking

5 minutes (1 min x 5)

Sitting - standing

20 times (in the duration of 7-8 minutes)

Shuffle walk

5 minutes (1 min x 5)

Falling

14 times (including falls in the four principal directions and on either of single knees)

As each individual may have a different way and style performing a daily activity, all of the activities for this analysis were repeated and recorded at different paces and in different ways so as to cover a wide range of styles. For example, while sitting on a chair, the activity was repeated once by taking the support from arm-rest of the chair, then sitting by taking support of the wall or nearby object and/or without taking any external support.

4|P a g e

When the plots were analyzed for extracting information, sharp negative peaks were also observed (in all the three axes) in the cases of falls which were missing in the cases for daily activities. This phenomenon can be more easily understood by the action-reaction law as stated by the Newton’s laws. When a person falls to the ground, a large acceleration is caused due to the stumble and falling of the human body. However, this acceleration is reversed once the body hits the ground, due to the reaction provided by the ground to bring the body to rest. Even for slower impacts, a reasonable amount of action-reaction forces are observed. Moreover, the slope of the acceleration, i.e. the change of acceleration in a particular period of time, seems to provide much better information about if the movement was dangerous or not. To calculate the differential acceleration that can effectively separate falls from all other activities, a suitable time window has to be chosen. The ideal window is supposed to have a length just big enough to contain the entire information about the fall, i.e. all the action and reaction forces experienced during the fall. Finding a suitable window is absolutely necessary because if the window is too short, it will not contain the entire information of the fall and will not be very effective. Moreover, if the window is too long, it may cover more than one activity and one may falsely interpret it as a fall. Though falls are unpredictable and can occur at a range of various speeds, the best possible window was found to be 1.5 s after repeated testing of a large data set taken from young graduate students. An important thing to note is that only the maximum and minimum acceleration experienced on each axis within the window duration is considered for calculating the differential resultant acceleration. This way, the maximum change of acceleration within a time period is approximately considered to be the total change of acceleration. The formula used for calculating the differential resultant acceleration is given as: )

(1)

Where: dx = xmax - xmin, dy = ymax - ymin, and dz = zmax - zmin Once a window length is determined for a set of experiments, the window is then moved (slid) across the entire data set to observe the high peaks and specific signature of the activities. For example, suppose a data-set constitutes 600 sample points for each X, Y and Z axis, where the sampling rate is 50 samples/second for each of the parameters, and a windowlength of 1.5 seconds is chosen and applied over the data-set. Therefore, as the number of samples transmitted and received in one second of duration is 50, the window length may also be called 75 samples long. Now, the 75 newest samples (data points) for X, Y and Z acceleration will stay in the cache memory all the time during the execution. Therefore, the first 75 samples (first window) are taken and executed according to the equation 1 shown above to determine the differential resultant acceleration. Once calculated, the window slides forward to include the 76th sample inside the window and drops off the first one. The differential resultant acceleration is again calculated for the new window position so as to find the change in acceleration within that time period. The window keeps on sliding forward by including one new sample each time and discarding the oldest one until the entire 600 samples in the data-set are covered. This method allows finding the maximum change of acceleration within any 1.5 second duration during a particular activity. Upon observing and analyzing these peaks for different sets of activities, threshold acceleration may be identified which can separate the fall from non-fall activities.

4. RESULTS Now, a base model for the threshold-based fall detection system is ready, which will distinguish unintentional movements from intentional movements using a peak detector. After analyzing results from the dataset explained in Table 2, the threshold acceleration was chosen to be 8.0 g (where ‘g’ refers to the acceleration due to gravity). By placing the threshold at 8.4 g, a wide gap separates the normal daily living activities from the dangerous falls. Table 2: Differential resultant acceleration peaks observed during different activities Activity Standing Walking Sitting Shuffle Walk Fall

5|P a g e

Min (Peak) 0.1 g 4.0 g 1.8 g 4.0 g 8.4 g

Max (Peak) 0.4 g 5.3 g 4.0 g 4.9 g 13.6 g

Median 0.2 g 4.6 g 2.4 g 4.3 g 10.1 g

Mean 0.26 g 4.71 g 2.49 g 4.42 g 10.27 g

Std. Deviation 0.10 g 0.62 g 0.63 g 0.41 g 1.51 g

The system was then tested for the sensitivity and specificity after simulating falls and daily living activities by three young healthy subjects. Though the system was not tested for all the possibilities, the system demonstrated good results for the ones that were performed. Different activities like sitting, walking, standing, shuffling, and falling were simulated and the data was recorded to calculate the sharp spikes in differential acceleration. After comparing them with the threshold of 8.4 g, eighteen out of twenty falls were successfully detected whereas four out of six of the stumbles were misidentified as falls. This provides us a system with the sensitivity of 90% and specificity of 87.8%. On further analysis, this algorithm will not detect a fall when the event occurs at a very slow speed. If a person takes a long time in going down by trying to sit or bend once the onset of fall is sensed by the individual, or takes support of an object to protect from hitting the ground hard, the acceleration equal or greater to threshold may not be generated and hence the fall may get missed. Moreover, after the onset of an unintentional movement, if the person gets the support of nearby standing person or nearby placed object preventing him/her to land at a lower level, the unintentional movement may be wrongly classified as a fall instead of a stumble. Therefore, to improve the specificity of the system, orientation of the subject should be checked after the high acceleration.

5. Algorithm Collaboration In order to provide collaboration in fall detection methods, an additional algorithm was developed. The primary motivation for the algorithm was to develop a method of gait quantification so that subject specific variables such as weight, height, and body mass are not considered as factors. Therefore, a frequency analysis of steady state gait was deemed to be the desired mode of quantification. However due to the non-periodicity of the accelerometry signals, classical spectral analysis does not accurately describe the resulting biosignal spectrum. Instead, wavelet transforms were applied to the accelometric signals from the belt clip. In addition to the obvious resolution benefits, the over-sampling inherent in the transformation would also allow for fewer samples to be collected while maintaining reliable fall detection capability. Belt clip data was taken from 30 trials on a single healthy young adult male. A single insole was used as a foot switch to further determine the correct time when the subject was no longer standing. The insole was placed snuggly in a right foot slipper. The left slipper was worn during the trials as well. In each trial, an approximate two minute sample of data was taken at a sampling rate of 85 Hz. The subject would perform a myriad of tasks including walking, shuffling the feet, sitting with strait posture in a chair, and cooking dinner. Each trial would contain a single type of fall. Falls were defined by how the person landed, therefore falls onto the back, forward with hands extended, left and right side, left and right single knee crouch, double knee forward, and left and right crumples were noted. Falls were not stunted, to the best of the subject's ability, and both concrete and carpeted surfaces were used. Discrete wavelet transforms were conducted using Mathworks Matlab with the Wavelet Toolbox. The scale was then used to quantify the discrepancies that would occur when a fall was noted in the accelerometry waveform. A debauchies4 mother wavelet was deconstructed to the fourth level, and the absolute value for each sample of the detail approximation scale was performed. The fourth deconstruction was found to remove many of the high frequency artifacts that occur in trunk accelerometry, while also retaining the majority of fall and gait characteristics. The debauchies-4 mother wavelet was noted to be of a similar pattern in regard to fall waveforms detected in accelerometry. This similarity would therefore produce high correlation coefficients at the moment in time of a fall. A threshold of scale was empirically determined and set at 1024 for the x-axis, 1024 for the y-axis, and 1536 for the z-axis. If the scale at any moment in the waveform ever appeared above the threshold for all three accelerometer axes, then a fall was considered to have happened at that time in the waveform. It was found that for 87% of the trials, a fall was detected at the correct moment in time. The remaining 13% detected the falls, but also reported false positives. In order to eliminate the false positives, the timestamp for the detected fall was correlated to the timestamp of when the footswitch turned off. If in addition to the correlation the activity on the footswitch data also remained zero for 3 seconds, then a true fall was reported as being detected. Using the footswitch data to eliminate false positives increased the fall detection reliability by 8% for the trials.

6. Conclusion It has been demonstrated that the system can detect falls using the threshold based differential acceleration peak detection algorithm. The system demonstrated good results of 90% sensitivity and 87.8 % specificity under the controlled lab environments on three healthy subjects. The system may not detect falls occurring at a very slow speed. Moreover, the system may detect stumbles occurring at high speed as falls because the differential acceleration may

6|P a g e

exceed the threshold. The specificity of the system can be further improved after observing the orientation of the subject once the threshold is exceeded. Furthermore, because the normal daily living activities (stumbles not included) and the events of falls display a wide gap in acceleration signatures, the threshold might be further reduced to increase the system sensitivity. The wavelet transform of the raw data can be used as an additional tool for confirming a fall. The initial results obtained from the wavelet transforms have shown satisfactory results in detecting a fall. It further motivates us in exploration of the potential of wavelet transforms in the area of gait analysis.

7. Acknowledgment The authors are deeply indebted to the generous funding support by the Texas Instruments (TI) Foundation, especially to Mr. Shekar Rao and Dr. Allen Bowling. The authors would also like to express their heartfelt thanks to AT&T and 24eight for their support and cooperation by providing the necessary devices without which the continuous development of the work would not have been possible. The authors would like to thank Dr. Bob Miller, Dr. Leo Razoumov, Dr. Lusheng Ji and Mr. Alex Kalpaxis for benefiting the work by providing their constructive criticism.

References [1]

[2]

[3] [4] [5] [6] [7] [8]

[9]

[10] [11] [12]

[13] [14]

[15]

A. Shumway-Cook, M. A. Ciol, J. Hoffman, B. J. Dudgeon, K. Yorkston, L. Chan, “Falls in the Medicare Population: Incidence, Associated Factors, and Impact on Health Care”, Phys. Ther., vol. 89, no. 4, pp. 324-332, April 2009. Centers for Disease Control and Prevention, National Center for Injury Prevention and Control. (September, 2009). Web–based Injury Statistics Query and Reporting System (WISQARS). [Online]. Available: www.cdc.gov/ncipc/wisqars J. A. Stevens, P. S. Corso, E. A. Finkelstein, T. R. Miller, “The costs of fatal and nonfatal falls among older adults”, Inj. Prev., vol. 12, no. 5, pp. 290–295, June 2006. Victorian Government Health Information, Australia. (2009, August). Definitions of a fall. [Online]. Available: http://www.health.vic.gov.au/agedcare/maintaining/falls/definition.htm M. N. Nyan, F. E. H. Tay, M. Manimaran, K. H. W. Seah, “Garment-based detection of falls and activities of daily living using 3-axis MEMS accelerometer”, in J. Phys.: Conf. Ser., Singapore, 2006, vol. 34, pp. 1059-1067. M.N. Nyan, F. E.H. Tay, M. Z. E. Mah, “Application of motion analysis system in pre-impact fall detection”, J. Biomech., vol. 41, no. 10, pp. 2297-2304, July 2008. M. N. Nyan, F. E. H. Tay, E. Murugasu, “A wearable system for pre-impact fall detection”, J. Biomech., vol. 41, no. 16, pp. 3475-3481, November 2008. M. Lan, A. Nahapetian, A. Vahdatpour, L. Au, W. Kaiser, M. Sarrafzadeh. Smartfall: An automatic fall detection system based on subsequence matching for the smartcane. Presented at Fourth International Conference on Body Area Networks (BodyNets) – Los Angeles, CA. [Online]. Available: http://bi.eng.utah.edu/pubs/2006_rss_musselman.pdf J. Musselman, M. Gandhi, S. J. M. Bamberg. Development of an in-shoe sensor insole for motion analysis, physical therapy, and rehabilitation. Presented at Robotics: Science and Systems Conference - Philadelphia, PA. [Online]. Available: http://bi.eng.utah.edu/pubs/2006_rss_musselman.pdf S. J. Morris, J. A. Paradiso, “Shoe-integrated sensor system for wireless gait analysis and real-time feedback”, in EMBS/BMES Conf.: Proc. Sec. Joint, Houston, TX, 2002 vol. 3, pp. 2468 – 2469. S. Bamberg, A. Y. Benbasat, D. M. Scarborough, D. E. Krebs, J. A. Paradiso, “Gait Analysis Using a ShoeIntegrated Wireless Sensor System”, IEEE Trans. on Inf. Technol. Biomed, vol. 12, pp. 413-423, July 2008. S. J. M. Bamberg, P. LaStayo, L. Dibble, J. Musselman, S. K. D. Raghavendra, “Development of a Quantitative In-Shoe Measurement System for Assessing Balance: Sixteen-Sensor Insoles”, in Conf. Proc. IEEE Eng. Med. Biol. Soc., New York City, NY, 2006, pp. 6041-6044. H. Lee, L. Chou, “Detection of Gait Instability Using the Center of Mass and Center of Pressure Inclination Angles”, Arch. Phys. Med. Rehabil., vol. 87, no. 4, pp. 569-575, April 2006. H. Noshadi, S. Ahmadian, F. Dabiri, A. Nahapetian, T. Stathopoulos, M. Batalin, W. Kaiser and M. Sarrafzadeh. Smart Shoe for Balance, Fall Risk Assessment and Applications in Wireless Health. Presented at Microsoft eScience Workshop - Indianapolis, IN. [Online]. Available: http://www.cs.ucla.edu/~ani/publications/SmartShoe2008-eScience-.pdf Jay Lindsay. (2008, July 31). 'iShoe' uses NASA tech to help prevent elderly falls. USA Today. [Online]. Available: http://www.usatoday.com/tech/news/techinnovations/2008-07-31-mit-ishoe_N.htm

7|P a g e

[16] [17]

24eight, “Qiq Insoles: Zigbee Enabled ‐ Wireless Motion and Pressure Sensing Insoles for Mobility and Gait Analysis”, Unpublished. Leo Razoumov, “Lua parser”, Unpublished.

8|P a g e