Activity Classification Feasibility Using Wearables ... - MDPI

Journal of

Sensor and Actuator Networks Article

Activity Classification Feasibility Using Wearables: Considerations for Hip Fracture Akash Gupta 1, * , Adnan Al-Anbuky 1 and Peter McNair 2 1 2

*

School of Engineering and Mathematical Sciences, Auckland University of Technology (AUT), Auckland 1142, New Zealand; [email protected] School of Clinical Sciences, Auckland University of Technology (AUT), North Shore Campus, Auckland 1142, New Zealand; [email protected] Correspondence: [email protected]

Received: 21 November 2018; Accepted: 11 December 2018; Published: 17 December 2018







Abstract: Falls in the elderly are a common health issue that can involve severe injuries like hip fractures, requiring considerable medical attention, and subsequent care. Following surgery, physiotherapy is essential for strengthening muscles, mobilizing joints and fostering the return to physical activities. Ideally, physiotherapy programmes would benefit from active home-based monitoring of the elderly patients’ daily activities and exercises. This paper aims at providing a preliminary analysis addressing three key research questions. First, what are the key involved activities (at-hospital, home exercises, and activities of daily living) during the post-operative hip fracture rehabilitation process? Second, how can one monitor and identify a range of leg exercises accurately? Last, what is the most suitable sensor location that can categorize the majority of the physical activities thought to be important during the rehabilitation programme? During preliminary testing, it was noted that a standard deviation of the acceleration signal was suitable for classification of static activities like sitting, whereas classification of the ambulatory activities like walking, both the frequency content and related amplitude of the acceleration signal, plays a significant role. The research findings suggest that the ankle is an appropriate location for monitoring most of the leg movement physical activities. Keywords: activity recognition; hip fractures; rehabilitation; sensor localization; wearables; tri-axial accelerometer

1. Introduction Hip fracture is a common event among the elderly population. This often leads to loss of independence with high morbidity and mortality rates [1]. It is a common health issue arising from osteoporosis. It is normally combined with a fall and is anticipated to become the forefront in terms of demand for healthcare and social welfare services [2]. The effect of physical therapy following surgery has been presented previously [3,4] and shows its potential to increase strength, range of motion, and mobility. Despite continual therapy, some analysis [4,5] reveals that there is considerable variability in responses to rehabilitation, and it becomes challenging for healthcare professionals to keep a close track of the patient’s daily activities/exercise and hence health progress. It has been expressed that following hip fracture surgery, the patient should be provided with an optimal, well-coordinated, and organized rehabilitation programme [6]. In order to be able to achieve such a program, we require active tracking. This will support the needs for the well-coordinated and organized rehabilitation programme. The patient can be tracked actively and in real time through wearable sensors attached on the body, remote communication, and the activity recognition approach. J. Sens. Actuator Netw. 2018, 7, 54; doi:10.3390/jsan7040054

www.mdpi.com/journal/jsan

J. Sens. Actuator Netw. 2018, 7, 54

2 of 17

In fact, wearable sensor-based activity monitoring has been considered quite effective for clinical applications in activity or movement measurement and classification [7,8]. The use of wearable sensors supports patients to live an independent life safely. It also helps healthcare professionals in actively tracking a patient’s activity levels on a regular basis once discharged from hospital [7,9]. Understanding the recovery progression stages and the activities involved during the hip fracture rehabilitation process is essential for planning the sensor-based monitoring system. No specific guidelines are accessible pertaining to the key activities involved at different stages of rehabilitation following hip fracture. Based on the information provided from the literature [10,11], it is apparent that there are different activities at each stage of treatment that occur for hip fracture patients. The post-operative hip fracture healing progressive stages involve a number of stages. These stages cover the rehabilitation requirements at hospital, indoor at-home activities, and outdoor activities. At hospital, bed-mobility, which involves turning and bridging activities, is the first stage of the rehabilitation. This is followed by the second stage, which is the functional tasks, range of motion, and strengthening exercises. The activity types associated here are: 1. 2. 3.

The transitions, like lying to sitting and sitting to standing. The ambulatory activities, like climbing the stairs and walking with walking aids. Minimizing impairments, like walking with weights, exercycle, and stretching routines.

Rehabilitation in the indoor environment that encourages independent living covers the third stage. The focus here is on the lower extremity physical daily assisted living. This involves four groups of activities: 1. 2. 3. 4.

Stationary exercise while lying on the back and stomach. Stationary exercise while sitting, i.e., straightening the knee from 90-degree flexion to fully extended and then returning to flexed. Stationary exercise while standing, like lifting the thigh upwards in front of the body, swinging a leg side to side, stepping up, and squats. Exercycle, i.e., time spent cycling on a stationary bike.

Apart from the continued improvement in activities described above, rehabilitation in the outdoor environment that helps the patient go out of the home and maintain as much independence as possible covers the final stage of rehabilitation. The focus here is on gait-related activity. This includes mostly walking and knowing the distance travelled, and speed is the key aspect here. The post-operative effects of physiotherapeutic exercises and mobility training during the hip fracture rehabilitation process for elderly patients have been discussed by a number of scholars [12–15]. Limited information is available that accurately describes the physical activities undertaken at key stages during the post-operative rehabilitation, particularly those stages that occur when the patient is at home. Appreciating the number of sets of different exercises and the time spent by the patient when sitting, standing, or walking during the course of the day is of significance as it has not been addressed sufficiently by other researchers. As a result, the development of a new active monitoring system that can monitor the different key activities like sitting, standing, climbing stairs, walking, exercycle, leg movement, and swinging the leg to the side as described above is valuable for monitoring physical activity during independent living [16]. This paper is organized as follows: The next section describes the methods used for activity sensing and data analysis. Section 3 provides the results of analysing the data collected on the selected activities. Section 4 discusses the findings, and Section 5 presents some conclusions. 2. Wearable Data Collection and Analysis Method This section discusses the wearable device used for activity data collection. It also discusses the approach for activity data analysis, presentation, and recognition. The subject used for this research is

J. Sens. Actuator Netw. 2018, 7, 54

3 of 17

J. Sens. Actuator Netw. 2018, 7, x FOR PEER REVIEW

3 of 17

a healthy young is a healthy youngindividual. individual.Although Althoughthis thisdoes doesnot notoffer offerprecise preciserepresentation representation of of aa post-operative post-operative hip fracture elderly patient, it does satisfy the early stages of studying the feasibility for the the approach approach hip fracture elderly patient, it does satisfy the early stages of studying the feasibility for before proceeding proceeding with before with trials trials on on elderly elderly people people and and then then elderly elderly patients. patients. 2.1. Wearable Activity Sensing Device System Used 2.1. Wearable Activity Sensing Device System Used Acceleration-based sensing has been considered predominant for activity classification [17]. Acceleration-based sensing has been considered predominant for activity classification [17]. As As a result, the wearable sensor hardware used is the micro-electromechanical sensor (MEMS) a result, the wearable sensor hardware used is the micro-electromechanical sensor (MEMS) triaxial triaxial accelerometer MPU6050 (InvenSense Inc., San Jose, CA, USA). It offers four different ranges accelerometer MPU6050 (InvenSense Inc., San Jose, CA, USA). It offers four different ranges of2 of acceleration: ±2 g, ±4 g, ±8 g, ±16 g, where g is the acceleration due to gravity in m/s . acceleration: ±2 g, ±4 g, ±8 g, ±16 g, where g is the acceleration due to gravity in m/s2. The acceleration The acceleration range of ±2 g is considered sufficient to detect ambulation activities [18]. The three range of ±2 g is considered sufficient to detect ambulation activities [18]. The three layers as shown layers as shown in Figure 1 below represent the wireless controller (Microduino Core+), the triaxial in Figure 1 below represent the wireless controller (Microduino Core+), the triaxial accelerometer, accelerometer, and the SD card. The stored test data are then processed off-line using MATLAB FFT and the SD card. The stored test data are then processed off-line using MATLAB FFT software software (8.6.0.267246 (R2015b), Mathworks, Natick, MA, USA. This software identifies the dominant (8.6.0.267246 (R2015b), Mathworks, Natick, MA, USA. This software identifies the dominant spectrum relevant to each of the activities. spectrum relevant to each of the activities.

Figure Figure 1. 1. Wearable Wearable activity activity sensing sensing device device system. system.

Activity Data Collection Process 2.2. Activity Real-time triaxial triaxial accelerometer accelerometer data data are are collected collected at at aa sampling sampling frequency frequency of of 128 128 Hz for each Real-time activity (i.e., static, ambulatory, and hip fracture activities) over a five-minute period. Five minutes is selected as the preference for the data collection due to the following reasons. reasons. willprovide provideaasufficient sufficientnumber numberofofactivity activity samples. Second, it offers good coverage First, itit will samples. Second, it offers good coverage for for the range of activities especially for slow movement activities likemovement leg movement sitting, the range of activities especially for slow movement activities like leg whilewhile sitting, slow slow walking, and swinging leg side to the sidestanding. while standing. It will help in analysing the walking, and swinging the legthe to the while It will also helpalso in analysing the dynamic dynamic switching onetoactivity to the other. switching from one from activity the other. The structure of the five-minute period is divided into 10 data collection tests where each test time is collects data for 24 24 ss followed followedby byaa6-s 6-shalt. halt.The Thetest testisissubsequently subsequentlyrepeated repeateduntil untilthe therequired required time completed. The 24-s epochs with 6-s rest are chosen ad-hoc. However, five other different times (1, 3, is completed. The 24-s epochs with 6-s rest are chosen ad-hoc. However, five other different times 6, 12, 18and s) are explored on a given Aftercase. threeAfter activities analysed movement, (1, 3, and 6, 12, 18also s) are also explored oncase. a given threeare activities are(leg analysed (leg slow, and fast walking at the ankle location only), the findings that no difference observed movement, slow, and fast walking at the ankle location only),show the findings show that was no difference on reducing collection 6, 12, or time 18 s for each activity recognition results was observedthe ondata reducing the time data to collection to 6, 12,set, or as 18 the s for each set, as the activity are the same. In fact, forthe three seconds, shown Figure 2, is observed that 2, forit fast walking that and recognition results are same. In fact,asfor threein seconds, asitshown in Figure is observed leg fast movement both maximum frequency and amplitude dataand seem to be more consistent for walkingactivities, and leg movement activities, both maximum frequency amplitude data seem to compared to slow walking. Moreover, on reducing one second, inconsistency observeddata for be more consistent compared to slow walking. toMoreover, ondata reducing to oneissecond, both maximum and frequency across all activities. inconsistency is amplitude observed for both maximum amplitude and frequency across all activities.

J.J. Sens. Sens. Actuator Actuator Netw. Netw. 2018, 2018, 7, 7, 54 x FOR PEER REVIEW

of 17 44 of 17

Figure 2.2.Three-second Three-second data collection comparison of activities three activities the frequency Figure data collection timetime comparison of three using theusing frequency spectrum spectrum with the maximum amplitude. with the maximum amplitude.

2.3. 2.3. Activity Activity Data Data Processing Processing Model Model This This section section describes describes the the mathematical mathematical expressions expressions used used for for the: the: mean mean (m), (m), standard standard deviation deviation (σ), skewness (S), kurtosis (K), amplitude (A), and frequency peak (FP) involved in data processing (σ), skewness (S), kurtosis (K), amplitude (A), and frequency peak (FP) involved in data processing algorithm algorithm techniques techniques for for activity activity classification. classification. These These are are as as follows follows [19]: [19]: K 11 K cci ⋅·nni mm== ∑ NN i =1 i i

(1) (1)

i =1

1 K 2 σv =  (ci − m) ⋅ ni u 1 = i N K u1 σ = t 1 ∑K (ci − m)32 · ni S = N i= 1 (c − m ) ⋅ n i

i

Nσ i =1 11 KK (c − m)43 ⋅ ni i K= 3∑ S= 4 ( ci i− m ) · n Nσ Nσi=1i =1 3

A = max K 128 (1 : (5 / FS )) | 1 |α K= ( c − m )4 · n i 4∑ i Nσ FP i==(1l − 1) × FS

(2) (2) (3)

(4) (3) (5) (4)

(6) A = max|α128 (1 : (5/FS))| (5) where N represents the number of total data samples, ci is the centre value for the ith interval of the = (l −i, 1K) is ×the FS number of intervals, α128 represents every (6) histogram, ni = the data sample number atFP interval fourth sample of the total FFT sample dataset, l = 1:(5/FS), and Frequency Scalar (FS) = (sampling where N represents the number of total data samples, ci is the centre value for the ith interval of the frequency/N4)/128. N4 is equal to a value of four. It takes every fourth sample from the collected histogram, ni = the data sample number at interval i, K is the number of intervals, α128 represents every accelerometer data. The value of 128 represents the length of α128. fourth sample of the total FFT sample dataset, l = 1:(5/FS), and Frequency Scalar (FS) = (sampling A frequency range of 20 Hz was considered sufficient to capture everyday activities [20]. With frequency/N4 )/128. N4 is equal to a value of four. It takes every fourth sample from the collected respect to this point, the proposed sampling frequency is down-sampled to 32 Hz by taking every accelerometer data. The value of 128 represents the length of α128 . fourth sample. This is represented in Equation 5 as α128. A frequency range of 20 Hz was considered sufficient to capture everyday activities [20]. Some of the activities investigated are part of a daily life routine, such as sitting, standing, and With respect to this point, the proposed sampling frequency is down-sampled to 32 Hz by taking every walking. Categorizing each of these activities is important to associate or discriminate posture within fourth sample. This is represented in Equation 5 as α128 . the context of the exercises prescribed by a physiotherapist. For example, if a physiotherapist needs Some of the activities investigated are part of a daily life routine, such as sitting, standing, to monitor the leg movement of a person while sitting, categorizing the posture irrespective of the and walking. Categorizing each of these activities is important to associate or discriminate posture limb movement is important. within the context of the exercises prescribed by a physiotherapist. For example, if a physiotherapist It has been acknowledged that, based on sensor orientation, it is possible to classify some static needs to monitor the leg movement of a person while sitting, categorizing the posture irrespective of activities [19]; however, this alone is not enough to discriminate all types of activities. However, for the limb movement is important. a detailed classification of static activity behaviour, Baek et al. [19] used four parameters, skewness,

J. Sens. Actuator Netw. 2018, 7, 54

5 of 17

It has been acknowledged that, based on sensor orientation, it is possible to classify some static activities [19]; however, this alone is not enough to discriminate all types of activities. However, for a detailed classification of static activity behaviour, Baek et al. [19] used four parameters, skewness, kurtosis, mean, and standard deviation, and categorized the activity based on a single-axis sensor orientation. This paper combines the three-axis raw accelerometer data and subsequently categorises static exercises (i.e., sitting and standing) based on these parameters. The purpose of combining the sensor axes is to make the algorithm immune to potential errors in aligning the sensors with body or anatomical axes. In fact, for ambulatory and hip fracture-related activities categorisation, the combined three-axis raw accelerometer data are subjected exclusively to FFT. This is to identify the dominant frequency peak corresponding to the maximum amplitude. Identification of the maximum amplitude and frequency would help in comparing and classifying each activity accurately, as discussed more in the Results Section. 2.4. Considerations for the Body-Worn Sensor The location of a sensor on the human body is significant from the perspective of human comfort, obtaining precise and accurate data, communication optimization, usability, and adaptability. It is acknowledged that sensor placement is highly dependent on the type of activity monitored and is application specific [21,22]. Sensors with single and multiple accelerometer sensors have been placed at different locations to explore the optimal sensor location for a range of activities [21–25]. Cleland et al. [21] investigated six different body locations to determine the optimal placement of an accelerometer sensor for recognition of basic everyday activities. Their analyses showed that sitting and standing results were poor when the sensor was on the foot. This, however, was more accurate for lying and jogging. Notably, the lower back is not considered suitable for classification, as the accelerometer tends to twist and causes irregularities in the data. Hence, Cleland et al. [21] concluded that the left hip (sensor located at the iliac crest) is the ideal location for classifying everyday activities. Atallah et al. [24] focused on investigating the ideal sensor location for a varied group of activities involving upper and lower limbs. Results showed that wrist location provided good precision and identification rates for activities like preparing food and eating, whereas a waist-mounted sensor showed good results for lower limb activities like walking and sitting. Unlike Cleland et al. [21], Atallah et al. [24] did not investigate the effect of combining accelerometer data on classification accuracy. Overall, there are limited evidence and information available about sensor placement in general. No one location can be viewed as perfect, and this factor is still a subject of debate that needs to be addressed [16,26]. Since hip fracture exercises and physical activities focus on lower limb activities, an ankle location was considered as it has the potential to quantify most exercises and particularly gait-related activities and related features like walking, energy expenditure, and distance. An important feature related to localization is sensor orientation. It may be directly attached to the skin by glue or tape or be placed within a strap. This becomes more complicated if developing a user-independent sensing device [27]. This is because it gives the user the freedom to adjust the sensor position and its placement to fit with their clothing choice [22]. Allowing such adjustments could alter signals from sensors like accelerometers quite dramatically, leading to erroneous categorisation of exercise and activities. In our study, as part of the preliminary testing, four different sensor locations (waist, hip, thigh, and ankle) were considered, as shown in Figure 3 below. Hip, thigh, and ankle sensors were attached on the right side of the body, whereas a waist sensor was located at the centre of the body.

J. Sens. Actuator Netw. 2018, 7, 54

6 of 17

J. Sens. Actuator Netw. 2018, 7, x FOR PEER REVIEW

6 of 17

Figure 3. 3. Sensor locations. Figure Sensor placement placement locations.

3. Results This section represents the experimental data analysis of various activities with respect to sensor localization. The description of each of the activities is as follows. 3.1. Static Activities Activities 3.1. Static Sitting vs. Standing 3.1.1. Sitting vs. Standing Data for Little difference difference Data for two two static static activities, activities, sitting sitting and and standing, standing, are are shown shown in in Figures Figures 44 and and 5. 5. Little is across thethe dominant frequencies observed for these activities. However, across is seen seenininthe theamplitude amplitude across dominant frequencies observed for these activities. However, sensor locations, there were notable differences in the dominant frequencies. Because the frequency across sensor locations, there were notable differences in the dominant frequencies. Because the was quite different and different the amplitude be considered together frequency was quite and was the minimal, amplitudethese wasparameters minimal, cannot these parameters cannot be for categorisation of sitting and standing. considered together for categorisation of sitting and standing. 2 can be utilized to categorise the It It seems seems appropriate appropriate that that aa low low amplitude amplitude (value (value