Posture Monitoring System for Context Awareness ... - Semantic Scholar

IEEE TRANSACTIONS ON INSTRUMENTATION AND MEASUREMENT, VOL. 59, NO. 6, JUNE 2010

1589

Posture Monitoring System for Context Awareness in Mobile Computing Jonghun Baek and Byoung-Ju Yun

Abstract—The posture of a user is one of the contextual information that can be used for mobile applications and the treatment of idiopathic scoliosis. This paper describes a method for monitoring the posture of a user during operation of a mobile device in three activities such as sitting, standing, and walking. The user posture monitoring system (UPMS) proposed in this paper is based on two major technologies. The first involves a tilt-angle measurement algorithm (TAMA) using an accelerometer. Unlike most previous studies, it is based on a relative computation using the dot product from the time-series acceleration data. Because TAMA does not require a physical calibration by a user, it is more robust and accurate compared to other methods that rely on absolute computations. The second technology is an effective signal-processing method that eliminates the motion acceleration component of the accelerometer signal using a second-order Butterworth low-pass filter (SLPF). Because the posture of a user is only related to the gravity acceleration component, the motion acceleration components should be removed. The TAMA and UPMS are implemented on a personal digital assistant (PDA). They are evaluated to verify the possibility of application to a mobile device. Additionally, a posture-based intelligent control interface in context-aware computing that reacts to the posture of a PDA user is implemented on the PDA to complement the poor user interface (UI) of the mobile device, and its results are presented. Index Terms—Accelerometer, angle measurement, Butterworth filter, context awareness, dot product, human–computer interaction, posture monitoring, user interface (UI).

I. I NTRODUCTION

C

ONTEXT awareness is a key factor for new applications in ubiquitous computing. The goal of context-aware computing is to offer relevant information and/or services to a mobile device user by capturing its contextual information in the changing situation of the user, the status of a device, or the surroundings. Unlike designing a user interface (UI) for traditional desktop applications, mobile devices should separate complex contextual information. Many kinds of contextual information should therefore be sorted within a context-aware computing environment. The general types of objects in context-aware computing can be sorted as follows: user, physical environment, Manuscript received March 1, 2008; revised January 2, 2009. First published April 5, 2010; current version published May 12, 2010. The Associate Editor coordinating the review process for this paper was Dr. Emil Petriu. J. Baek is with Samsung Electronics Company Ltd., Gyeongbuk 730-722, Korea (e-mail: [email protected]). B.-J. Yun is with the School of Electrical Engineering and Computer Science, Kyungpook National University, Daegu 702-701, Korea (e-mail: [email protected]). Color versions of one or more of the figures in this paper are available online at http://ieeexplore.ieee.org. Digital Object Identifier 10.1109/TIM.2009.2022102

computing system, and user–computer interaction. The contexts for each object can be subdivided into act/body/identity for a user, space/time/environment for a physical environment, available resources/access context for a computing system, and interaction context/obstacle context for a user–computer interaction, respectively. These contexts can also be subdivided into contextual information. For example, the act for a user can be subdivided into activity, posture, gesture, and so on, and the environment for a physical environment can be subdivided into temperature, humidity, intensity of illumination, vibration, and so on [1]–[4]. Traditional desktop-based UIs have been developed based on that the activity of a user is in a static state. To design a UI for desktop devices, one can use all of its visual resources. Representative desktop-based interaction mechanisms are the keyboard, mouse, and joystick. In general, these are very graphical and still more detailed for desktop-based applications. In contrast, for the mobile-device-based UI, one cannot utilize all or any of their visual resources, because of the fact that not only the activity of the users is in a dynamic state [5] but also mobile devices have limited resources, small keypads, and small liquid crystal displays. UIs of mobile devices rely on a keypad, a stylus pen, and an input panel. When using these mechanisms to manipulate mobile devices, malfunctions and time delays—because of a small button and repeated manipulation—may occur. Therefore, it may inconvenience users. In traditional desktop applications, a computer does not input data by itself. However, in mobile applications with context awareness, the mobile device does, because the information can be delivered or requested by nonconscious input of a user. Context-aware computing analyzes contexts in a ubiquitous computing environment and discriminates whether the information is valid or not, and then, if the information is valid, the mobile device generates the control event that requests or delivers information to execute the application. A mobile device, although its computing power is increasing, provides a limited computing environment when compared with a personal computer or an embedded system with a digital signal processor. In addition, a mobile device cannot dedicate its full computing power to auxiliary applications when its primary role is communication. In particular, an accelerometer application for monitoring the posture of a user should run all the time without demanding too much central-processing-unit time. This is not only a matter of reducing computing loads but also a matter of preserving the limited battery power in a mobile device. We should therefore develop a lightweight signal-processing algorithm for monitoring the posture of a user.

0018-9456/$26.00 © 2009 IEEE Authorized licensed use limited to: Kyungpook National University. Downloaded on May 13,2010 at 03:33:13 UTC from IEEE Xplore. Restrictions apply.

1590

IEEE TRANSACTIONS ON INSTRUMENTATION AND MEASUREMENT, VOL. 59, NO. 6, JUNE 2010

Although previous research has studied the posture during mobile computing, there has not been a detailed analysis of the situation [6]. In our paper, the tilt-angle measurement for posture monitoring has been studied to improve the accuracy of the tile angle and to usefully utilize it more than other previous studies. This paper proposes a user posture monitoring system (UPMS) that monitors the posture of a user watching a movie played out by a mobile device. The UPMS was implemented on a personal digital assistant (PDA) by using the tilt-angle measurement algorithm (TAMA) to measure the tilt angle and a second-order Butterworth low-pass filter (SLPF) to eliminate the motion acceleration component of the accelerometer signal generated while a user moves. It is evaluated to verify the possibility of application to a mobile device. Additionally, through the creation of a “Display ON/OFF” function on a PDA, a posture-based intelligent control interface in contextaware computing that reacts to the posture of a PDA user is implemented to complement the poor UI of the mobile device. “Display ON/OFF” is a function that automatically turns on the display of a mobile device when a user watches a movie played out by a mobile device. The user-posture-based input method as interaction between a human and a computer enables such a user to more easily approach tasks than a desktop-based input method. In our posture-based mobile interaction technique, we show the possibility of using a two-axis accelerometer that is embedded in a mobile device, and it has an effect on the input. The structure of this paper is given as follows. Section II describes the sensors used for capturing the contextual information. In Section III, the TAMA is presented. Section IV introduces the UPMS using the TAMA and the SLPF. In Section V, the experimental results for the performance evaluation of the posture-based intelligent control interface are presented. Finally, Section VI summarizes the most important conclusions and offers some discussions for the proposed methods.

charge, containing recording data, requiring long-time analysis, and being very sensitive to surroundings [8], [9]. In particular, these devices are hard to apply to some mobile applications. The mobile device therefore requires a new method that can detect posture easier and faster in a mobile environment. An accelerometer can easily be embedded in a mobile device because an accelerometer has advantages such as being lightweight, small, inexpensive, and relatively impervious to the surroundings. For these reasons, an accelerometer is one of the most widely used sensors in diverse applications such as counting steps [15], posture monitoring [7], [13], activity estimation [16]–[20], and other applications as a primary sensor for capturing contextual information regarding motion [6]. Many new applications of an accelerometer are now being considered by mobile device designers, and in fact, mobile device models with an accelerometer are already available in the market. An accelerometer is therefore a suitable sensor that coincides with this purpose. A two-axis accelerometer (ADXL202 EB) was used to measure tilt angles from the gravity acceleration component. The accelerometer uses the force of gravity as an input vector to determine the orientation of an object in space (see Fig. 1) [21]. The accelerometer is most sensitive to tilt when its sensitive axis is perpendicular to the force of gravity, i.e., parallel to the Earth’s surface. When the accelerometer is oriented on its axis to gravity, i.e., near its +1- or −1-g reading, the change in output acceleration per degree of tilt is negligible [4]. In the proposed system, the analog filtering for the X- and Y -axes are set by X-CAP and Y -CAP capacitors used to control the noise component of the accelerometer signal, respectively. Table I gives the typical noise component of the accelerometer for various X-CAP and Y -CAP capacitors [21]. In our case, those values were chosen to be both 0.47 uF to reduce the noise of the accelerometer because the tilt-angle measurement uses gravity acceleration. III. TAMA

II. S ENSORS For capturing the contextual information related to a user, additional devices that can detect activity, posture, and gesture are required. Advances in mobile computing and microelectromechanical systems (MEMSs) enable us to embed various sensors into mobile devices such as cellular phones, PDAs, portable media players, and other portable devices. The contextual information can therefore be captured through various sensors and analyzed through recognition technologies. In the field of monitoring the posture, many research groups have studied the tilt-angle measurement method for monitoring the posture of humans in various fields such as medical and rehabilitation [7]–[12], mobile computing [6], [13], music [14], and so on. This technology is an important part in the field of determining the posture of persons, robots, or other objects. Various devices for sensing posture have been used such as electromyographs, digital photographs, videos, ultrasonic, gyroscopes, and magnetic sensors. Most of them may not be suitable for measuring the posture in real life because they have disadvantages such as having a comparatively high-end setup

The method for measuring the tilt angle using the twoaxis accelerometer has widely been used in UIs [13] and healthcare such as for physiotherapeutic scoliosis treatment [7] and monitoring of spinal posture [8]. Most existing anglemeasuring algorithms depend on the absolute computation by using the Earth’s gravity as a reference input. To obtain high accuracy for the tilt angle, these should have the improved pulsewidth modulation decoding scheme and the high-accuracy calibration method recommended by the manufacturer [21]. However, for the high-accuracy calibration of the accelerometer, a user should physically calibrate the output signals of the accelerometer by rotating it. In contrast, the proposed method is based on the relative computation using the dot product from the output signals of the accelerometer. It used the reference vectors defined as the acceleration values measured at 0◦ (the mobile device with the accelerometer is parallel to the Earth’s surface; see Fig. 2) of the X- and Y -axes compensated at the datum angle, respectively. The compensation is used to obtain a signal close to the ideal acceleration values. At this time, the datum angle has been used to compensate the time-series

Authorized licensed use limited to: Kyungpook National University. Downloaded on May 13,2010 at 03:33:13 UTC from IEEE Xplore. Restrictions apply.

BAEK AND YUN: POSTURE MONITORING SYSTEM FOR CONTEXT AWARENESS IN MOBILE COMPUTING

Fig. 1.

Typical output values of the accelerometer due to gravity.

TABLE I NOISE FOR FILTER CAPACITOR SELECTION, X-CAP AND Y -CAP

it is parallel to the Earth’s surface. The scale factor is 12.5% duty cycle change/g. However, because the accelerometer may not always be level, offset errors must be considered. To get X- and Y -axis acceleration values close to the ideal ones, we investigated which datum angle (for example, 0◦ , 90◦ , and 180◦ ) is proper for compensating the X- and Y -axis acceleration data. The values of the offset errors and the reference vectors for each axis obtained at the selected datum angle are used to compensate the time-series acceleration data and in the equation of the dot product, respectively. The compensated signals are passed through a tilt-angle computation. The tilt angle (θ) is determined by the following equation using the dot product:  θ = cos

Fig. 2.

1591

Orientation of the accelerometer embedded in the mobile device.

acceleration data. In the proposed method, there is no necessity for doing any physical calibration. A. Signal Processing for Measuring the Tilt Angle The flowchart of the proposed signal processing for the tiltangle measurement is given in Fig. 3. The X- and Y -axes of the accelerometer are pointing toward the downward and forward directions, respectively. The output signals of the accelerometer are passed through a low-pass filter to minimize inaccuracy for the tilt angle due to the hand trembling or unnecessary motion of a user. If the accelerometer is in an ideal state, the X-axis digital output of the accelerometer is 1 g when it is equal to the force of gravity, and the Y -axis digital output is 0 g when

−1

R × Cx + Ry × Cy  x  ( Rx + Ry ) × ( Cx + Cy )

 ×

180 π

(1)

where Rx and Ry are the reference vectors, and Cx and Cy are the current input vectors for the X- and Y -axes of the accelerometer, respectively. Finally, the sign discrimination determines the sign of the tilt angle. The positive or negative value of the tilt angle depends on the direction of the Y -axis of the accelerometer. We observed the output signals of the accelerometer, and the result was that if the compensated acceleration value for the Y -axis is positive, the sign of the tilt angle is negative, and vice versa. Because the reference vectors are defined as the values measured at 0◦ (when the direction of the accelerometer is parallel to the Earth’s surface, namely, the directions of the X- and Y -axes are downward and forward, respectively) of the X- and Y -axes, acceleration values are compensated at the datum angle, respectively, and the angle between the reference vectors and current input vectors is the tilt angle (θ). Therefore, if the Y -axis is opposite to the force of gravity, the compensated acceleration value for the Y -axis is negative, and the tilt angle is positive.

Authorized licensed use limited to: Kyungpook National University. Downloaded on May 13,2010 at 03:33:13 UTC from IEEE Xplore. Restrictions apply.

1592

IEEE TRANSACTIONS ON INSTRUMENTATION AND MEASUREMENT, VOL. 59, NO. 6, JUNE 2010

Fig. 3. Signal-processing steps for measuring the tilt angle.

Fig. 4. Algorithm for compensating the time-series acceleration data and for obtaining the reference vectors.

angle are noted in Table II, respectively. Table III shows the values of the model parameters obtained at each datum angle using the training data set. After this, the values of the offset errors for each axis at the selected datum angle are used to compensate the time-series acceleration data, and the values of the reference vectors for each axis are used in (1). To select the datum angle, we compared the accuracy of tilt angles measured by the TAMA without/with compensation at different datum angles such as 0◦ , 90◦ , and 180◦ . The average error value was 12.9676◦ for the TAMA without compensation and 1.5814◦ for that with compensation in the range of −90◦ to +90◦ , as shown in Fig. 5. Most significant of all, the TAMA compensated at 180◦ was superior to the other cases. The average errors with compensation at 0◦ , 90◦ , and 180◦ were 2.0000◦ , 2.1030◦ , and 0.6411◦ , respectively. Therefore, the values of the model parameters (Xoffset , Yoffset , Rx , and Ry ) compensated at 180◦ were selected.

B. Data Collection Method The accelerometer was mounted on an anglemeter to measure the tilt angle. The angles were measured both at 180◦ and in the range −90◦ to +90◦ with 10◦ increments. These angular ranges are suitable for the application to a UI, as well as for the posture monitoring system for the treatment of idiopathic scoliosis [7], [13]. The time-series acceleration data from the accelerometer was gathered for approximately 30 s for each degree at a sampling rate of the 100 samples/s, and it is termed the training data set. C. Compensation and Reference Vectors Fig. 4 shows the algorithm for compensating the time-series acceleration data and for obtaining the values of the reference vectors (Rx and Ry ) for the X- and Y -axes, respectively. The process for compensating the time-series acceleration data is composed of three steps: 1) calculation of the average for the total time-series acceleration data filtered by the low-pass filter; 2) obtainment of the average offset values of the X- (Xoffset ) and Y -axes (Yoffset ) at the datum angle; and 3) compensation of the filtered time-series acceleration data by the addition or subtraction of the offset values to obtain time-series acceleration values close to the ideal ones at each datum angle. Furthermore, the reference vectors for each axis are obtained by averaging the time-series acceleration values compensated when the X- and Y -axis directions are equal to the force of gravity and parallel to the Earth’s surface, respectively. We define the offset errors and the reference vectors as the model parameters of the TAMA proposed in this paper. The equations for the model parameters and compensation (Xcompensation and Ycompensation ) for each axis in each datum

D. Estimation Time To estimate the posture of a user during mobile computing, the accelerometer was attached to a PDA, and the TAMA was implemented on it. We investigated the estimation time to measure the tilt angle from the time-series acceleration data. Test data sets were gathered at a sampling rate of the 20 samples/s by the data collection method presented in this paper, and these were collected five times before and after attaching the accelerometer to the PDA. All compensated time-series acceleration data were computed in windows—whose sizes were 10, 20, and 40 samples corresponding to estimation times of 0.5, 1, and 2 s, respectively. Table IV shows the comparison of the average errors before/after attaching the accelerometer to the PDA and the standard deviation when measuring the repeatability (five times) of the TAMA in the range of −90◦ to +90◦ using the test data sets. At every estimation time before/after attaching the accelerometer to the PDA, the differences in the average error values and the standard deviation values of the TAMA were no more than 0.0405◦ and 0.0200◦ , respectively. The longer the estimation time, the higher the accuracy and the lower the standard deviation of the TAMA; however, the difference between every estimation time remains small, as seen in Table IV. To find out the proper estimation time, a “Display ON/OFF” function was implemented on the PDA. This function is controlled by the posture of a PDA user. When the user watches a movie played out by the PDA, the display of the PDA is ON, and vice versa. The estimation time was evaluated from five subjects (five males, ages 25 to 34). From the experimental results, all of the subjects had the best response at the estimation of 1 s.

Authorized licensed use limited to: Kyungpook National University. Downloaded on May 13,2010 at 03:33:13 UTC from IEEE Xplore. Restrictions apply.

BAEK AND YUN: POSTURE MONITORING SYSTEM FOR CONTEXT AWARENESS IN MOBILE COMPUTING

1593

TABLE II EQUATIONS OF THE MODEL PARAMETERS FOR THE TAMA

TABLE III VALUES OF THE MODEL PARAMETERS OBTAINED AT EACH DATUM ANGLE USING THE TRAINING DATA SET

1.6337◦ , respectively, and the repeatabilities were good with standard deviations under 0.2379◦ and 0.2088◦ , respectively, in the range of −90◦ to +90◦ , as seen in Table V. These results were compared with the previous research [7] in the range of 0◦ to 70◦ using evaluation factors such as the average errors, the undistinguishable angles, the standard deviation, and the level of usability. In Table VI, the average error values and the standard deviation values of the TAMA were lower than those of the previous research. The previous research also had the undistinguishable angles around ±90◦ , but the TAMA can measure tile angles in all ranges. In particular, because the TAMA does not make physical calibrations, it may beneficially be utilized. As a result, the TAMA significantly outperformed the previous research in all aspects.

IV. U SER P OSTURE M ONITORING Fig. 5. Accuracy comparisons of the tilt angles measured by the TAMA without/with compensation at different datum angles.

E. Performance Evaluation When the test data sets were collected, the tilt angles for each degree measured by the TAMA were also simultaneously stored in the PDA. Table V shows the tilt angles measured by the TAMA with 1-s estimation time and 180◦ datum angle. The values of the absolute errors before and after attaching the accelerometer to the PDA were no more than 1.9956◦ and

In this section, we introduce the UPMS to monitor the posture of a mobile device user. For a UI on a mobile device platform, it monitors the posture of a user watching a movie played out by a mobile device during mobile computing in three activities, i.e., sitting, standing, and walking, by using the TAMA.

A. System Architecture The system prototype and the system architecture of the UPMS are given in Figs. 6 and 7, respectively. The UPMS is

Authorized licensed use limited to: Kyungpook National University. Downloaded on May 13,2010 at 03:33:13 UTC from IEEE Xplore. Restrictions apply.

1594

IEEE TRANSACTIONS ON INSTRUMENTATION AND MEASUREMENT, VOL. 59, NO. 6, JUNE 2010

TABLE IV AVERAGE ERRORS AND AVERAGE VALUES OF THE STANDARD DEVIATION FOR TILT ANGLES MEASURED BY THE TAMA BEFORE AND AFTER ATTACHING THE ACCELEROMETER TO THE PDA OVER DIFFERENT WINDOW SIZES

TABLE V MEAN ANGLE, ERROR, AND STANDARD DEVIATION VALUES FOR TILT ANGLES MEASURED BY THE TAMA (ESTIMATION TIME: 1 S, DATUM ANGLE: 180◦ )

TABLE VI PERFORMANCE COMPARISON BETWEEN THE TAMA AND [7]

additionally comprised of the activity state estimation module and the SLPF in the TAMA. If the activity level of a user is in a dynamic state, the motion acceleration should be removed because the time-series acceleration data used in the TAMA only relate to the gravity acceleration component. We have therefore included the activity state estimation module for deciding whether it is static or dynamic and the SLPF eliminating the motion acceleration of the accelerometer’s output. The dynamic/walking states of the user can be distinguished from static/running states by using the following statistics as signal features: standard deviation and eccentricity, respectively [19].

B. Data Collection Method Fig. 8 shows the posture of a user watching a movie played out by a PDA in standing, sitting, and walking states, respectively. θ is the angle between the reference vectors and current

Fig. 6.

System prototype of the UPMS.

Authorized licensed use limited to: Kyungpook National University. Downloaded on May 13,2010 at 03:33:13 UTC from IEEE Xplore. Restrictions apply.

BAEK AND YUN: POSTURE MONITORING SYSTEM FOR CONTEXT AWARENESS IN MOBILE COMPUTING

Fig. 7.

1595

System architecture of the UPMS.

Fig. 8. Postures of a user watching a movie played out by the PDA and directions of the accelerometer in three activities. (a), (c), and (e) Initial states. (b), (d), and (f) Postures watching a movie in the standing, sitting, and walking states, respectively.

Fig. 9. Frequency response curve of the SLPF. The pole values are (a) complex numbers, (b) real numbers and moved to the left half-plane in the z-plane, and (c) real numbers and moved to the right half-plane. (d) Pole–zero plot on the z-plane.

Authorized licensed use limited to: Kyungpook National University. Downloaded on May 13,2010 at 03:33:13 UTC from IEEE Xplore. Restrictions apply.

1596

IEEE TRANSACTIONS ON INSTRUMENTATION AND MEASUREMENT, VOL. 59, NO. 6, JUNE 2010

input vectors. The reference vectors and current input vectors are the time-series acceleration data when the mobile device is perpendicular to the force of gravity and when a user watches a movie played out by a mobile device, respectively. In other words, θ is the angle between the mobile device and the Earth’s surface, as shown in Fig. 8. To investigate the tilt angle for the posture of a user in three activities, the two-axis accelerometer was attached to a PDA, and the time-series acceleration data were collected from five subjects ranging in age from 26 to 34 with a mean of 27 and a standard deviation of 4.1. The movie was played on a PDA during experimentation. The training data sets were collected in our scenario from five subjects that were asked to perform a test: after the initial state of about 5 s, the subjects watched the movie played out by the PDA for about 15 s. Then, they repeated this procedure three times and returned to the initial state for each activity. The initial states in sitting, standing, and walking involved the PDA being placed on the desk, gripped by subjects in an attention state, and gripped by subjects in a walking state, respectively, as shown in Fig. 8. We also considered that a PDA should be placed on the cradle. C. Motion Acceleration Component Elimination As the posture is only related to the gravity acceleration component, the motion acceleration components should be removed to measure the tilt angle when the activity of the user is in a dynamic state. In this paper, motion acceleration, as represented by a signal in the high-frequency component (alternating current signal), is defined as noise in the walking state. An accelerometer signal (a) can be written as a = ag + am

(2)

where ag and am are the gravity and motion acceleration components, respectively. The SLPF is designed to completely eliminate the noise component with no effect on the gravity acceleration component, and its frequency characteristics are also analyzed. The transfer function and gain (K) of the SLPF are given by H(z = ejω ) = K

(z − z1 ) · (z − z2 ) (z − p1 ) · (z − p2 )

(3)

where z1 and z2 are zeros, and p1 and p2 are the poles of the SLPF, respectively K=

(1 − p1 ) · (1 − p2 ) . 4

(4)

This transfer function is transformed into the following recurrence equation: af [n] = (p1 + p2 )af [n − 1] − p1 p2 af [n − 2] +K (a[n] + 2a[n − 1] + a[n − 2])

(5)

where a is the original signal of the accelerometer, and af is the filtered signal.

Fig. 10. Original signal and signal filtered by the SLPF for the X-axis of the accelerometer in walking. (a) Original time-series acceleration data. (b)–(e) Time-series acceleration data after filtering: (b) p1 = −0.5 − j0.5, p2 = −0.5 + j0.5; (c) p1 = p2 = −0.6; (d) p1 = p2 = 0.7; and (e) p1 = p2 = 0.97.

Fig. 9(a)–(c) shows the frequency response curves of the SLPF according to the movement of the pole values in the z-plane, and Fig. 9(d) shows the pole–zero diagram. We observed the frequency response by moving poles. The frequency response curves have their peak values at a specific frequency component when the pole values were complex numbers, as shown in Fig. 9(a). If the pole values were real numbers and the poles were moved to the left half-plane in the z-plane, then the gain of the SLPF was increased, and frequency response curves have maximum flat response at the passband, as shown in Fig. 9(b). For these reasons, the SLPF had an effect on the gravity acceleration component, but the noise was also not eliminated. On the contrary, when poles were moved to the right half-plane, the skirt characteristic of the SLPF was better, and the SLPF allowed passing the very small low-frequency component, so the SLPF may eliminate only the noise component, except the gravity acceleration of the accelerometer outputs, as shown in Fig. 9(c). An experiment was conducted to eliminate the motion acceleration component according to moving of the pole values of the SLPF using (5). Fig. 10 contains the training data set and shows the results of the experiment for one representative subject among the five subjects. In the experimental results, the pole values are the closer to “1” in the z-plane, the SLPF can eliminate the motion acceleration component and has no effect on the gravity acceleration. This coincides with the results for the frequency characteristic analysis of the SLPF (see Fig. 9). To find out the proper pole values of the SLPF, the pole values were investigated in the range of 0.95 to 0.99. We selected p1 = p2 = 0.97 as the value for both of the poles in consideration of the delay and noise removal characteristics when filtering the training data set by the SLPF. Fig. 9(d) shows that all poles are inside the unit circle and therefore implies a stable filter.

Authorized licensed use limited to: Kyungpook National University. Downloaded on May 13,2010 at 03:33:13 UTC from IEEE Xplore. Restrictions apply.

BAEK AND YUN: POSTURE MONITORING SYSTEM FOR CONTEXT AWARENESS IN MOBILE COMPUTING

1597

Fig. 11. Posture (watching the movie played out by the PDA) analysis of five subjects who do three activities (sitting, standing, and walking) and angle of the PDA placed on the cradle for about 1 min, respectively. Postures are estimated at every 1 s.

D. Posture Recognition in Three Activities To determine the range of θ for the posture of a user, a series of threshold analysis tests were run. The θ in each activity was calculated by the TAMA with the training data set. The interesting observation here is that though the angles among each posture are different from each other, all the output patterns are similar. In each activity condition during the experiments, the postures of all subjects were observed by an administrator. From the observation results, it was observed that the subjects tended to tilt the PDA toward the opposite of the force of gravity in the sitting state, parallel to the Earth’s surface in the standing state, and approximately parallel to the Earth’s surface in the walking state, as shown in Fig. 8. In the walking condition, the subjects looked down to find an acceptable route and avoid obstacles on the path; hence, they tilted the PDA toward a position approximately parallel to the Earth’s surface [6]. These observation results are consistent with the experiment results, as shown in Fig. 11. The direction of the accelerometer in Fig. 8(c) is identical to it in the reference vectors. The threshold analyses were performed on the training data sets to estimate the posture of a user in each activity, and we examined the values of the optimal threshold to determine the convergence of the posture. Fig. 12 shows the mean, standard deviation, maximum, and minimum values of the tilt angle for each posture using the training data sets. The threshold values for differentiating among the postures in the sitting, standing, and walking activities were found and annotated on the plot in Fig. 12. The angular ranges of the threshold values for the postures in the three activities are 36.9872◦ to 60.0885◦ in the sitting condition, 20.2383◦ to 30.7822◦ in the standing condition, and 0.1198◦ to 14.3088◦ in the walking condition. The angle of the PDA is 60.4939◦ when it is placed on the

Fig. 12. Bar showing the threshold values for the postures of the subjects in each activity. Mean, standard deviation (SD), maximum (Max), and minimum (Min) values were obtained from the training data set.

cradle. From the experimental results, the sitting state can be distinguished from the standing state through an estimation of the posture during mobile computing. V. R ECOGNITION R ESULTS The UPMS was applied to the “Display ON/OFF” function to be used in a UI on a mobile device. The recognition accuracy rate was measured to evaluate the performance of the UPMS. Two evaluation factors were used as follows: 1) the ratio of the number of “Display ON” to the number of trials and 2) the ratio of the number of “Display ON” to the number of

Authorized licensed use limited to: Kyungpook National University. Downloaded on May 13,2010 at 03:33:13 UTC from IEEE Xplore. Restrictions apply.

1598

IEEE TRANSACTIONS ON INSTRUMENTATION AND MEASUREMENT, VOL. 59, NO. 6, JUNE 2010

TABLE VII RECOGNITION RESULTS OF THE POSTURES ESTIMATED BY THE UPMS

malfunctions (“Display OFF”) when the subjects continuously watch the movie played out by the PDA for about 1 min in each activity. The sampling rate was about 20 samples/s in the PDA with the two-axis accelerometer. The posture was estimated at intervals of 1 s. Ten subjects, five adapted subjects and five first-time subjects (10 males, aged 22 to 34), participated in this experiment, and they were asked to perform a test: a movie was played on the PDA during experimentation. The subjects tried to watch the movie played out by the PDA ten times and tried to continuously watch the movie for about 1 min in each activity. Table VII shows the recognition accuracy of the UPMS. The two numbers in parentheses are the number of successes using the UPMS and the total number of trials, respectively, as seen in Table VII. The recognition accuracy rate is greater than 99.7% for all subjects. The results are useful in terms of the posture-based control interfacing and technique that provides the convenience of control of mobile devices, because the control event is automatically generated by the posture of a user, as human–computer interaction technology for context-aware computing. Although our posture recognition algorithm using the decision tree had high accuracy, we may need to consider using statistical approach methods such as support vector machines, linear discriminant analysis, and principal component analysis in further work. Because the statistical approach method has the possibility that it not only can optimally discriminate between more classes of a user’s postures or gestures but also can easily adapt to more and more people. VI. C ONCLUSION The TAMA measuring the tilt angle using the two-axis accelerometer and the UPMS have been designed to monitor the posture of a user. Additionally, the posture-based intelligent

control interface reacting to the posture of a PDA user has been implemented on the mobile device to complement the poor UI of the mobile device. Our approach in measuring the tilt angle may allow for more accurate field-based recognition of the posture using the accelerometer and warrants more study in larger and more various populations of subjects and postures. The TAMA can be used to estimate not only the posture of users with a mobile device, as mentioned in this paper, but also the posture of scoliosis patients and the bent spine posture of musicians, athletes, or public people. The proposed UI using context-aware computing can automatically recognize the posture of a mobile device user with good accuracy. The prototype may usefully be utilized because it is no longer doing the physical calibration and makes user interaction with a mobile device easier in a mobile computing environment. It can therefore be used in consumer products such as digital protractors, mobile games, and wearable computers for posture correction of public people and athletes in the real world. Although a two-axis accelerometer does not express the posture of a user inasmuch detail as those of a three-axis accelerometer, the two-axis accelerometer can detect the postures used in this paper. For this reason, a two-axis accelerometer was used, but it could later easily be combined to form a three-axis accelerometer. R EFERENCES [1] G. Chen and D. Kotz, “A survey of context-aware mobile computing research,” Dartmouth Comput. Sci., Hanover, NH, pp. 1–16, Tech. Rep. TR2000-381, 2000. [2] B. N. Schilit, N. Adams, and R. Want, “Context-aware computing applications,” in Proc. 1st Int. Workshop Mobile Comput. Syst. Appl., 1994, pp. 85–90. [3] A. K. Dey and G. D. Abowd, Towards an Understanding of Context and Context-Awareness, 1985. [4] D. Salber, A. K. Dey, and G. D. Abowd, “The context toolkit: Aiding the development of context-enabled applications,” in Proc. CHI, 1999, pp. 434–441. [5] M. Ehreumann, T. Lutticke, and R. Dillmann, “Dynamic gestures as an input device for direction a mobile platform,” in IEEE Int. Conf. Robot. Autom., 2001, vol. 3, pp. 2596–2601. [6] J. S. Yi, Y. S. Choi, J. A. Jacko, and A. Sears, “Context awareness via a single device-attached accelerometer during mobile computing,” in Proc. MobileHCI, 2005, pp. 303–306. [7] M. Bazzarelli, N. G. Durdle, E. Lou, and V. J. Raso, “A wearable computer for physiotherapeutic scoliosis treatment,” IEEE Trans. Instrum. Meas., vol. 52, no. 1, pp. 126–129, Feb. 2003. [8] R. J. Nevins, N. G. Durdle, and V. J. Raso, “A posture monitoring system using accelerometers,” in Proc. IEEE Can. Conf. Elect. Comput. Eng., 2002, pp. 1087–1092. [9] H. Hsiao and W. M. Keyserling, “A three-dimensional ultrasonic system for posture measurement,” Ergonomics, vol. 33, no. 9, pp. 1089–1114, Sep. 1990. [10] C. S. Lenhart, “Introduction to the three-dimensional scoliosis treatment according to Schroth,” Physiotherapy, vol. 78, no. 11, pp. 810–815, Nov. 1992. [11] H. R. Weiss, “The progression of idiopathic scoliosis under the influence of a physiotherapy rehabilitation programme,” Physiotherapy, vol. 78, no. 11, pp. 815–821, Nov. 1992. [12] M. Vergara, J. L. Sancho-Bru, and A. Perez-Gonzalez, “Description and validation of a non-invasive technique to measure the posture of all hand segments,” J. Biomech. Eng., vol. 125, no. 6, pp. 917–922, Dec. 2003. [13] J. Baek, I. Jang, and B. Yun, “Recognizing and analyzing of user’s continuous action in mobile systems,” IEICE Trans. Inf. Syst., vol. E89-D, no. 12, pp. 2957–2963, Dec. 2006. [14] K. Ueno, K. Frukawa, M. Nagano, and T. Asami, “Good posture improves cello performance,” in Proc. 20th Annu. Int. Conf. IEEE Eng. Med. Biol., 1998, vol. 5, pp. 2386–2389.

Authorized licensed use limited to: Kyungpook National University. Downloaded on May 13,2010 at 03:33:13 UTC from IEEE Xplore. Restrictions apply.

BAEK AND YUN: POSTURE MONITORING SYSTEM FOR CONTEXT AWARENESS IN MOBILE COMPUTING

[15] G. Lee and J. Baek, “Counting steps using impact oscillator,” Electron. Lett., vol. 42, no. 17, pp. 960–961, Aug. 2006. [16] J. Parkka, M. Ermes, P. Korpipaa, J. Mantyjarvi, J. Peltola, and I. Korhonen, “Activity classification using realistic data from wearable sensors,” IEEE Trans. Inf. Technol. Biomed., vol. 10, no. 1, pp. 119–128, Jan. 2006. [17] L. Bao and S. S. Intille, “Activity recognition from user-annotated acceleration data,” in Proc. 2nd Int. Conf. Pervasive Comput., 2004, vol. 3001, pp. 1–14. [18] S. W. Lee and K. Mase, “Activity and location recognition using wearable sensors,” IEEE Pervasive Comput., vol. 1, no. 3, pp. 24–32, Jul. 2002. [19] J. Baek, G. Lee, W. Park, and B. Yun, “Accelerometer signal processing for user activity detection,” in Knowledge-Based Intelligent Information and Engineering Systems, vol. 3215. Berlin, Germany: Springer-Verlag, 2004, pp. 610–617. [20] D. Siewiorek, A. Smailagic, J. Furukawa, N. Moraveji, K. Reiger, and J. Shaffer, “SenSay: A context-aware mobile phone,” in Proc. IEEE Int. Symp. Wearable Comput., 2003, pp. 248–249. [21] “Low-Cost ±2 g/±10 g dual axis iMEMS accelerometers with digital output,” Analog Devices Data Sheet, 1999.

1599

Byoung-Ju Yun received the B.S. degree in electronics engineering from Kyungpook National University, Daegu, Korea, in 1993 and the M.S. and Ph.D. degrees in electrical engineering and computer science from the Korea Advanced Institute of Science and Technology, Daejeon, Korea, 1996 and 2002, respectively. From 1996 to May 2003, he was a Senior Engineer with Hynix Semiconductor Inc. Since June 2003, he has been an Assistant Professor with the School of Electrical Engineering and Computer Science, Kyungpook National University. His current research interests include image processing, MPEG-4, H.264, digital image watermarking, computer vision, SVC, HCI, and multimedia communication systems.

Jonghun Baek received the B.S. degree in electronics engineering from Daegu University, Daegu, Korea, in 2001 and the M.S. and Ph.D. degrees in information and communication from Kyungpook National University, Daegu, Korea, 2003 and 2008, respectively. Since March 2008, he has been a Senior Researcher with Samsung Electronics Company Ltd., Gyeongbuk, Korea. His current research interests include human–computer interaction, wearable computing, context-aware computing, and affective computing.

Authorized licensed use limited to: Kyungpook National University. Downloaded on May 13,2010 at 03:33:13 UTC from IEEE Xplore. Restrictions apply.