A Fatigue Detect System Based on Activity Recognition Congcong Ma, Wenfeng Li, Jingjing Cao, Shuwu Wang, and Lei Wu School of Logistics and Engineer, Wuhan University of Technology, Wuhan, 430077, China
[email protected], {liwf,caojingjing,wangsw,wulei}@whut.edu.cn
Abstract. Fatigue is considered a key factor to accidents and illnesses in our daily life. Detecting fatigue is therefore useful to prevent accidents and keep our body healthy. It is useful to the people who usually sit many hours a day performing office jobs, it can remind people to have a rest and do some exercises, so to help them developing good working habits. In this paper, we propose a non-invasive way to monitor people’s activity. By applying Activity Recognition using Body Sensor Network technologies, we made a smart cushion to monitor people’s activities; we acquire pressure data and analyze it in MATLAB to infer whether a subject is suffering fatigue. With the proposed method, we learnt that subjects are getting tired after about an hour only. Experimental results show that pressure data for left-right orientation can clearly judge whether a sitting subject is suffering fatigue. Keywords: Activity Recognition, Body Sensor Networks, Fatigue Detection, Smart Cushion.
1
Introduction
Fatigue detection is becoming a hot topic in the smart-health domain. Increasing number of people is working in front of visual display terminals (VDTs), such as computer, smartphone or other display terminals, and, in many cases such workers feel fatigue without realizing it or properly addressing it. Fatigue is indeed often wrongly overlooked, because it can lead to mental and physical problems and negatively impact study/work efficiency and safety. Detecting fatigue of people who always sit in front of VDTs is useful, e.g. to prevent various related illnesses. It is possible to distinguish two kinds of fatigue: physical fatigue and mental fatigue [1]. The former is due to muscle activity when people perform manual works, the latter is caused by a variety of psychological stressors. In this paper we focused on mental fatigue. Caused by excessively prolonged labor tasks, mental tension can negatively affect efficiency state. Mental fatigue is the origin of many diseases; if a person perform excessive or heavy overloaded mental work, he/she will undergo long-term fatigue, compromising the correct operation of physiological functions and causing a variety of diseases, and eventually leading to decreased immune system, endocrine disorders, G. Fortino et al. (Eds.): IDCS 2014, LNCS 8729, pp. 303–311, 2014. © Springer International Publishing Switzerland 2014
304
C. Ma et al.
etc. In this case, people easily get diseases such as colds, but also seriously exposed to cardiovascular diseases [2], diabetes, etc. Therefore, fatigue prediction and early detection is of key importance to prevent various medical situations. According to an U.S. epidemiological survey, among the adult population, about 14% of males and 20% of females suffers symptoms of fatigue performance [3]. Because of work intensity, fatigue illness and even deaths are increasing year by year. Fatigue is the main factor causing adults’ body condition decline and chronic illnesses appearance. Prolonged workload, mental stress, short time resting periods and poor physical activity cause many illnesses even suffer the dangers of karoshi. According to another survey by World Health Organization, about 35% of the world population has suffered fatigue. Since fatigue is usually not associated with other significant symptomatology it is difficult to be properly assessed using traditional clinical examination. Body Sensor Networks (BSNs) enable to measure many important human physiological characteristics, including physical activity status [4], body temperature [5], muscle activity, heart rate, and brain activity. Based on BSN technology, we can develop many kinds of human-centered applications in diversified domains such as mobile entertainment, health care and fitness [6, 7, 8]. In this paper we formed the assumption that if a person is tired, he/she may become very quiet, or he/she might feel anxiety and conversely become unsettled. Based on this assumption, we analyzed subjects during daily working activities to investigate the difference between the condition of non-fatigue and fatigue. Specifically, we acquired data from a device with pressure sensors to monitor user posture; these data are analyzed to judge whether the user is feeling fatigue. Experimental result confirmed that if a person is tired, he/she often acts as our assumption. With this method, we can predict fatigue earlier and remind people to have a rest or perform proper stretching exercises.
2
Methodology
Activity recognition can be used to recognize basic postures such as standing, sitting, lying and squatting [9]. It can also be used in the field of fatigue detection by analyzing the data of physiological phenomena and activity behavior. Automatic fatigue detection has gained much relevance in the field of car driving; however, researches have not yet been focused on fatigue detection of office workers. The technique to detect driving attention level is essentially the same, so the methods that used in the field of monitor people’s status of car driving can also be used to monitor the fatigue condition of office workers. 2.1
Related Work
Hiroshi and Masayuki developed a system to detect fatigue [10] using a camera to monitor people’s activity and image processing to analyze driver’s facial expressions, to observe the extent of driver’s eyes closed and wide open as an alarm of fatigue. Li
A Fatigue Detect System Based on Activity Recognition
305
also proposed an image processing method to classify fatigue-related facial expressions [11]. These approaches have, however, the disadvantage to be influenced by ambient light. Iampetch S. proposed the use of EEG signal to detect whether people was suffering fatigue [12]. This approach, however, is quite invasive as the user has to wear a device on his/her head to measure the signal of the brain activity. Patterson used a method based on a three-axis accelerometer sensor to quantity people’s activities with the aim of detecting fatigue symptoms [13]. This approach requires the user to wear smart objects too. 2.2
Proposed Method
In this paper, we propose a non-invasive way to monitor people’s activity while seated. This method has advantages because it just need to place on the chair a cushion with a suit of embedded pressure sensors. We use a FSR (Force Sensing Resistor) pressure sensor produced by Interlink Electronics [14]. It is ultra-thin, weight light, and highly accurate. Its’ size is 1.75x1.5" (approximately 45x38mm). As force is applied on the sensing areas, the resistance value of the FSR will be correspondingly altered. The more the forcing power, the smaller the resistance is. This sensor can detect a pressure power from 0 to 20kg. This pressure sensor can be easily fixed and embedded into the cushion textile or foam filling. As the sitting pressure distribution is closely related to the person sitting posture, in this paper we design a model to distinguish fatigue and non-fatigue based on the sitting posture. More precisely, we designed a pressure chair cushion based on four FSRs as shown in Fig. 1.
Fig. 1. FSR pressure sensor (left) and cushion equipped with FSR sensors (right)
When a person is sitting on the cushion, the FSR sensors on the cushion can detected the pressure data. If he/she slants to the right, the pressure value of the right sensor will greater than the left sensor. Use the smart cushion, we can measure the value of pressure that caused by our body weight. According to the pressure distribution, we can calculate the center of gravity of body that deviate from the center of the cushion. It can infer that people is sitting upright or swing to an orientation. The sitting posture images can be exampled as in Fig. 2.
306
C. Ma et al.
Fig. 2. Sitting posture images with the four directions: Left, Right, Forward, Backward
As our system model indicated in Fig. 3(a), four pressure sensors was fixed on the cushion and marked as fsr1, fsr2, fsr3, fsr4. The four sensors was evenly placed on the cushion, they have the same distance to the center of the cushion. Pressure sensors’ data are marked as , , , , in the coordinate system we use Z axis to represent the pressure value as shown in Fig. 3(b). Here we use the sensors on X axis to explain our method, as we can indicated that the sensors on the X axis can represent the body posture lean left and right.
Fig. 3. (a) System model of the sensors in the chair cushion, (b) Propose Method
The minus of and can represent the body center of gravity deviate from the Y axis. When people sit on the center of the cushion, the body center of gravity is on the point of zero point. Fig. 3(b) is an example that people lean to the right side, the lean angle is a, and it can be described by the value of . = To the same case, the minus of deviate from the X axis.
and =
− can represent the body center of gravity −
A Fatigue Detect System Based on Activity Recognition
2.3
307
System Architecture
In this section, we propose our system architecture of the posture recognition system based on Body Sensor Networks, the system architecture is depicted in Fig .4. The system is composed of several cushions with pressure sensors, the raw data was collected to the central coordinator for processing.
Fig. 4. Fatigue Detection System Architecture Based on BSNs
Each sensor node is composed of three modules: cushion module (we use the FSR sensors to acquire the raw data), processing module (we use Arduino board to process the data from the FSR) and transmitting module (we use CC2530 to send the signals to the coordinator). The processing module of each sensor node is an Arduino MEGA 2560 MCU, it has the function of low energy consumption, high sampling rate and high processing speed, so it can be widely used in the field of sensor data acquisition and industrial automation. The sensor data is collected and sent to the coordinator node (i.e. a computer or smart phone acting as the BSNs coordinator) through the wireless communication use CC2530. The coordinator is in charge of further data processing and giving the results of the human posture recognition.
3
Experiments
In this paper, we mainly focused on two directions: Fx (the pressure force between left and right), Fy (the pressure force between front and back). There is an important point of educational psychology "the adolescent can continue focus the attention about 10 to 30 minutes, adult can continue focus attention about 30 to 50 minutes", that’s why there is only 45 minutes of a lesson, even the adult’s classroom is not exceed an hour. In our experiments, we found out the subject gets tired after about an hour of working activity. The experiment were carried out on 4 subjects working at computers. The average age is 24. To avoid any bias, we requested subjects to refrain from caffeine for 4 hours before taking the experiment and alcohol for 24 hours. As the participant sits on the chair, he/she work just as usual for a whole duration of two hours. The smart cushion was placed at the center of the chair, people was
308
C. Ma et al.
sitting on it to cover every sensor, in order to make each sensor can detect the value of the pressure. The smart cushion acquires pressure sensory data, and the sampling frequency is 20Hz. We eventually analyzed the data in MATLAB. We have collected 66 thousand samples for analyzing. Since the cushion pressure data produced by people didn't make great changes in one second, we calculate the mean value of every 20 data, i.e., every second. In Fig. 5-Fig. 6, the axis-X utilize one second as a unit, and the axis-Y represents the force value of the pressure on the cushion, it was measured by voltage. The compared results are shown in Fig.5-Fig.6.
Fig. 5. Fluctuations of Fx (a) in the first hour, (b) in the second hour
Fig. 6. Fluctuations of Fy (a) in the first hour, (b) in the second hour
Fig. 5(a) shows the lateral pressure data of every second in the first hour, we can see that at the beginning of nearly 15 minutes, people do not work in the state. Later people is working in the state, the pressure data is steady. Fig. 5(b) shows the pressure data of the second hour. After about an hour later, people has suffering a short period of fatigue. Then people still working as normal, after about 90 minutes people will suffering a bit long period of fatigue. Fig. 6 shows the frontal pressure data of the cushion. Comparing the two figures, there are no clear signal to infer whether people is suffering fatigue. For the sake of graphical clearness, in the following we choose data of a time segment of 5 minutes (thus containing about 6000 samples) about one person. The
A Fatigue Detect System Based on Activity Recognition
309
axis-X represent the sample data we have collected, and the axis-Y represent the value of the pressure on the cushion, it was measured by voltage. We have interview the subject after two hours of work, he said he has great efficient at the beginning of working, after worked for an hour, he’s suffering fatigue. So in the following pictures, we choose two segments of period to analyze. In each of the following figures, plots on the left are referred to the beginning of the experiment (working period from 10th minute to the 15th), while plots on the right corresponds to a working period from 70th minute to the 75th.
Fig. 7. Fluctuations of Fx (a) at the beginning, (b) an hour later
Fig. 8. Fluctuations of Fy (a) at the beginning, (b) an hour later
Specifically, Fig. 7(a) shows lateral pressure data of the cushion. Compared with Fig. 7(a), Fig. 7(b) shows the activity level after over an hour: we can see that the activity level is lower with respect to the beginning; we can infer that people is suffering fatigue. Fig. 8 depicts frontal pressure of the cushion: in this case there are not clear changes in the signal. Through the analysis of the two sets of data we know that the subject is initially active on the chair (pressure data fluctuates) while starts gradually to move less on the chair (pressure data are more stable to a constant value) as the fatigue appears and he/she might stay still or rely on the back of the chair.
310
4
C. Ma et al.
Conclusion
On the basis of our results, we know that pressure data on front-back chair direction is not useful in determining user fatigue, but the pressure data from the sides of the chair can be effectively used to detect fatigue condition. The experimental results describe that when users are suffering fatigue their posture is stable and with lower activity. In this paper, firstly, the mental fatigue experiment was designed and the hardware of smart cushion was introduced. Secondly, the proposed method was used to acquire pressure sensor data from the instrumented cushion and the system architecture based on BSNs was introduced. Furthermore, data were analyzed in MATLAB and experiment results have been discussed. Results prove that a kind of fatigue can actually be detected by a chair equipped with a smart cushion such as the proposed one. Future research will be devoted to acquire more data to quantify people’s normal activities; in addition we plan to use Cloud technologies [15, 16] to perform online data processing, in order to get a precise real-time estimation of fatigue condition. Acknowledgments. This paper is supported by National "Twelfth Five-Year" Plan for R&D Technology (No.2012BAJ05B07).
References 1. Lal, S., Craig, A.: A critical review of the psychophysiology of driver fatigue. Biological Psychology 55(3), 173–194 (2001) 2. Nakane, H., Toyama, J., Kudo, M.: Fatigue detection using a pressure sensor chair. In: 2011 IEEE International Conference on Granular Computing, pp. 490–495 (2011) 3. Holmes, G.P., Kaplan, J.E., Gantz, N.M., et al.: Chronic fatigue syndrome: a working case definition. Ann Intern. Med., 387–389 (1988) 4. Lo, B., Yang, G.: Body Sensor Networks – Research Challenges and Opportunities. In: 2007 IET Seminar on Antennas and Propagation for Body-Centric Wireless Communications, pp. 26–32 (2007) 5. Kuroda, M., Tamura, Y., Kohno, R., Tochikubo, O.: Empirical evaluation of zero-admin authentication for vital sensors in body area networks. In: 30th Annual International Conference of the IEEE Engineering in Medicine and Biology Society, pp. 2349–2352 (2008) 6. Chulsung, P., Chou, P.H.: Eco: ultra-wearable and expandable wireless sensor platform. In: International Workshop on Wearable and Implantable Body Sensor Networks, pp. 165– 168 (2006) 7. Bellifemine, F., Fortino, G., Giannantonio, R., Gravina, R., Guerrieri, A., Sgroi, M.: SPINE: A domain-specific framework for rapid prototyping of WBSN applications. Software Practice and Experience 41(3), 237–265 (2011) 8. Fortino, G., Giannantonio, R., Gravina, R., Kuryloski, P., Jafari, R.: Enabling Effective Programming and Flexible Management of Efficient Body Sensor Network Applications. IEEE Transactions on Human-Machine Systems 43(1), 115–133 (2013) 9. Li, W., Bao, J., Fu, X., Fortino, G., Galzarano, S.: Human Postures Recognition Based on D-S Evidence Theory and Multi-sensor Data Fusion. In: 2012 12th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing (CCGrid), May 13-16, pp. 912– 917 (2012)
A Fatigue Detect System Based on Activity Recognition
311
10. Ito, T., Mita, S., Kozuka, K., Nakano, T., Yamamoto, S.: Driver blink measurement by the motion picture processing and its application to drowsiness detection. In: 5th IEEE International Conference on Intelligent Transportation Systems, pp. 168–173 (2002) 11. Xing, L., Guang, H., Guangteng, M., Yanshan, C.: A new method for detecting fatigue driving with camera based on OpenCV. In: International Conference on Wireless Communications and Signal Processing, pp. 1–5 (2011) 12. Iampetch, S., Punsawad, Y., Wongsawat, Y.: EEG-based mental fatigue prediction for driving application. In: Biomedical Engineering International Conference, pp. 1–5 (2012) 13. Patterson, M., McGrath, D., Caulfield, B.: Using a tri-axial accelerometer to detect technique breakdown due to fatigue in distance runners: A preliminary perspective. In: Annual International Conference of the IEEE Engineering in Medicine and Biology Society, pp. 6511–6514 (2011) 14. http://www.interlinkelectronics.com/FSR406.php 15. Fortino, G., Di Fatta, G., Pathan, M., Vasilakos, A.V.: Cloud-Assisted Body Area Networks: State-of-the-Art and Future Challenges. ACM Wireless Networks, 1–20 (to appear, 2014) 16. Fortino, G., Parisi, D., Pirrone, V., Di Fatta, G.: BodyCloud: A SaaS Approach for Community Body Sensor Networks. Future Generation Computer Systems 35(6), 62–79 (2014)
