Awareness Home Automation System Based on User ...

Download PDF
29 downloads 150403 Views 442KB Size Report
Comment
In this research we exploit two of android sensors. First, accelerometer sensor ... Keywords: Mobile sensing, awareness home automation system. 1 Introduction.
Awareness Home Automation System Based on User Behavior through Mobile Sensing Rischan Mafrur, M Fiqri Muthohar, Gi Hyun Bang, Do Kyeong Lee, Deokjai Choi School of Electronics and Computer Engineering Chonnam National University Gwangju, South Korea Email: {rischanlab, fiqri.muthohar }@gmail.com, [email protected], [email protected], [email protected] Abstract. This paper proposed awareness home automation system (HAS) based on mobile sensing. Some of the ideas have been proposed in HAS but most of them still requires human intervention such as click the button, voice commands, etc. We want to design and develop HAS which can understand and comprehend the user desires without having to wait for commands from the user (awareness HAS). In this research we exploit two of android sensors. First, accelerometer sensor for identification and activity recognition, second, magnetic field for user indoor positioning system and defined the context related to physical environment. This paper presenting the result of used both of the sensors for developing awareness HAS. Keywords: Mobile sensing, awareness home automation system

1

Introduction

Internet of Things (IoT) is one of the hot topics discussed by researchers. This idea made all of things (wide variety of devices) connected with Internet. One of interesting example of IoT is HAS. HAS make people live more comfortable because its will help people to doing their housework or household activity. There are many of solution for HAS in industry and market such as X-10, ZigBee, UPB, INSTEON, and Z-Wave. Our lab also would not miss to take this big challenge and opportunity in this field. In our research lab we divide two teams, First team [1] have been presented their research about optimized design and energy aware for wireless sensor network (WSN) based on IPv6-USN. We are second team who want to implement that design for HAS but we focus on artificial intelligence issue. In the case of artificial intelligence (AI) we believe the Moravec’s paradox. Maravec’s paradox that explained by Steven Pinker in his book [2] defined two type of AI problems: "hard but easy" and "easy but hard". First term "hard but easy" means human assume a problem is very hard to be solved such as playing chess, theorem proving but actually machine can solved easily. Otherwise, second term "easy but hard" means human assume a problem is very easy but it is too hard to solved by machine (i.e, image processing, human identification, natural language processing, perception and sentiments, etc). The most of problems in HAS as second term "easy but hard". We think the majority of research in this field are divided into three part : HAS design

such as optimize design for communication and low power [1,3-5], authentication and security such as using PIN, biometrics, etc[6,7], and human interaction such as using android client application, voice commands, etc [8,9]. Our research focus to develop awareness HAS which can automatically give response as user want based on mobile sensing. The core of this research is we try to exploit accelerometer and magnetic field sensors in android phone. The functions of accelerometer are for user identification [10] and user activity recognition [11,12,13]. We use magnetic field to defined user position (indoor positioning) and to aware the zone especially the context related to physical environment [14,15,16].

2

System architecture, scenario, and methodology

2.1 System Architecture In this research we use Raspberry Pi as our server and use two types of android devices for client (Galaxy Note II and Google Nexus 7 Tablet).

Fig. 1. HAS Client App

Fig. 2. Lab Room 445 engineering building CNU

Table 1. Alice and Bob behavior Alice behaviors He has desk A and chair A. Every day he enter to lab room around 09:00 AM - 11:00 AM. He always use jean trouser, the percentage he put his phone in the front hip his trousers is 97 %, 3 % he bring his phone with his hand. He always use elevator, after enter to the lab room he turn on the light, take his shoes and walking and sit on the chair A. After sit on the chair A, he playing music with his headset and favorite songs in his play list and then he start working.

Bob behaviors He has desk B and chair B. Every day he enter to lab room around 08:00 AM - 09:00 AM. He use tablet (Google Nexus 7), he always put his tablet on his small side bag. Sometime he use elevator or stairs, after enter to this room he turn on the light, take his shoes and sit on the chair B. After sit on the chair B, he always open the browser and open his favorite news portal.

We use Heimcontrol2 library in server as command receiver and modify Heimdroid3 application in client as data sensor collector and processing. User can give command directly using the button in client application, in this research we only use two of actions, there are: turn on light and play media. To use automatic mobile sensing user can check the check box, after that this application will automatically record the user sensor data, processing, and then sending the request to the server based on user behavior. The GUI of client application can be shown in Fig. 1. 2.2 Scenario We performed our experiments in our lab room. Fig.2. shows the design of our lab. In our experiment, we have two subjects called : Alice and Bob. Actually most of people, they have similar schedule for every day, for example: after he wake up and then preparing to go to office, they have arrived to their office 09.00 AM, and it happened continue over and over again in the working day. So, in this research we use the scenario according to the subjects behavior, what are their behavior when they come to lab from their house until enter to the lab and what they are doing when they arrived to lab, etc. More details about their behavior can be seen in Table 1. We have three scenario as follows : 1. We perform experiment for 5 days, every day we ask the subjects to enter the lab room with theirs behavior 10 times, finally, we have 100 data from two subjects. 2. We also try to inverse it, what happened if Alice enter the lab and he walk and sit down to Bob’s chair and vice versa, we also get 100 data from two subjects. 3. In Fig. 3, we have "chair C", we also use it for the testing, we ask the subjects to doing same thing but after they enter the lab room they not sit down in theirs chair but they sit on chair C. Finally, we have 100 data from each scenario and the total is 300 data. We doing this scenario for two times, first, when we collect the data for developing our system (training data) and the second we use this scenario again for testing after we finished our system, for the second time we count how many our HAS system give right and wrong response. 2.3 Methodology We have 4 types of signal for accelerometer and magnetic field data including X,Y,Zaxis signal and the magnitude M. To extracting features we use time and frequency domain features, time domain features such as average maximum and minimum acceleration, average absolute difference and standard deviation. We use Fast Fourier transform (FFT) to transform the data from time to frequency domain, we use the first 50 FFT coefficients. Support Vector Machines (SVM) with Radial Basis Function (RBF) was applied on feature vectors for classification. 3

2 3

Result and Discussion

http://ni-c.github.io/heimcontrol.js/ http://jorism.github.io/Heimdroid/

The previous work [10] from our lab have been presented that they can achieved 92.7% accuracy for user identification based on gait pattern. In this research for identification system we implement that method. The next are about activity recognition and indoor position. Fig. 3. shows the magnitude (M) from accelerometer data, the first figure is data generated by Alice and the second by Bob. We have five events on the figure [A,B,C,D,E]. A means user walk from elevator to the door lab, B means user stand and open the door, C means user take his shoes, continue walking to his chair, D means user standing before sit down, E means user sit down.

Fig. 3. Magnitude (M) accelerometer Alice/Note II (first), Bob/Nexus 7 (second) The events A and B for the both user is similar because they doing same activity walking from the elevator and then open the door. Bob has longer time than Alice for the event C, if we look at again in Fig. 2, we show Bob has longer distance from the door to his chair than Alice. The event D means standing before they sit down but the data from Bob look strange, it is because of he put his tablet in side bag, when he standing before he sit down, he move his side bag and put it to his desk. We use magnetic field to determine user position based on our experiment we can achieved high accuracy but this sensor easily influenced by other events that occurred in mobile phone. Fig. 4 shows the example FFT signature values of magnetic field data, the data we collected on the same activity (when user sit down), the blue point means the FFT values of the magnetic field data when user only sit down and the red point means when user sit down and get incoming call. In this research, we perform two times based on our scenario. First scenario is subjects enter the lab with their behavior such as Alice enter the lab and sit down in his chair (50 data) and Bob also enter the lab and sit down in his chair (50 data), second scenario is Alice enter the lab and then sit on the Bob’s chair (50 data) and Bob sit on the Alice’s chair (50 data), the last scenario is Alice and Bob enter the lab and then sit down in “chair C” (50+50 data). In this research we have two types of responses, turn on the light and play media. Based on all of our experiment, the HAS give 100% correct response for the turn on the light when user enter the lab (open the door) but for the second response in the first time experiment when we analyze the data in local PC, we achieved 94%

accuracy but in the second time when we applied in real environment we only get 88% accuracy, the detail result can be seen in Table 2. From the result we saw that the accuracy is quite different between when we applied our approach in local PC and when we tried in real environment (mobile phone).

Fig. 4. FFT magnetic field, blue point (sit down), red point (sit down+incoming call) Table 2. The number of correct response of our HAS when analyzed in PC and when tested in real environment (A= Alice, B=Bob, c= chair, C= chair C) A ⇒ Ac B ⇒ Bc A ⇒ Bc B ⇒ Ac A ⇒ C B ⇒ C Total PC 45 44 49 47 48 49 282 Real env. 43 40 46 44 46 45 264 We thought that caused by many reasons such as the environment in mobile phone like subjects get SMS or call, it will affect the magnetic field sensor and another reason like limited resources in mobile phone to do some complex processing, it will be our challenge for the next research.

4

Conclusion

This paper shows that with mobile sensing we can create awareness HAS which can automatically give response as users want based on theirs behavior. We have been presented that by using accelerometer and magnetic field we can create awareness HAS and using our approach we achieved 88 % accuracy. The contributions of this paper can conclude to as follows: 1. We provide our testbed architecture of awareness HAS and the realistic scenario which according with subject behavior. 2. We implement user identification, user activity recognition and indoor positioning (mobile sensing) to get automatic response from our awareness HAS.

5

Acknowledgement 1. This research was supported by the MSIP (Ministry of Science, ICT and Future Planning), Korea, under the ITRC(Information Technology Research Center) support program (NIPA-2014-H0301-14-1014) supervised by the NIPA(National IT Industry Promotion Agency).

2. Basic Science Research program through the National Research Fund of Korea (NRF) funded by the Ministry of Education, Science, and Technology (MEST), Korea (2012-035454).

References [1] A. M. Efendi, S. Oh, A. F. P. Negara, and D. Choi, “Battery-less 6lowpan-based wireless home automation by use of energy harvesting,” International Journal of Distributed Sensor Networks, vol. 2013, 2013. [2] S. Pinker, The Language Instinct. plus 0.5em minus 0.4em Harper: Perennial Modern Classics, 2007. [3] U. Sharma and S. Reddy, “Design of home/office automation using wireless sensor network,” International Journal of Computer Applications, vol. 43 No.22, April 2012. [4] Q. Sun, W. Yu, N. Kochurov, Q. Hao, and F. Hu, “A multi-agent-based intelligent sensor and actuator network design for smart house and home automation,” Journal of Sensor and Actuator Networks, 2013. [5] D.-M. Han and J.-H. Lim, “Design and implementation of smart home energy management systems based on zigbee,” IEEE Transactions, vol. 56(3), 2010. [6] M. H. Assaf, R. Mootoo, S. R. Das, E. M. Petriu, V. Groza, and S. Biswas, “Sensor based home automation and security system,” in 2012 IEEE International Instrumentation and Measurement Technology Conference (I2MTC), 13-16 May 2012. [7] Ali Fahmi P.N, Elyor Kodirov, Ardiansyah, Deokjai Choi and Gueesang Lee, “Hey home, open your door, i’m back! authentication system using ear biometrics for smart home,” International Journal of Smart Home, vol. 7(1), 2014. [8] S. R. Das, S. Chita, N. Peterson, B. A. Shirazi, and M. Bhadkamkar, “Experimental evaluation of speech recognition technologies for voice-based home automation control in a smart home,” in 4th Workshop on Speech and Language Processing for Assistive Technologies, French, 21-22 August 2013. [9] R.A.Ramlee, M.H.Leong, R.S.S.Singh, M.M.Ismail, M.A.Othman, H.A.Sulaiman, M.H.Misran, and M. Said, “Bluetooth remote home automation system using android application,” The International Journal of Engineering And Science (IJES), vol. 2(1), 2014. [10] Hoang Minh Thang, Vo Quang Viet, Nguyen Dinh Thuc, and Deokjai Choi, “Gait Identification Using Accelerometer on Mobile Phone, ICCAIS, 2012. [11]Y. Kwon, K. Kang, and C. Bae, “Unsupervised learning for human activity recognition using smartphone sensors,” Journal Expert Systems with Applications, vol. 41, 2014. [12]O. Banos, M. Damas, H. Pomares, A. Prieto, and I. Rojas, “Daily living activity recognition based on statistical feature quality group selection,” Journal Expert Systems with Applications, vol. 39, 2012. [13]A. M. Khan, A. Tufail, A. M. Khattak, and T. H.Laine, “Activity recognition on smartphones via sensor-fusion and kda-based svms,” International Journal of Distributed Sensor Networks, vol. 2014, 2014. [14]K. Yun, Y. Jo, N. Kim, U. Jo, and Y. Kim, “Experimental performance evaluation of a simple indoor positioning scheme with two commonly used sensors,” in SERSC proceedings 2013, 2013. [15]B. Li, T. Gallagher, C. Rizos, and A. G. Dempster, “Using geomagnetic field for indoor positioning,” in International Global Navigation Satellite Systems Society IGNSS Symposium 2013, Australia, 2013. [16]C. E. Galvan-Tejada, J. P. Garcıa-Vazquez, and R. F. Brena, “Magnetic field feature extraction and selection for indoor location estimation,” Sensors (ISSN 1424-8220; CODEN: SENSC9), vol. 2014, 2014.