Design and implementation of real-time image ...

Design and implementation of real-time image monitoring system based on Android mobile terminal Wang Yong

Fan Da Lei

College of Engineering & Information Technology University of Chinese Academy of Sciences Beijing China [email protected]

College of Engineering & Information Technology University of Chinese Academy of Sciences Beijing China [email protected]

Abstract—This paper designs an Android-based mobile terminal real-time image monitoring system, making the remote monitoring of Android-based mobile terminal through Short Message Service (SMS) come into being, and thus gaining the image of Remote Mobile Terminals (RMT). The overall function of the process is: install this system on an Android-based RMT; register monitoring number and monitoring code; when the Local Monitor Terminal (LMT) sends SMS commands to RMT, SMS monitoring module of this system captures SMS, separates the incoming number and content and compares them respectively with the registered records in the SQLite database. If the match is successful, a message is replied and the LMT sends the information code needed to acquire according to the content of the message. The RMT takes photos and replies Multimedia Messaging Service (MMS) according to the code. This system can be operated normally after testing, and shows promising application value.

II.

RELATED TECHNOLOGIES

A. Android System Android is a mobile phone operating system established by Andy Rubin [3]. It was purchased by Google in 2005, and in 2007 Android operating system was formally opened to display, meanwhile a global Alliance organization - Open Handset Alliance was set up. In 2008, Google proposed Android HAL architecture diagram, and in September officially launched the Android 1.0 system, which is the earliest Android version. Since then, Google has released a series of Android versions. The Android system is a kind of free and open source code operating system based on Linux, including operating system, middleware, and key applications [4]. The level of the Android system can be simply described as: the underlying is Linux, the middle is a Java virtual machine called Dalvik, and the above is the Android runtime library [5]. The Android applications are Java programs running in the Dalvik. In general, Android applications consist of four components, as are the Activity, Service, BroadcastReceiver, and ContentProvider. The Activity is component responsible for Android applications interaction with users, providing visual user interface for the application; Service owns the same status as the Activity, but it has no visual interface, no interaction with the user, running in the background; BroadcastReceiver is responsible for monitoring the broadcast news of Android other components; ContentProvider provides a data exchange standard for Android across application. Because the Android system is based on Linux kernel and has the characteristics of open source and free, it quickly get the broad masses of fans and the support of many vendors.

Keywords—real-time; video monitoring system; Android; mobile terminal; SMS

I. INTRODUCTION In the mobile network widely coverage and smartphone highly popular era, people's life style has been changed. In mobile intelligent era, people are no longer restricted by time and space, and can contact with friends following one's inclinationsly, acquiring information at anytime and anywhere. Video monitoring is an important topic in computer vision. In the engineering field need of professional monitoring, users want to reduce labor costs. At any time to be able to see the monitor screen is the most need of customers using of monitoring and control system [1]. From the development trend of the monitoring system architecture, distributed system architecture, as preposes the data processing function, improves the computation ability of cameras, reduces the functions of control center and the control center only performs the function of organization and coordination, are the keys to improve the system’s processing ability [2].

B. Mobile Video Monitoring Video monitoring is mainly experienced three stages, namely the simulation video monitoring, network video monitoring and mobile video monitoring.  Simulation video monitoring stages: before the early 90s, the video transmission medium is analog cable and is mainly used in monitoring a small scale.

In this context, SMS control system based on Android mobile terminal can achieve message control video monitoring, provide a new way of mobile video monitoring, and have a good application prospect.

 Network video monitoring stages: in the late 90s, the technologies of network video monitoring are based on

This work is supported by National Natural Science Foundation of China (#61371155).

____________________________________ 978-1-4799-4419-4 /14/$31.00 ©2014 IEEE

53

embedded WEB server with the gradually mature of Ethernet technology. The specific working principle of this kind system is: video information is compressed by cameras and transmitted to a WEB server through internal bus; network users can watch the monitoring information on the WEB server through the client browser. As Internet application, network video monitoring also realizes the multi-user concurrent access [6].

through the message center the message is forwarded to the user.

 Mobile video monitoring stages: compared to the other two monitoring methods, mobile video monitoring adopts newer technology. Mobile video monitoring system is a combination of computer, the video transmitter, mobile network and video coding technology. It can transmit on-site real-time information of different locations through mobile network to the monitoring center or handheld devices [7]. With mobile communication transmission into the 3G era, bandwidth of mobile data transmission is increased sharply, as solved the transmission bandwidth bottlenecks of video monitoring business on the mobile network. Application range of 3G mobile monitoring is very wide, and it can be divided into security monitoring and interactive monitoring, as can be used in intelligent transportation, home care, remote medical treatment and other fields.

III. SYSTEM DESIGN This paper proposed a SMS monitoring system installed on Android mobile terminal. The system runs above Android2.0, interacting with the outside world in the form of SMS and MMS.

With the development of multimedia technology, and the enhancement of automation level of remote monitoring and remote data acquisition, the data transmission mode is changing from SMS to MMS, so that the remote supervisory control and data acquisition are becoming more direct, more visual, more accurate, and more comprehensive [12].

A.

System Operation Process The system operation process (shown in Figure 1) is: install the system on the Android Remote Mobile Terminals (RMT), and register the terminal number having the monitoring permission and the agreed password for monitoring in the system. When the authorized Local Monitoring Terminal (LMT) sends the agreed password for monitoring, the system makes the certification match, if the match is successful, RMT replies a hint of asking LMT what to query. When RMT receives the query code from LMT, RTM starts the camera and takes pictures automatically. Finally RMT replies the query of LMT through MMS or SMS to implement the remote monitoring of RMT.

A whole business operation era is coming for telecom operators now, and smart phone monitoring terminal will become universal mobile monitoring methods, as has a broad market prospect [8]. At present, mobile video monitoring system based on Android system generally uses the C/S structure. The client software is installed on the mobile receiving terminal, as logins the remote hard disk video recorder or server with specific IP address through telecom operator networks. By receiving the video deployed on remote device cameras and conformed to the H.264 coding standard (the standard adopts high precision, multiple model prediction technologies to improve compression ratio for reducing the code flow), the mobile receiving terminal implements remote video monitoring. C. Short Message Service of Mobile Phone Because the Short Message Service (SMS) of mobile phone has convenient and economic advantages, more and more people are used to use SMS to convey information [9]. SMS develops rapidly in China, and being the main business of telecom operators, SMS has become a most widely used communication service in addition to the call service [10].

Install system on RMT

Register monitor number and password

SMS monitor module for SMS monitoring

Separate SMS terminal number & content

RMT replies monitoring information

LMT sends query code according to hint

If match is successful RMT replies a hint

Match with SQLite database records

Fig. 1. System operation process

B. Function implementation To realize the goal of system operation, the system is divided into several functional modules in detail. The mutual calling process and the data transmission relations of the modules are shown in Figure 2.

SMS are divided into two kinds: one kind is directly sending or receiving text messages via mobile phones or other user terminals, and each message is composed of 160 characters in English or numeric, or 70 Chinese characters. The other kind is called Multimedia Messaging Service (MMS), and it is the upgrade of SMS. MMS is a messaging system where messages may include multimedia elements such as pictures, video and audio [11]. The message sending mode of MMS and SMS is the same: store-and-forward business. That is the message is sent to the message center firstly, and then

1) Register Interface: as a portal interface, Register Interface sets “the monitor password”, “agreed SMS content”, “delete number”, and the other UI controls, and it is responsible for collecting the corresponding SMS content, adding the “register”, “query” and “delete” button, receiving

54

the “Click” event, and realizing the monitor number and contract SMS content to be added, deleted, and checked. 2) Establish a SQLite Database: in SQLite database building the corresponding fields of authorized monitor number and contract SMS content. When clicking the “register”, “query” or “delete” button on the registration screen, execute SQL statements, to realize the collected monitoring number and the contract SMS content to be added, deleted, and queried in the database. 3) Monitor Module: When clicking the “monitor” button on the registration screen, traverse all the records of monitor numbers and contract SMS contents stored in the database and extract them to be encapsulated in the Buddle object, as is passed to the monitor module through the Intent object. Monitoring module is a derived class of BroadcastReceiver components, as is using onReceive () function to monitor. 4) SMS for Matching: LMT sends a SMS to RMT, and the SMS content should be the contract SMS content registered in the SQLite database. 5) Match Check: Check whether the LMT number and SMS content match with the records in the database. 6) Match: The PDU (protocol data unit) model has gradually become the main model used for sending and receiving SMS. General SMS contains SMS center number, mobile terminal number, and SMS content. Thus it can obtain the terminal number and SMS content from the PDU of SMS, and make a comparison with each record in SQLite database. If the records are matched, the parameter of match in monitor module is set to TRUE, otherwise the system halt. 7) Hint SMS: RMT sends hint SMS to LMT, its content is “match successfully, reply 1 for querying monitoring picture, reply 6 for resetting”. 8) SMS with Query Code: LMT sends SMS to RMT with query code listed in the hint SMS. 9) Code Judge: If LMT number has been matched, Match Check module invokes the specified module according to the query code in replied SMS. The specified module will extract relevant data, and send data to LMT. 10) Camera Module: If the query code is 1, open camera and take pictures. A new object of Camera class is used to take pictures, calling a time delay function, delaying for a period of time (such as 5 seconds), calling picture function of Camera class to take pictures. The pictures are in the form of JPG format images through file output stream, and are put in the embedded memory card on the specified path. 11) MMS Sending Module: MMS sending module reads images from the memory card and sends MMS to LMT. 12) Cancel Match Module: If the query code is 6, the parameter of match in monitor module is set to FALSE and let LMT send SMS to rematch.

Fig. 2. Mutual calling process of the modules

C. Security Mechanism The MMS and SMS replied from RMT involve camera photographs, and other important information, and the information is sent to LMT in the form of SMS or MMS, so to ensure information security and not affect the normal SMS communication between RMT and LMT or other mobile terminals, it is necessary to set up security certification for the SMS monitoring process. The system designs the security mechanism mainly by setting authorized LMT number and monitoring password. The security certification process is shown in Figure 3.

Fig. 3. The flow chart of system security certification

1) Certification SMS: When LMT sends certification SMS to RMT, RMT extracts the code form SMS first, comparing it with the authorized LMT registration code list. If the code matches with the authorized code list, LMT has the right to monitor RMT. Then system checks whether the monitor parameter is set to true. When it is true, the SMS content is extracted and compared with the monitor password list. If the password matching is success, RMT sends confirm SMS to LMT, or the certification SMS is regarded as normal SMS. 2) Confirm SMS: When RMT accepts the certification SMS from LMT, it automatically sends confirm SMS to LMT,

55

Android devices running Android application on the PC; DDMS is a very powerful debugging environment to help developers observe the running process of Android applications. Based on the table of test cases, the above test modes are used for each test case and the typical test results are as follows:

whose content can be set to: “match successfully, reply 1 for querying monitoring picture, reply 6 for resetting.” 3) Query SMS: LMT replies RMT with query SMS including query code according to confirm SMS. 4) Data Information: RMT extracts the data of related modules according to the query code, and automatically sends them to the LMT in the form of MMS or SMS. 5) End Query: After LMT finishes monitoring, it needs to cancel its surveillance authority. So LMT sends end query SMS to RMT to change its monitoring state. 6) Cancel Monitor Authority: RTM sets the monitoring state to False, modifies the sign of matching success, and cancels the monitor authority of LMT.

A. Test on Item 3.1 Add code “Log.i("smsreceiver","----oncreate----");” in the onReceive() function in the SmsReceiver.java file, and press the monitor button in AVD, then its DDMS status is shown in figure 4.

IV. SYSTEM TEST System test plays an important role in software development. In this link, developers can find problems in the system operation, including bugs and expected results not conform to the expected output of design, and find the source of errors and faults. As improves the program to ensure that the developed system can meet the demand of design with high performance and stability. In this paper, the system is tested under the debug simulation environment combined with the actual operating conditions. According to the operation process and system design target, test cases are designed, as cover all the implementation functions of the system, to fully test the correctness of the system operation. Typical test cases are shown in Table 1. TABLE I. No. 1.1

2.1

2.2

2.3

Test Item

Test Module

Fig. 4. Screenshot of DDMS test on Item 3.1

Figure 4 implies the monitor module starts successfully, and the test on item 3.1 is finished. B. Test on Item 3.2 Through the Emulator Control window of DDMS, set the visual LMT whose number is 13233 send SMS to the visual RMT. The content of SMS is “11qwe”, as has already been registered in the system. Add code “Log.i("smsreceiver","---match----");” following the assignment statement of match sign in match module, and start system. Figure 5 and figure 6 show the process.

TABLE OF TEST CASES Module Description

Test Mode

Expected Result

Open the main User Display system DDMS Successfully display interface Interface interface AVD system interface Number authorization and password registration Delete authorized number Retrieve authorized number and registered password

Storage Module

Add, delete, retrieve the input data

AVD

Successfully add authorized record in registration list

Storage Module

Add, delete, retrieve the input data

DDMS AVD

Successfully delete authorized records from registration list

Storage Module

Add, delete, retrieve the input data

AVD

Successfully show the registration list

3.1

Start monitor match

Monitor Module

Analyze and match SMS

DDMS

Successfully start monitor module

3.2

SMS matching

Monitor Module

Analyze and match SMS

DDMS

Successfully match the authorized number

3.3

Send hint SMS

Monitor Module

Analyze and match SMS

AE

Successfully send hint SMS

4.1

Camera monitoring

Camera Module

Start camera, take picutres and store pictures

AE

Successfully start camera, take pictures, store pictures, and send MMS

Fig. 5. Screenshot of sending SMS in DDMS

The Test mode includes Dalvik Debug Monitor Service (DDMS) test, Android Virtual Device (AVD) test and Actual Environment (AE) test. Among them: AVD is the virtual

Fig. 6. Screenshot of DDMS test on Item 3.2

56

The above figures imply the match function works successfully, and the test on item 3.2 is finished.

 In the present version, monitoring system starts by sending SMS manually. But in the subsequent version, monitoring system will be set to run automatically when system boots, in order to realize the active monitoring of RMT.

C. Test on Item 4.1 Figure 7 shows the test result of item 4.1. When LMT receives the hint SMS from RMT, it replies SMS with query code “1”. When RMT receives the query SMS, it starts camera module to take pictures and sends MMS with pictures to LMT. Figure 7(a) shows RMT starts to take pictures, and figure 7(b) shows LMT receives the MMS with pictures from RMT.

 In the present version, monitoring system replies query with static camera pictures in the form of MMS. But in the subsequent version, monitoring system will add control unit of video and microphone, so as to realize video and audio monitoring function. REFERENCES [1]

(a) RMT takes picture

Q. Xin, P. Olofsson, Z. Zhu, B. Tan, and C. E. Woodcock. “Toward near real-time monitoring of forest disturbance by fusion of MODIS and Landsat data,” Remote Sensing of Environment, vol. 135, pp. 234-247, 2013. [2] P. Garrido et al. “An Alert System for People Monitoring Based on Multi-Agents using Maps,” Journal of Universal Computer Science, vol. 19(9), pp. 1257-1274, 2013. [3] F. Ableson. “Introduction to Android development,” 2009. www.ibm.com/developerworks/opensource/library/os-androiddevel/index.html [4] http://www.android.com/ [5] M. Bazard. “Overview on Android – The New Mobile Operating System,” International Journal of Science, vol. 2(1), pp. 25-34, 2011. [6] G. Gualdi, A. Prati, and R. Cucchiara. “Video streaming for mobile video surveillance,” IEEE Trans. on Multimedia, vo. 10(6), pp. 11421154, 2008. [7] R. Cucchiara, and G. Gualdi. “Mobile video surveillance systems: An architectural overview,” Mobile Multimedia Processing. Springer Berlin Heidelberg, 2010, pp. 89-109. [8] Z. Liang. “Research on the Brand Communication Media and Channels of Mobile Internet Age,” Proceedings of the 9th International Conference on Innovation & Management, pp. 248-254, 2012. [9] K. Church, and R. Oliveira. “What's up with whatsapp?: comparing mobile instant messaging behaviors with traditional SMS,” Proceedings of the 15th ACM international conference on Human-computer interaction with mobile devices and services, pp. 352-361, 2013. [10] T. Chen, and M. Y. Kan. “Creating a live, public short message service corpus: The NUS SMS corpus,” Language Resources and Evaluation, vol. 47(2), pp. 299-335, 2013. [11] B. Kwan. “A Strategic Analysis of Mobile Data Service Offerings for the Vancouver 2010 Winter Olympic Games,” Doctoral dissertation, SIMON FRASER UNIVERSITY, 2008. [12] M. Mikulec, L. Kapicak, M. Voznak, et al. “Implementation of voice, SMS and MMS services into Interactive Video Audio System (IVAS),” Proceedings of the 36th IEEE International Conference on Telecommunications and Signal Processing (TSP), pp. 204-207, 2013.

(b) LMT receives MMS with picture

Fig. 7. Screenshot of AE test on Item 4.1

Figure 7 implies the camera module starts successfully, and the test on item 4.1 is finished. V. CONCLUSION With the continuous development of intelligent mobile phone hardware and software technologies, it becomes more and more easily for mobile phones and other mobile terminals accessing to the network and getting information. This paper proposes a monitoring system based on Android mobile terminals, with SMS or MMS as the medium. After authorized LMT sending certification SMS to RMT, LMT gets the control authority of RMT and access its information, so as to realize RMT monitoring. By passing the simulation software test and actual environment test, the proposed system is proved running normally and reaches the design objectives. But due to the rapid development of Android application technologies as well as the continuous improvement of monitoring service experience of users, there are many needs to perfect the system. The next version of the system will be perfected from the following aspects:

57