Design and Development of a Fully Automated Consumer-based Wireless Communication System For Fruit Grading Wong Bing Yit, Nur Badariah Ahmad Mustafa, Zaipatimah Ali, Syed Khaleel Ahmed, Zainul Abidin Md Sharrif Electronics & Communication Department, College of Engineering, Universiti Tenaga Nasional Km7, Jalan Puchong-Kajang, 43009 Kajang, Selangor, Malaysia
[email protected],
[email protected],
[email protected],
[email protected],
[email protected]
Abstract – As a smart consumer, there is always a need for us to identify the quality of the goods we want to buy, such as trying to choose the better mango amongst the others in a fruit market. With the popularity of the mobile phone and the maturity in wireless messaging technology, it is no more a dream for us to get the information we want anywhere, anytime. This paper proposes a new mobile data service that includes signal processing service via MMS (Multimedia Messaging Service) and SMS (Short Messaging Service). This paper discusses the study, design, construction and analysis of a fully automated consumerbased prototype wireless system which interconnects signal processing tools with consumers using MMS and SMS technology. The system with signal processing application processes the information sent by the consumer using MMS. Then it sends back the grading result to the consumer using SMS.
processing. The automated prototype system attempts to utilize MMS and SMS technology to connect the consumers with the remote signal processing application, by processing the received image, and letting the consumer know the grade with price suggestions within minutes via SMS. It would be beneficial for the consumers who need the information and grading of certain products while they are in the market. It could also serve as a type of system architecture design for wireless signal processing networks.
Keywords- wireless communication system, MMS, SMS, remote signal processing, automated system, interconnection, network planning, mobile data service.
II. DESIGN & TECHNOLOGY PREVIEW In order to design and develop this system, a lot of literature review had been conducted in order to understand, how MMS works, how SMS works, and how to establish communication, using wireless communication technology, between the cell phone and signal processing application. In the following sections, the operation of wireless communication technology is discussed.
I. INTRODUCTION & MOTIVATION In this project, a prototype wireless communication system is designed and built which enables the consumer in the market to send the image of a fruit (such as banana, mango, apple) using MMS to our server through GPRS (General Packet Radio Service) network. The MMS is received by the GPRS modem connected to the server. The server interconnects the GPRS modem with a signal processing application and database. The server processes the result from signal processing application and sends back the grading result of the fruit to the consumer using SMS. This project looks into the available wireless communication technologies, and the study and design of the system architecture and components. The development and construction of the system, and interconnection issues between components are discussed. The possibilities of implementation for this system are reviewed. The performance analysis between different wireless communication technologies is conducted. The data of time spent for each process is collected and analysis of GPRS network between mobile service providers is done. Finally, an approach for image pre-processing for performance optimization is developed. This paper proposes innovative use of mobile data service with wireless messaging technology for remote signal
A. MMS Short Review MMS is a telecommunications standard for sending messages that includes multimedia objects (images, audio, video, rich text) [1], [2]. MMS messages are typically transported over WAP (Wireless Application Protocol), and are encoded using WAP MIME (Multipurpose Internet Mail Extensions) formats. Normally, MMS is sent and received using GPRS or EDGE (Enhanced Data rates for Global Evolution) network. Here is the process of how MMS works [3] [4] [10]: (a) The sending phone initiates a data connection that provides TCP/IP network connectivity, usually over GPRS or EDGE network. (b) The sending phone performs an HTTP (Hypertext Transfer Protocol) POST method to an MMSC (Multimedia Messaging Service Centre) of the MMS message encoding in the MMS Encapsulation Format. (c) The MMSC receives the MMS message submission and validates the message sender. (d) The MMSC stores the content of the MMS message and makes it available as a dynamically generated URL link. (e) The MMSC generates an MMS notification message, which is sent via WAP Push over SMS
978-1-4244-4522-6/09/$25.00 ©2009 IEEE
364
ISCIT 2009
to the message recipient(s). This MMS notification message contains a URL pointer to the dynamically generated MMS content. (f) The recipient receives the MMS notification message. It then initiates a data connection that provides TCP/IP network connectivity (usually over GPRS). (g) The recipient phone performs an HTTP (or WSP) GET method to retrieve the MMS message content URL from the MMSC.
Fig. 2: System Network Architecture for Fully Automated Consumer-based Wireless Communication System with Signal Processing Grading Tools.
B. Overall Transaction and Functions of Components From the system network architecture, the mobile user (consumer) captures the fruit image in the market and sends Fig. 1: A general MMS network architecture [1]. the MMS through the base station transmission to mobile Since the project is at developmental stage, MM1 protocol network MMSC, as discussed in section II. Then the MMS is received on the GSM/GPRS modem (interface between Mobile Station and MMSC) is used in this connected to the phone. The GPRS modem communicates project. It is the MMS standard for phone/GPRS modem to with server using MMS/SMS gateway (software for send and receive MMS messages from an MMSC [8] [9]. interconnection between GPRS modem and computer) through AT command (Hayes command set). The GPRS B. SMS modem functions in a manner similar to a normal mobile SMS stands for Short Message Service. It is a technology phone with no battery required. that enables the sending and receiving of messages between The MMS gateway communicates with the server mobile phones. application using HTTP protocol. The function of MMS One SMS message can contain up to 160 characters if 7-bit gateway is to listen to new MMS notification messages, character encoding is used. (7-bit character encoding is decodes MMS message and stores the data into a folder in the suitable for encoding Latin characters like the English server. When MMS notification message is received, the alphabet). It could encapsulate 1120 bits of data [3]. gateway initiates a GPRS connection to the mobile operator GSM network (Global System for Mobile Communications) and a connection to the operator MMS WAP gateway over is used for SMS sending and receiving. In Malaysia most GPRS is established. In order to retrieve the content of the GSM providers are using GSM 900MHz and GSM 1800MHz MMS message from the mobile operator's MMSC. After frequency bands. This project uses SMS to send back the retrieving the MMS, the MMS gateway reconnects the GPRS result to the consumer. modem to operator’s GSM network to continue listening incoming SMS. III. NETWORK DESIGN & COMPONENTS Visual Basic.net programming language has been chosen After understanding the requisite wireless communication for rapid application development and user friendliness. The technology, the system architecture was designed for the application programmed using VB.net is the main control prototype of this consumer-based wireless communication application for the automation process, messaging application, system. storing and retrieving data from database, calling and stopping signal processing application. The main application initiates A. System Architecture the process of signal processing when it detects a new Fig. 2 is the proposed system network architecture. The incoming MMS. system details are explained in following subsections.
365
The Signal Processing Application is developed in MATLAB (a technical computing software) and compiled as an executable file. It consists of image processing for feature extraction of the fruit, SVM (support vector machine) for fruit classification, and fuzzy logic for fruit grading. It is able to judge the types of fruits among banana, apple, mango and carrot, and grade the quality of the fruit. A MYSQL database is used to store the data, date and time for data verification and performance analysis. C. Flow of Processes in Main Application The flowchart of the process in the Main Application is shown in Fig. 3.
extract the image’s path along with the sender number to send back the result later using SMS. Then from the main application, the image data is inserted into MYSQL database along with the date and time. A modified path for the image is needed as MYSQL database will automatically erase the first \ of the string. At first, the image was stored into the database, but later, to optimize performance, the image path was inserted instead. The ID of the image is retrieved from the MYSQL table as it is a unique ID created by the database for the image. Next, the program initiates the process of signal processing by passing the ID to the signal processing application. The signal processing application connects to the database and retrieves the image path to process the image. After processing, the signal processing application stores the result in the database. Meanwhile, the main application waits until the signal processing application has finished. Then it retrieves the result from the database using the ID. Result string verification subfunction in the main application then verifies whether the result string is valid or else it regards it as a processing failure and sends a processing failure message to the user. The processed result needs to be encoded to URL string format to be sent to the MMS/SMS gateway as the gateway is using HTTP protocol. The MMS/SMS gateway processes the HTTP GET request and sends back the result using AT command to communicate with GSM/GPRS modem. D. Signal Processing Application Flow The flowchart for the interconnection between signal processing applications is shown in Fig. 4.
Fig. 3: A flowchart for main control and automation program.
When the main application starts, the MMS gateway service is examined whether it is running. This is to verify whether the GPRS modem is listening to incoming SMS and MMS. If not, the program will start the MMS gateway service. Next, the program starts to watch for incoming MMS with jpg file as most images taken by phone camera are encoded in jpg file format. This is to ensure the signal processing program will only process an image file. The problem encountered here is that most of the time the file system watcher fires an event too early, causing multiple file detections in a new coming file. So another open file loop needs to be added to ensure the file has been completely loaded before proceeding. After the file is finished loading, the program starts extracting the information from the decoded MMS file to
Fig. 4: Flowchart of signal processing application.
When the program starts, it first receives the ID from VB.net as input argument. Then, it retrieves the image path from the MYSQL table. The image path would be the input argument for the image processing feature extraction program. The program extracts the features from the image and puts it as an input argument for the SVM (support vector machine, an AI method for classification) program for fruit
366
classification. After the program has determined the type of fruit it opens the relevant fuzzy logic program to determine its grade [12] - [15].
studied. The mobile operator MMS services are studied only for Malaysia. TABLE I MMS CHARGES BETWEEN MOBILE OPERATORS IN MALAYSIA
E. MMS Gateway Flow Fig. 5 is a simplified flowchart of the process in the MMS gateway. The events occurring in the MMS gateway while receiving an MMS and sending an SMS are shown.
Plan Prepaid & Postpaid
MMS Prepaid Charges between Service Provider Maxis Celcom Digi U Mobile 25 sen/msg
35 sen/msg
25 sen/msg
18 sen/msg
The basic charges of MMS within same operator for different mobile operators are given in Table I. It is subjected to change depending on the mobile service provider and hence serves only as a rough reference. From the table, most of the MMS charges are about 25 sen per message. Since this is a consumer-based and home-user based communication system, our argument is that the service could be profitable to both service provider and consumer if the value of the item to be identified is much more than the cost of MMS message (or the cost of item identification). Therefore, spending 25 sen more to get probably the best quality amongst the others is workable for some cases, as long as the item identification cost is reasonable and acceptable for the consumer. Let the total number of qualities of a product to be represented in Qt and quality factor of a product is Qi (less than 1 is low quality, 1 is equivalent to the normal quality and more than 1 is high quality). Therefore the overall quality distribution of a product is equivalent to:
Fig. 5: Flowchart of MMS gateway.
First the computer needs to initialize the serial port communication with the GSM/GPRS modem. Then the gateway communicates with the GPRS modem using the AT command. The gateway points to the message storage location and sets the modem to start listening for an incoming message. When a new SMS is delivered, the gateway processes the command and decodes the SMS. Next it identifies whether the message is a WAP push MMS notification message. After it has detected a new MMS notification message, the gateway stops the SMS message listening and initializes the GPRS network connection. The gateway retrieves the data stream from the MMSC and decodes the MMS. The decoded MMS is stored inside the server and GPRS modem will be reinitialized to listen to the network again.
Assume the market price of a product is Po and the real hidden price Pr of a product is market price multiplied by its quality factor; i.e., The grading service will only be beneficial to the user if the total price the user pays, Pt (including market price Po and service price, Ps) is less than the true value of the product. In other words,
Therefore if the service price is too high, making the total cost of the product higher than the cost of the best quality of the product, the service will not be beneficial to the user.
IV. PERFORMANCE ANALYSIS & OPTIMIZATION A basic implementation analysis, performance evaluation and network optimization has been conducted to further improve the system usability and responsiveness. A. Possibilities of Implementation In this scenario, MMS rates in Malaysia in relation with the possibilities of implementation of this MMS service are
367
B. Result & Performance Analysis An example screenshot of the main application and the result received via SMS is shown in Fig. 6.
Fig. 6: GUI of the VB.net application
In Fig. 6, the first tab is to activate the MMS watching and automation process. The detected MMS and processing flow is displayed in the list box and the process is also stored in a text file using date and time as filename. The second tab is to tweak the configuration setting. The link of the MMS folder and signal processing application can be changed here. The third tab is for testing the connection of the GPRS modem to the server, connection of the GPRS modem to the GSM or GPRS network, and switching on/off of the MMS/SMS gateway service. The fourth tab is the overview of the system architecture and the explanation of the software.
TABLE II PROCESSING TIME
Process MMS sending and receiving Image Processing Time Result sending and receiving
Avg. Time Spent 2 minute 36 seconds
Std. Dev. 57.74
4.17 seconds
1.27
13 seconds
3.88
From Table II, it can be seen that the time spent for the consumer to send MMS and receive the grading result is roughly 2-3 minutes. Most of the time was spent in MMS sending & receiving and SMS sending & receiving. Obviously this depends on the wireless network traffic congestion, and is beyond our control. From the standard deviation of the MMS sending and receiving process, it can be seen that the condition of the GPRS network varies a lot. This is the key issue that affects the processing time for the consumer. The actual process in the server normally takes around 3-5 seconds, which is good and could be an excellent service to the consumer if the network connection could be faster. A time of 2-3 minutes could only be acceptable for some consumers, while a processing time within 1 minute could be ideal. Therefore, a suggested method here would be having data pre-processing program on the phone to send SMS data only to the server. Another area of concern here is to compare the image processing result between images compressed in MMS and images taken in a camera. Such a comparison of result between MMS image and camera image is shown in Fig. 10.
Fig. 7: Example of the MMS message & SMS result to the consumer
The system performance was tested by sending test messages using different mobile operators at different times. The results for 16 such messages are shown in Table II.
Fig. 8: Comparison result between camera image (above) and MMS image (below).
The camera image looks much sharper and clearer (~1.7MB 2816x2112), while MMS image (~10kb 160x120) looks a little blurry. The feature extraction result produced
368
between each image is still very close to each other. Therefore the fruit sorting using SVM and the grading using fuzzy logic is unaffected. There is a small concern here since for development purposes GPRS modem is used. When the GPRS modem is connected to GPRS network to retrieve the MMS, the modem will not be able to listen to new coming SMS. Therefore the new coming SMS at that time will be stored at SMSC and wait until the GPRS modem can receive SMS again which reduces the performance of the system. The option here could be adding extra GPRS modem and distribute the tasks, such as one for receiving MMS notification message, one for GPRS connection and another one for sending SMS message. C. Network & Performance Optimization In order to maximize the network performance, the fewer the components involved the better the performance. The components involved has been minimized as much as possible and the faster connector are chosen for faster processing. In an automation system, if problems arise most of the errors would be difficult to identify and determine without an error log. Therefore most of the coding is written using try and exception loops for better error solving and is recorded in a log file. For faster signal processing, less complexity of the application normally provides better speed. A profiler is used to simplify the program as much as possible. Further, the connection with the database could slow down the process. Therefore the system architecture will try to minimize the connection to database as much as possible such as saving the data temporarily into a text file instead and forcing an ideally one time connection to the database. A suggested method here is to separate the process of listening to incoming SMS and retrieving MMS as doing the same thing on a single modem will slow down the process of listening to more incoming MMS notification messages. V. CONCLUSION & ACCOMPLISHMENT In this paper, a new MMS-based system has been proposed, designed, and developed with signal processing for fruit grading for consumers. The prototype network architecture, integration of wireless messaging system with signal processing between mobile consumers for development purposes was studied, proposed and designed. The flowchart of the whole process is discussed. The interconnection and automation issues between each component has been discussed and solved. The possibilities of implementation, performance evaluation methods and system optimization methods have been discussed. In future a data preprocessing method will be designed and developed. ACKNOWLEDGMENT This project is funded by MOSTI under e-science project number 01-02-03-SF0094.
REFERENCES [1] [2] [3] [4] [5] [6] [7] [8] [9] [10] [11]
[12]
[13]
[14]
[15]
[16]
[17]
[18]
[19]
[20]
369
WAP MMS Architecture Overview Specification Version 25-April-2001, Open Mobile Alliance. WAP-205-MMSArchOverview-20010425-a, 2009. Multimedia Messaging Service. Wikipedia. 6 April 2009. Accessed: 15 April 2009. Available: http://en.wikipedia.org/wiki/Multimedia_Messaging_Service A. Henry-Labodere and V. Jonack, “SMS and MMS interworking Mobile Network,” Chapter 9, Artech House Inc. 2004. How MMS works. NowSMS. 2009. Accessed: 15 April 2009. http://www.nowsms.com/howmmsworks.htm Multimedia Messaging Service Encapsulation Protocol, Version 1.1 30-October-2002, Open Mobile Alliance. OMA-MMS-ENC-v1_120021030-C, 2009. MMS Message Format. Symbian Developer Library http://www.symbian.com/developer/techlib/v70sdocs/doc_source/devg devgu/cpp/Messaging/MMS/format.html Multimedia Messaging Service (MMS) Functional description Stage 2, 3GPP. 3GPP TS 23.140 version 3.1.0 Release 1999. MM1, The NowSMS Tech Support Files, 17 February 2009. Accessed: 15 April 2009. http://blog.nowsms.com/search/label/MM1 Overview of MMS. Mbuni. 2009. Accessed: 15 April 2009 http://www.mbuni.org/userguide.shtml#Section_.1.1.1 World GSM Frequency Bands. Simoncells. 2009. Accessed: 15 April 2009. http://www.simoncells.com/scripts/gsmzone.asp Connecting to an Operator MMSC. NowSMS. 2008. Accessed: 16 April 2009. http://www.nowsms.com/documentation/ProductDocumentation/mms_ notifications_and_content/Connecting_to_operator_MMSC.htm B.Y. Wong, Z.Abidin, Z. Ali, S.K. Ahmed, N.B. Mustafa. Image Processing of a Banana: Area Determination via Edge Detection using MATLAB: The 4th International Colloquium on Signal Processing and its Application (CSPA 2008), 7-9 March 2008. B.Y. Wong, Z.Abidin, Z. Ali, S.K. Ahmed, N.B. Mustafa. Image Processing of an Agriculture Produce: Determination of Size and Ripeness of a Banana: The International Symposium on information Technology 2008 (ITSIM ’08), 26-29 August 2008 N.B. Mustafa, Z.Abidin, Z. Ali, S.K. Ahmed, B.Y. Wong. Fuzzy Classification of Agriculture Produce: Application to Banana Sorting by Size: 4th International Conference on Information Technology & Multimedia at UNITEN (ICIMU 2008), 18-19 November 2008. Min Min Kyaw, S.K. Ahmed, Z. Abidin. Shape-Based Sorting of Agriculture Produce using Support Vector Machines in a MATLAB/Simulink: 2009 5 th International Colloquium on Signal Processing and Its Applications (CSPA 2009) 6-8 March 2009. Siang, B.K., Bin Ramli, A.R. Prakash, V. Bin Syed Mohamed, S.A.R. SMS gateway interface remote monitoring and controlling via GSM SMS. Telecommunication Technology, 2003. NCTT 2003 Proceedings. 4th National Conference on 14-15 Jan. 2003 Page(s):84 - 87. Zhao Ji-Chun, Sun Su-Fen, Yu Feng, Luan Ru-Peng, Zhang Wei. The Design and Realization of Embedded Wireless Video Monitoring System Based on GPRS. Wireless Communications, Networking and Mobile Computing, 2008. WiCOM '08. 4th International Conference on 12-14 Oct. 2008 Page(s):1 - 4. Jung-Hoon Lee, Min-Jae Seo, Gwan-Su Kim, Hong-Hee Lee. IEC 61400-25 interface using MMS and web service for remote supervisory control at wind power plants. Control, Automation and Systems, 2008. ICCAS 2008. International Conference on 14-17 Oct. 2008 Page(s):2719 - 2723. Tahat, A.A. Mobile personal electrocardiogram monitoring system and transmission using MMS. Devices, Circuits and Systems, 2008. ICCDCS 2008. 7th International Caribbean Conference on 28-30 April 2008 Page(s):1 - 5. Jamal, A., Mannan, H., Maqsood, M., Ilyas, M.S. Remote Monitoring and Control using Wireless Messaging. Engineering Sciences and Technology, 2005. SCONEST 2005. Student Conference on 27-27 Aug. 2005 Page(s):1 - 5.