Mobile Phone Enabled Barcode Recognition for ... - Springer Link

Mobile Phone Enabled Barcode Recognition for Preferences Monitoring Eman Yasser Daraghmi, Chia-Feng Lin, and Shyan Ming Yuan Distributed Computing System, Institute of Computer Science and Engineering, National Chiao Tung University Hsinchu, Taiwan {Eman.yasser85,teralin,smyuan}@gmail.com

Abstract. In this paper, we present an innovative software system called Preferences Monitoring that makes use of modern middleware technologies and available hardware and communication systems. People can use the system to easily buy products that fit with their preferences. The system consists of mobile-phone side to scan product barcodes, submit user inquiries to a server, and display results. Results are related to whether users can eat or drink the product based on their preferences such as religion (no meat, beef, pork, or wine), or based on health settings such as allergies, chronic diseases or calories. Also, the system includes web services to facilitate user access to online profiles for editing accounts, viewing history-shopping and subscribing to other services. The preliminary results of testing our system show that the system can provide useful services for consumers. We also highlight some technical issues that are related to reading barcode using mobile phones. Keywords: Barcode recognition, consumer preferences, J2ME middleware.

1 Introduction Consumer products packaging has introduced a noticeable amount of product-related information. This includes nutritional information, ingredients, country of origin, and contents which may cause allergy. There is also a plenty of additional product-related information that is not directly printed on the product packaging due to size constraints and possibly commercial considerations, e.g. reviews by consumer watch groups or price comparisons. For these reasons, the information cannot be customized for consumers who find it difficult sometimes to choose products that fit with their preferences. In this paper, we address the problem of people shopping according to their preferences such as culture and health. People who travel to new countries usually face difficulties to adapt themselves to the new life. One of the most important difficulties is food/drink shopping especially when the language written on products is not the same as their native language. People will find it difficult to find the desired product, to know the ingredients of a product, and read warnings and other important information. Furthermore, people may come from different cultures with religious restrictions on food and drinks as people might not be allowed to eat meat, beef, or pork or drink wine. In addition to shopping in a foreign country, people in their native M. Zhou and H. Tan (Eds.): CSE 2011, Part II, CCIS 202, pp. 297–302, 2011. © Springer-Verlag Berlin Heidelberg 2011

298

E.Y. Daraghmi, C.-F. Lin, and S.M. Yuan

countries may need to monitor what they buy. People with chronic diseases or who have allergy to a specific food or drink need to watch carefully what they eat and drink. Also, people who are on diets need to calculate the number of calories in their food. Nowadays, manufacturers and markets use barcode to track a product so that storing and selling become easier. A barcode is a small image of lines and spaces that represent numbers and other symbols. A barcode symbol typically consists of five parts: a quiet zone, a start character, data characters (including an optional check character), a stop character, and another quiet zone [1]. It represents data by combining these parts that containing products information. This information can be extracted using an optical scanner barcode reader. Different standards have been used in the barcode systems. One of these standards is the EAN-13 barcode which is a 13 digit (12 + check digit) ant it is a superset of the original 12-digit Universal Product Code (UPC) system developed in the United States. The EAN-13 barcodes are used worldwide for marking products often sold at retail point of sale. The numbers encoded in EAN-13 bar codes are product identification numbers. All the numbers encoded in UPC and EAN barcodes are known as Global (GTIN). The barcode system can be integrated with new technologies such as mobile phones enabling more useful services. Mobile phones comprise display, long-range communication capabilities, processing, and user profile storage capabilities [1]. Employing mobile phones and image recognition technologies to identify a product barcode in shopping offers simple and faster interaction. In this paper we present a system that enables consumers to know with a single click more information about products and find an answer for their question “Is this product fine for me?”

2 Related Work In this section we review several systems developed based on barcode, mobile phones and image recognition technologies. Junaini and Abduallah developed a system to automatically scan barcode to recognize Halal food (food can be eaten by Muslims) using mobile phones [2]. Another system was proposed in [3] for visual impaired people. The system provides barcode scanning and text to speech conversion. A management system of grocery production based on barcode identification technology was proposed in [4]. The authors of [5] introduced an algorithm for 1D and 2D barcode using embedded digital signal processor (DSP). The above studies stated some related problems to mobile-phone enabled barcode recognition. Consumers sometimes cannot capture a barcode precisely due to limited code image size, blur code image due to reflections or poor lighting conditions, or out-of focus mobile phone distance [6],[7]. Solutions to this problem vary between titling, skewing or rotating a barcode image [8]. Another solution is an algorithm proposed to rectify the distorted code image [9]. Our system is different from the previous systems in three aspects. The first aspect is using efficient barcode recognition that supports camera zoom-in and zoom-out functionality. This reduces the effect of the size and distance constraints. The second aspect is using middleware technologies that support web services to allow users to add any service they want and enable it on their mobile phones. Finally, we include

Mobile Phone Enabled Barcode Recognition for Preferences Monitoring

299

different types of communication technologies such as 3G and SMS. The 3G communication offers high data transfer speed and capacity. The SMS can be used if a mobile phone doesn’t have 3G communication since SMS can provide useful information at real time [10].

3 System Overview The system is called “Preferences Monitoring”. It is based on EAN-13 barcode recognition. It allows users to predefine their profile from their account that contain user food culture, e.g. Christianity, Judaism, Islam, etc, the substances they are allergic to, calories based diet, user likes and dislikes. If a user is then holding a mobile phone in front of a certain product, e.g., in the supermarket, the phone displays a message like “This product is fine for you”, “Warning, this is not suitable for you!” or” No information found” as shown in figure 1.

Fig. 1. Screen-shots of Preferences Monitoring system

Users also can write comments on their profiles regarding a certain product, and view other users’ comments. They can also receive notifications about similar products to the one they usually buy or scan its barcode. Furthermore, users are able to see history of last mobile recognition. Our system was initially developed for nonnative consumer who cannot read Chinese in Taiwan, and then it included health monitoring services. The first part of the system has completed, while the second part is still under testing and will be ready at the end of May, 2011.

4 System Architecture The general architecture of our system is shown in Figure 2. The system contains three parts: the user profile side, the mobile phone side - barcode recognition

300


Fig. 2. System Architecture

component running entirely on J2ME enabled-mobile phone that supports the MMAPI9 (Mobile Media APIextension) extension, and the server side including web server and information server component which is located on a separate server. 4.1 User Profile Side The user pages are designed by ASP.NET and communicate with web server and database by the internet. The pages allow users to predefine their profile, enters user food culture, e.g. Muslims can eat only Halal food, Hindus are vegetarian, pork and shellfish are strictly forbidden in Judaism, etc, the substances they are allergic to, calories based diet, consumer likes and dislikes and any details the consumers want. Users also can view and define other services on their accounts. 4.2 Mobile-Phone Side The mobile phone enabled barcode recognition offers functionalities to recognize an EAN-13 barcode, communicate with servers and display results. The mobile phone side contains predefined J2ME components (libraries) for the major tasks: Object recognition, communication with a remote server as well as an XML parser for presenting the final information to the user using the available graphical user interface. The algorithm which is responsible for barcode recognition is based on the Batoo-Toolkit [11]. It is designed for Nokia phones. Additional components (functional modules) can be easily added without any required knowledge about the image recognition task. For example, the communication between the mobile phone and servers are based mainly on the 3G, other communication types such as SMS can be easily added.

Mobile Phone Enabled Barcode Recognition for Preferences Monitoring

301

4.3 Server Side The server side contains the web server that is responsible for communicating with the web server and providing the user profile side with the required information and functionalities. The second part is the information server which is responsible for communicating with mobile phones and product database. It receives requests from users and matches the request with user profile and product information. Then, it returns the result to the mobile phone as an XML file when the communication is 3G. Sending an XML file instead of bytes reduces the communication delay time.

5 Prototype Testing We have finished the first part of testing. We tested the system on a simulated database of markets with foreign students living in Taiwan. International students in Taiwan are from diverse cultures and they have different preferences. The students used their Nokia mobile phones to scan barcodes of different products, send their request to a server using 3G communication and get immediate results. The preliminary testing results are satisfying since the student showed a considerable willing to use the system because it eases their lives in Taiwan. Results also show that the process of barcode recognition was easy. The quality of imageprocessed barcode was enough to reduce the error rate of scanning and matching the product in the database. The communication speed was high with minimum delay. One limitation deserves mentioning here is battery consumption. We notice that some mobile phones consume more energy during the process of scanning barcodes and that is because of allocating more power to the camera. The second part of testing will focus on integrating all parts of the system. Testing will contain real database of some supermarkets after making agreement with them. It will also contain testing health preferences with local people from Taiwan.

6 Conclusions In this paper, we present a system that eases consumer shopping and maintains consumer preferences. Because people recently have been willing to see more services that solve their daily problems on their mobile phones, we propose “Preferences Monitoring” system. The system enables foreign and local people in a country to maintain their preferences, cultures, religions and health routines while shopping. After testing the system, we conclude that integrating the used barcode system in markets with modern mobile phones and middleware technologies has the potential for improving people lives. However, there are still issues in this field requiring more consideration such as supporting different mobile phone platforms, implementing efficient barcode recognition algorithm, and deploying up-to-date middleware technologies. Our future work will contain testing more services of the system in real databases and integrating the system with multiple mobile phone platforms such as Android and Windows.

302


References 1. International Organization for Standardization: Information Technology International Symbology Specication-Data Matrix. ISO/IECI6022 (2000) 2. Junaini, S., Abduallah, J.: MyMobiHalal 2.0: Malaysian Mobile Halal Product Verification using Camera Phone Barcode Scanning and MMS. In: Proceedings of the International Conference on Computer and Communication Engineering (2008) 3. Damper, R.I., Garner, D., Jordan, G., Rahman, A., Saunders, C.: A Barcode-Scannaeird for Visually-Impaired People. In: 18th Annual International Conference of the IEEE Engineering in Medicine and Biology Society, Amsterdam (1996) 4. Liu, X., Lin, C.: Information Management System of Grocery Production Processing Based on a Bar Code Identification Technology. In: IEEE International Workshop on Anti-Counterfeiting, Security, Identification, pp. 164–168 (2007) 5. Ohbuchi, E., Hanaizumi, H., Hock, L.A.: Barcode Readers Using the Camera Device in Mobile Phones. In: International Conference on Cyberworlds, pp. 260–265 (2004) 6. Liang, Y.-H., Wang, Z.-Y., Cao, X.-Y., Xu, X.-W.: Real Time Recognition of 2D Bar Codes in Complex Image Conditions. In: International Conference on Machine Learning and Cybernetics, vol. 3, pp. 1699–1704 (2007) 7. Adelmann, R.: Mobile Phone Based Interaction with Everyday Products - On the Go, Next Generation Mobile Applications. In: The 2007 International Conference on Services and Technologies, NGMAST 2007, pp. 63–69 (2007) 8. Falas, T., Kashani, H.: Two-Dimensional Bar-Code Decoding with Camera-Equipped Mobile Phones. In: Fifth Annual IEEE International Conference on Pervasive Computing and Communications, pp. 597–600 (2007) 9. Sun, A., Sun, Y., Liu, C.: The QR-code Reorganization in Illegible Snapshots Taken by Mobile Phones. In: Gervasi, O., Gavrilova, M.L. (eds.) ICCSA 2007, Part III. LNCS, vol. 4707, pp. 532–538. Springer, Heidelberg (2007) 10. Awwad, Y., Lin, H.T., Yuan, S.M.: Hybrid Teaching Assistant for Developing Students Interactivity and Comprehension. In: Ninth IEEE International Symposium on Multimedia Workshops, ISMW 2007 (2007) 11. Adelman, R., Langheinric, M., Floerkemeier, C.: A toolkit for Barcode Recognition and Resolving on Camera Phones - Jump Starting the Internet of Things. In: Workshop Mobile and Embedded Interactive Systems (MEIS 2006) at Informatik (2006)