Machine Learning Techniques for Mobile Intelligent Systems: A Study Archana Chaudhary Asstt. Professor, School of Computer Science & IT, Devi Ahilya University, Khandwa Road, Indore-452017, Madhya Pradesh, India
[email protected],
Dr. Savita Kolhe Scientist(S.S), Computer Application, Directorate of Soybean Research (ICAR), Khandwa Road, Indore-452017, Madhya Pradesh, India
[email protected],
Dr. Raj Kamal Professor, School of Computer Science & IT & School of Electronics, Devi Ahilya University, Khandwa Road, Indore-452017, Madhya Pradesh, India (IEEE Member)
[email protected]
Abstract—The field of Machine learning has developed tremendously with the development of Mobile technology. This paper presents the concept of machine learning and presents a concise study on different machine learning techniques and their applications on mobile devices. It also presents a brief overview of performance parameters of machine learning techniques useful for mobile devices.
Keywords- ANN; Bluetooth; CART; GSM; KNN; Mobile computers; Mobile Intelligent System; Mobile phones; PDA; Pocket PC; Wireles networks; 3G+.
I. INTRODUCTION The Machine Learning field has evolved from the field of Artificial Intelligence, which aims to imitate intelligent abilities of humans by machines. Mitchell [2] emphasizes machine learning as a study of computer algorithms that performs better through experience. It implies computer programs use their experience of past tasks to improve their performance. But description does not incorporate the concept of knowledge acquisition process for the stated computer programs. Alpaydin [3] defines Machine Learning as Èthe capability of the computer program to acquire or develop new knowledge or skills from existing or non-existing examples for the sake of optimizing performance criterionŠ. The field of Machine Learning has grown tremendously since last 50 years. According to Alpaydin [3] the two important factors for growth in this field are removing tedious human work and cost reduction. Machine Learning techniques when applied to different fields as Natural Language Processing, Medical diagnosis, Bioinformatics, Efficient search engine design, Mobile Learning, Classifying DNA sequences, Speech and handwriting recognition, object identification in Computer vision, Game playing, Robot Locomotion, Stock
978-1-4673-1989-8/12/$31.00 ©2012 IEEE
Market Analysis, Cheminformatics, detecting credit card fraud in financial institutions have proved to work with huge amounts of data and provide outputs in a matter of seconds. Thus, machine learning has great potential in improving the efficiency and accuracy of decisions drawn by intelligent computer programmes. Machine learning field is also useful for mobile devices such as Smartphones, smartcards, sensors, handheld and automotive computing systems [20]. Mobile Technology has fostered development with the help of increasing mobile terminals (e.g. computers, mobile computers, mobile phones, Pocket PC, PDA) and mobile networks (GSM, 3G+, wireless networks, Bluetooth, etc.). Machine learning techniques like C4.5, Naïve Bayesian, Decision trees etc are helpful for mobile devices. Machine learning applications for mobile devices include Sensor based activity recognition, Mobile text categorization, Malware detection on mobile devices, Language understanding etc. But the application of traditional machine learning approaches have several limitations with Mobile devices. These problems include: • A mobile device has limited computational and memory capacity which restricts the application of machine learning algorithms that work with large data set. • A mobile learning environment exhibits concept drift, for e.g. the concept being learned changes as the userÑs mode of operation changes. • A userÑs intolerance of any hindrance that machine learning techniques may introduce in the normal mobile phone operation. • Therefore, a study of different machine learning techniques on different parameters is of much importance in order to pave a way for future works in this direction.
II.
TYPES OF MACHINE LEARNING
Machine learning can be categorized into major groups as supervised, unsupervised machine learning and reinforcement learning as shown in Fig. 1. These groups represent how the learning method works.
Figure 1. Machine learning types
1) Supervised learning It consists of algorithms that reason or learn from externally supplied instances to result in a general hypothesis that makes prediction about future instances [1]. There is an outcome or output variable to guide the learning process in this category. There are many supervised learning algorithms as decision trees, K-Nearest Neighbour (KNN), Support Vector Machines (SVM) and Random Forests [4].
that are further used for classification of new instance [2]. It is useful to consider more than one neighbor at a time while classifying hence it is referred as K-Nearest Neighbor. This nearest neighbor is measured in terms of Euclidean distance that measures the dissimilarities between examples represented as vector inputs and some related measures. B. Decision Tree A decision tree is defined as a hierarchical model based on nonparametric theory where local regions are identified in a sequence of recursive splits in a smaller number of steps that implements divide and conquer strategy used in classification and regression tasks [3]. A decision tree takes input data and applies decision rules for predictions or future course of actions. The working of a simple decision tree is shown in Fig. 2.
Figure 2.
2) Unsupervised learning Unlike supervised learning where there is presence of an outcome or output variable to guide the learning process unsupervised learning makes models without predefined classes or examples [5]. It further implies that here no supervisor is present and learning relies on guidance obtained heuristically by the system testing different data samples or environment. The states of outputs are defined implicitly by specific learning algorithm that is used and built in constraints [5] of the problem. 3) Reinforcement learning The machine interacts with its environment by producing certain actions in reinforcement learning. These actions affect the state of the environment, which in turn results in the machine receiving some scalar rewards (or punishments). The goal of machine is to learn to act in a way that maximizes the future rewards it receives (or minimizes the punishments) over its lifetime. This learning is related to decision theory (in statistics and management science) and control theory (in engineering). III.
MACHINE LEARNING ALGORITHMS
There are various machine learning algorithms depending upon the domain of their application [1]. The following Machine learning algorithms are considered to give readers a basic understanding of machine learning algorithms. A. K-Nearest Neighbour(KNN) This is abbreviated as KNN and is also called as instance based learning which is a type of supervised learning algorithm. KNN simply stores in memory present training dataset and when a new query is executed, a set of similar related instances or neighbours are retrieved from memory
Decision tree working
They are powerful and popular tools for classification and prediction. They represent rules, which can be understood by humans and used in knowledge system such as database. The strengths of using decision tree learning include • It generates understandable rules. • It can perform classification without much of computation. • It can handle categorical variables. • It provides a clear indication of which fields are most important for prediction or classification. The weaknesses of using decision tree learning include • It is not suitable for prediction of continuous attribute. • It performs poorly with many classes and small data. • It is computationally expensive to train. C. Support Vector Machine (SVM) It is a relatively new machine learning technique proposed by Vladimir Vapnik and colleagues at AT&T Bell laboratories in 1992 and it represents the state of art in machine learning techniques. The concept of SVM is used in Computer Science and Statistics as a set of supervised learning methods. SVM is useful for data analysis and identification of patterns that are useful for classification and regression analysis. The basic idea of SVMs is to make use of a (nonlinear) mapping function ( ) that transforms data in input space to data in feature space in such a way as to make a problem linearly separable. SVMs are gaining importance because they got a good theoretical basis and are successfully used in real world applications. SVMs are useful in bioinformatics in DNA sequencing and modeling
protein structure. There are two types of software that provide SVM training algorithms. The first type is specialized software whose main objective is to provide an SVM solver. LIBSVM [6] and SVMlight [6] are two popular examples of this class of software. The other class of software is machine-learning libraries that provide a variety of classification methods and other facilities such as methods for feature selection, preprocessing etc. D. Artificial Neural Network (ANN) Artificial Neural Networks or simply neural nets have many names such as connectionist models, paralleldistributed models and neuromorphic systems [7]. All these models attempt to achieve good performance through interconnection of simple processing units. ANN is based on Biological nervous system. A simple ANN is shown in Fig. 3.
Figure 3. Artificial Neural Network
These networks are used for approximating continuous/discontinuous functions and therefore they have applicability where we have basic information about inputoutput map to be approximated. A set of data (input-output information) is used for training the network. Once the network has been trained it can be given any input (from the input space that is to be approximated) and will produce an output, which would correspond to the expected output from the approximated mapping [8]. The quality of output that is produced is arbitrarily close to the actual output desired as regards to the generalization capabilities of these networks. Neural networks can be used for pattern identification, to test oracle, project effort and cost estimation, disease diagnosis, handwriting recognition, pattern classification etc. E. Classification and Regression Trees (CART) It is a popular technique, as it requires small input data for analysis. This is in contrast to other machine learning techniques where extensive input from the analyst, the analysis of intermediate results and interpretation of results are required [8]. CART designs classification and regression trees for predicting continuous dependent variables (regression) and categorical predictor variables (classification). The problems solved by this technique are either regression type problems or classification type problems. In regression type problems, one attempts to assign the values of a continuous variable from one or more continuous and/or categorical predictor variables [8]. In classification type problems, one attempts to conclude values
of a categorical dependent variable from one or more continuous and/or categorical predictor variables [8]. CART analysis is a tree building technique, which is different from other traditional data analysis methods. It consists of tree building process and finally selecting an optimal tree that fits the information in the learning dataset. F. Case Based Reasoning Case based reasoning is a common technique by which we solve new problems by using the solutions from the problems solved earlier. We take the instances of solutions from the problems solved in past and try to solve new problems by using these cases. Each solution available to us is termed as a case [8]. A new case is defined by the initial problem description. A case is retrieved from a set of past cases and this retrieved case is then combined with the new case through reuse strategy into solved case. This solved case becomes a proposed solution to the defined problem. Once this solution is identified, it is practically implemented to the real world problems and hence it is tested. This process of testing is termed as revision of the problem. The retain process starts after testing process. In retain process the useful experience is retained for future reuse and case base is updated by a newly learned case or by modification of some existing cases. G. Rule Induction It is an efficient machine learning technique and easy as rules are crystal clear to interpret than a regression model or a trained neural network. It consists of condition-action rules, decision trees or similar knowledge structures. In this technique the performance part sorts the instances down the branches of a decision tree or searches for the very first rule whose conditions match the instance, using an all-or-none match process [8]. Learning algorithms in rule induction process use greedy search method. There are two main approaches to rule induction-propositional learning and relational learning [8]. H. Genetic Algorithms and Genetic Programming These are a form of evolutionary computing that are used for problem solving techniques based on the principle of biological evolution like natural selection. Genetic algorithms deal with genes, chromosomes and population. They are based upon the Darwin theory Èthe survival of the fittestŠ depending upon the fitness function where the best possible solutions are chosen from the pool of individuals. The fitter individuals have good chance of their selection. Once selection process is complete, new individuals have to be formed. These new individuals are formed by the process of crossover or mutation. In the process of crossover, combining of the genetic makeup the two solution candidates (producing a child out of two parents) creates new individuals. In mutation process, few randomly chosen parts of genetic information are changed to obtain a new individual. This process of generation does not conclude until one of the conditions like minimum criteria is met or the desired fitness level is attained or a specified number of
generations are reached or any of the combinations above is attained [9]. I . Random Forest Random forests are used for classification. Breiman [10] defines a random forest as a classifier consisting of a collection of tree-structured classifiers {h (x), Qk, k=1,2..} where the {Qk} are independent identically distributed random vectors and each tree casts a unit vote for most popular class at input x. This technique consists of generation of group of trees that vote for the most common class. There are two important characteristics. Firstly, the generalization error converges as the number of trees increases in the forest and secondly this learning technique does not suffer from over fitting [10]. IV.
MACHINE LEARNING TECHNIQUES FOR MOBILE APPLICATION : A STUDY
Machine learning techniques have been used for mobile devices. An extension to the mobile MP3 player, XPod was developed by Sandor et al. [11]. It automated the process of selecting a song that best suited the current activity of the user. This system played contextually correct music using different machine learning techniques. In this work, SVMs were found to be well suited to different tasks for mobile music player as they begin to classify new instances having very less training data to start on. This is a desirable feature from a userÑs point of view because a mobile user spends very small time slice in setting up the system and more time in enjoying the benefits. But SVMs canÑt be created on a constrained device. SVMs can be created on an unconstrained device such as a PC and then be transferred to a portable device. Hence a constrained device can evaluate an SVM. Whereas decision trees are computationally feasible classifiers as these can be easily converted into a rule set that can be evaluated quickly. Decision trees perform better with boosting strategy. Machine learning techniques are also useful in data mining problems [12]. They showed that ANN technique has good ability to solve data mining tasks than the other techniques. Machine learning techniques are vital for the areas of network management and surveillance [13]. Machine learning techniques such as reinforcement learning and instance-based learning can also be used with a mobile music platform for different forms of music expressions [15]. It was further implied that Soar cognitive architecture is efficient to support mobile music interactions. Mobile learning is a novel and useful forms of learning as it has all the benefits of e-learning as well as eradicates the gap of time and space occurring in classroom learning [16]. To provide a better learning environment for English learning [16] presented a Personalized Intelligent Mobile Learning System (PIMS). PIMS suggested English news articles to the learners according to the learnerÑs reading abilities [16] using fuzzy item response theory. PIMS has been successfully deployed on Personal Digital Assistant (PDA) to provide personalized mobile learning for enhancing the reading ability of learners. PIMS also
enhanced English vocabulary of learners. The development of mobile technologies and easy access to multimedia web resources are two important factors that led to development of mobile learning systems [17]. A mobile learning system was developed that used existing web resources that resulted in learning material according to the needs of the learner. Mobile agents are agents that move in a computer network from one host to another for accomplishing different tasks [18]. A mobile agent is capable of taking intelligent decisions regarding its itinerary and modifies the decision(s) dynamically as per the information that becomes available as it moves from one host to another [18]. Machine learning techniques for classification like Naïve Bayesian and ANN can be used for document classification and retrieval [18] for mobile agents. Mobile devices offer a broad range of services over several wireless network access technologies [19]. Mobile devices have many security threats and machine learning techniques like Bayesian Networks-Nearest Neighbour, Random Forests are helpful in intrusion detection for mobile devices [19]. Machine learning techniques are useful for mobile devices in many ways. V. MOBILE INTELLIGENT SYSTEM The development of novel applications for mobile devices for mobile communication has influenced today the development of mobile information and wireless communication services. These novel technical developments give rise to new applications and related application areas. It is vital for the acceptance of new technologies, that new applications that are created add additional values and use the advantages of basic technologies or techniques and should be designed according to the needs of user. An intelligent system is a system that performs those activities that normally requires human intelligence. An intelligent system designed for mobile devices is called mobile intelligent system. The basic architecture of mobile intelligent system is shown in Fig. 4.
Figure 4. Basic architecture of mobile intelligent system
Mobile intelligent system applications include game playing, language translation, intelligent agents etc. A mobile application has following characteristics that make them popular • Mobile applications have very small startup time as most of the users use mobile devices frequently and for short duration of time.
•
They have good responsiveness. These devices acknowledge immediately as and when an action is performed. • They have intended or focused purpose. These applications have clearly defined set of tasks that are to be done with minimum clicks or taps. • They have set of style guidelines for the device running the application. Mobile Intelligent System architecture was discussed by Jawad et al. [21] that was designed with the help of sensor network. This system integrated major functions of acquisition, processing and routing of information. Additionally multi criteria approaches were used to find the best path between the Mobile Intelligent System source and Mobile Intelligent System destination. The success of any mobile intelligent system depends upon the performance of learning techniques implemented on mobile devices. For measuring performance of machine learning techniques for a mobile intelligent system there are some basic criteria like • Prediction accuracy • Sensitivity • Specificity The prediction accuracy of an algorithm determines how an algorithm predicts results for a given dataset accurately. The sensitivity of an algorithm determines how an algorithm classifies correctly the positive instances [14]. The specificity of an algorithm determines how much better is the algorithm in classifying the negative instances. For achieving good performance the prediction accuracy should be good and sensitivity should be fair. Therefore, the choice of most appropriate algorithm plays a vital role in the successful development of mobile intelligent system. VI. CONCLUSION This paper gives a brief review of machine learning techniques. It shows that each machine learning technique has some strengths and weaknesses. It gives brief study of emerging areas of use of machine learning techniques for mobile applications for the development of mobile intelligent system. It also presents some performance measures for a mobile intelligent system. It is difficult to directly state that one of the techniques performs better than the other. The performance of any machine learning technique for mobile devices depends upon the application domain and the application requirement. Mobile application areas have great potential for improving performance by using machinelearning techniques. REFERENCES [1]
[2] [3] [4]
Zanifa Omary, Fredrick Mtenzi, ÈMachine Learning approach to identifying the dataset threshold for the performance estimators in supervised learningŠ, International Journal for Infonomics (IJI), 3: 3, September 2010. T.Mitchell, Machine Learning, MIT Press, 1997. Alpaydin Ethem, Introduction to Machine Learning, MIT Press, Second Edition, 2004. L.Rokach and O.Maimon. Part C, ÈTop down induction of decision trees classifiers μ surveyŠ, Applications and Reviews, IEEE
[5] [6] [7] [8]
[9]
[10] [11]
[12]
[13]
[14]
[15] [16]
[17]
[18]
[19]
[20] [21]
Transactions on Systems, Man and Cybernetics,vol.35, pp 476-487, 2005. T. Caelli, W.F. Bischof, Machine Learning and Image Interpretation, Plenum Press, 1997. Asa Ben-Hur, Jason Weston, A user’s guide to Support Vector Machines. Richard P. Lippmann, ÈAn introduction with computing with Neural NetsŠ, ASSP IEEE 4:2, pp. 4-22, 1987. Yogesh Singh, Pradeep Kumar Bhatia, Omprakash Sangwan,ÈA Review of studies on Machine Learning TechniquesŠ, International Journal of Computer Science and Security 1:1, 2007. Stephen M Winkler, Michael Aenzeller and Stefan Wagner, ÈAdvances in applying Genetic Programming to Machine Learning, Focusing on Classification ProblemsŠ, Parallel and Distributed Processing Symposium, IEEE, IPDPS, 2006. L.Breiman, Random Forests, Kluwer Academic Publishers, Vol. 45, 2001. Sandor Dornbush, Jesse English, Tim Oates, Zary Segall, Anupam Joshi, ÈXPod: A Human Activity Aware Learning Mobile Music PlayerŠ, Proceedings of conference on Advances in Ambient Intelligence, pp. 107-122, IOS Press Amsterdam, The Netherlands, 2007. Areej Shhab,Gongde Guo, Daniel Neagu, ÈA study on applications of Machine Learning Techniques in Data MiningŠ, 22nd BNCOD Workshop, 2005. Nigel Williams,Sebastian Zander,Grenville Armitage, ÈA preliminary Performance comparison of Five Machine Learning Algorithms for practical IP traffic flow classificationŠ, ACM SIGCOMM Computer Communication Review, 36:5, 2006. S. Kotsiantis, C. Pierrakeas, P. Pintelas, ÈPredicting studentÑs performance in distance learning using machine learning techniquesŠ, Artificial Intelligence, Taylor and Francis Group, 2004. Nate Derbinsky,Georg Essl, ÈCognitive Architecture in Mobile Music InteractionsŠ, NIME’11, Oslo, Norway May-June 2011. Chih-Ming Chen, Shih-Hsun Hsu, ÈPersonalized Intelligent Mobile Learning System for Supporting Effective English LearningŠ, Educational Technology and Society, 11:3, pp. 153-180, 2008. Mohammed Alzaabi, Jawad Berri, Mohammed Jamal Zemerly, ÈWeb-based Architecture for Mobile learningŠ, International Journal for Infonomics(IJI) 3:1, March 2010. Jihoon Yang, Prashant Pai, Vasant Honavar, Les Miller, ÈMobile Intelligent Agents for Document Classification and Retrieval : A Machine Learning ApproachŠ, Citeseerx , 2011. Dimitrios Damopoulos, Sofia A. Menesidou, Georgios Kambourakis, Maria Papadaki, Nathan Clarke, Stefanos Gritzalis, ÈEvaluation of Anomaly-Based IDS for Mobile Devices Using Machine Learning ClassifiersŠ, Security Comm. Networks, 2011. Rajkamal, Mobile Computing, Oxford University Press, 2007. Jawad Oubaha, Ahmed Habbani and Mohammed Elkoutbi, ÈMobile Intellient System (MIS) and a multi-criteria in MPLS networksŠ, Internationla Journla of Next Generation Networks (IJNGN), 2:4, pp. 61-71, 2010.
