A Business Classifier to Detect. Readability Metrics on Software. Games and Their Types. Yahya M. Tashtoush, Department of Computer Science, Computer and ...
International Journal of E-Entrepreneurship and Innovation, 4(4), 47-57, October-December 2013 47
A Business Classifier to Detect Readability Metrics on Software Games and Their Types Yahya M. Tashtoush, Department of Computer Science, Computer and Information Technology Faculty, Jordan University of Science and Technology (JUST), Irbid, Jordan Derar Darwish, Department of Computer Science, Computer and Information Technology Faculty, Jordan University of Science and Technology (JUST), Irbid, Jordan Motasim Albdarneh, Department of Computer Science, Computer and Information Technology Faculty, Jordan University of Science and Technology (JUST), Irbid, Jordan Izzat M. Alsmadi, Department of Information Systems, Prince Sultan University, Riyadh, Kingdom of Saudi Arabia Khalid Alkhatib, Department of Computer Information Systems, Computer and Information Technology Faculty, Jordan University of Science and Technology (JUST), Irbid, Jordan
ABSTRACT Readability metric is considered to be one of the most important factors that may affect games business in terms of evaluating games’ quality in general and usability in particular. As games may go through many evolutions and developed by many developers, code readability can significantly impact the time and resources required to build, update or maintain such games. This paper introduces a new approach to detect readability for games built in Java or C++ for desktop and mobile environments. Based on data mining techniques, an approach for predicting the type of the game is proposed based on readability and some other software metrics or attributes. Another classifier is built to predict software readability in games applications based on several collected features. These classifiers are built using machine learning algorithms (J48 decision tree, support vector machine, SVM and Naive Bayes, NB) that are available in WEKA data mining tool. Keywords:
Data Mining, Decision Tree, Games Clasifiers, Readability Metrics, Vector Machine
DOI: 10.4018/ijeei.2013100104 Copyright © 2013, IGI Global. Copying or distributing in print or electronic forms without written permission of IGI Global is prohibited.
48 International Journal of E-Entrepreneurship and Innovation, 4(4), 47-57, October-December 2013
INTRODUCTION Software companies are interested in covering the fundamental stages of software development process, by stressing all the practical principles that are applicable during software project development stages. It’s essential for both developers and customers to get comprehensive knowledge about their project. Many research papers proposed ideas and methods to predict software cost estimation or defect prediction based on collecting software attributes of the software product, process or any other relevant parts. In addition, the primary concern for software engineers is to identify a variety of factors that may have an impact on software development process. Software readability is considered as a critical factor especially in software maintenance and evolution stages. This is since the more readable code reduces the significant effect on software maintainability. This will eventually reduce overall expenses required for future project extensions for updates, enhancements, support, etc. Readability of code is considered as one of the essential issues for software developers. Therefore, the level of code readability can clearly affect maintainability and reuse of code. Code readability has a global effect on the code development process. Since, development cost for some projects depends on the level of readability. Many researchers investigated the relation between code readability and the quality of software by proposing readability measures to predict the degree of code readability that may eventually help improve such aspects in future. In this work, a new approach is proposed to estimate the readability metric using multiple attributes such as: number of statements, number of lines of codes, number of control statements, percentage of comments and other attributes that may affect the process of code readability. The developed readability metric and other metrics are used to predict the type of game source code given a set of games assembled for the evaluation purposes.
Data mining is the process of extracting the information from datasets (Basili & Perricone 1984). Data mining techniques motivate researchers to use their scalable and efficient data analysis methods to analyze software engineering data in diverse areas such as business, entertainment, financial, scientific data...etc. (Zhang & Zhou 2004). Data mining techniques are used to discover the most valuable information in the organized data by utilizing data analysis methods under adequate computational restrictions in order to find particular patterns over the selected data. Many research projects were conducted in software engineering filed using data mining methods, where data mining classification techniques such as Support Vector Machines (SVM) (Karatzoglou & Meyer, 2006), (J48) decision tree method, Naïve Bayes (NB) and Artificial Neural Network (ANN), are used in order to come up with effective prediction and identification methods for software modules defects prediction. This work utilizes data mining techniques to analyze software metrics especially in games domain. We have built a dataset from open source games projects written in C++ and Java, important software features are extracted from them using Source monitor tool (Liu et al. 2008). SVM is a supervised learning machine algorithm that is used for binary dataset classification (Hastie et al., 2004). Support vector machine aims to build a vector that separates dataset into two categories by applying a simple linear method in high-dimensional space. Moreover, it can show the optimal value for that dataset according to the decision function f(x) = sign (hw, _(x)i + b) (Alsmadi & Magel, 2006; Campwood Software, 2013). A decision or classification tree represents a sequence of policy that can be converted easily into a set of rules. Decision tree helps in recognizing a strategy to achieve a specific target and predict software complexity (Sivaprakasam & Sangeetha, 2012). In this work, decision tree method (J48) is used as a classification method for the software
Copyright © 2013, IGI Global. Copying or distributing in print or electronic forms without written permission of IGI Global is prohibited.
International Journal of E-Entrepreneurship and Innovation, 4(4), 47-57, October-December 2013 49
dataset. The objective of J48 is to reduce the ambiguity in the dataset by partitioning the data according to specific classes. Therefore, certainty achieved if all instances in the subset of data belong to the same class. NB classifier is based on conditional probabilities by counting the frequency of multiple values and the combinations of those values in the historical data. NB has the ability of predicting the probability of a specific event based on the probability of another already occurred event. In addition to its useful characteristics like the simplicity in training, since it only requires assigning a specific instance of each class related to that instance independently. NB assumes that all the attributes are independent of each other. Therefore, this classifier assumes the independency of assumptions. Artificial Neural Network is considered as one of the adaptive systems that have the ability of changing its behavior according to any change in the interacting parts. It is used to find the relationships between inputs and outputs, consequently, produces specific patterns in data. The rest of the article is structured as follows: literature review is discussed in the second section; while the third section presents the research methodology and the results; the conclusion is shown in the fourth section.
LITERATURE REVIEW We will introduce several examples of papers in the software metrics field who utilized data mining techniques particularly for classification and prediction. Major areas include: cost estimation or prediction, quality and defect prediction. Connor et al. (2011) utilized data mining techniques to identify the most efficient and useful metrics that can be used to predict the success or failure of software products. The paper described methods of extracting number of source code metrics from Jazz repository and investigated different slices of code in order
to extract useful metrics in an attempt to build an operational instance of software products. Najadat et al. (2012) proposed a new approach in order to achieve a realistic prediction of the cost and the complexity of future software projects. The paper utilized data mining classification techniques to study the association between different software attributes and software cost estimation. Raymond and Westley (2008) proposed a new readability metric. This metric has the ability to predict readability judgments efficiently in comparison with human annotators. In addition, this new readability metric exhibits a significant level of association with more predictable metrics of software quality; such as, defects, code churn, and self-reported stability. Yu and Zhou (2010) provided a survey on software metrics complexity. The authors studied some efficient software complexity metrics such as: LOC, Halstead and McCabe metrics. This survey concludes the advantages and disadvantages for each metric. Orme et al. (2006) proposed a set of coupling metrics. Those metrics are commonly used to produce a high quality software product by measuring the data passed across module interfaces to verify couplings between modules in a given system. The new proposed metrics are: The number of External Classes (NEC), Reference to External Classes (REC) and Referenced Includes (RI). Jyothi and Babu (2011) proposed a new readability measure. The paper provided an effective platform for performing predictable readability experiments. In this work, the authors determined the most effective features in order to predict the code readability. Then, they proposed a new model that has the ability to analyze other codes. Liu et al. (2008) proposed Bayesian Network (BN) model in order to provide an effective relation analysis between a set of metrics and software quality in the software development process. Further, the authors explored the
Copyright © 2013, IGI Global. Copying or distributing in print or electronic forms without written permission of IGI Global is prohibited.
50 International Journal of E-Entrepreneurship and Innovation, 4(4), 47-57, October-December 2013
dependency between the selected metrics to improve the development process. This work applied BN model to solve the problem of classifying software modules into defect or defect-free modules. Alsmadi and Magel (2006) developed software metrics tool in order to extract some software metrics from a collection of source codes written in C/C++, C# and Java. The tool focused on some metrics related to complexity such as: max. nesting, MCDC, cyclomatic complexity, SLOC, etc. Major goal was to judge the validity and value of such metrics on software codes and their ability as indicators for software quality and complexity. Sivaprakasam and Sangeetha (2012) investigated the relation between the software quality in general and software code readability in particular. The authors developed a readability measure to predict software code readability. They concluded that, the proposed readability measure is better than human judgment in predicting readability judgment. In addition to that, the proposed measure provided an excellent platform for assessing readability. Basili and Perricone (1984) analyzed the associations between the distribution of errors and the frequency of those errors during software development process. The authors concentrate on the maintenance of the software already developed. In addition to studying other environmental factors such as: the complexity of the software, the ability to reuse the existing design, and the existing experience of the software developers. The authors concluded that, there are multiple factors that may affect the software development. Aggarwal et al. (2002) proposed an integrated measure of software maintainability. Software maintainability was measured based on three aspects; Readability of Source Code (RSC), Documentation Quality (DOQ), and Understandability of Software (UOS). The authors found that the readability and understandability of both, source code and documentation should
be considered to measure the maintainability, and are both critical to the project maintainability. Collar and Valerdi (2006) proposed the code’s propositional structure to be crucial for programming code comprehension, through which the reader constructs an understanding of the semantic relations between propositional elements of the code. Comprehension at the text base level relies on the effectiveness of the propositional organization in supporting the reader’s ability to construct meanings from the propositions in the text of the programming code. Raymond and Westley (2010) suggested that the readability of source code can be affected by: the identifier names, comments, and overall appearance. The authors presented a technique for modeling code readability based on the judgments of human annotators. The model correlates strongly with human annotators and also with external (widely available) notions of software quality, such as software changes and defect detectors. A Web-based tool for annotating the readability of code snippets was used. Chen et al. (2010) presented (C-Decompiler) a refined decompiler to generate C code with high readability, where four evaluation criteria were used in the experiments. Those are: 1. Functional analysis, which evaluates the quality of identifying functions; 2. Variable expansion rate (expansion %) which reflects how many additional variables are used in the decompiled code, compared with the original code; 3. (Reduction percentage %) which depicts the reduction percentage of code lines when the program is represented by a high level language such as: C/C++ instead of assembly code; 4. (Cyclomatic Complexity) CC, which is a software metric that directly measures the number of linearly independent paths through a program’s source code.
Copyright © 2013, IGI Global. Copying or distributing in print or electronic forms without written permission of IGI Global is prohibited.
International Journal of E-Entrepreneurship and Innovation, 4(4), 47-57, October-December 2013 51
The C-Decompiler can generate highly readable source code. It can also extract the switch case structure, identify the arrays and increase the readability of the decompiled source code greatly. Even more it adds comments to the decompiled code to increase readability. Namani and J (2012) proposed a new readability metric based on a set of rules collected from software engineers. Some existing readability metrics such as: Automated Readability Index (ARI) and Gunning’s Fox Index (FOG) were used. The proposed readability metric uses some rules obtained from senior software engineers including: Lines of Code (LOC), Line Length (LL), presence of comment lines in the program (NOCL), No. of Blank Lines (NOBL), breaking the line after a semicolon (NLAS), blank space after directive statements (BSAD) and No. of Methods (NOM). Paper conducted an assessment study to validate the value of proposed readability metrics. Wang et al. (2011) proposed that blank lines in source code can help in making code more readable. So, a tool (SEGMENT) is presented as a solution to the automatic blank line insertion problem, using program structure and naming information to identify “meaningful blocks” and consecutive statements that logically implement a high level action. SEGMENT takes as input a Java method, and outputs a segmented version that separates meaningful blocks by blank line insertions. Not only the resulting segmentation can help in readability, it can provide hints for where to place internal comments. Sivaprakasam and Sangeetha (2012) proposed that analysis of different software development activities shows that software readability has a global effect on Software development cost, and is independent of software size (i.e., KSLOC). So, based on associations between a simple set of local code features and human notions of readability, an automated readability measure that uses the total number of words, syllables and sentences as the basic counts is presented, and shows that it can be 80% effective, and better than a human on average, at predicting readability judgments.
READABILITY AND GAME CLASS PREDICTION We conducted a study to collect readability metrics from several software games and then perform statistical analysis using WEKA tool for classification and prediction purposes. This research follows the following steps: 1. Collect a dataset of several game programs for desktop and mobile applications written in: C++, and Java from open source websites; 2. The source monitor tool is used in order to extract several software metrics related to object orientation, complexity, size and readability; Metrics are normalized between 0 and 10 values according to the following equation: Xn = (((Xi-min)/max-min) *10) where: Xn: normalized x Xi is the original value of the metric. Min is the minimum value of metric and Max is the maximum value of metric: 4. A weight is assigned for each metric according to the significance or impact of readability (Tashtoush et al, 2013) and the comments weight is assigned to be positive because the increase of its value increases the software readability while the other metrics weights are set to be negative because they have a positive correlation with software readability; 5. The readability is calculated according to the following equation: Readability = Lin * -0.06 + stm * -0.06 +bch * -0.93 + com * 1.66 + cd * -0.06 + mc * -0.06 + asm * -0.06 + ad * -0.75 + ac * -0.93 + fun * -0.06
Copyright © 2013, IGI Global. Copying or distributing in print or electronic forms without written permission of IGI Global is prohibited.
52 International Journal of E-Entrepreneurship and Innovation, 4(4), 47-57, October-December 2013
where: • • • • • • • • • •
Lin is the normalized value for lines; Stm is the normalized value for statements; Bch is the normalized value for branches; Com is the normalized value for comments; Cd is the normalized value for class definitions; Mc is the normalized value for methods per class; AvgStmM is the normalized value for average statements per method; Ad is the normalized value for average depth; Ac is the normalized value for average complexity; and Fun is the normalized value for functions.
According to this equation the highest value of readability for the games which are used in this research is 9.0 and the lowest value is -16.28. The quarter’s method is used to classify the games into three classes based on readability: highly readable, med. readable and low read-
able. The quarter’s method is applied on the games readability values. Quarter one value is (-7.0) and the quarter three values is (-2.45). According to these quarters the following assumptions were made: a. Readability values which are less than -7.0 are set to be low readable; b. Readability values which are between -7.0 and -2.45 are set to be med readable; c. Readability values which are more than -2.45 are set to be high readable. The data mining results for these class labels are shown in Tables 1, 2 and 3: 6. Decision Tree, Naive-Base, and Support Vector machine are applied to the game data set in order to extract patterns and rules. Tables 1, 2 and 3 show the results of Decision Tree, Naive-Base, and Support Vector
Table 1. J48 decision tree classifier (low readable, med readable and high readable) Training Dataset
Correct
Percentage
M
L
H
Mean Error
66% training
33
73.33
0.77
0.82
0.89
0.17
2 folds
92
69.70
0.84
0.86
0.89
0.22
5 folds
96
72.73
0.74
0.83
0.77
0.2
10 folds
100
75.76
0.81
0.89
0.89
0.17
Table 2. Naive-base classifier (low readable, med readable and high readable) M
L
H
66% training
Training Dataset
28
Correct
62.22
Percentage
0.71
0.82
0.82
0.27
Mean Error
2 folds
85
64.39
0.75
0.89
0.82
0.25
5 folds
96
72.73
0.72
0.88
0.79
0.22
10 folds
91
68.94
0.80
0.71
0.88
0.24
Copyright © 2013, IGI Global. Copying or distributing in print or electronic forms without written permission of IGI Global is prohibited.
International Journal of E-Entrepreneurship and Innovation, 4(4), 47-57, October-December 2013 53
Table 3. Support vector machine classifier (low readable, med readable and high readable) Training Dataset
Correct
Percentage
M
L
H
Mean Error
66% training
25
55.56
0.58
0.91
0.72
0.32
2 folds
90
68.18
0.68
0.91
0.83
0.29
5 folds
91
68.94
0.68
0.93
0.82
0.29
10 folds
84
63.64
0.68
0.93
0.82
0.30
machine respectively, when the class label is “Low, Med, and High”. Best inJ48 case is in 10-folds with highest correctly classified instances and good distribution of instances. In NB case, 5-folds case is shown in Table 2 to be a better option.
In case of SVM, 2 and 5 folds are the best options in terms of correctly classified instances. By large, J48 is the best classifier in terms of the results in all cases. Table 4, 5, 6 show the results of Decision Tree, Naive-Base, and Support Vector machine
Table 4. J48 classifier (C++ game, Java PC game and Java mobile game) Training Dataset
Correct
Percentage
JP
C++
JM
Mean Error
66% training
38
84.44
0.87
1
0.89
0.13
2 folds
95
71.97
0.81
0.95
0.79
0.20
5 folds
108
81.82
0.82
1
0.84
0.14
10 folds
108
81.82
0.81
1
0.84
0.14
Table 5. Naive-base classifier (C++ game, Java PC game and Java mobile game) Training Dataset
Correct
Percentage
JP
C++
JM
Mean Error
66% training
37
82.22
0.91
1
0.89
0.13
2 folds method
104
78.79
0.86
0.99
0.86
0.15
5 folds method
102
77.27
0.85
0.99
0.84
0.16
10 folds method
98
74.24
0.85
0.99
0.84
0.17
Table 6. Support vector machine classifier (C++ game, Java PC game and Java mobile game) Training Dataset
Correct
Percentage
JP
C++
JM
Mean Error
66% training
37
82.22
0.81
1
0.85
0.27
2 folds method
95
71.97
0.75
0.95
0.8
0.3
5 folds method
101
76.52
0.80
0.99
0.81
0.28
10folds method
100
75.76
0.78
0.99
0.80
0.29
Copyright © 2013, IGI Global. Copying or distributing in print or electronic forms without written permission of IGI Global is prohibited.
54 International Journal of E-Entrepreneurship and Innovation, 4(4), 47-57, October-December 2013
respectively, when the class label is “Type of software game based on language and platform”. Percentage in correctly predicting the type of programming language or environment in J48 is less than that to predict in the first case (LMH). However, predicting in the case of C++ is the highest in all cases. Table 5 shows also that predicting instances of C++ was higher in terms of prediction accuracy in comparison with other languages or environments. Figure 1 shows the J48 Decision Tree (low readable, med readable and high readable). Also, it shows that readability is high for two cases: •
When: Comm.
