American Journal of Scientific Research ISSN 1450-223X Issue 12 (2010), pp.06-15 © EuroJournals Publishing, Inc. 2010 http://www.eurojournals.com/ajsr.htm
Predict and Approximate Software Quality with Bayesian Networks and Quality Factors Homayun Motameni Department of Computer Sciences, Islamic Azad University, Sari Branch, Sari, Iran E-mail:
[email protected] Hadi Kamfar Department of Computer Sciences, Islamic Azad University, Zanjan Branch, Iran Alireza Khanteimori Department of Computer Sciences, Amir Kabir University of Technology, Tehran, Iran Abstract Software quality is one of important field in software engineering that related to software satisfaction. Some models and methods have made to calculating quality. Almost all of these models use quality factors or metrics, but calculate them is another problem because we can not calculate them exactly or can not determine some of them in some software project and usually our data about quality factors and metrics are incomplete or uncertain. Also these models can not predict software quality before calculate all of quality metrics and factors. Bayesian Networks have become a popular tool for modeling many kinds of statistical problems over the last decade. In this paper we proposed a model for software quality with BNs and ISO9126 quality model. ISO9126 is one of best and complete quality model. This new model doesn't have last quality model's problems and can predict software quality with incomplete and uncertain data. Also by this model we can reduce time and cost of calculating software quality.
Keywords: Software Quality Model, Bayesian Networks, ISO9126
Introduction Software quality is "Conformance to explicitly defined functional and implicit characteristics that are expected of professionally developed software". Software is now used in many demanding applications and software defects have caused serious damage and even physical harm. This software can be software to fly airplanes or to drive automobiles, software to control air traffic, run factories or operate power plants. People have been killed by defective software. The aim of software quality engineering is to investigate the relationships among in-process metrics, project characteristics, and end-product quality. Therefore software quality is very important that can improve software satisfaction. By determine software quality the manager or stakeholder can decide to using software or improving the aspects and factors that have low quality. We don't use or sell software with low quality because it makes much wasted time and cost for both producer and consumer. Software quality has more effect when we predict or determine quality among the project because we can reduce cost and time with reviewing or ending the software project.
Predict and Approximate Software Quality with Bayesian Networks and Quality Factors
7
There are many efforts to determining Software quality and have made some models and methods for calculating it like McCall Model, Boehm Model, FURPS Model, ISO9126 Model and Dromey Model. Almost all of these models use quality factors or metrics, but calculate these quality factors and metrics are another problem because we can not calculate them exactly or can not determine some of them in some software project and usually our data about quality factors and metrics are incomplete or uncertain. Also these models can not predict software quality before calculate all of quality metrics and factors. In this paper we proposed a model for software quality with BNs and ISO9126 quality model. ISO9126 is one of best and complete quality model. In fact we model ISO9126 with BN. By this work we can use probability of quality factors and metrics. This new model doesn't have last quality model's problems and can predict software quality with incomplete and uncertain data. Also by this model we can reduce time and cost of calculating software quality. In chapter 2 we introduce Bayesian Network with an example. In chapter 3 we discuss about software quality model and ISO9126 quality model. In chapter 4 we proposed SWQBN (Software Quality with Bayesian Network).
Introduction to Bayesian Networks Bayesian Networks (BN) derive from the convergence of statistical methods that permit one to go from information (data) to knowledge (probability laws, relationship between variables,…) with Artificial Intelligence (AI) that permits computers to deal with knowledge (not only information). The main purpose of BN is to integrate uncertainty into expert systems (SIMON, WEBER & LEVRAT, 2007). An expert, most of the time, has only approximate knowledge of the system, that he/she formulates in terms like: A has an influence on B; if B is observed, there is a great chance that C occurs and so on. It becomes obvious that this tool is well-suited to deal with the problem concerned, as expert knowledge can be needed to estimate, for instance, the reliability of a component or how components of a subsystem interact in terms of reliability (Khaddaj & Horgan, 2004). Bayesian Networks are an increasingly popular formalism for reasoning and decision-making in problems that involve uncertainty and probabilistic reasoning. A Bayesian Network is an acyclic directed graph that represents dependencies between variables and their associated probabilities (Zalewski, Kornecki & Pfister, 2006). Nodes represent variables and arcs represent relationships between variables. Nodes of a Bayesian network are usually drawn as circles or ovals. The semantics of a link can be described in the following way: there is a link from X to Y, if X has a direct influence on Y. Each node has a conditional probability table (CPT) associated that quantifies the effects that the parents have on the node. A Bayesian network also represents the quantitative relationships among the modeled variables (Dahl & Gran, 2000). Numerically, it represents the joint probability distribution among them. This distribution is described efficiently by exploring the probabilistic independence among the modeled variables. Each node is described by a probability distribution conditional on its direct predecessors. Nodes with no predecessors are described by prior probability distributions (Kornecki & Zalewski, 2006). In figure 1, there is a burglar alarm that can be fired by a burglar or by mistake due to an earthquake. The probability of happening a burglary (P(B)) is 0.001, and the probability of happening an earthquake (P(E)) is 0.002. The probability of the alarm going off depends on the nodes Burglary and Earthquake, which is depicted in the table near the alarm node (Neil & Fenton, 1996). Supposing that there are two persons that can call when the alarm goes off, John and Mary, the probability of any of them calling because the alarm went off is also represented in the tables for nodes John Calls and Mary Calls.
8
Homayun Motameni, Hadi Kamfar and Alireza Khanteimori Figure 1: An example of a Bayesian network
Inference in a BN is the computation of the posterior probability distribution for the set of query variables. Given a set of evidence variables for which the exact value is known, an inference algorithm is used to propagate these values through the BN, according to Bayes rule. There are several inference algorithms, but basically they fall in two categories: exact inference and approximate inference. Exact inference algorithms perform the exact computation of the probability distributions. Belief propagation in trees is linear, while belief propagation in multiply connected graphs has been shown to be NP-hard. Approximate inference algorithms try to deal with this difficulty in a more tractable way, trying to provide an approximation to the probability distribution for query variables (Kaner & Bond, 2004).
Software Quality Model and Metrics Quality isn’t about zero defects or measurable improvements in defect rates, nor is it about meeting documented requirements. Its no more and no less that satisfying customer needs (whether or not the needs are properly documented). The ultimate measure of design quality is fitness to purpose. This means that to measure software quality we have to understand what the purposes are for which it is intended. Note this also means that quality is not a measure of software in isolation; it is a measure of the relationship between software and its application domain. This means that an assessment of its quality depends on the context, and if you change the context, the assessment of quality may change – software that’s good for one purpose may not be so good for another purpose. Measurement of quality generally starts by identifying a set of critical quality factors, and then refining these down to measurable attributes of the software (Fenton & Pfleeger, 1997). There is no agreed way of doing this. Different quality assessors disagree about what various measurable attributes of software actually tell you about its quality. No quantitative approach to software quality assurance can be complete without a measurable definition of software product quality (Mustafa & Khan, 2005). We can never know whether the quality is satisfactory or improving if we cannot measure it. Moreover, we need quality measures if we are to improve our resource estimation and productivity measurement. In the case of resource estimation, higher quality requirements may lead to greater resources. In the case of productivity measurement, speed of production is meaningless without an accompanying assessment of product quality. Thus work on resource estimation and productivity assessment inspired software engineers to develop models of quality which took into account various views of software quality. For example, Boehm’s advanced COCOMO cost estimation model is tied to a quality model. Similarly, the McCall quality model is related to productivity (Firesmith, 2003). Quality Model is the set of characteristics and the relationships between them which provide the basis for specifying quality requirements and evaluating quality (Gran, 2002). some type of quality
Predict and Approximate Software Quality with Bayesian Networks and Quality Factors
9
model are McCall Model, Boehm Model, FURPS Model, ISO9126 Model (Korb & Nicholson, 2004) and Dromey Model. These models are usually constructed in a tree-like fashion. The upper branches hold important high-level quality factors of software products, such as reliability and usability that we would like to quantify. Each quality factor is composed of lower-level criteria, such as modularity and data commonality (Ankush & Ashraf, 2007). The criteria are easier to understand and measure than the factors; thus, actual measures (metrics) are proposed for the criteria. The tree describes the pertinent relationships between factors and their dependent criteria, so we can measure the factors in terms of the dependent criteria measures. Quality models are expected to be used at the specification and design phase of software quality assurance. The idea is that targets for the high level factors are set, while assessments of the likeliness of meeting these targets is based on measuring the lower level criteria during design. We use The ISO9126 Quality Model for predict software quality in this paper that is more popular than other and canonical.
ISO9126 Model Based on previous attempts for defining software quality, the International Organization for Standardization (ISO) and the International Electro technical Commission (IEC) have published the multipart standard ISO/IEC 9126 which defines a software product quality model, quality characteristics, and related metrics. These constituents can be used to both evaluate and set goals for the quality of a software product (ISO/IEC 9126, 2001). Part 1 of ISO/IEC 9126 contains a two-part quality model: one part of the quality model is applicable for modeling the internal and external quality of a software product, whereas the other part is intended to model the quality in use of a software product. These different quality models are needed to be able to model the quality of a software product at different stages of the software lifecycle (Fenton & Pfleeger,1997).Typically, internal quality is obtained by reviews of specification documents, checking models, or by static analysis of source code. External quality refers to properties of software interacting with its environment. In contrast, quality in use refers to the quality perceived by an end user who executes a software product in a specific context. These product qualities at the different stages of development are not completely independent, but influence each other. Thus, internal metrics may be used to predict the quality of the final product – also in early development stages. For modeling internal quality and external quality, ISO/IEC 9126 defines the same model. This generic quality model can then be instantiated as a model for internal quality or for external quality by using different sets of metrics. The model itself is based on the six characteristics functionality, reliability, usability, efficiency, maintainability, and portability. Figure 2 shows the ISO9126 Quality Model's factors and metrics. Figure 2: The ISO9126 Quality Model's factors and metrics Quality Functionalit
Reliability
Security
Maturity
Accuracy Interoperability
Recoverability Fault-tolerance
Usability
Efficiency
Maintainabilit y
Learnability
Time behavior
Analyzability
Resource Utilization
Stability
Installability
Changeability
Co-existence
Compliance
Adaptability
Understandability
Operability Attractiveness
Suitability
Testability
Portability
Replaceability
10
Homayun Motameni, Hadi Kamfar and Alireza Khanteimori
As shown in Figure 2, each of these characteristics has further sub-characteristics. ISO9126 Quality Model is an internationally recognized model and being followed by majority of the organization around the world. During the Requirement Analysis phase, categorize the requirements as per the characteristic and the sub-characteristic in the ISO9126 Model Quality Factor is obtained by using a series of formulas (Mustafa & Khan, 2005). The definition of ISO9126 Quality Model's characteristics are: Functionality: A set of attributes that bear on the existence of a set of functions and their specified properties. The functions are those that satisfy stated or implied needs. Reliability: A set of attributes that bear on the capability of software to maintain its level of performance under stated conditions for a stated period of time. Usability: A set of attributes that bear on the effort needed for use, and on the individual assessment of such use, by a stated or implied set of users. Efficiency: A set of attributes that bear on the relationship between the level of performance of he software and the amount of resources used, under stated conditions. Maintainability: A set of attributes that bear on the effort needed to make specified modifications. Portability: A set of attributes that bear on the ability of software to be transferred from one environment to another. The model of quality in use is based on the characteristics effectiveness, productivity, safety, and satisfaction and does not elaborate on further sub-characteristics. In the further parts of ISO/IEC 9126, metrics are defined which are intended to be used to measure the attributes of the (sub)characteristics defined in Part 1: The provided metrics are quite abstract which makes them applicable to various kinds of software products, but they cannot be applied without further refinement. Table 1 show defining the sub-characteristics in the ISO9126 Quality Model. Table 1:
Defining the sub-characteristics in the ISO9126 Quality Model
Characteristic name
Sub-Characteristic name Suitability Accuracy
Functionality
Interoperability Security Maturity
Reliability
Fault-tolerance Recoverability Understandability
Usability
learn ability Operability Attractiveness Time behavior
Efficiency Resource Utilization Compliance Maintainability
Analyzability
Definition The capability of the software to provide an appropriate set of functions for specified tasks and user objectives The capability of the software to provide the right or agreed results The capability of the software to interact with one or more specified systems The capability of the software to protect information and data from unauthorized access The capability of the software product to avoid failure as a result of faults in the software The capability of the software to maintain a specified level of performance in cases of software faults The capability of the software to re-establish a specified level of performance and recover the data in case of a failure The capability of the software product to be understand, learned, used and appear attractive to the user The capability of the software to provide appropriate response while performing its function The capability of the software to enable the user to operate and control The capability of the software to be attractive to the user The capability of the software to provide appropriate response while performing its function The capability of the software product to use appropriate amounts and types of resources The capability of the software to follow standards of efficiency The capability of the software to be diagnosed for causes of failures in the software, or for the parts to be modified to bee identified
Predict and Approximate Software Quality with Bayesian Networks and Quality Factors Changeability Stability Testability
Portability
Adaptability Install ability Co-existence Replace ability
11
The capability of the software to enable a specified modification to be implemented The capability of the software to avoid unexpected effects from changes made in the software The capability of the software product to enable modified software to be validated The capability of the software to be adapted for different environments The capability of the software to be installed in a specified environment The capability of the software to co-exist with other independent software The capability of the software to be used in place of another software for the same purpose
Data Collection Measurement is the key factor in any software quality assurance program. But effective use of measurement is dependent on careful data collection, which is difficult, especially when data must be collected across a diverse set of projects. Thus, data collection is becoming a discipline in itself, where specialists work to ensure that measures are defined unambiguously, that collection is consistent and complete, and that data integrity is not at risk. But it is acknowledged that metrics data collection must be planned and executed in a careful and sensitive manner. Data collection is also essential for scientific investigation of relationships and trends. Good experiments, surveys and case studies require carefully-planned data collection, as well as thorough analysis and reporting of the results.
SWQBN: Modeling software quality with BN We must to obtain or compute every software quality factor to calculate software quality with ISO9126 or any other quality model. But it is impossible to compute all of quality factor in different software for some reasons: • Some of quality factor (in different software) can't be computed • Sometimes cost of compute all of quality factors are very high • We need long time for compute all of quality factors • In some cases we need to a lot of people (user, quality manager, member of quality assurance's team) to compute quality factors Because of these reasons we can't compute all of software quality factors for all software. Also we usually have uncertainty in quality factors and metrics. Therefore it is better to find a way to compute software quality with only some of quality factors. Our new model needs data of quality factors and metrics from different software and different company. We can use Bayesian Networks to compute software quality. By this way we don't need to know all of quality factors or metrics. Bayesian network graph for software quality has three levels. Quality metrics are in first level and their arcs go to quality factors in second level. Quality factor's arcs go to third level in quality node. Each node (quality, quality factor, quality metric) has five states: Very High, High, Medium, Low and Very Low. The data (probability) for first level (quality metric) is different from one company or organization to another. For example security in a company is more important than another one. Therefore the quality experts of each company can fill them in CPT according to their quality metrics, knowledge, and experience of previous software in Thier Company. Using the previous knowledge of the similar project and software is one of the most important factors to filling CPT. Bayesian network graph is shown in figure 3.
12
Homayun Motameni, Hadi Kamfar and Alireza Khanteimori Figure 3: Modeling software quality with Bayesian Networks
Because of large amount of data for CPT in second and third level and to filling them faster it is almost impossible to fill CPT by hand. We can use BN programming software like BNT (in Matlab) or BNJ (in JAVA). We can use this way to fill CPT of node in second and third level. We generate different state of CPT by programming algorithm. For example there are 54 = 625 states (row in CPT) for a quality factor by 4 quality metrics. By table2 we can calculate the amount of each CPT's rows. Table 2:
Degree of quality state State
degree
Very High High Medium Low Very Low
5 4 3 2 1
Also we can suppose priority for each quality metrics and their affect on quality factor. But quality experts don't agree on a unique model for it and most of them agree on this model (model with equal priority of quality metrics). After that we modulate these amounts for each row to domain 0 to 1. For this task we can use this formula: p=d/(5*n). In this formula 'd' is degree of each row and 'n' is the number of quality metrics that go to quality factor. And 'p' is calculated percent of each row. In some BN's software 'p' is in range of 0 to 100. We don’t multiply 100 to calculate 'p' in them and formula p = ( d /( 5 * n )) * 100 ⇒ p = 20 * d / n become . For example 'Reliability' and its three metrics (Maturity, fault-tolerance, Recoverability) are in table 3 (part of CPT). Table 3:
An example of a row of CPT (Reliability factors) Maturity
Very High
Fault-tolerance Low
Recoverability Medium
Predict and Approximate Software Quality with Bayesian Networks and Quality Factors
13
d = 5 + 2 + 3 = 10 ⇒ p = 20 * d / n = 20 * 10 / 3 = 66.67 It means we have Reliability in this example by probability of 66.67 percent. We can see the code of filling CPT for a factor with three metrics like 'Reliability'. Figure 4 shows Code of filling CPT for a factor with three metrics. Figure 4: Code of filling CPT for a factor with three metrics
For i = 1 to 5 For j = 1 to 5 For k = 1 to 5 d = i + j + k; p = d / (5 * n); CPT( i , j , k) = p; End For End For End For
Set to CPT
This algorithm will implement for all of quality factors and software quality. Some of nested 'For' is equal to some of metrics that effect on each quality factor. Therefore we need 7 nested loops for filling CPT of Software quality. We can use one set of nested 'For' for the factors with equal amount of metric. Therefore we can use one set of four nested 'For' for 'Functionality', 'Portability' and 'Usability'. Figure 5 shows Code of filling CPT for three factors each of them with four metrics. This code constructs CPTs faster. Figure 4: Code of filling CPT for a factor with three metrics
For i = 1 to 5 For j = 1 to 5 For k = 1 to 5 For l = 1 to 5 d = i + j + k + l; p = d / (5 * n); CPT( i , j , k , l) = p; CPT( i , j , k , l) = p; CPT( i , j , k , l) = p; End For End For End For End For
Set to 'Functionality' CPT Set to 'Usability' CPT Set to 'Portability' CPT
Profits of SWQBN Using SWQBN model for predicts software quality has some benefit that different it from other models. We can predict software quality with a few evidences. In other models for compute quality we had to gain all of quality factors and metrics. This trait reduces cost and time wasting because if we determine software quality earlier, we can make a suitable decision to increase software quality earlier. Therefore we can produce software earlier, cheaper and by higher quality. In other models we had to
14
Homayun Motameni, Hadi Kamfar and Alireza Khanteimori
compute quality by some math relation and asking question from and filling questionnaire by quality experts, users, manager, stakeholders, members of software quality assurance team and members of software development team. Therefore SWQBN decreases time and cost of computing software quality. Also SWQBN need to much fewer people to calculating quality than other models. By SWQBN we can predict software quality before end of software project (in construction time). With this ability we can avoid project from failing. Also by these reason we can decrease cost of quality assurance. One of best advantage of SWQBN is that we can view affect of one or more quality factors or metrics on software quality. This ability helps us to make a suitable decision among the project to increasing software quality.
Conclusion Quality models are methods to calculate software quality. Usually quality models are made of by quality factors and quality metrics. ISO9126 is one of the newer, standard and more popular of quality models. This model contains six quality factors. These factors are determined by some quality metrics. But ISO9126 and other quality models have some limit to compute software quality. The main problem of them is limitation to calculate all of quality factor in different software. We choose ISO9126 model because it is more efficient than other quality models. We model software quality with BN and ISO9126. This model reduces cost and time wasting that is high in other quality model (Bertoa, Troya & Vallecillo, 2006). We had to compute all of quality factors and metrics exactly but it is not possible to compute all of them all the time and in each software project. But this proposed model can work with incomplete and uncertain data. Also by this model we don't have to compute all of the quality factors and metrics. We can compute software quality by having some of the quality factors and metrics. Also SWQBN decreases time and cost of quality assurance and computing software quality. SWQBN can predict quality among the project therefore we can increase software quality by make a suitable decision. For this reason SWQBN decrease time and cost of developing software. One of the most important advantages of using SWQBN is improvement of software quality and customer satisfaction (Yao & Yang, 2007). For further work we can implement software to predict quality with our new model. This software will get default (primary) probability of quality metrics in user's software company from user. To predict software quality for each software project, the calculated qualities metrics will enter to quality predict software. Then software calculates probability of other quality factors, metrics and software quality. Also we can use different effect of quality factors and metrics for software quality. Project managers, quality experts, stakeholders, software development teams and quality assurance teams will be the user of this software.
References [1] [2] [3]
[4]
SIMON, C.H., WEBER, P.H., & LEVRAT, E. (2007). Bayesian Networks and Evidence Theory to Model Complex Systems Reliability, Journal of Computers, 2(1), 1-12. Khaddaj, S., & Horgan, G. (2004). The Evaluation of Software Quality Factors in Very Large Information Systems, Electronic Journal of Information Systems Evaluation, 7(1), 43-48. Zalewski, J., Kornecki, A.J., Pfister, H.L. (2006). Numerical Assessment of Software Development Tools in RealTime Safety Critical Systems Using Bayesian Belief Networks, Proceedings of the International Multiconference on Computer Science and Information Technology, pp. 433 – 442. Dahl, G., & Gran, B.A. (2000). The Use of Bayesian Belief Nets in Safety Assessment of Software Based Systems, International Journal of General Systems, 29(2), 205-229.
Predict and Approximate Software Quality with Bayesian Networks and Quality Factors [5]
[6]
[7] [8] [9] [10] [11]
[12] [13] [14] [15] [16]
15
Kornecki, A., & Zalewski, J. (2006). The Qualification of Software Development Tools From the DO-178B Certification Perspective. Crosstalk, The Journal of Defense Software Engineering, 19(4), 19-23. Neil, M., & Fenton, N. (1996). Predicting Software Quality using Bayesian Belief Networks, Proceedings of 21st Annual Software Engineering Workshop NASA/Goddard Space Flight Centre, December 4-5, pp.1 – 8. Kaner, C., & Bond, W.P. (2004). Software Engineering Metrics: What Do They Measure and How Do We Know? 10th international software metrics symposium, metrics, pp.1 – 12. Fenton, N.E., & Pfleeger, S. L. (1997). Software Metrics: A Rigorous and Practical Approach,2nd Edition Revised ed. Boston: PWS Publishing. Mustafa, K., & Khan, R.A. (2005). Quality Metric Development Framework (MDF), Journal of Computer Science, 1 (3), 437-444. Firesmith, D. (2003). Using Quality Models to Engineer Quality Requirements, Journal of Object Technology, 2(5), 67-75. Gran, B.A. (2002). The use of Bayesian Belief Networks for combining disparate sources of information in the safety assessment of software based systems, Dr.Ing. Thesis, Norwegian University of Science and Technology, pp.1 – 100. Korb, K.B., & Nicholson, A.E. (2004). Bayesian Artificial Intelligence, Chapman & Hall/CRC. Ankush, M., & Ashraf, K. (2007). Bayesian Network Technologies: Applications and Graphical Models, IGI PuBlIshING, New York. ISO/IEC 9126. (2001). Software Engineering-Product Quality. Parts 1 to 4. Bertoa, M.F., Troya, J.M., & Vallecillo, A. (2006). Measuring the usability of software components, Journal of Systems and Software ,79(3), 427-439. Yao, L., & Yang, B. (2007).An Approach to Early Prediction of Software Quality, Journal of Electronic Science and Technology of China, 5(1), 1-6.
