Dynamically Weighted Combination Model for Describing Inconsistent ...

Indian Journal of Science and Technology, Vol 9(35), DOI: 10.17485/ijst/2016/v9i35/90211, September 2016

ISSN (Print) : 0974-6846 ISSN (Online) : 0974-5645

Dynamically Weighted Combination Model for Describing Inconsistent Failure Data of Software Projects Subburaj Ramasamy and C. A. S. Deiva Preetha* School of Computing, SRM University, Kattankulathur, Chennai - 603203, Tamil Nadu, India; [email protected],[email protected]

Abstract Background/Objective: The Software Reliability Engineering discipline is witnessing continued research in the field of Software Reliability Growth Models (SRGMs), in order to develop suitable models so as to match with the phenomenal developments in the software sector. The objective of this paper is to propose a combined dynamically weighed model for reliability analysis. Methods: In general, the ability of a model to describe a dataset adequately is assessed by Goodness of Fit (GoF) measures that a model achieves. We used one of the difficult data sets for evaluation of the GoF performance and the non-linear regression was carried out using Curve fitting application of the MATLABTM software tool. The coefficients of determination R-Square (R2), Sum of Squared Error (SSE), Root Mean Square Error (RMSE) are the different GoF metrics used. Findings: The major challenge for the software projects today, is to measure, analyze and control the level of quality of the delivered software. Undesirably long testing cycles adversely influence time to market and hence software development organizations are very keen on having a good control on the testing cycle. In the last few years many reliability models have been identified and recommended. However, no single model can be considered suitable for all situations. Some models assume the growth of mean value function to be exponential, while other models assume it to follow S-type growth. This poses a challenge to the reliability modeling. The dynamically weighted combination model that we propose in this paper helps to address both the phenomena. The superposition and time transformation property of Non-Homogenous Poisson Process (NHPP) model was considered to derive the proposed model. Application: Using SRGM helps to improve the reliability performance. We have derived a powerful model by combining two well-known models. The proposed model gives excellent GoF performance and provides more confidence both for the customer and for the software development organizations.

Keywords: Dynamically Weighted Combination Models, Goodness of Fit Statistics, Software Reliability Growth Models

1. Introduction

Depending on the set of assumptions about the failure pattern, software reliability models are categorized as below1,2: • Markovian Model • Bayesian Models • Non-Homogenous Poisson Process (NHPP) Models The Markovian model3 illustrates that the failure state of software, is one of the many countable states that the software has, at a given point of time. Only the current state of the software would influence the failure intensity. Knowledge on the previous performances of the

* Author for correspondence

software is used by Bayesian model. Prior to the testing, some details on the software are collected. This is used along with the data gathered during the testing. This approach helps in arriving on a more precise estimation and prediction of reliability. A major attribute of the NHPP model is that every process has a mean value function. This is the number of failures expected up to a given time. Using multiple options for the mean value function, various models may be constructed for use in varied situations. While some projects depict exponential growth of mean value function, others depict S shaped growth. In order to address both the phenomena, a few flexible

Dynamically Weighted Combination Model for Describing Inconsistent Failure Data of Software Projects

Generalized NHPP models were proposed4–7 which use Generalized exponential and Weibull function. However, we could achieve similar results with a combination of NHPP models with simple functions for ROCOF. In this paper, we propose to combine two NHPP models and use a dynamically weighted combination model for the reliability analysis. The superposition and time transformation property of NHPP model is considered to derive such a combined weighted model. The Goodness of Fit (GoF) statistics for the base models are compared with that of the combined weighted model to illustrate the improved GoF performance of the latter. This paper is structured as below: Second section describes the base characteristics of the NHPP models. Third section provides details on the two NHPP models that we have combined and the failure dataset that we have chosen. Further, we are also discussing about those key properties of NHPP models, which has been a driving source for this analysis. In the fourth section, we have provided the Gof (Goodness of Fit) metrics that we estimated from the base models and the combined model. Findings are summarized in the fifth section.

2. NHPP Models NHPP describes the cumulative failures up to a time t. Most of the reliability models fall under this category. The model basically assumes that the cumulative failures that happen in a time t, adapts a Poisson process and the time between failures follows exponential distribution.4,5 Some of the assumptions of NHPP models are as follows: • A software system randomly fails due to the software faults that are identified. • When t=0, no failures are there. • The probability of a single failure to occur in a small interval g is λ(t) g + o(g) where λ(t) is instantaneous failure intensity at time t. In a small interval g, the probability for more than one failure to occur is o(g). The chance of more failures to happen in such a short time interval is very less.

3. NHPP Models Selected for the Weighted Combination Model We chose one of the earliest NHPP models (1972) with 2

Vol 9 (35) | September 2016 | www.indjst.org

exponential growth of mean value function and another early model with S-shaped growth of mean value function as per the details below. • Musa’s BET4model addresses failure intensity pattern as exponential function • Yamada’s S-type model4 addresses increasing/ decreasing failure intensity pattern

3.1 Musa’s Basic Execution Time Model (BET)

Musa BET model illustrates that that for every failure, the decrement in failure intensity function remains unchanged. Musa also assumes one fault will cause one failure4,5. Musa’s BET model basically assumes: • Faults are not dependent and are distributed at a constant rate in which they are encountered • When compared with execution time of instruction, the execution time between failures is more. • Correction is applied instantaneously or the repeated occurrence of that failure is left uncounted. • It assumes exponential growth of mean value function µ(t), the mean value function and λ(t), the failure intensity are defined as below µ(t) = f◦ [1 – exp(-λ◦t / f◦)] (1) λ(t) = λ◦ exp(-(λ◦t)/ f◦) (2) where, t is the CPU execution time, f◦ is the total number of failures that would be experienced in infinite execution time, λ◦is the initial software failure intensity.

3.2 Delayed S-shaped Model of Yamada

The failure rates in a project can follow different trends as testing phases progresses. It can be increasing, decreasing or constant. In some projects, an increase in failure rate is seen initially and it starts reducing later. This is because the testing team is on the process of learning. Thereby, they are less efficient during the initial phases of the testing cycle and slowly improve down the line. This learning process of the team could be due to several factors like, skill level of the programmers, new technology of the development environment, sequence of test cases chosen and unique features of the product under test. Yamada proposed the S-shaped SRGM based on this learning process of the team. This model illustrates S-type growth of the mean value function8.

Indian Journal of Science and Technology

Subburaj Ramasamy and C. A. S. Deiva Preetha

The following are some of the basic assumptions of Yamada’s Delayed S-shaped model8,9 • There is a learning curve involved at the beginning of the reliability testing • The testing skills of the team involved in the testing, slowly improves and there is an increase in the test effectiveness, later • The S form of the failure pattern shows that, the reliability of the product under test decreases in the beginning due to low test effectiveness, and increases later10. The equations for mean value function µ(t) and failure intensity λ(t) are given below

f(x) = a*(1-exp(-b*x/a))

µ(t) = c[1 – ( 1+dt) exp( -dt)]

4. Goodness of Fit (GoF) Statistics

c,d> 0

(3)

λ(t) = cd2t exp(-dt) (4) where, the expected number of errors to be eventually detected is indicated by c and the error detection rate per error in steady state, is indicated by d.

3.3 Selection of Failure Data Set

Usually most researchers choose Musa P1 dataset11 to prove their models, as it is, rather an ideal dataset without any fluctuation. We have particularly chosen Musa’s P14C11 dataset for this performance analysis as this respective dataset depicts a failure intensity pattern that increases first and then reduces due to the learning process of the testing team. We have estimated the model parameters with this data and GoF statistics was estimated for the two base models and was compared with the combined weighted model.

3.4 D ynamic Weighted Combination of NHPP Models

The important benefit of the NHPP model is its superposition property. If the underlying processes are independent, when there are many failure processes, the superposed failure process, will still be NHPP. Using this property, two or more NHPP models can be combined, by summing up the respective mean value functions. We can arrive on the failure intensity of the superposed process by just adding the failure intensity of the underlying processes12–15. The new weighted model was derived by combining Musa BET and Yamada delayed S shaped model16. The three models are parameterized and listed as: Vol 9 (35) | September 2016 | www.indjst.org

(5)

f(x) = c*(1-(1+d*x)*exp(-d*x)) (6) f(x) = p*(a*(1-exp(-b*x/a)))+(1-p)*(c*(1-(1+d*x)*exp(d*x))) (7) Mean value functions of Musa BET model, Yamada delayed S-Shaped model and the dynamically weighted combination model are given above. As can be seen, p is the weight for Musa’s model and (1-p) is weight for the Yamada’s model. Since the value of p is estimated every time for a better fit, we call this dynamically weighted model.

Goodness of fit measure achieved by the model for a given dataset is used to assess how well a model can describe that dataset adequately. The value of GoF measure achieved by each model will vary for each dataset. The GoF statistics estimated by us for the three different models are provided below. Parameters for all three models for Musa’s P14C dataset were estimated using the curve fitting application of the MATLAB software. R-Square(R2), Sum of Squared Error(SSE) and Root Mean Square Error(RMSE) are the different GoF metrics used. The squared difference between the actual data and estimated value is summed up to get the SSE. The smaller value of SSE indicates better fit. R-Square is the ratio of sum of square in the regression about the mean. The value of R-Square can vary from 0 to 1, where the value which is proximate to 1 indicates better fit. To measure the difference between the actual and predicted values Root Mean Square Error is used and lower values of RMSE indicate better fit. The GoF statistics for all three models are given in the Tables 1 to 3. Table 1. GoF statistics for Musa Betmodel f(x) = a*(1-exp(-b*x/a)) Musa P14C For 36 Failures

SSE 344.4

R-square 0.9114

RMSE 3.183

Table 2. GoF statistics for Yamada Delayed S model f(x)=c*(1(1+d*x)*exp(d*x)) Musa P14C For 36 Failures

SSE 128.5

R-square 0.9669

RMSE 1.916


3

Dynamically Weighted Combination Model for Describing Inconsistent Failure Data of Software Projects

Table 3. GoF statistics for Weighted Combination model f(x) = p*(a*(1-exp(-b*x/a)))+(1-p)*(c*(1-(1+d*x)*exp(-d*x))) Musa P14C For 36 Failures

SSE 85.59

R-square 0.9780

4.

RMSE 1.635

Tables 1, 2 and 3 confirm that the weighted combined model performs better than the base models used for the combination.

5. Summary and Conclusions Some SRGMs describe failure intensity pattern as exponential function and others describe S-shaped growth. There by getting a better GoF measure in a consistent fashion for any model is a challenge. Initially we estimated the GoF statistics for Musa BET model using P14C dataset. Subsequently using the same dataset, we did the estimation for Yamada Delayed S-shaped model. We observed a relative improvement in GoF metrics. We further estimated the GoF statistics of the combination model and found it to be far superior when compared to the constituent models. Generally, researchers use models such as Generalized exponential and Weibull function with Rate of Occurrence of Failures for better goodness of fit. However, we could achieve similar results with a combination of two NHPP models with simple functions for ROCOF using the superposition property of NHPP models. The R2 value for the proposed combination model is close to 1 and the values for Root Mean Square Error and Sum of Squared Error values are smaller. Thereby we could conclude that combination models which are arrived by combining suitable SRGMs depicts significant improvement in GoF measures.

5. 6.

7.

8. 9. 10. 11.

12. 13.

14.

15.

6. References 1. Musa JD. Software reliability engineering. McGraw-Hill Publication; 1999. 2. Subburaj R. Software reliability engineering. McGraw Hill Education (India) Private Limited, New Delhi; 2015. p.61– 80. 3. Xie M. Software reliability modelling. World Scientific

4

Vol 9 (35) | September 2016 | www.indjst.org

16.

Publishing Co Ltd., Singapore, Elements of Software Reliability Modelling;1999,Chapter 2. Ramasamy S, Govindasamy G. Generalized exponential Poisson model for software reliability growth. International Journal of Performability Engineering.2006 July; 2(3):291–301. Ramasamy S, Govindasamy GPK. A software reliability growth model for vital quality metrics. South African Journal of Industrial Engineerng.2007; 18(2):93–108 Ramasamy S, Govindasamy GPK. A software reliability growth model for estimating Debugging and the Learning Indices. International Journal of Performability Engineering. 2012;8(5):539–49. Ramasamy S, Govindasamy G. A software reliability growth model addressing Imperfect Debugging and learning.Proceedings of International Conference on Reliability and Safety Engineering. 2006;156–64. Yamada S, Osaki SOM. S-shaped reliability growth modeling for software error detection. IEEE Transactions on Reliability.1983 Dec;32(5):475–78. Yamada S, Osaki S. Nonhomegenous error detection rate models for software realibility growth. Stochastic Models in Reliability Theory.1984:120–43 Ramasamy S, Govindasamy G. A software reliability growth model addressing learning. Journal of Applied Statistics.2008;35(10):1151–68. Musa JD. DACS Software reliability dataset, data and analysis center for software [Internet]. [Cited 2005 Sep 19]. Available from: www.dacs.dtic.mil/databases/sled/swrel. shtml. Xie M. Software reliability modelling. World Scientific Publishing Co Ltd.,Singapore, Non Homogenous Poisson Process (NHPP) Models.1991;4:110. Gayathry G, Selvi RT. Classification of software reliability models to improve the reliability of software. Indian Journal of Science and Technology.2015 Nov; 8(29). DOI: 10.17485/ijst/2015/v8i29/85287. Rashid E, Patnayak S, Bhattacherjee V. Estimation and evaluation of change in software quality at a particular stage of software development. Indian Journal of Science and Technology. 2013 Oct; 6(10).DOI: 10.17485/ijst/2013/ v6i10/38797. Molani M, Ghaffari A, Jafarian A. A new approach to software project cost estimation using a hybrid model of radial basis function neural network and genetic algorithm. Indian Journal of Science and Technology. 2014 Jan; 7(6). DOI: 10.17485/ijst/2014/v7i6/46537. Subburaj R, Kumaran AMJM. Dynamically weighted combination of fault based software reliability growth models. Indian Journal of Science and Technology. 2016 Jun; 9(22). DOI: 10.17485/ijst/2016/v9i22/93967.
