An Imperfect Debugging Model with Two Types of ... - ScienceDirect

COMPUTER MODELLING

PERGAMON

Mathematicaland ComputerModelling31 (2000) 343-352

www.elsevier.nl/locate/mcm

An Imperfect Debugging Model with Two Types of Hazard Rates for Software Reliability Measurement and Assessment K.

TOKUNO

AND

S.

YAMADA

Department of Social Systems Engineering, Faculty of Engineering Tottori University, Tottori-shi, 680-8552, Japan Qsse.tottori-u.ac.jp Abstract-we propose a software reliability model which assumes that there are two types of software failures. The first type is caused by the faults latent in the system before the testing; the second type is caused by the faultsregeneratedrandomlyduringthe testing phase. The former and latter software failure-occurrence phenomena are described by a geometrically decreasing and a constant hazard rate, respectively. Further, this model describes the imperfect debugging environment in which the fault-correction activity corresponding to each software failure is not always performed perfectly. Defining a random variable representingthe cumulative number of faults successfully corrected up to a specified time point, we use a Markov process to formulate this model. Several quantitative measures for software reliability assessment are derived from this model. Finally, numerical examples of software reliability analysis baaed on the actual testing data are presented.@ 2000 Elsevier Science Ltd. All rights reserved. Keywords-software reliability, Latent faults, Regenerated random faults, Hazard rates, Imperfect debugging, Markov process.

1.

INTRODUCTION

Many software reliability models have been developed for the purpose of estimating software reliability quantitatively.

Among these, it is said that software reliability

growth models

have

high validity and usefulness. These models can describe a software fault-detection or a software failure-occurrence phenomenon during the testing phase in the software development process and/or the operation phase. A software failure is defined as an unacceptable departure from program operation caused by a fault remaining in the system. Most representative software reliability growth models are based on the following assumptions. l l

The debugging is perfect. The hazard rate for software failures is proportional to the residual current fault content.

The above assumptions imply that all of the faults latent in the system are corrected, and a hazard rate converges to zero. However, in general, it is impossible to remove all faults from the system and deliver a fault-free software system to the customers. Proficient programmers recognize the existence of the regenerative faults (see [l]). A ccordingly, the software system may This work was supported in part by the Foundation for C&C Promotion and the Secom Science and Technology Foundation. 0895-7177/00/$ - see front matter @ 2000 Elsevier Science Ltd. All rights reserved. PII: SO895-7177(00)00105-9

Qpe=t

by 44-W

K. TOKUNO AND S. YAMADA

344

show the same failure-occurrence testing

phase.

Further,

errors or misunderstandings an imperfect

of the two types prior to testing. phase.

failures.

a geometrically

and latter

decreasing

That

process

quantities

for software

likelihood

estimation

reliability

measurement

types of software

assessment

we assume failures

F2. Software

failures

that

environment

is

[2-41.

model with the existence

faults

remaining

or regenerated Defining

in the system

during

phenomena

rate, respectively.

the testing

are described a random

by

variable

are derived.

Further,

Finally,

based on actual

testing

the method

numerical

of maximum-

examples

of software

data are illustrated.

DESCRIPTION

the following

two types of software

due to faults originally due to faults

corrected up to a specified time point, (see [5]). Fr om this model, several interesting

is presented.

2. MODEL

Software

debugging

growth

introduced

this model

and assessment

Fl.

reliability

failure-occurrence

hazard

of model parameters

In this paper,

the last stage of the such as typographical

of faults successfully

to formulate reliability

is, the actual

One is due to the original

number

during

factors

models have been proposed

a software

and a constant

the cumulative

we use a Markov

system

human-error

The other is due to faults randomly

The former

representing

debugging

[4], we discuss

of software

contain

the test results.

imperfect

modifying

as a hardware

activities

about

one. Several

In this paper,

phenomenon

debugging

latent

randomly

in the system

introduced

failures

exist.

prior to the testing.

or regenerated

during

the testing

phase. Next, we describe the definition of the perfect debugging. The purpose of the debugging activity is to detect and correct the faults latent in the system and to improve software quality/reliability. Therefore, we consider that the debugging activity is perfect when the debugging corresponding to each software failure has contributed to the improvement of software reliability, and then one fault is corrected. Accordingly, perfect debugging increases the mean time between software failures (MTBSF) and decreases the hazard rate; imperfect and decrease the hazard rate. We consider that the increase of other

new faults during

The software Al.

reliability

The debugging

debugging growth

activity

can be negligible

model constructed

is performed

debugging does not increase MTBSF in the hazard rate by the introduction

(see [6]).

here is based on the following

as soon as the software

failure

assumptions.

occurs.

A2. The debugging activity for the fault which has caused the corresponding software failure succeeds with probability p (0 5 p 5 l), and fails with probability q(= 1 - p). We call p the perfect A3. The hazard

debugging

rate.

rate for Fl is constant

as each fault is corrected. (see PI). A4. The debugging

activity

between

The hazard is performed

software

failures

rate for F2 is constant without

distinguishing

and decreases throughout between

geometrically

the testing

phase

Fl and F2.

A5. The probability that two or more software failures occur simultaneously is negligible. A6. At most one fault is corrected when the debugging activity is performed, and the faultcorrection time is not considered. Let X(t) be a counting process representing the cumulative number of faults corrected up to testing time t. From Assumption A2, when i faults have been corrected by an arbitrary testing time t, after the next software failure occurs, X(t)

=

1’

27 i + 1,

(with probability

q),

(with probability

p),

(see Figure 1). Further, from Assumptions A3 and A4, when i faults hazard rate for the next software failure-occurrence is given by Zi(7-) = Dki + 0,

(i=O,l,2

I...,

have been corrected,

D > 0, 0 < k < 1, 0 10,

r 2 0),

the

(2)

Imperfect Debugging Model

345

Figure 1. A diagrammatic representation of transitions between states of X(t). zi(z) A D+8--

....................,, .............II ........................... e-1, 1,

,I “A

2

01

3

,I V”

*

v> . . .

4

Testing Time (A: perfectdebugging, v: imperfect debugging) Figure 2. A sample realization of hazard rate z~(T).

where

D

the hazard

is the initial

hazard

rate for F2.

rate for Fl,

In the actual

k is the decreasing

testing

environment,

ratio of the hazard it is difficult

to specify

when the faults are introduced or regenerated and whether the software failure regenerative fault. Therefore, we assume that the faults are introduced randomly and the software

failures caused by the introduced

faults occur randomly

during

rate,

is due to the in the system

the testing

Accordingly, the hazard rate for F2 is on average constant through the testing phase. function of ~~(7) is shown in Figure 2. We can see that the hazard rate pi decreases debugging

succeeds,

but remains

constant

when debugging

and B is

immediately

phase.

A sample when the

fails.

The expression of equation (2) is from the point of view that software reliability depends on the debugging efforts, not the residual fault content. We do not note how many faults remain in the software system. Equation (2) describes a software failure-occurrence phenomenon where a software contributes

system has a high frequency of software failure-occurrence, and perfect debugging largely to the improvement of software reliability during the early stage of the testing

phase; later in the testing phase, the decrease in the hazard rate is slower even if debugging perfect (see [&lo]). From (2), the distribution function for the next software failure is given by

is

Further, let Qi,j(r) denote the one step transition probability that after making a transition into state i, the process {X(t), t 2 0) makes a transition into state j by time 7. Then, Qi,j(r), which represents the probability that if i faults have been corrected at time zero, j faults are corrected by time r after the next failure occurs, is given by Qi,j(~) = pij

[l - ,-(Dk’+e)T] ,

K. TOKUNO AND S. YAMADA

346

where Pij are the transition probabilities from state i to state j and given by

Pij=

Q,

(j = i),

P,

(j=i+l),

{ 0,

3. DERJVATION 3.1.

Distribution

(i,j=O,l,2

(5)

(elsewhere),

OF SOFTWARE

of the First

,... ).

Passage

RELIABILITY

Time

to the Specified

MEASURES

Number

of Corrected

Faults Let S, be the random variable representing the time spent in removing n faults and Gi,+(t) be the distribution function associated with the probability that n faults are corrected in the time interval (O,t] on the condition that i faults have already been corrected at time zero. Then, we obtain the following renewal equation: t Gi,&)

=

Gi+l,pZ(r - u)

I

dQi,i+&) + f

G,n(t - u>dQi,i(u>

= &i+, * G+l,n(t) + Qi,i* G,d;,

(i=O,1,2

)...,

(6)

n-l),

where * denotes a Stieltjes convolution and Gn,+(t) = 1 (n = 1,2,. . . ). We use Laplace-Stieltjes

(L-S) transforms to solve (6), where the L-S transform of Gi,n(t) is

defined as G+(s)

EE O”emstdGi,,(t). s0

(7)

Prom (6), we get &I(s)

= &,i+l(s)Gi+I,,(s)

From (4), the L-S transforms of Q++r(r)

(i=O,l,2

+ S&)G!i,,(s),

)...)

n-l).

(8)

and Qi,i(r) are, respectively, given as

(9) (10) Substituting (9) and (10) into (8) yields p (Dki + e) = s + p (Dki + e) G+l,n(S),

G&)

(i = 0, 1,2,. . . , n - 1).

Solving (11) recursively, we obtain the L-S transform of Go,,(t) eo,n(s)

=

yj i=.

P Pz

+ 4

s+p(Dki+8)

n-1 c A?%

= i=.

(11)

as

p w

+ 0)

(12)

“s+p(Dki+e)’

where A; z 1,

n-l kj + B/D A; = n j=. kj - ki ’

(n = 2,3,. . . , i = 0,1,2,.

. . ,n - 1).

(13)

j#i By performing the reverse transform of (12) and rewriting Go,,(t) and Gn(t), we have the distribution function of the first passage time when n faults are corrected n-1

G,(t)

= Pr{S,

< t} = c A; [l - e-p(Dk’+e)c] i=o

,

(t 2 0, n = 1,2,. . _).

(14)

Imperfect

Debugging

Model

347

‘. E[S,] = c +a P (Ok2 + 0) ’

(15)

n-l ’ Var[&] = C i=c p2 (Dlci + e>2 ’

(16)

Further, the mean and the variance of S, are given by n-l

respectively. 3.2. Distribution

of the Number

of Faults Corrected

up to a Specified

Time

Since X(t) is a counting process, we have the following equivalent relation: {STZI t} *

{X(t)

> n}.

(17)

Therefore, we get Pr{S, Let P,(t)

2 t} = Pr{X(t)

> n}.

(18)

denote the probability that n faults are corrected up to testing time t. Prom (14)

and (18), we obtain the probability mass function Pn(t) as P,(t)

= Pr{X(t)

= n} = G,(t)

- G,+r(t).

(19)

Using (19), we can derive the expectation and the variance of X(t),

E[X(t)l =

respectively, as

2 G(t),

(20)

n=l

1 2

Var[X(t)]

= F(2n

- l)G,(t)

3.3. Expected

Number

of Software

-

2 [ n=l

n=l

G,(t)

.

(21)

Failures

We introduce a new random variable H(t) representing the cumulative number of software failures occurring in the testing time-interval (0, t]. Let A&(t) be the expected number of software failures up to testing time t on the condition that i faults have been already corrected at time zero, that is, h&(t) = E[H(t) ] X(0) = i], which is called a Markov renewal function (see [5]).

(22)

Then, we obtain the following renewal

equations: Mi(t) = &(t) +

Qi,i * M(t)

-I- Qi,i+l * Mi+l(t),

(i = O,l, 2,. . .).

(23)

Using the L-S transforms of A&(t) (i = O,l, 2,. . .), from (12) we get

i&(s) =

(24)

Inverting (24) with respect to t and rewriting A&(t) as M(t), we have

M(t) = ;

z

n-1

G,(t) = ;E[X(t)].

(25)

K. TOKUNO AND S. YAMADA

348

3.4.

Distribution

of the Time

Let Xl (I = 1,2,...

between

Software

Failures

) be the random variable which represents the time interval between the

(1 - l)st and the Zth software failure occurrences and @Z(z) be the distribution function of Xl. It is noted that Xl depends on the number of the faults corrected up to the (1- l)st software failure occurrence, which is not explicitly known. However, the random variable Cl, which represents the number of the faults corrected up to the (I - l)st software failure occurrence, follows a binomial distribution having the following probability mass function: Pr{Cl = i} = where (“i’)

(

Z-I i

>

i l-1-i P4 T

(i=O,1,2

,...,

Z-l),

G (I - l)!/[(Z - 1 - i)!i!] d eno t es a binomial coefficient. Further, it is evident that Pr{XZ 5 z ] Cl = i} = 1 - e-(Dk’+e)r,

(27)

which is given by (3). Accordingly, we can get the distribution function for Xl as l-l @l(z)

=

5 2 ) Cl = i} . Pr{Cl = i}

Pr{Xr 5 z} = CPr{Xr i=o =

l-l

(28)

z-

1

c( >

pzql-l-i

1 _ [

i

i=o

e -(Dki+e)z

. I

Then, we have the reliability function for Xl as

(29) Further, the hazard rate for XZ is given by

‘2

(l;l)piql-l-i l-l tz

(ski

+ 0) e-(Dk’++

(39) (l,l)piql-l-ie-(Dk”+e)”

.

The expectation of Xl, that is, MTBSF, is defined by 03 E[XZ] ss

s0

Rl(x) dx.

(31)

From (29), we can derive MTBSF as

E[&]

= g

(” ; ‘) ‘s,

(32)

i=o 4.

PARAMETER

ESTIMATION

In this section, we discuss the estimation method of the unknown parameters D and k. The hazard rate for Xl can be regarded as a random variable depending on the cumulative number of the corrected faults. Let Zl (Z = 1,2,. . . ) be the random variable representing the hazard rate for Xl. From (2) and (26)) we have Pr{Cl = i} = Pr {Zr = Dki + O} =

piq1-1-i,

(i = 0,1,2,.

. . ,z - 1).

(33)

Imperfect

Then,

the expectation

Debugging Model

349

of 2~ is given by l-l

E[Zl] = c

piq”-1-i = D($

(Dk? + 0) .

+ ,$1-l + 0.

(34)

i=O

Accordingly, probability

using

(34), we approximate

density

function

the hazard

for Xl is approximated

rate for XI expressed

by (30).

Then.

by

&(3;) = [D(plc + C.&i + e] c-W~+q)‘-‘+~ls. Suppose realization probability

that

the data

set on m software

failure-occurrence

(35)

time-intervals

xl, that

of Xl (I = 1,2,. . . , m), is observed during the testing phase. density function, that is, the likelihood function, is given by

The

is, the

simultaneous

~=~9~(~~)=~[~(pk+q)~-~+e]exp -f$bk+d-l+e]~l . l=l

Taking

kl

the natural

logarithm

1nL = 2

the

l=l

(36)

1

of (36) yields

{In [D&k + q)l-’

+

e] - [D(pk

+ t&l

+ e] Q} .

(37)

l=l

The maximum-likelihood estimates b and & for the unknown parameters D and k can be obtained by solving the simultaneous likelihood equations w = w = 0, that is, m

(pk+ d1-1 Dbk + qF1 + 0

CL (11=1

l)(pk + q)1-2

D(pk + q)‘-1 + 6’

(pk + q)+rl

- (1 - l)(pk + q)1-2x1

1 1

= 0,

(33)

= 0,

(39)

which can be solved numerically.

5. NUMERICAL Applying several

the imperfect

numerical

The data

debugging

illustrations

set consists

EXAMPLES

model discussed

of software

of 26 software

reliability

above to the actual measurement

failure-occurrence

testing-data,

we show

and assessment.

time-interval

data

ICY(days with

1=

,26) cited by Goel and Okumoto [ll]. The testing termination time is 250 days, that is g&;l = 250. For these data, we assume that model parameters p and 0 can be prespecified. Let d = BT be the expected number of F2 in the testing time-interval (0, T]. Assuming that 10% of the cumulative number of software failures at the testing termination time T = 250 are F2, we can determine that t9 = 0.0104. In case of p = 0.9, the maximum-likelihood estimates of unknown parameters D and k are estimated as

L?= respectiv% faults

E[X(t)]

The estimated in (T

0.189, expected

k = 0.947, numbers

are shown in Figure

(e = 0.0104, p = 0.9), of software

failures

MT)

in (25) and corrected

3.

The estimated Gzs(t) in (14), which represents the distribution function of the time spent in correcting 26 faults, are shown in Figure 4. Though 26 software failures are observed at the testing

351

Imperfect Debugging Model

0

600

4o044g 500

200 250300

100

Testing Time (days) Figure 5. Estimated cv[X’i‘;i]. RU(X)

10

20 x(dw)

Figure 6. Estimated Rz). Table 1. MTBSF

E[Xl].

I=1 1=5 I-1”

I=15

0.1'

‘=201,25 1=27

0.05

0

20

10 x

(days)

Figure 7. Estimated 3.

30

352

K. TOKUNO

AND

6. CONCLUDING

S. YAMADA

REMARKS

In this paper, we have developed a software reliability growth model considering the software failure-occurrence due to the faults introduced during debugging of the testing phase. This model considers the imperfect debugging environment in which both fault removal is uncertain and debugging may introduce other new faults. Accordingly, this imperfect debugging model is superior to the earlier model by Yamada et al. [4]. Several quantities for software reliability measurement have been derived for this model, and the numerical examples of software reliability analysis based on the actual testing-data have been illustrated. The hazard rate for F2, 8, and the perfect debugging rate, p, have been determined experimentally. It is necessary to analyze observed testing data in detail in order to estimate 0 and p. Considering the method for measuring and collecting testing data, we need to verify the validity of this model.

REFERENCES 1. M.L. 2. 3. 4. 5. 6. 7. 8. 9. 10. 11.

Shooman,

Software

Engineering:

Design,

Reliability,

and Management,

McGraw-Hill,

New York,

(1983). M. Ohba and X.-M. Chou, Does imperfect debugging affect software reliability growth?, In Proc. llth Znt. Conf. Software Engineering, pp. 237-244, (1989). J.G. Shanthikumar, A state- and time-dependent error occurrence-rate software reliability model with imperfect debugging, In Proc. National Computer Conf., pp. 311-315, (1981). S. Yamada, K. Tokuno and S. Osaki, Software reliability measurement in imperfect debugging environment and its application, Reliability Engineering and System Safety 40 (2), 139-147, (1993). S. Osaki, Applied Stochastic Modeling, Springer-Verlag, Heidelberg, (1992). A.L. Goel, Software reliability models: Assumptions, limitations, and applicability, IEEE tins. Softwarn Engineering SE-11 (12), 1411-1423, (1985). P.B. Moranda, Event-altered rate models for general reliability analysis, IEEE nans. Reliability R28 (5), 376-381, (1979). J.D. Musa, I. Iannino and K. Okumoto, Software Reliability: Measurement, Prediction, Application, McGraw-Hill, New York, (1987). S. Yamsda, Software quality/reliability measurement and assessment: Software reliability growth models and data analysis, Journal of Information Processing 14 (3), 254-266, (1991). S. Yamada, Software Reliability Models: Fundamentals and Applications, (in Japanese), JUSE Press, Tokyo, (1994). A.L. Goel and K. Okumoto, Time-dependent error-detection rate model for software reliability and other performance measures, IEEE Trans. Rgliability R-28 (3), 206-211, (1979).