QUICKEST DETECTION WITH EXPONENTIAL ... - Project Euclid

The Annals of Statistics 1998, Vol. 26, No. 6, 2179 ᎐ 2205

QUICKEST DETECTION WITH EXPONENTIAL PENALTY FOR DELAY 1 BY H. VINCENT POOR Princeton University The problem of detecting a change in the probability distribution of a random sequence is considered. Stopping times are derived that optimize the tradeoff between detection delay and false alarms within two criteria. In both cases, the detection delay is penalized exponentially rather than linearly, as has been the case in previous formulations of this problem. The first of these two criteria is to minimize a worst-case measure of the exponential detection delay within a lower-bound constraint on the mean time between false alarms. Expressions for the performance of the optimal detection rule are also developed for this case. It is seen, for example, that the classical Page CUSUM test can be arbitrarily unfavorable relative to the optimal test under exponential delay penalty. The second criterion considered is a Bayesian one, in which the unknown change point is assumed to obey a geometric prior distribution. In this case, the optimal stopping time effects an optimal trade-off between the expected exponential detection delay and the probability of false alarm. Finally, generalizations of these results to problems in which the penalties for delay may be path dependent are also considered.

1. Introduction. Quickest detection is the problem of detecting, with as little delay as possible, a change in the probability distribution of a sequence of random measurements. This problem arises in a great variety of applications, such as seismology, speech and image processing, biomedical signal processing, machinery monitoring and finance. Overviews of existing techniques for quickest detection can be found in Basseville and Nikiforov Ž1993., Brodsky and Darkhovsky Ž1992., Carlstein, Muller and Siegmund Ž1994. and ¨ Kerestecioglu ˇ Ž1993.. A useful formulation of the quickest detection problem is to consider a sequence X 1 , X 2 , . . . of random observations, and to suppose that there is a change point t G 1 Žpossibly t s ⬁. such that, given t, X1 , X 2 , . . . , X ty1 are drawn from one distribution and X t , X tq1 , . . . , are drawn from another distribution. The set of detection strategies of interest corresponds to the set of Žextended. stopping times with respect to the observed sequence, with the interpretation that the stopping time T decides that the change point t has occurred at or before time k when T s k. We will be more specific about this model in subsequent sections.

Received March 1997; revised June 1998. 1 Supported in part by the IDA Center for Communications Research, Princeton, New Jersey. AMS 1991 subject classifications. Primary 62L10; secondary 60G40, 62L15, 94A13. Key words and phrases. Quickest detection, change point problems, optimal stopping, exponential cost.

2179

2180

H. V. POOR

The design of quickest detection procedures typically involves the optimization of a trade-off between two types of performance indices, one being a measure of the delay between the time a change occurs and it is detected w i.e., ŽT y t q 1.q, where xqs max 0, x 4x , and the other being a measure of the frequency of false alarms Ži.e., events of the type T - t 4.. In essentially all such extant designs, detection delay is penalized via a linear function of delay. w An exception is found in Pelkowitz Ž1987., in which nonlinear delay penalties are proposed, but corresponding optimal stopping times are not derived.x This type of penalty is suitable for many applications. For example, the earliest applications of quickest detection involved the monitoring of manufacturing processes to detect possible declines in quality of the manufactured goods. In this situation, the cost of delay is accurately measured by a linear penalty, reflecting the fact that the economic cost of discarded defective goods will be proportional to the quantity produced. However, in other applications, linear cost does not capture the true cost of delayed action. Consider, for example, financial applications in which the change point may represent a time at which a fundamental shift in the performance, or expected performance, of some type of investment occurs. In this situation, the compounding of investment growth or the short shelf lives of investment opportunities point to exponential penalties as more suitable measures of the cost of delay. Similarly, in the health monitoring of components in interconnected systems Že.g., aircraft systems, communication networks, power grids, biological populations, etc.., the effects of undetected faults can exponentiate with time, again suggesting a more aggressive cost structure than is captured with a linear delay penalty. Motivated by these types of applications, this paper considers the problem of quickest detection with exponential penalty for delay. In particular, we consider penalties on the detection delay of the form q

Ž 1.

␣ ŽTytq1. y 1 ␣y1

,

where ␣ / 1 is a positive constant. Note that, with ␣ ) 1, this penalty quantifies an exponential growth of costs as a function of delay, reflecting the type of exponentiating costs mentioned in the preceding paragraph. Alternatively, with ␣ - 1, Ž1. quantifies a saturating, sublinear cost of delay. Since q

Ž 2.


T

s

Ý ␣ lyt , lst

the exponential penalty in this latter case can be viewed as a discounted version of the traditional linear penalty ŽT y t q 1.q. Of course, as ␣ ª 1, the quantity of Ž1. approaches the linear delay penalty. For this cost function, we consider two traditional formulations of the quickest detection problem. The first of these is a minimax formulation, first proposed in the linear-delay-penalty case by Lorden Ž1971., in which the delay penalty is a worst-case measure of delay, and false alarms are con-

EXPONENTIAL QUICKEST DETECTION

2181

strained through a lower bound on the allowable mean time between false alarms. In this formulation, the worst-case delay is taken over all possible realizations of the observations leading up to the change point and over all possible values of the change point. The second formulation is a Bayesian formulation, first proposed in the linear-delay-penalty case by Kolmogorov and Shiryayev w see Shiryayev Ž1963.x , in which the change point is endowed with a prior distribution, and the opposing performance indices are expected detection delay and false-alarm probability. We also consider modifications of these problems in which the delay penalty depends explicitly on the observed sample path of the random sequence. We develop optimal detection procedures for each of these formulations. Performance analysis is also considered in each case, with the first of the formulations offering perhaps the most interesting results in this regard. For example, among other results, it is seen that the classical Page CUSUM test Žwhich is minimax optimal for linear delay penalty. can have infinite minimax exponential delay if the rate at which delay penalty accumulates is too large relative to the rate at which discrimination information between prechange and postchange distributions accumulates. The remainder of this paper is organized as follows. The minimax solution is presented in Section 2, and Section 3 is devoted to performance analysis in this case. Section 4 develops the Bayesian solution. Section 5 considers the extension of the results of Sections 2 through 4 to the case of path-dependent cost of delay. Finally, Section 6 contains some concluding remarks. Appendices containing the more detailed elements of the required proofs are also included. 2. A minimax solution. We begin by considering the situation in which the change point t is a fixed, nonrandom quantity that can be either ⬁ or any value in the positive integers. To model this situation, we consider a measurable space Ž ⍀, F ., consisting of a sample space ⍀ and a ␴-field F of events. We further consider a family Pi ; t g w 1, 2, . . . , ⬁x4 of probability measures on Ž ⍀, F ., such that, under Pt , X1 , X 2 , . . . , X ty1 are independent and identically distributed Ži.i.d.. with a fixed marginal distribution Q b , and X t , X tq1 , . . . are i.i.d. with another marginal distribution Q a and are independent of X1 , X 2 , . . . , X ty1. For simplicity, we assume that Q a and Q b are mutually absolutely continuous, that the likelihood ratio L s dQ ardQ b has no atoms under Q b and that 0 - DŽ Q b 5 Q a . - ⬁, where DŽ Q b 5 Q a . denotes the Kullback᎐Leibler divergence of Q a from Q b ,

Ž 3.

D Ž Q b 5 Q a . s y log L dQ b .

H

For technical reasons, we also assume the existence of a random variable X 0 that is uniformly distributed in w 0, 1x and that is independent of X1 , X 2 , . . . under each Pt . We would like to consider procedures that can detect the change point, if it occurs Ži.e., if t - ⬁., as quickly as possible after it occurs. As a set of

2182

H. V. POOR

detection strategies, it is natural to consider the set T of all Žextended. stopping times with respect to the filtration Fk 4k G 0 where Fk denotes the smallest ␴-field with respect to which X 0 , X 1 , . . . , X k are measurable. Thus, when the stopping time T takes on the value k, the interpretation is that T has detected the existence of a change point t at or prior to time k. Following Lorden Ž1971., it is of interest to penalize exponential detection delay via its worst-case value d Ž T . s sup d t Ž T .

Ž 4.

tG1

with q

d t Ž T . s ess sup Et

Ž 5.

½


5

Fty1 ,

where Et ⭈4 denotes expectation under the distribution Pt . ŽRecall that the essential supremum of a random variable is the greatest lower bound of the set of constants that bound the random variable with probability one.. Note that d t ŽT . is the worst-case average delay under Pt , where the worst case is taken over all realizations of X 0 , X 1 , . . . , X ty1. The desire to make dŽT . small must be balanced with a constraint on the rate of false alarms. The rate of false alarms can be quantified by the mean time between false alarms, f Ž T . s E⬁ T 4 ,

Ž 6.

and a useful design criterion is then given by

Ž 7.

subject to f Ž T . G ␥ ,

inf d Ž T .

Tg T

where ␥ is a constant. That is, we seek a stopping time that minimizes the worst-case delay within a lower-bound constraint on the mean time between false alarms. The solution to Ž7. for the linear-delay-penalty Ž ␣ s 1. case was demonstrated in Moustakides Ž1986.. Here, we extend this solution to the case of general ␣ . To do so, for h G 0 we define a stopping time Th s inf k G 0 < Sk G h4 ,

Ž 8. where k

Ž 9.

Sk s max

1FjFk

ž

Ł ␣ L Ž Xl . lsj

/

s ␣ L Ž X k . max Sky1 , 1 4 ,

and S0 s 0. We then have the following result. THEOREM 2.1.

Ž 10.

Suppose h G 0 and P⬁Ž ␣ L Ž X1 . ) 1 . ) 0.

Then Th solves Ž7. with ␥ s f ŽTh .. That is,

Ž 11.

f Ž T . G f Ž Th . « d Ž T . G d Ž Th . .

k G 1,


2183

REMARK 2.2. Note that condition Ž10. is trivially satisfied if ␣ G 1. PROOF. In proving this result, we will make heavy use of Moustakides’ method of proof for the linear-delay case w Moustakides Ž1986.x . For the case in which log ␣ - DŽ Q b 5 Q a ., the extension to exponential cost is rather straightforward. For larger ␣ , some variations are needed. These considerations will arise in the proof of Lemma 2.4 below. ŽSince the case h s 0 is trivial, we consider only h ) 0.. It is easily seen that, in seeking solutions to Ž7., we can restrict attention to stopping times that achieve the constraint on f ŽT . with equality. This follows since, if ␥ - f ŽT . - ⬁, then we can produce a stopping time that achieves the constraint with equality without increasing the worst-case exponential delay, simply by randomizing between T and the stopping time that is identically zero. ŽSuch randomized stopping times are in T by virtue of the inclusion of F0 in the filtration.. Stopping times for which f ŽT . s ⬁ can be eliminated from consideration, since for this case we can choose sufficiently large n so that f ŽminT, n4. G ␥ , and we always have dŽminT, n4. F dŽT .. w That f ŽTh . - ⬁ follows from Theorem 3.1 below.x We now state the following two intermediate results, whose proofs are given in Appendix A and from which the theorem follows. LEMMA 2.3.

Ž 12.

Suppose T g T is such that 0 - f ŽT . - ⬁. Then dŽ T . G dŽ T . J

1 E⬁ ÝTy ks0 max S k , 1 4 4

E⬁ ÝTy1 ks0 Ž 1 y S k .

q

4

with equality if T s Th . LEMMA 2.4. Suppose Ž10. holds. Then Th solves the following maximization problem for all continuous nonincreasing functions g: w 0, ⬁. ª ⺢: Ty1

Ž 13.

sup Tg T

½Ý

ks0

g Ž Sk .

5

subject to f Ž T . s ␥ .

Taking g Ž x . s ymax x, 14 and g Ž x . s Ž1 y x .q, respectively, Lemma 2.4 asserts that Th simultaneously minimizes the numerator and maximizes the denominator of dŽT . within the constraint f ŽT . s ␥ . Since dŽTh . s dŽTh ., the theorem follows. I Theorem 2.1 asserts the optimality of the stopping time based on the first exit of Sk from the interval w 0, h.. We henceforth assume that h G 1, in which case the stopping time Th can be written equivalently as

Ž 14.

Th s inf k G 0 < m k G log h4 ,

where

Ž 15.

m k s log max Sk , 1 4 ,

k s 0, 1, . . . .

2184

H. V. POOR

It is easily seen that the sequence m k 4 can be computed recursively via

Ž 16.

m k s max m ky1 q log L Ž X k . q log ␣ , 0 4 ,

k s 1, 2, . . . ,

with m 0 s 0. That is, the test based on Th accumulates the adjusted log-likelihoods, log LŽ X k . q log ␣ , resetting the accumulation to zero whenever it goes negative. The alarm is sounded when this accumulation crosses the upper threshold log h. With ␣ s 1 the stopping time Th thus reduces to the classical CUSUM test of Page Ž1954.. For ␣ / 1, the test is more or less aggressive in sounding alarms than is Page’s test, depending on whether ␣ ) 1 or ␣ - 1. This is, of course, completely consistent with intuition, since larger values of ␣ correspond to greater penalties on delay. 3. Performance analysis. In the preceding section, we showed that the stopping time Th is optimal in the sense of Theorem 2.1. In this section, we consider the performance of this stopping time by determining the quantities dŽTh . and f ŽTh .. We begin with the following result, which gives exact expressions for these two quantities. ŽHere, and throughout this paper, 1A denotes the indicator function of the event A.. THEOREM 3.1.

Suppose h ) 1, and Ž10. holds. Then f Ž Th . s

Ž 17.

E⬁ N 4 1 y P⬁Ž F0 .

-⬁

and d Ž Th . s

Ž 18.

E1 ␣ N 4 y 1

Ž 1 y ␣ . ž 1 y E1 ␣ N 1 F 0 4 /

,

where N is the stopping time

Ž 19.

n

½

N s min n G 1

Ý

5

log L Ž X l . q log ␣ f Ž 0, log h . ,

ls1

and where F0 denotes the event

Ž 20.

½

N

Ý ls1

5

log L Ž X l . q log ␣ F 0 .

REMARK 3.2. The proof of this result relies on the renewal properties of the accumulated sum m k of Ž16., arising from the resetting of this sum each time it crosses zero. This analysis is similar to the classical analysis of Page’s CUSUM w e.g., Basseville and Nikiforov Ž1993., pages 195᎐197 or Siegmund Ž1985., Section II.6x . However, a distinction between this result and that for Page’s CUSUM arises in the treatment of the exponential delay, dŽTh ., and so a proof is included in Appendix B.

2185


Since N of Ž19. is the first exit time of a random walk from an interval, its statistical behavior can be analyzed via the classical methods of Wald approximation, diffusion approximation, and so on w cf. James, James and Siegmund Ž1988., Khan Ž1978. or Siegmund Ž1985.x . However, even without Theorem 3.1, f ŽTh . and dŽTh . can be estimated directly by approximating the behavior of Th with that of the stopping time

Ž 21.

T˜h s inf u G 0 < Zu y min Z s G log h ,

½

5

0FsFu

where Zu ; u G 04 is a Brownian motion approximating the random walk n

Ž 22.

Ý

log L Ž X l . q log ␣ ,

n s 1, 2, . . . .

ls1

Approximation of this type for linear delay penalty and the classical Page test has been considered by Reynolds Ž1975.. Analogous results can be obtained for the exponential-penalty case, as we now develop. Consider the case in which the observations are in ⺢ n , and Q b and Q a are Ž N ␮ 0 , ⌺ . and N Ž ␮ 1 , ⌺ . distributions, respectively, with ⌺ positive definite. Then, under P⬁ we may use the model

Ž 23.

Zu s '2 D Bu q D Ž ␤ y 1 . u,

u G 0,

and under P1 we may use the model

Ž 24.

Zu s '2 D Bu q D Ž ␤ q 1 . u,

u G 0,

where Bu ; u G 04 is a standard Brownian motion, D s DŽ Q b 5 Q a ., with DŽ Q b 5 Q a . given by Ž3., and the constant ␤ is defined as

␤s

Ž 25.

log ␣ D Ž Qb 5 Qa .

.

Here, of course, D s Ž ␮ 1 y ␮ 0 .X ⌺y1 Ž ␮ 1 y ␮ 0 .r2. w Note that the model Ž23. and Ž24. can also be used in the local testing case if the two distributions Q a and Q b are sufficiently close to one another that the asymptotes Var⬁Žlog LŽ X 1 .. ; Var1Žlog LŽ X 1 .. ; 2 D and E1 log LŽ X 1 .4 ; yE⬁ log LŽ X 1 .4 ' D give accurate approximations.. The statistics of stopping times of the form Ž21. have been analyzed in several works, including Kennedy Ž1976., Lehoczky Ž1977. and Taylor Ž1975.. We can conclude from this analysis w cf. equations Ž3.1.104. and Ž3.1.105. of Basseville and Nikiforov Ž1993.x that, under the model Ž23. and Ž24., we have

¡Ž log h . r2, 2

Ž 26.

D = f Ž T˜ . s ~ h

¢

ž

1y ␤

h

y1

1y␤

␤ s 1, y log h

/

Ž1 y ␤ . ,

␤/1

2186

H. V. POOR

and

Ž e ␤ D y 1 . = d Ž T˜h . Ž 27.

s

½

Ž h y 1 y log h . r Ž 1 q log h . , Ž Ž 1 y ␤ . h1q ␤ y h q ␤ h ␤ . r Ž h y ␤ h ␤ . ,

␤ s 1, ␤ / 1.

ŽRecall that e ␤ D s ␣ .. As ␤ ª 0 Ži.e., ␣ ª 1. these expressions reduce to those of Reynolds Ž1975. for the diffusion approximation to the performance of Page’s test under linear delay penalty. Note that, for large ␥ Žand consequently large h., Ž26. implies D = f Ž T˜h . ;

Ž 28.

½

h1y ␤r Ž 1 y ␤ . , log hr Ž ␤ y 1 . , 2

␤ - 1, ␤)1

and so d Ž T˜h . ;

Ž 29.

½

O Ž ␥ ␤ rŽ1y ␤ . . , O Ž exp

Ž ␥ DŽ ␤ y1..

0 - ␤ - 1,

.,

␤ ) 1.

Thus, the asymptotic behavior of the expected delay penalty is fundamentally different depending on whether the rate of delay-penalty increase is greater or less than the rate at which discrimination information between prechange and postchange distributions accumulates. It is interesting to compare Ž27. with the performance of Page’s test under exponential delay penalty. To do so, let us denote by T˜P the stopping time Ž21. where the Brownian motion Zu ; u G 04 approximates the random walk n

Ž 30.

Ý log L Ž X l . ,

n s 1, 2, . . . .

ls1

Again assuming Gaussian observations as above, this Brownian motion behaves statistically as Ž23. with ␤ s 0 and as Ž24. with ␤ s 0, respectively, under prechange and postchange conditions. The statistical behavior of T˜P thus approximates the statistical behavior of Page’s test. Applying Ž26. with ␤ s 0 yields h y 1 y log h f Ž T˜P . s . Ž 31. D To analyze dŽT˜P . for ␣ / 1, we consider two cases, ␤ - 1r4 and ␤ ) 1r4. For ␤ - 1r4, equation Ž3.1.104. of Basseville and Nikiforov Ž1993. straightforwardly yields d Ž T˜P . s

Ž 32.

2 ␰ Ž 'h .

1q ␰

y Ž ␰ q 1 . Ž 'h . ␰

␰

2

y␰q1

2

Ž e ␤ D y 1 . Ž ␰ q 1 . Ž 'h . q ␰ y 1

,

with ␰ s 1 y 4␤ . It follows that, asymptotically in the false-alarm constraint ␥ , we have

'

Ž 33.

d Ž T˜P . s O Ž ␥ Ž ␰q4 ␤ .r2 . ,

0 - ␤ - 1r4.


2187

Alternatively, for ␤ ) 1r4, we can show that dŽT˜P . s ⬁. In particular, we note that, for any x g Ž0, log h., T˜P is no smaller than the first exit time of Zu ; u G 04 from the interval Ž0, log h. after the first time that Zu s x. Statistically this latter time is no smaller than the first exit of Zu q x; u G 04 from Ž0, log h., a stopping time that we denote by T˜ x . It follows from page 258 of Dvoretsky, Kiefer and Wolfowitz Ž1953. w see also equation ŽA:194. of Wald Ž1947.x that, under the postchange conditions, T˜ x has a probability density that is a mixture of densities of the form

Ž 34.

␭i 2 ⌫ Ž 1r2 . t

3r2

½

exp y

␭2i 4t

y

Dt 4

5

,

t G 0,

where the ␭ i ’s are positive constants. Clearly, then, we have ˜

E1 ␣ T P 4 s ⬁,

Ž 35.

if log ␣ ) Dr4. So we see that the delay penalty incurred by the continuous-time version of Page’s test in this case is infinite if the rate of penalty increase is greater than one-fourth the rate at which discrimination information between prechange and postchange distributions accumulates. Even for smaller ␤ , Ž29. and Ž33. imply that

Ž 36.

d Ž T˜P . d Ž T˜h .

s OŽ␥ ␧ . ,

0 - ␤ - 1r4,

with ␧ s ␰r2 q ␤ Ž1 y 2 ␤ .rŽ1 y ␤ .. Since ␧ is strictly positive, the optimization problem posed in the preceding section clearly yields a significantly better test under exponential penalty than does the classical linear-penalty formulation. 4. A Bayesian solution: the exponential disorder problem. We now turn to a Bayesian version of the quickest detection problem, in which the change point t is assumed to be a random variable with a known prior distribution on the nonnegative integers. In particular, we consider the general set-up of Section 2, with an additional probability distribution P on Ž ⍀, F . under which t has the given prior distribution and the Pk ’s considered previously are the conditional distributions given the events t s k 4 . Here we do not need the assumptions that LŽ X k . is nonatomic under Q b and that DŽ Q b 5 Q a . is finite, and so we drop them. Additionally, as randomization will not be needed here either, we can replace the uniform random variable X 0 with a constant. In this situation, for a stopping time T, as a measure of delay we can adopt the expected exponential delay, q

Ž 37.

E

½


5

,

2188

H. V. POOR

where E ⭈4 denotes expectation under the measure P. Similarly, as a measure of false-alarm rate we can adopt the false-alarm probability, PŽT - t. .

Ž 38.

Analogously with the case of minimax design, we would like to determine stopping times T that effect optimal trade-offs between the two objectives of small detection delay and small false-alarm rate. A convenient way of implementing such a trade-off is to seek T g T to solve the optimization problem q

Ž 39.

inf Tg T

P Ž T - t . q cE

½


5

,

where c ) 0 is a constant controlling the relative importance of the two performance indices. q Note that, if we replace Ž ␣ ŽTytq1. y 1.rŽ ␣ y 1. with its ␣ ª 1 limit ŽT y t q q 1. , then the criterion Ž39. reduces to the classical Kolmogorov᎐Shiryayev criterion for detection of a ‘‘disorder’’ w see Shiryayev Ž1963.x , with the exception that Shiryayev Ž1963. uses ŽT y t .q in place of ŽT y t q 1.q. We could have equivalently considered the delay ŽT y t .q rather than ŽT y t q 1.q. In particular, since T and t are integer valued, it is easy to see that Ž39. is equivalent to q

Ž 40.

inf Ž 1 y c . P Ž T - t . q ␣ c E

Tg T

½

␣ ŽTyt . y 1 ␣y1

5

.

Interestingly, Ž40. implies that, with c G 1, the optimal stopping time for Ž39. is T ' 0. It is also noteworthy that, for ␣ - 1, a delay penalty of the ‘‘opportunityloss’’ form, q 1T - t4 y ␣ ŽTytq1. , Ž 41. is easily treated via Ž39. by appropriate adjustment of the constant c. In particular, the problem

Ž 42.

inf Tg T

P Ž T - t . q cY E 1T - t4 y ␣ ŽTytq1.

q

4

,

with cY ) 0, is equivalent to Ž39. with c s cY Ž1 y ␣ .rŽ1 q cY . - 1. As in previous analyses of the Bayesian disorder problem, we will assume a prior distribution on the change point t of the form P Ž t s 0 . s ␲ and P Ž t s k < t G k . s ␳ , Ž 43. where ␲ and ␳ are two constants lying in the interval Ž0, 1.. That is, there is a probability ␲ that a change has already occurred when we start observing the sequence; and there is a conditional probability ␳ that the sequence will transition to the postchange state at any time, given that it has not done so prior to that time. This model gives rise to a geometric prior distribution

Ž 44.

PŽ t s k. s

½

␲, ky 1 Ž1 y ␲ . ␳ Ž1 y ␳ . ,

if k s 0, if k s 1, 2, . . . .

2189


The solution to problem Ž39. with the geometric prior Ž44. is summarized in the following result. For appropriate chosen threshold RU G 0, the stopping time

THEOREM 4.1.

TB s inf k G 0 < R k G RU 4

Ž 45. with

Ž 46.

Rk s

␣ L Ž Xk . 1y␳

Ž R ky1 q ␳ . ,

k s 1, 2, . . . , R 0 s

␣␲

,

1y␲

is Bayes optimal w i.e., it solves Ž39. with the geometric prior Ž44.x . Moreover, if c G 1, then RU s 0. REMARK 4.2. The stopping time TB can be written equivalently as TB s inf k G 0 < r k G RU r Ž 1 q RU . 4 ,

Ž 47.

where r k s R krŽ1 q R k . satisfies the recursion

Ž 48.

rk s

␣ L Ž X k . r ky1 q ␳ Ž 1 y r ky1 . ␣ L Ž X k . r ky1 q ␳ Ž 1 y r ky1 . q Ž 1 y ␳ . Ž 1 y r ky1 .

,

k s 1, 2, . . . , with r0 s

Ž 49.

␣␲ 1 y ␲ q ␣␲

.

With ␣ s 1 it is easily seen that r k ' ␲ k J P Ž t F k < Fk ., and the result of Theorem 4.1 reduces to that of Shiryayev Ž1963, 1978.. REMARK 4.3. A proof of Theorem 4.1 is given in Appendix C. The basic idea of this proof is to first convert Ž39. to a standard optimal stopping problem by rewriting the objective of Ž39. as E YT 4 , where q

Ž 50.

½

Yk s E 1k - t4 q c

␣ Ž kytq1. y 1 ␣y1

5

Fk ,

k s 0, 1, . . . , ⬁.

This can be done because of the nonnegativity of the Yk ’s and the monotone convergence theorem. The sequence Yk 4 is given explicitly by c Yk s Ž 1 y ␲ k . l Ž R k . y , k s 0, 1, . . . , Ž 51. ␣y1 where l is the line

Ž 52.

lŽ R. s

␣ y 1 q c q cR ␣y1

and where ␲ k is defined as in Remark 4.2; that is, ␲ k s P Ž t F k < Fk .. w We also have Y⬁ s ⬁ if ␣ ) 1 and Y⬁ s crŽ1 y ␣ . if ␣ - 1.x Since Ž␲ k , R k . forms a homogeneous Markov process, Ž39. thus reduces to a Markov optimal stop-

2190

H. V. POOR

ping problem, which can be solved by standard methods Ži.e., dynamic programming.. In particular, the pay-off resulting from the initial condition Ž␲ 0 , R 0 . s Ž␲ , R . can be shown to be c inf E YT 4 s Ž 1 y ␲ . s Ž R . y , Ž 53. ␣y1 Tg T where s is a function satisfying the condition

Ž 54. s Ž R . s l Ž R . m R G RU where RU s inf R G 0 < s Ž R . s l Ž R . 4 . Markov optimal stopping theory then implies that the optimal stopping time is given by

Ž 55. Topt s inf k s 0, 1, . . . < l Ž R k . s s Ž R k . 4 s inf k s 0, 1, . . . < R k G RU 4 , which equals TB . REMARK 4.4. It follows from the proof of Theorem 4.1 given in Appendix C that the function s appearing in the payoff Ž53. is the pointwise monotone limit from above of the sequence of functions Q n l; n s 0, 1, . . . 4 where l is the line Ž52. and where the operator Q is defined in Ž125.. It can be shown Žsee Appendix C. that Q n l; n s 0, 1, . . . 4 is a monotone nonincreasing sequence of continuous functions, from which it follows that the sequence RUn 4 defined by

Ž 56.

RUn s inf R G 0 < Q n l Ž R . s l Ž R . 4 ,

n s 0, 1, . . . U

converges upward to the decision threshold R . Thus, computation of the threshold and optimal cost can be performed iteratively. 5. Quickest detection with path-dependent exponential costs. Thus far, we have considered delay penalties that depend on the sample path of observations only through the stopping time T. It is also of interest to allow for delay penalties that depend on the sample path in more direct ways. For q example, we might wish to replace the exponential penalty ␣ ŽTytq1. with T

Ž 57.

Ł ␾k ,

kst

where ␾ k 4 is a nonnegative sequence adapted to the observations. ŽWe adopt the conventions Ł ba s 1 if b - a, and Ý ba s 0 if b - a.. Such a penalty might arise, for example, in the detection of changes in financial time series, where the quantity ␾ k is the return that an investment would have generated during time period k had it been in force then. w For a related problem, see Beibel and Lerche Ž1997..x It is straightforward to show that the replacement of the linear penalty on ŽT y t q 1.q with path-dependent penalties of the form ÝTks t ␾ Ž X k ., where ␾ is a real-valued, measurable function satisfying 0 - H␾ Ž x . Q aŽ dx . - ⬁, does not materially change the form of the solutions of the Lorden and Shiryayev problems. In this section, we provide results analogous to those of Sections 2

2191


through 4 for certain problems of this type in which the path-dependent costs are exponential. We first consider the cost structure of Ž57. in the Bayesian case. In particular, we generalize Theorem 4.1 as follows. THEOREM 5.1. Consider the model of Section 4 and the problem

Ž 58.

inf Tg T

P Ž T - t . q cE

½

Ł Tks t ␾ Ž X k . y 1

␣y1

5

,

where c ) 0, and ␾ is a real-valued, nonnegative function satisfying

␣ J ␾ Ž x . Q a Ž dx . - ⬁

Ž 59.

H

Û G 0, the stopping and ␣ f 0, 14 . Then, for appropriately chosen threshold R time ˆk G RÛ TˆB s inf k G 0 < R

Ž 60.

½

5

with

Ž 61. Rˆk s

␾ Ž Xk . L Ž Xk . 1y␳

Ž Rˆky1 q ␳ . ,

ˆ0 s k s 1, 2, . . . , R

␾ Ž X0 . ␲ 1y␲

,

is Bayes optimal w i.e., it solves Ž58. with the geometric prior Ž44.x . Moreover, Û s 0. if c G 1, then R PROOF. The key to this theorem is the following result, a proof of which is found in Appendix D. LEMMA 5.2. Consider the model of Section 4 with the constant ␣ s H␾ dQ a â given by / 1 and the post-change distribution Q a replaced with Q

â Ž dx . s Q

Ž 62.

␾ Ž x . Q a Ž dx . ␣

.

Let Pˆ denote the new probability measure on Ž ⍀, F . defined by this change. Then Ž58. is solved by the solution to the following problem: q

Ž 63.

inf Tg T

ˆ PˆŽ T - t . q cE

½


5

,

ˆ⭈4 denotes expectation under P. ˆ where E Since TˆB is the optimal stopping time for Ž63. Žvia Theorem 4.1., the theorem follows. I We now turn to the analogous problem in the minimax setting of Section 2. We have not been able to generalize Theorem 2.1 to the case of a direct q substitution of Ž57. for ␣ ŽTytq1. in Ž5.. However, from Ž2. we see that an

2192

H. V. POOR

alternative substitution of interest is to replace Ž1. with T ly1

Ý Ł ␾ Ž Xj . ,

Ž 64.

lst jst

where ␾ is a positive, real-valued measurable function on the range of the observation X k such that H␾ Ž x . Q aŽ dx . - ⬁ and such that ␾ Ž X k . LŽ X k . has no atoms under Q b . That is, we can consider the following problem: inf dˆŽ T .

Ž 65.

Tg T

subject to f Ž T . G ␥ ,

where T ly1

Ž 66.

dˆŽ T . s sup ess sup Et tG1

½Ý Ł

lst jst

5

␾ Ž X j . Fty1 .

Of course Ž65. reduces to Ž7. when ␾ ' ␣ . In this connection, we consider stopping times of the form Tˆh s inf t s 0, 1, 2, . . . < Sˆk G h 4 ,

Ž 67. with k

Ž 68.

Sˆk s max

1FjFk

ž

Ł ␾ Ž Xl . L Ž Xl . lsj

/

s ␾ Ž X k . L Ž X k . max Sˆky1 , 1 4 ,

k G 1,

where Sˆ0 s 0 and L s dQ ardQ b . We then have the following result. THEOREM 5.3. Consider the probability model of Section 2 and suppose that h G 0 and P⬁Ž ␾ Ž X 1 . L Ž X 1 . ) 1 . ) 0.

Ž 69.

Then Tˆh solves Ž65. with ␥ s f ŽTˆh .. Moreover, if h ) 1, the quantities f ŽTˆh . and dˆŽTˆh . are given by Ž17. and Ž18., respectively, with N replaced with the random variable

Ž 70.

½

ˆ s min n G 1 N

n

Ý

5

log L Ž X l . q log ␾ Ž X l . f Ž 0, log h . .

ls1

PROOF. Analogously to the situation with Theorem 5.1, Theorem 5.3 follows as a corollary to Theorems 2.1 and 3.1, after we note the following result, which is proved in Appendix D. LEMMA 5.4. Consider the model of Section 2 with ␣ chosen as in Lemma â as in Lemma 5.2. Suppose T g T is such 5.2 and with Q a replaced with Q that 0 - f ŽT . - ⬁. Then q

Ž 71.

ˆt dˆŽ T . s sup ess sup E tG1

½


5

Fty1 ,


2193

ˆt⭈4 denotes expectation under the measure Pˆt on Ž ⍀, F . corresponding where E â . to a change point at t and a postchange distribution Q The theorem follows immediately. I 6. Conclusion. We have considered the quickest detection problems of Lorden and Shiryayev when the linear penalty on detection delay is replaced with a possibly path-dependent exponential delay penalty. We have seen that each of these problems is solved by replacing, in the corresponding linearpenalty optimal stopping rules, the likelihood ratio between pre- and postchange distributions with a scaled version of itself. We have also explored the issue of performance analysis in each of these problems. In the minimax problem this involves separate analysis of the exit statistics of the optimal stopping rule; whereas in the Bayesian problem this involves the computation of the payoff function, which is an adjunct to the determination of the optimal stopping rule. In each of the problems considered, we have examined the tradeoff of opposing performance indices: dŽT . and f ŽT . in the minimax formulation q and P ŽT - t . and EŽ ␣ ŽTytq1. y 1.r␣ y 14 in the Bayesian formulation. In the minimax case, the tradeoff was effected by minimizing one of these indices with a constraint on the other, and in the Bayesian formulation we opted for the minimization of a linear combination of the two indices. These optimization criteria were chosen because they are the traditional ones for their respective change-detection formulations. However, we could of course have considered alternatives such as a linear combination of the performance indices in the minimax case, or a false-alarm constrained minimization in the Bayesian case. As with their linear-delay counterparts, these alternative problems should have essentially the same solutions Žaside from the choice of threshold. as the problems presented here w cf. Theorem 4.10 of Shiryayev Ž1978.x . Similarly, one might also introduce other combinations of performance indices, such as trading false-alarm probability against minimax delay w e.g., Yakir Ž1996.x . Several other problems of interest are suggested by this work. For example, it may be interesting to explore formal connections between the minimax and Bayesian formulations of this problem, as has been done in the case of linear delay penalty in Beibel Ž1996., Bojdecki and Hosza Ž1984. and Ritov Ž1990.. Also, continuous-time versions are of interest; in the minimax case, they formalize the connection between results such as Ž26. and Ž27. and optimal stopping solutions w cf. Beibel Ž1996.x ; and in the Bayesian case, they can give rise to closed-form solutions for the payoff w e.g., Theorem 4.9 of Shiryayev Ž1978.x . Moreover, continuous-time solutions are sometimes particularly simple when viewed as ‘‘generalized parking’’ problems w see, e.g., Beibel Ž1994. or Beibel and Lerche Ž1997.x . Finally, alternative ways of invoking exponential penalties may be of interest. For example, it is common to consider problems of optimal stopping in which the rewards are exponen-

2194

H. V. POOR

tially discounted by a deflator w see, e.g., Dubins and Teicher Ž1967.x . A similar formulation in which the deflator changes at an unknown time could be used to model an exponential cost of delay in quickest detection problems. APPENDIX A. Proofs for Section 2. PROOF OF LEMMA 2.3. This result is the exponential-delay analog of Lemma 3 of Moustakides Ž1986., the proof of which can be adapted straightforwardly to this case. In particular, we define q

b t Ž T . s Et

Ž 72.

␣ ŽTytq1. y 1

½

5

Fty1 .

␣y1

On applying the identity Ž2. we can write

Ž 73.

bt Ž T . s

⬁

kyt

Ý Ý ␣ l Pt Ž T s k