Certain Results Concerning Filtering and Control ... - Semantic Scholar

Certain Results Concerning Filtering and Control of Diusions in Small White Noise Charalambos D. Charalambous Robert J. Elliotty

Abstract

controlled diusion models. In particular, under each hypothesis our model is described during a time interval, The purpose of this talk is twofold. First, we examine in [0 equations of the form detail the binary hypothesis decision and/or estimation ; T ]; T > 0, by stochastic p i i i i problem using a risk-sensitive cost criterion, when the dx (t) = f (x (t))dt +  (xi (t))dwi (t); xi (0) 2 0; is a small noise covariances. Here, fxi (s); 0  s  tg; fy(s); 0  this Feynman-Kac stochastic equation. state, observation processes, respectively. Second, we examine in detail the behaviour of our cal- sfwi (tsg);, 0are the s  and fb(s); 0  s  tg, are independent culations, in the limit as the covariances of the random standard Wienertgprocesses, which are independent of the inputs tend to zero. The procedure employs large devi- random variable, xi (0). Under of the two-hypothesis ations techniques. This approach enables us to establish an exponential-of-integral cost each function of the form relations between stochastic and deterministic methods in n R 1 tackling the binary decision problem. The latter reveals J0i;;T (^ui ) = E~ i exp  0T jQi; 2 (xi (t)
a natural formulation of the binary decision problem in ?x^i(t; y))j2 dt ; i > 0; terms of an H 1 -disturbance attenuation framework. is used to grade the performance of each estimator, x^i(t; y); i = 1; 2. Speci cally, an estimator x^i (t; y) of the Key words. Hypothesis Testing, Filtering, Risk- state at time t, xi (t), given the data fy(s); 0  s  tg, Sensitive, Error Probabilities, H 1 Estimation and Con- is a non-anticipative functional of the data up to time trol. t. The optimal estimator x^i;(t; y), is obtained by mini , (see imizing over the set of admissible estimators, U^ad De nition2.3), its cost function J0i;;t (^xi ): J0i;;t (^xi;) = inf^ J0i;;t (^xi ); i = 1; 2: (3) x^ 2U This talk discusses the following typical problem. We are given hypothesis Hi ; i = 1; 2, each representing ob- We now wish to describe the traditional decision problem, served data, fy(s); 0  s  tg, that originated from dif- for uncontrolled diusions in order to obtain an analogy to ferent models. The decision strategist receives the data the risk-sensitive approach. Let fy(s); 0  s  tg, repreand wishes to decide which of the two hypothesis is in ef- sent the observed data collected during the time interval, fect. This problem arizes in many disciplines within engi- [0; t], and de ne neering, such as, communications, fault detection, signal ^hi (t) =: E~ i hi (xi (t))jfy(s); 0  s  tg ; 1  i  2: (4) and/or image processing. The underlying mathematical models can be diverse as well, for example, controlled and The traditional least-squares likelihood-ratio (LR) test for uncontrolled diusions, jump point process, discrete pro- the above problem can be formulated by  cess. ^ i0;t = exp 1 R0t h^ i;0 (s)N ?1 dy(s)  In this talk we focus our attention on uncontrolled and  (5) R ? 21 0t h^ i;0 (s)N ?1 h^ i (s)ds ; i = 1; 2:  Department of Electrical Engineering, McGill Univeri

1 Introduction

i

i ad

sity, Montreal, P.Q. Canada H3A 2A7, email: [email protected]. This author's work was supported by the This represents the LR of each hypothesis, Hi , against Natural Science and Engineering Research Council of Canada the null hypothesis, 8 p under grant OGP0183720. < dxi (t) = f i (xi (t))dt +  i (xi (t))dwi (t); y Department of Mathematical Sciences, University of Alberta, Edmonton, P.A., Canada T6G 2G1, email: rel- H0i : : [email protected] dy(t) = pN 21 db(t);

for 1  i  2. The general M-hypothesis problem is a straight forward extension. For both decision and performance one chooses a threshold, i , associated with each ^ i0;t , and performs the following test: ^ 20;t > 2 ; ^ 10;t 1

^ 20;t < 2 ; ^ 10;t 1

and

(6)

^ 20;t

2 ^ 10;t > 1 and in ^2 2 if ^ 10;t =

1 , then 0;t

We decide in favour of hypothesis H2 if ^ 2;t

Similarly, one can show that the Cherno bounds can be represented in terms of expectations with respect to a Wiener measure of functionals of ^ i0;t ; i = 1; 2. In this talk we address the following issues: 1. Formulate the LR test and Cherno bounds using risk-sensitive estimators; 2. Describe the behaviour of the LR and Cherno bounds, in the limit as  ! 0; 3. Generalize 1, 2 above to controlled diusions; 4. Discuss the importance of the risk-sensitive parameter i ; i = 1; 2.

favour of hypothesis H1 if ^ 10 <

1 ; 0 H1 and H2 are equally probable. Once the decision strategy is established the next step is to derive an assessment on the error performance of the LR test. In our case there are two possibilities of making errors. The rst is called a \false alarm" and is denoted by PF , and the second is called a \miss" and is denoted by PM . However, in2 theory, to compute these one needs 2.1 Absolute Continuity of Measures ^ the distribution of ^ 10 , and this is often unknown. Hence 0 one is forced to consider bounding techniques which can Notation 2.1 be computed explicitly. The bounding technique we are concerned with is the so-called Cherno bound, (see [8]). Ik denotes k  k identity matrices; The extension of this bound to nonlinear binary decision \0" denotes transposition of a matrix; problems as presented in [6] is the following: !s # "  \(
)i " and \(
)i;j " denote the ith and ijth components of 2 2  ^ 

0 ;t 2 ~ vectors and matrices, respectively; PF  0min exp ? s log( ; (7) E ) s1

1 ^ 10;t \L(V1 ; V2 )" denotes the space of linear transformations of a vector space V1 into a vector space V2 ; " !s+1 #   2 2 ^ PM  ?1min exp ?s log(

1 ) E~ 1 ^ 01;t : (8) For  :





< S i;D (s); 1 i > ; (18)

i;D i i @ i;D @t W (x;1t) = ?Dx W (x; t):f (t; x; u (t; y )) i 2 i i ? 1 ? 2i jNn 2 (y_ (t) ? h (t; x))j + ` (t; x; u (t; y)) o + supv(t)2Rn ?Dx W i;D (x; t):i (xi ) 12 v(t) ? 21 i jv(t)j2 ;

W i;D (x; s) = 1 i (x); i

having representation

W i;Dn(x; t) = Jwi;0 (t; ui ; di;w ) =: supd 2L2 ([s;t];R ) 1 i (xi (s)) + R t ?`i (xi ( ); ui (; y ))  s i h 1 i 2 ? 1 2 ? 2 jdw ( )j + jN (y_ ( ) ? hi (xi( )))j2 d ; xi (t) = x ; x_ i ( ) = f i (xi ( ); ui (; y)) +i (xi ( ))diw ( ); xi (t) = x;  2 [s; t); i w

Proof. See [12].

3.1

y_ 2C~ ([s;T ];Rd )

(15) where W i;D (
) is the unique viscosity solution of the forward HJ equation

for i = 1; 2. i the cost function has the equivaMoreover, for ui 2 Uad lent representation R

=




dqi; (x; t) = Li +  `i (x; ui ) qi; (x; t)dt + 1 hi (x)qi; (x; t):N ?1 dy(t);

R

 I i; (s; u; i; ) I (s; ui ) =: lim 0 !0 i  = sup < W i;D (t); S i;D (t) >

n

i

i

Limiting Deterministic Dynamic Games

and S i;D (
) is the unique viscosity solution of the backward Next we wish to introduce the main theorem derived in HJ equation [12]. An application of this theorem will enable us to i;D i i @ i;D describe the behaviour of the LR and error probabilities, @t S (x; t) + Dx S (x;1t):f (x; u (t; y )) ? i i 1 in the limit as the covariances of the random inputs tend +` (x; un(t; y)) ? 2 jN 2 (y_ (t) ? hi (x))j2 o to zero. + supv(t)2R Dx S i;D (x; t):i (x)vi(t) ? 21 jvi (t)j2 = 0; S i;D (x; T ) = 'i (x); Notation 3.3 De ne the quantities i

n

 C~ ([0; T ]; Rd ) =: z; z 2 C ([0; T ]; Rd ) \ C 1 ([0; T ]; Rd ) ;

< 1 ; 2 >= sup f1 (x) + 2 (x); x2Rn

1 ; 2 2 C (Rn )g :

These arize in the application of the Laplace-Varadhan lemma [9, 5]; the second represents the \sup pairing" between 1 and 2 .

i

having representation

S i;D (x; t) =nJRsi;0 (?t; ui ; di;s  ) =: supd 2L2 ([t;T ];R ) tT `i (xi ( ); ui (; y)) i h ? 21 jdis ( )j2 + jN ? 12 (y_ ( ) ? hi (x( )))j2 d +'i (x(T )); xi (t) = x ; x_ i ( ) = f i (xi( ); ui (; y)) +i(xi ( ))dis( ); xi (t) = x;  2 (t; T ]: i s

i

n

Moreover, I i;0 (s; ui; ) is equivalent to the following dy- Theorem 4.2 The risk-sensitive LR is given by R namic game: L^ 20;;t inf x^2 2U^2 RR q2; (z; t)dz ; (20) = J i;0 (ui; ) = inf u 2U sup supx (s)2R 2 L^ 10;;t inf x^1 2U^1 R q2; (z; t)dz nRb 2L? ([s;T ];R ) supd 2L2 ([s;T ];R ) sT `i (xi (t); ui (t; y)) o where q i; (
) is a solution of the Feynman-Kac DMZ equa  ? 21 jdis (t)j2 + jbs (t)j2 dt + 1 i (xi (s)) + 'i (xi (T )) ; tion   x_ i (t) = f i (xi(t); ui (t; y)) + 1i (xi (t))dis(t); xi (s) = x; i; (x; t) = Li +  jQi; 21 (x ? x^i (t; y ))j2 q i; (x; t)dt dq i i  2 y_ (t) = h (x (t)) + N bs (t); t 2 (s; T ]: + 1 hi (x)qi; (x; t):N ?1 dy(t); (21) These hold for i = 1; 2. qi; (x; 0) = 0i; (x); Proof. See [12]; in this talk we shall give a derivation i = 1; 2. which is independent of those appeared in the literature. Proof. Follows from above comments and Theorem 3.2. n

ad

i

i ad

i s

d

s

i

n

n

ad

n

i

i

i

4 RS Likelihood-Ratio

Theorem 4.3 The performance bounds are given by 

 2 

o

n

    De nition 4.1 Consider (1), (2). The risk-sensitive es- PF  exp ?s log 1 E L^ 20;;t s L^ 10;;t 1?s ; (22) i  timator of the state at time t, namely, x (t), given the data, fy(s); 0  s  tg, is de ned by   2  n s+1  1; ?s o   i;  i; i ; (23) L^ 0;t x^ (s; y) = arg inf^ J0;t (^x ); i = 1; 2: (19) PM  exp ?s log 1 E L^ 20;;t i x^i 2Uad

By Remark 2.4, under the system ( i ; Fti ; P i ), we have n

R

?

Proof. The derivation will be given in the nal paper.

J0i;;t (^xi ) = E i i0;t exp  0t jQi; 12 xi (s)

? x^i (s; y) j2 ; i = 1; 2: i

Moreover, if the measure-valued process de ned similar to Notation 3.1 has a density function n

R

?

qi; (z; t)dz = E i Ix(t)2dz i0;t exp  0t jQi; 12 xi (s)

?x^i(s; y) j2 jFty ; i = 1; 2; then we have

J0i;;t (^xi ) = E i

i

Z

x^i; (t; y) = arg inf E i i x^i 2Uad

R



n

qi; (z; t)dz ;

Z

R



n

where E denotes expectation with respect to the Wiener measure y (dy).

qi; (z; t)dz ; i = 1; 2:

Example 4.4 Consider the linear-Gaussian system

dxi(t) = F i (t)xi(t)dt + Gi (t)dwi (t); dy(t) = H i (t)xi(t)dt + N (t) 21 db(t); for i = 1; 2. Then, 



exp ? 12 jP i (t)? 21 (x ? ri (t))j2 i; 1 q (x; t) =  ^ i0;t 1 i 2 2 (2) jP (t)j n

Z i t

 exp 2

0

 ? i
i Tr P (s)Q ;

R

^ i0;t = exp 0t H i (s)ri (s):N ?1 dy(s)  R ? 12 0t jN ? 21 H i (s)ri (s)j2 ds ;

Note that qi; (
) is by construction an in nite-dimensional  h n information state and so, formally we may write x^i (t) = i;1 (^xi ) = E~ i exp  R t jQi; 12 (ri (s) ? x^i (s))j2 i i i; J 0 ;t 2  (t; fy(s); 0  s  tg) = ~ (t; q (x; t)); 8t 2 [0; T ]. 0 i (s)Qi ) ds
; Moreover, if qi; (
) is expressed in terms of a nite-number + Tr ( P of quantities such that the dependence of qi; (
) on fx^i (s);  s  T g is only through them, then these nite- for i = 1; 2. It can be shown that x^i (t) = ri (t); 8t 2 [0; t], number of quantities are sucient statistics for the risk- where sensitive ltering problem. That is, fx^i; (t; y); t 2 [0; T ]g i; i ;i i i;0 ?1 i is a nite-dimensional risk-sensitive estimator, which is a dx^ (t) = F (t)^x (t)dt + P (t)H (t)N d^b (t); d^bi (t) = dy(t) ? H i (t)^xi;(t)dt; x^i; (0) = ; functional of the sucient statistics. i

P_ i (t) = F i (t)P i(t) + P i (t)F i;0 (t) + i P i (t)QiP i (t) ? P i (t)H i;0(t)N ?1 H i (t)P i (t) + Gi (t)Gi;0 (t); i P (0) = P0i ; for i = 1; 2. The LR is given by R

?




^ 20;t  exp 22 0t Tr P 2 (s)Q2 R : ^ 10;t  exp 21 0t Tr (P 1 (s)Q1 ) With the above calculations at hand, in this talk we will address the issues outlined in the introduction.

References [1] D. Jacobson, \Optimal stochastic linear systems with exponential performance criteria and their relation to deterministic dierential games," IEEE Trans. Automat. Contr., vol. 18, pp. 124{131, 1973. [2] A. Bensoussan and J. H. van Schuppen, \Optimal control of partially observable stochastic systems with an exponential-of-integral performance index," SIAM J. Contr. Optim., vol. 23, pp. 599{613, 1985. [3] M. James, \Asymptotic analysis of nonlinear stochastic risk-sensitive control and dierential games," Mathem. of Contr., Sig., and Syst, vol. 5, no. 3, pp. 401{417, 1992. [4] P. Whittle, \A risk-sensitive maximum principle: The case of imperfect state observations," IEEE Trans. Automat. Contr., vol. 36, pp. 793{801, 1991. [5] M. James, J. Baras, and R. Elliott, \Risk-sensitive control and dynamic games for partially observed discrete-time nonlinear systems," IEEE Trans. Automat. Contr., vol. 39, pp. 780{792, 1994. [6] J.L Hibey, D.L. Snyder, and J.H. van Schuppen, \Error Probability Bounds for Continuous-Time Decision Problems," IEEE Trans. Informat. Theory, vol. 24, pp. 608{622, 1978. [7] W.H. Fleming and E. Pardoux, \Optimal control for partially observed diusions," SIAM J. Contr. Optim., Vol. 20 , pp. 261-285, 1992. [8] H.L. Van Trees, Detection Estimation and Modulation Theory: Part I. 1968. [9] M. Friedlin and A. D. Wentzell, Random Perturbations of Dynamical Systems. NY: Springer-Verlag 1984. [10] C. Charalambous, \The role of information state and adjoint in relating nonlinear output feedback risksensitive control and H1 robust control," TR 94/95 014, Measurement and Contr. Resear. Cent., Idaho State Univer., 1995. [11] M. R. James and J. S. Baras, \Nonlinear ltering and large deviations: A PDE-control theoretic approach," Stochastics, vol. 23, pp. 391{412, 1988.

[12] C. Charalambous, \The Role of Information State and Adjoint in Relating Nonlinear Output Feedback Risk-Sensitive Control and Dynamic Games ," IEEE Trans. Automat. Contr., vol.42, pp.1163-1170, Aug. 1997.