THE STOCHASTIC MAXIMUM PRINCIPLE IN OPTIMAL CONTROL OF DEGENERATE DIFFUSIONS WITH NON SMOOTH COEFFICIENTS FARID CHIGHOUB y , BOUALEM DJEHICHE z , AND BRAHIM MEZERDIx Abstract. For a controlled stochastic di¤erential equation with a …nite horizon cost functional, a necessary conditions for optimal control of degenerate di¤usions with non smooth coe¢ cients is derived. The main idea is to show that the SDE S admit a unique linearized version interpreted as its distributional derivative with respect to the initial condition, we use technique of Bouleau-Hirsch on absolute contunuity of probability measures in order to de…ne the adjoint process on an extension of the initial probability space. Key words. stochastic di¤erential equation, optimal control, maximum principle, non smooth coe¢ cients. AMS sub ject classi…cation. 93 E20, 65 N30
1. Introduction. Let ( ; F; Ft ; P ) be a …ltered probability space, on which a d-dimensional Brownian motion (Bt ) is de…ned with the …ltration (Ft ), we consider a controlled stochastic di¤erential equation dxt x0
= b (t; xt ; ut ) dt + (t; xt ) dBt ; = ;
for t 2 [0; T ] ;
(1.1)
where u is a suitable control process adapted to the …ltration (Ft ) : The solution xt of the above SDE is called the response of the control ut ; and (xt ; ut ) is called an admissible pair. The objective of the optimal control problem is to minimize the cost functional "Z # T
J (u) = E
f (t; xt; ut ) dt + g (xT ) ;
(1.2)
0
where b, ; f; and g are given maps taking values in Euclidean spaces Rd ; Rd d ; R and R, respectively. A control process that solves this problem is called optimal. A classical approach for control problem is to derive necessary conditions satis…ed by an optimal solution. The argument is to use an appropriate calculus of variations of the cost functional J (u) ; with respect to the control variable in order to derive a necessary condition of optimality. The maximum principle initiated by Pontryagin, states that an optimal state trajectory must solve a Hamiltonian system, together with a maximum condition of a function called a generalized Hamiltonian. The Pontryagin’s maximum principle was derived …rst for deterministic problems. The …rst version of the stochastic maximum principle was established by Kushner [14] (see also Bismut [4], Bensoussan [3] and Haussmann [12]). However, at that time, the results where essentially obtained under the condition that is independent Partially supported by PHC Tassili 07 MDU 705 of Applied Mathematics, University Med Khider, Po. Box 145 Biskra (07000) Algeria (
[email protected]) z Dept of Mathematics, Royal Institute of Technology, S 100 44, Stockholm, Sweden. (E-Mail:
[email protected]) x Laboratory of Applied Mathematics, University Med Khider, Po. Box 145 Biskra (07000) Algeria (E-mail:
[email protected]) y Laboratory
1
of control and b, ; f and g are bounded, continuously di¤erentiable in the space variable: The basic idea is to perturb an optimal control and to use some sort of Taylor expansion of the state trajectory around the optimal control. The maximum principle is expressed in terms of an adjoint process and a variational inequality as follows. Let pt ; qt be processes adapted to the natural …ltration of Bt ; and satisfying the backward stochastic di¤erential equation dpt = Hx (t; xt ; ut ; pt ) + qt dBt ; pT = gx (xT ) ;
(1.3)
where the Hamiltonian H is de…ned by H (t; x; u; p) = p:b (t; x; u)
f (t; x; u) :
(1.4)
The maximum principle then states that, if (^ xt ; u ^t ) is an optimal pair, then one must have max H (t; x ^t ; ut ; pt ) = H (t; x ^t ; u ^ t ; pt ) u
a:e: t 2 [0; T ] ; P
a.s.
(1.5)
The …rst version of the stochastic maximum principle when the di¤usion coe¢ cient depends explicitly on the control variable and the control domain is not convex; was obtained by Peng [16], in which he studied the second order term in the Taylor expansion of the perturbation method arising from the Itô integral. He then obtained a maximum principle for control-dependent di¤usion, which involves in addition to the …rst-order adjoint process, a second-order adjoint process. Recently, the smoothness conditions on the coe¢ cients have been weakened. The …rst result has been derived by Mezerdi [15], in the case of a SDE with a non smooth drift, by using Clarke generalized gradients and stable convergence of probability measures. The method performed by Bahlali-Mezerdi-Ouknine in [2] is intimately linked to the Krylov estimate, they proved that (1:3) and (1:5) remain true when the coe¢ cients are only Lipschitz but not necessarily di¤erentiable and the di¤usion coe¢ cient is uniformly elliptic. However, If b, are Lipschitz continuous and f and g are C 1 in space variable, Bahlali-Djehiche-Mezerdi [1] proved a stochastic maximum principle in optimal control of a general class of degenerate di¤usion processes. This result was established by using techniques introduced by Bouleau and Hirsch [5; 6]. This property (on absolute continuity of probability measures) was the key fact to de…ne a unique linearized version of the stochastic di¤erential equation (1:1) : The objective of this paper is to extend the results of [1] to the case where b, ; f and g are only Lipschitz continuous. We prove a stochastic maximum principle for this problem. The idea is to de…ne a slightly di¤erent stochastic di¤erential equation de…ned on an enlarged probability space, where the initial condition will be taken as a random element. This paper is organized as follows. The assumptions, notations and formulation of the problem are given in section 2. In section 3, we de…ne a family of smooth control problems with approximate the original one, and we establish the stochastic maximum principle for the original control problem. 2. Assumptions and the main result. 2
2.1. Assumptions. In this section we will make some preliminaries. For any x; y 2 Rd ; we use x:y to denote the inner product of these two vectors. We put @x = @ ; and note that if : Rd ! Rd then @x , @xj i i;j=1;:::;d 2 Rd d . @xj j=1;:::;d From now on, let = C0 R+ ; Rd be the space of continuous functions w such that w (0) = 0; endowed with the topology of uniform convergence on compact subsets of R+ . F is the Borel -…eld over ; P is the Wiener measure on ( ; F ) ; (Ft )t 0 is the …ltration of coordinates augmented with P -null sets of F: We de…ne the canonical process Bt (w) = w (t) ; for all t 0: Thus, ; F; (Ft )t 0 ; P; Bt is a Brownian motion. Let T be a …xed strictly positive real number, we consider the set of admissible controls U de…ned as the collection of A-valued, Ft -adapted measurable process u: = fut : 0 t T g. A is a given closed set in some Euclidean space Rp : Now, for each u 2 U; let xt be the solution of the controlled stochastic di¤erential equation dxt = b (t; xt ; ut ) dt + (t; xt ) dBt ; x0 = ;
for t 2 [0; T ] ;
and the objective is to minimize over controls u 2 U the cost functional "Z #
(2.1)
T
J (u) = E
f (t; xt; ut ) dt + g (xT ) :
(2.2)
0
We introduce the standing assumptions: Maps b : [0; T ] Rd U ! Rd ; : [0; T ] Rd ! Rd d ; f : [0; T ] Rd U d ! R; g : R ! R; satisfy the following: b; f are B [0; T ] Rd U -measurable, d is B [0; T ] R -measurable, and g is B Rd -measurable, where B (G) is the Borel -…eld of the metric space G. There exist M > 0; such that for all (t; x; y; a) in R+ Rd Rd A jb (t; x; a) b (t; y; a)j + j (t; x) (t; y)j M jx yj ; jf (t; x; a) f (t; y; a)j + jg (x) g (y)j M jx yj ; jb (t; x; a)j + j (t; x)j M (1 + jxj) ; jf (t; x; a)j + jg (x)j M (1 + jxj) ;
(2.3) (2.4) (2.5) (2.6)
b (t; x; a) and f (t; x; a) are continuous in a uniformly in (t; x) :
(2.7)
and
Assumptions (2:3) and (2:5) guarantee the existence and uniqueness of strong solution for (2:1) ; such that for any p > 0, E
p
sup jxt j
< +1:
0 t T
Since b, j (the jth column of the matrix ), f and g are Lipschitz continuous functions in the state variable, they are di¤erentiable almost everywhere in the sense 3
of Lebesgue measure (Rademacher Theorem see [8]): Let us denote by bx , gx any Borel measurable functions such that
x;
fx and
@x b (t; x; a) = bx (t; x; a) dx-a:e:; @x f (t; x; a) = fx (t; x; a) dx-a:e:; @x (t; x) = x (t; x) dx-a:e:; @x g (x) = gx (x) dx-a:e: It is clear that these almost everywhere derivatives are bounded by the Lipschitz constant M: Finally, assume that bx (t; x; a) ; fx (t; x; a) are continuous in a uniformly in (t; x) : We assume throughout this paper that an optimal control u ^ of the control problem assosiated with (2:1) and (2:2) exists. That is J (^ u) = inf J (u) : u2U
R Let h be a continuous positive function on Rd such that h (x) dx = 1 and R 2 @f 2 L2 (hdx) ; where jxj h (x) dx < 1: We set D = f 2 L2 (hdx) ; such that @xj @f denotes the derivative in the sense of distributions. @xj Equipped with the norm
kf kD
2 Z X Z = 4 f 2 hdx + Rd
1 j d d R
@f @xj
3 12
2
hdx5 ;
D is a Hilbert space, which is a classical Dirichlet space (see [6]). Moreover D is a 1 subset of the Sobolev space Hloc Rd : et (x; w) = Let e = Rd ; and Fe the Borel -…eld over e and Pe = hdx P: Let B e e e Bt (w) and Ft the natural …ltration of Bt augmented with P -negligible sets of Fe: It is clear that
e ; Fe; Fet
t 0
et ; Pe; B
x ~t de…ned on the enlarged space di¤erential equation
is a Brownian motion. We introduce the process
e ; Fe; Fet
t 0
et ; Pe; B
solution of the stochastic
et ; d~ xt = b (t; x ~t ; u ~t ) dt + (t; x ~ t ) dB x ~0 = ;
(2.8)
associated to the control u ~t (x; w) = ut (w) : Since the coe¢ cients b and are Lipschitz continuous and grow at most linearly, equations (2:8) has a unique Fet -adapted solution, with continuous trajectories. Equations (2:1) and (2:8) are almost the same except that uniqueness of the solution of (2:8) is slightly weaker. One can easily prove that the uniqueness implies that for each t 0; x ~t = xt ; Pe-a.s: 2.2. The main result. The main result of this paper is stated in the following Theorem. 4
Theorem 2.1. (Stochastic maximum principle) Let (^ u; x ^) be an optimal pair for the controlled system (2:1) and (2:2) ; then there exist an Ft -adapted process (the adjoint process) satisfying
pt :=
2 T Z e4 E
(s; t) :fx (s; x ^s ; u ^s ) ds +
(T; t) :gx (^ xT )
t
3
Fet 5 ;
(2.9)
for which the following stochastic maximum principle holds: dt-a.e; Pe-a.s:
H (t; x ^t ; u ^t ; pt ) = max H (t; x ^t ; a; pt ) a2A
(s; t) ; (s (
(2.10)
t) is the fundamental solution of the linear equation
d (s; t) = bx (s; x ^s ; u ^s ) : (s; t) ds +
P
1 j d
(t; t) = Id:
j x
esj ; (s; x ^s ) : (s; t) dB
(2.11)
and the Hamiltonian H is de…ned by H (t; x; u; p) = p:b (t; x; u) Here
denotes the transpose of the matrix
f (t; x; u) :
(2.12)
:
3. Proof of the main result. Let us recall some preliminaries and notation on the Bouleau-Hirsch ‡ow property, which will be applied in this paper to establish the stochastic maximum principle. Theorem 3.1. (The Bouleau-Hirsch ‡ow property) For Pe-almost every w (1) For all t 0; x ~t is in Dd . (2) There exists a Fet -adapted, GLd (R)-valued continuous process e t such t 0
that for every t
0
@ (x (w)) = e t ( ; w) @x t
dx-a:e:
@ denotes the derivative in the sense of ditributions. @x (3) The distributional derivative e t is the unique solution of the linear (matrix) stochastic di¤ erential equation 8 P j e j ; s t; < d e (s; t) = bx (s; x ~s ) : e (s; t) dB ~s ; u ~s ) : e (s; t) ds + x (s; x s 1 j d (3.1) : e (t; t) = Id; where
where bx and xj are versions of the almost everywhere derivatives of b and j : Remark 3.2. It is proved in [5] that the image measure of Pe by the map x ~t is absolutely continuous with respect to the Lebesgue measure. Let us recall Ekeland’s variational principle which will be used in the sequel. Lemma 3.3. (Ekeland principle [9]) Let (S; d) be metric space and : S ! R [ f+1g be lower-semicontinuous and bounded from below. For " 0; suppose 5
u" 2 S satis…es
(u" )
inf
(u) + ": Then for any
u2S
that
> 0; there exists u 2 S such
(u" ) ;
u d u ; u" u
; "
(u) +
d u; u
; for all u 2 S:
From now on, let us assume that the initial time s = 0 and initial state system are …xed. De…ne a metric on the space U of admissible controls d (u: ; v: ) = P~ f(t; w) 2 [0; T ]
: ut (w) 6= vt (w)g ;
of the
(3.2)
where P~ is the product measure of the Lebesgue measure and P: Since A is closed, it can be shown similarly to [10] ; that U [0; T ] is a complete metric space under d: Lemma 3.4. (1) For any p 0; there is a constant C1 > 0 such that for any u, v 2 U along with the corresponding trajectories xu , xv it holds that 1
2p
sup jxut
E
xvt j
0 t T
C1 d (u; v) 2 :
(2) The cost functional J : (U; d) ! R is continuous. More precisely, there is a constant C2 > 0 for any u, v 2 U such that jJ (u)
1
J (v)j
C2 d (u; v) 2 :
See [15; 20] for the proof. 3.1. The maximum principle for a Family of perturbed control problems. Now, let ' be a nonZ negative smooth function de…ned on Rd ; with support in the unit ball such that functions by convolution
' (y) dy = 1: For n 2 N de…ne the following smooth
Rd
bn (t; x; a) = nd
Z
b (t; x
y; a) ' (ny) dy;
Rd n
f (t; x; a) = n
d
Z
f (t; x
y; a) ' (ny) dy;
Rd
j;n
(t; x) = nd
g n (x) = nd
Z
Z
j
(t; x
y) ' (ny) dy;
Rd
g (x
y) ' (ny) dy:
Rd
In the next Lemma we list some properties satis…ed by these functions. Lemma 3.5. (1) The functions bn (t; x; a), j;n (t; x) ; f n (t; x; a) ; and g n (x) are Borel measurable bounded functions and Lipschitz continuous with constant K in x: 6
(2) There exists a constant C positive independent of t, x and n such that for every t in [0; T ] jbn (t; x; a)
b (t; x; a)j +
j;n
jf n (t; x; a)
f (t; x; a)j + jg n (x)
j
(t; x)
(3) The functions bn (t; x; a) ; f n (t; x; a) ; in x; and for all t in [0; T ] ; we have
j;n
(t; x) and g n (x) are C 1 -functions
lim bnx (t; x; a) = bx (t; x; a)
dx-a:e:;
lim fxn (t; x; a) = fx (t; x; a)
dx-a:e:;
n!+1
lim
n!+1
j;n x
j x
(t; x) =
(t; x)
lim gxn (x) = gx (x)
1 and R > 0 ZZ sup jbnx (t; x; a)
p
bx (t; x; a)j dxdt = 0;
n!1 a2A B(0;R) [0;T ]
lim
ZZ
dx-a:e:;
dx-a:e:
n!+1
lim
C ; n
C : n
g (x)j
n!+1
(4) For every p
(t; x)
p
sup jfxn (t; x; a)
fx (t; x; a)j dxdt = 0:
n!1 a2A B(0;R) [0;T ]
where B (0; R) denotes a ball in Rd of radius R: Proof. Statements (1), (2) and (3) are classical facts (see [11] for the proof). (4) is proved as in [1]. Now, consider the process yt ; t 0; solution of the stochastic di¤erential equation, et by de…ned on the enlarged probability space e ; Fe; Fet ; Pe; B t 0
dyt = bn (t; yt ; ut ) dt + y0 = ;
and de…ne the cost functional
n
et ; (t; yt ) dB
2 T 3 Z e 4 f n (t; yt ; ut ) dt + g n (yT )5 ; J n (ut ) = E
(3.3)
(3.4)
0
where bn ; n ; f n and g n be the regularized functions of b; ; f and g: The following result gives the estimates which relate the original control problem with the perturbed ones. Lemma 3.6. Let (xt ) and (yt ) the solutions of (2:1) and (3:3) respectively, corresponding to an admissible control u: Then there exist positive constants M1 and M2 such that: 2 e sup jxut ytu j2 (1) E M1 : ( n ) : 0 t T
(2) jJ n (ut )
J (ut )j
M2 : n ;
where
7
n
=
C : n
Proof. This lemma follows from standard arguments from stochastic calculus and lemma 3.5: Let u ^ be an optimal for the original control problem (2:1) and (2:2) : Note that u ^ is not necessarily optimal for the perturbed control problem (3:3) and (3:4) : However, by Lemma 3:6 we obtain the existence of ( n ) (2M2 : n ) ; a sequence of positive real numbers converging to 0 such that: J n (^ u)
inf J n (u) +
n:
u2U
That is u ^ is n -optimal for the perturbed control problem. According to Lemma 3:4 it is easy to see that J n (:) is continuous on U endowed with the metric d de…ned by (3:2) : By Ekeland’s variational principle (Lemma 3:3), for u ^ with is an admissible control un such that
n
=
2 3
n;
there
2 3
d (^ u; un )
n;
and J n (un )
J n (u) ; for any u 2 U;
where J n (u) = J n (u) +
1 3
n d (u; u
n
):
This means that un is optimal for the perturbed system (3:3) with a new cost function J n . Denote by xn the unique solution of (3:3) corresponding to un ; and let n (s; t) (s t) ; be the unique solution of the linear matrix equation (
n
d
(s; t) = bnx (s; xns ; uns ) :
n
(s; t) dt +
P
j;n x
(s; xns ) :
n
1 j d n
(t; t) = Id:
ej ; (s; t) dB s
(3.5)
Remark 3.7. Since un is optimal for J n ; and the functions bn ; n ; f n and g n are smooth, we can use the spike variation technique to derive a maximum principle for un : Proposition 3.8. For each integer n, there exists an admissible control un and a Fet -adapted process qtn given by qtn =
e E
"Z
T
n;
(s; t) :fxn (s; xns ; uns ) ds +
n;
(T; t) :gxn (xnT )
t
and a Lebesgue null set N such that for t 2 N c e [H n (t; xnt ; unt ; qtn )] E
e [H n (t; xnt ; v; qtn )] E
#
Fet ;
(3.6)
1 3
n;
for every A-valued Ft -measurable random variable v; where the Hamiltonian H n is de…ned by H n (t; x; u; p) = p:bn (t; x; u) 8
f n (t; x; u) ;
(3.7)
where n; denotes the transpose of the matrix n : Proof. Let t0 2 [0; T ] and v an A-valued Ft -measurable random variable. For any " 0; de…ne un" 2 U by un" =
v unt
t 2 [t0 ; t0 + "] ; t 2 [0; T ] [t0 ; t0 + "] :
The fact that J n (un )
J n (un" ) ;
and d (un" ; un )
";
imlpy that J n (un" )
1 3
J n (un )
n :":
However, according to Lemma 3:5; the data de…ning the perturbed control problem (3.3), (3.4) are di¤erentiable. Therefore the map " ! J n (un" ) is di¤erentiable at " = 0; and that dJ n (un" ) e [H n (t; xnt ; unt ; qtn )] j"=0 = E d"
e [H n (t; xnt ; v; qtn )] + E
1 3
n
0;
for every A-valued Ft -measurable random variable v: Remark 3.9. This inequality can be proved for every near optimal control u , using the stability of the state equation and adjoint process with respect to the control variable (see Zhou [20]). Let n (s; t) (s t) the d d-matrix valued process, satisfying the following linear equation ( P j;n esj ; d n (s; t) = bnx (s; x ^ns ; u ^s ) : n (s; t) dt + ^ns ) : n (s; t) dB x (s; x 1 j d (3.8) n (t; t) = Id; where x ^nt is the unique solution of (3:3) corresponding to the optimal control u ^ d^ xnt = bn (t; x ^nt ; u ^t ) dt + n x ^0 = :
n
et ; (t; x ^nt ) dB
(3.9)
Corollary 3.10. there exists an Fet -adapted process satisfying pnt :=
2 T Z e4 E
n;
(s; t) :fx (s; x ^ns ; u ^s ) ds +
n;
(T; t) :gx (^ xT )
t
and a Lebesgue null set N such that, for t 2 N c ; e [H n (t; x E ^nt ; u ^t ; pnt )]
e [H n (t; x E ^nt ; v; pnt )]
for every A-valued Ft -measurable random variable v: 9
1 3
n;
3
Fet 5 ;
(3.10)
(3.11)
3.2. Passing to the Limit. Our aim is now to give a maximum principle of di¤usion processes with Lipschitz coe¢ cients (problem (2.9) and (2.10)). To pass to the Limit in (3.10) and (3.11), we will use Egorov and Portmanteau-Alexandrov theorems, and the Bouleau - Hirsch ‡ow property. Lemma 3.11. We have e lim E
n!+1
e lim E
n!+1
n
(s; t)
sup jpnt
pt j
sup j
2
(s; t)j
= 0;
(3.12)
s t T
2
0 t T
= 0;
e [jH n (t; x lim E ^nt ; u ^t ; pnt )
(3.13)
H (t; x ^t ; u ^t ; pt )j] = 0;
n!+1
(3.14)
where t ; pt and H are determined by the fundamental solution (2.11), the adjoint process (2.9) and the associated Hamiltonian (2.12), corresponding to the optimal pair (^ x; u ^) : nt ; pnt and H n are determined by the fundamental solution (3.8), the adjoint process (3.10) and the associated Hamiltonian (3.7), corresponding to the approximating sequence x ^nt ; given by (3.9): Proof. Using Burkholder, Schwartz inequalities and Gronwall Lemma, we have e E
n
sup j
2
(s; t)
(s; t)j
t s T
8 " < RT n e e sup j n (s; t)j : E jbx (t; x ^nt ; u ^t ) ME : t s T 0 " # 12 9 = P e RT j;n 4 j + E ^nt ) ^t ) dt : x (t; x x (t; x ; 1 j d 0 4
1 2
4
bx (t; x ^t ; u ^t )j dt
# 12
Since the coe¢ cients in the linear stochastic di¤erential equation (3.8) are bounded, e it is easy to see that E
sup j
prove the following " RT n e jbx (t; x ^nt ; u ^t ) E 0
and
e E
"
RT
j;n x
n
4
(s; t)j
< +1: To derive (3.12), it is su¢ cient to
t s T
(t; x ^nt )
j x
0
4
#
bx (t; x ^t ; u ^t )j dt ! 0
4
#
(t; x ^t ) dt ! 0
as n ! +1;
as n ! +1; j = 1; 2; :::::; d.
Let us prove the …rst Limit. We have " # RT n 4 n e jbx (t; x ^t ; u ^t ) bx (t; x ^t ; u ^t )j dt E
M (I1n + I2n ) ;
0
where
I1n
e =E
"
RT
sup jbnx a2A 0
(t; x ^nt ; a) 10
4 bx (t; x ^nt ; a)j
#
dt ;
and "
e =E
I2n
RT
sup jbx (t; x ^nt ; a) a2A 0
4
#
bx (t; x ^t ; a)j dt ;
Since the law of x ^nt is absolutely continuous with respect to the Lebesgue measure, n let t (y) its density. Then I1n
=
ZT Z
0 Rd
Let us show that, for all t 2 [0; T ] Z lim sup jbnx (t; y; a) n!+1 a2A Rd
e For each p > 0; E
4
sup jbnx (t; y; a)
bx (t; y; a)j
a2A
p
sup j^ xnt j
0 t T
4
n t
bx (t; y; a)j
< 1: Thus,
n t
(y) dydt:
(y) dy = 0:
lim Pe
R!+1
then it is enough to show that for every R > 0; Z 4 lim sup jbnx (t; y; a) bx (t; y; a)j n!+1 a2A B(0;R)
n t
sup j^ xnt j > R
= 0;
0 t T
(y) dy = 0:
According to Lemma 3.5 sup jbnx (t; y; a)
a2A
4
bx (t; y; a)j ! 0 dy-a:e;
at least for a subsequence. Then by Egorov’s Theorem, for every > 0; there exists a measurable set F with (F ) < ; such that sup jbnx (t; y; a) bx (t; y; a)j converges a2A
uniformly to 0 on the set F c : Note that, since the Lebesgue measure is regular, F may be chosen closed. This implies that Z 4 lim sup jbnx (t; y; a) bx (t; y; a)j nt (y) dy n
Fc
a2A
sup sup jbnx (t; y; a)
lim n
y2F c
4
bx (t; y; a)j
= 0:
a2A
Now, by using the boundness of the derivatives bnx ; bx by the Lipschitz constant M , we have Z 4 sup jbnx (t; y; a) bx (t; y; a)j nt (y) dy a2A
F
e sup jbnx (t; x =E ^nt ; a)
4
bx (t; x ^nt ; a)j
a2A
2M 4 Pe (^ xnt 2 F ) :
11
f^ xn t 2F g
Since (^ xnt ) converges to x ^t in probability, then in distribution. Applying the Portmanteau-Alexandrov Theorem, we obtain Z 4 lim sup jbnx (t; y; a) bx (t; y; a)j nt (y) dy 2M 4 lim sup Pe (^ xnt 2 F ) n!+1
a2A
F
2M 4 Pe (^ xt 2 F ) Z = 2M 4 t (y) dy < ": F
where t (y) denotes the density of x ^t with respect to Lebesgue measure. Now, since Z 4 sup jbnx (t; y; a) bx (t; y; a)j nt (y) dy a2A
B(0;R)
= +
Z
ZF
Fc
we get lim
n!+1
I1n
4
n t
(y) dy
4
n t
(y) dy;
sup jbnx (t; y; a)
bx (t; y; a)j
sup jbnx (t; y; a)
bx (t; y; a)j
a2A
a2A
= 0:
0 be a …xed integer, then it holds that I2n C J1k + J2k + J3k ; where " # RT 4 k n k n e J =E bx (t; x ^ ;u ^t ) b (t; x ^ ;u ^t ) dt ;
Let k
1
J2k
J3k
t
e =E
e =E
"
"
x
t
0
RT
bkx
(t; x ^nt ; u ^t )
bkx
4
(t; x ^t ; u ^t ) dt ;
0
RT
bkx
(t; x ^t ; u ^t )
#
4
#
bx (t; x ^t ; u ^t ) dt :
0
Applying the same argements used in the …rst limit (Egorov and PortmanteauAlexandrov Theorems), we obtain that lim J1k = 0: We use the continuity of bkx n!+1
^t ) in x and the convergence in probability of x ^nT to x ^T to deduce that bkx (t; x ^nt ; u k ^t ; u ^t ) in probability as n ! 1; and to conclude by using the converges to bx (t; x dominated convergence theorem that lim J2k = 0: Since bkx ; bx are bounded by the n!+1
Lipschitz constant and by using the absolute continuity of the law of x ^t with respect to the Lebesgue measure, the convergence of bkx to bx ; and the Dominated convergence Theorem, we get lim J3k = 0: n!+1
Next, let use prove the limit (3:13) : Clearly h i e jpnt pt j2 E C1 (
where
n 1
2 T Z e 4 j( =E
n;
(s; t) :fxn (s; x ^ns ; u ^s )
t
12
n 1
+
n 2);
3
2 (s; t) :fx (s; x ^s ; u ^s ))j ds5
and n 2
h e j =E
n;
2
(T; t) :gxn (^ xnT )
(T; t) :gx (^ xT )j
i
:
Since fx is bounded by the Lipschitz constant M , and applying the Schwartz inequality, we obtain for all n 2 N n 1
e CE
sup j
1 2
4
n;
e :E
(s; t) dsj
t s T
e + CM:E
n;
sup j
(s; t)
"Z
T
4
jfxn (s; x ^ns ; u ^s )
0
# 12
fx (s; x ^s ; u ^s )j ds
2
(s; t)j
t s T
e It is easy to see that E
4
n;
sup j
(s; t) dsj
< +1: Applying the same argu-
t s T
ments used in the …rst Limit (Egorov and Portmanteau - Alexandrov Theorems) it holds that "Z # 12 T 4 n n e lim E jfx (s; x ^s ; u ^s ) fx (s; x ^s ; u ^s )j ds = 0: n!+1
0
On the other hand, since gx is bounded by the Lipschitz constant, and applying the Schwartz inequality we get n 2
n h e j C E h e j + CM:E
n; n;
io 12 n h e jgxn (^ : E xnT ) i 2 (T; t) (T; t)j ; 4
(T; t)j
4
gx (^ xT )j
io 12
where M is a positive constant. Let k 0 be a …xed integer, then it holds that h i h i h 4 4 n n e jgx (^ e g n (^ e g k (^ E xnT ) gx (^ xT )j E gxk (^ xnT ) + E x xT ) x xT ) h i 4 e g k (^ +E gx (^ xT ) : x xT )
gxk (^ xT )
4
i
The law of x ^nT is absolutely continuous with respect to the Lebesgue measure, let (y) it is density, and by the same manner (by applying Egorov and Portmanteau - Alexandrov theorems), we get Z h i 4 4 e gxn (^ lim E xnT ) gxk (^ xnT ) = lim gxn (y) gxk (y) nT (y) dy = 0: n T
n!+1
n!+1 Rd
We use the continuity of gxk in x and the convergence in probability of x ^nT to x ^T k n k to deduce that gx (^ xT ) converges to gx (^ xT ) in probability as n ! 1; and to conclude by using the dominated convergence theorem that h i 4 e gxk (^ lim E xnT ) gxk (^ xT ) = 0: n!+1
Since
e E
h
gxk (^ xT )
gx (^ xT )
4
i
=
Z
Rd
13
gxk (y)
gx (y)
4 T
(y) dy;
gxk , gx are bounded by the Lipschitz constant, and gxk converges to gx dx-a:e; we conclude by the Dominated convergence Theorem that h i 4 e gxk (^ lim E xT ) gx (^ xT ) = 0: n!+1
Finally, by using Burkholder-Davis-Gundy inequality, we obtain (3:13) :Now, let us prove that e [jH n (t; x lim E ^nt ; u ^t ; pnt )
n!+1
H (t; x ^t ; u ^t ; pt )j] = 0:
Applying the Schwartz inequality we get o1 o1 n n 2 2 e jbn (t; x e [jH n (t; x e jpnt pt j2 2 : E ^nt ; u ^t )j E ^nt ; u ^t ; pnt ) H (t; x ^t ; u ^t ; pt )j] E n o1 n o1 2 2 e jbn (t; x e jpt j2 2 E e jf n (t; x + E ^nt ; u ^t ) b (t; x ^t ; u ^t )j : E ^nt ; u ^t ) f (t; x ^t ; u ^t )j :
Lemma 3.5 and (3.13) imply that the …rst expression in the right hand side converges to 0 as n ! 1: Since e jf n (t; x E ^nt ; u ^t )
f (t; x ^t ; u ^t )j
e jf n (t; x E ^nt ; u ^t ) f n (t; x ^t ; u ^t )j e jf n (t; x +E ^t ; u ^t ) f (t; x ^t ; u ^t )j ;
f n being continuous and bounded, x ^nt converges uniformly in probability to x ^t ; we conclude by the dominated convergence theorem that e jf n (t; x lim E ^nt ; u ^t )
f n (t; x ^t ; u ^t )j = 0:
e jf n (t; x lim E ^t ; u ^t )
f (t; x ^t ; u ^t )j = 0:
n!+1
Using Lemma 3.5 and the dominated convergence theorem to conclude that n!+1
The convergence of the second term in the right hand side can be performed in a similar way. Proof. of Theorem 2.1. Use Corollary 3.10 and Lemma 3.11. Remark 3.12. 1) If we assume that b, ; f and g are continuously di¤ erentiable, we get the classical maximum principle [3] : Remark 3.13. 2)If we assume that g is continuously di¤ erentiable and is non degenerate, we get the maximum principle proved in [2] : Remark 3.14. 3) If we assume that f and g are continuously di¤ erentiable and is degenerate, we get the maximum principle proved in [1] : REFERENCES [1] Bahlali, K., Djehiche, B., Mezerdi, B.: On the Stochastic Maximum Principle in Optimal Control of Degenerate Di¤usions with Lipschitz Coe¢ cients. Appl. Math. Optim., Vol. 56, 364-378 (2007). [2] Bahlali, K., Mezerdi, B., Ouknine, Y.: The maximum principle in optimal control of a di¤usions with nonsmooth coe¢ cients. Stoch. Stoch. Rep. 57, 303-316 (1996) 14
[3] Bensoussan, A.: Lectures on stochastic control. In: Lect. Notes in Math., vol. 972, pp. 1-62. Springer, Berlin (1983) [4] Bismut, J.M.: An introductory approach to duality in optimal stochastic control. SIAM Rev. 20(1), 62-78 (1978) [5] Bouleau, N., Hirsch, F.: On the derivability with respect to the initial data of the solution of a stochastic di¤erential equation with Lipschitz coe¢ cients . Lect. Notes in Math. 1393, 39-57 (1989) [6] Bouleau, N., Hirsch, F.: Sur la propriété du ‡ot d’une équation di¤érentielle stochastique. C.R. Acad. Sci. Paris 306, 421-424 (1988) [7] Cadenillas, A., Karatzas, I.: The stochastic maximum principle for linear convex optimal control with random coe¢ cients. SIAM J. Control Optimal. 33(2), 590-624 (1995) [8] Clarke, F.H.: Optimization and Nonsmooth Analysis. Wiley, New York (1983) [9] Ekeland, I.: Non convex minimization problems. Bull. Am. Math. Soc. (N. S.) 1, 443-474 (1979) [10] Elliott, R.J., Kohlmann, M.: The variational principle and stochastic optimal control, Stochastics 3, 229-241 (1980) [11] Frankowska, H.: The …rst order necessary conditions for nonsmooth variational and control problems. SIAM J. Control Optim. 22(1), 1-12 (1984) [12] Haussmann, U.G.: A stochastic maximum principle for optimale control of di¤usions. In: Longman Scienti…c & Technical, Harlow. Pitman Research Notes in Mathematics Series, vol. 151. Wiley, New York (1986) [13] Krylov, N.V.: Controlled Di¤usion Processes. Springer, Berlin (1980) [14] Kushner, N.J.: Necessary conditions for continuous parameter stochastic optimization problems. SIAM J. Control Optim. 10, 550-565 (1972) [15] Mezerdi, B.: Necessary conditions for optimality for a di¤usion with a non smooth drift. Stochastics 24, 305-326 (1988) [16] Peng, S.: A general stochastic maximum principle for optimal control problems. SIAM J. Control Optim. 28, 966-979 (1990) [17] Rockafellar, R.T.: Conjugat convex functions in optimal control problems and the calculus of variations. J. Math. Anal. Appl. 32, 174-222 (1970) [18] Warga, J.: Necessary conditions without di¤erentiability assumptions in optimal control. J. D¤er. Equ. 18, 41-62 (1975) [19] Yang, B.: Necessary conditions for optimal controls of forward_bakward stochastic systems with non smooth cost functionals. J. Fudan Univ. Nat. Sci. 39(1), 61-67 (2000) [20] Zhou, X.Y.: Stochastic near optimal controls: necessary and su¢ cient conditions for near optimality. SIAM J. Control Optim. 36(3), 929-947 (1998)
15
