BSDE formulation of combined regular and singular stochastic control ...

6 downloads 0 Views 137KB Size Report
Jan 10, 2018 - where (Wt) is an n-dimensional Brownian motion, (αt) is a predictable process taking ..... Probabilistic aspects of finite-fuel, reflected follower.
arXiv:1801.03336v1 [math.OC] 10 Jan 2018

BSDE formulation of combined regular and singular stochastic control problems Bruno Bouchard∗

Patrick Cheridito†

Ying Hu‡

January 2018

Abstract In this paper we study a class of combined regular and singular stochastic control problems that can be expressed as constrained BSDEs. In the Markovian case, this reduces to a characterization through a PDE with gradient constraint. But the BSDE formulation makes it possible to move beyond Markovian models and consider path-dependent problems. We also provide an approximation of the original control problem with standard BSDEs that yield a characterization of approximately optimal values and controls.

Keywords: singular stochastic control, constraint backward stochastic differential equation, minimal supersolution. Mathematics Subject Classification. 93E20

1

Introduction

We consider a class of continuous-time stochastic control problems involving two different controls: a regular control affecting the state variable in an absolutely continuous way, and a singular control resulting in a cumulative impact of finite variation. For standard stochastic control in continuous time, we refer to the textbooks [13, 22, 14, 27, 26]. Singular stochastic control goes back to [2, 3] and has subsequently been studied by e.g. [4, 16, 17, 19, 18, 20, 9, 1, 23, 11, 21, 15]. For a typical Markovian singular stochastic control problems, it can be deduced from dynamic programming arguments that the optimal value is given by a viscosity solution of a PDE with ∗

Universit´e Paris-Dauphine, PSL Research University, CNRS, CEREMADE, Paris, France ([email protected]). Partially supported by CAESARS - ANR-15-CE05-0024. † Department of Mathematics, ETH Zurich, R¨ amistrasse 101, 8092 Zurich, Switzerland. ‡ Univ Rennes, CNRS, IRMAR - UMR 6625, F-35000 Rennes, France ([email protected]) and School of Mathematical Sciences, Fudan University, Shanghai 200433, China. Partially supported by Lebesgue center of mathematics “Investissements d’avenir” program - ANR-11-LABX-0020-01, by CAESARS - ANR-15-CE05-0024 and by MFG - ANR-16-CE40-0015-01.

1

gradient constraint. On the other hand, it has been shown that PDEs with gradient constraints are related to BSDEs with Z-constraints; see, e.g. [8, 25]. In this paper we directly show that a wide variety of combined regular and singular stochastic control problems can be represented as Z-constrained BSDEs1 . This has the advantage that it allows to study path-dependent problems. More precisely, we consider optimization problems of the form  Z T Z T α,β α,β α,β g(t, X , αt )dβt + h(X ) (1.1) f (t, X , αt )dt + sup E α,β

0

0

for a d-dimensional controlled process with dynamics dXtα,β = µ(t, X α,β , αt )dt + ν(t, X α,β , αt )dβt + σ(t, X α,β )dWt ,

X0 = x ∈ Rd ,

(1.2)

where (Wt ) is an n-dimensional Brownian motion, (αt ) is a predictable process taking values in a compact subset A ⊆ Rk (the regular control) and (βt ) is an l-dimensional process with nondecreasing components (the singular control). The coefficients µ, ν, σ and the functions f, g, h are all allowed to depend in a non-anticipative way on the paths of X α,β . Our main representation result is that the optimal value of (1.1) is given by the initial value of the minimal supersolution of a BSDE Z T Z T Zs dWs (1.3) p(s, X, Zs )ds − Yt = h(X) + t

t

subject to a constraint of the form q(t, X, Zt ) ∈ Rl− , where (Xt ) is the unique strong solution of an SDE dXt = η(t, X)dt + σ(t, X)dWt , X0 = x, with the same σ-coefficient as (1.2). In addition, we show that the original problem (1.1) can be approximated with a sequence of standard BSDEs Z T Z T j j j Zsj dWs . (1.4) p (s, X, Zs )ds − Yt = h(X) + t

t

While the minimal supersolution of the constrained BSDE (1.3) gives the optimal value of the control problem (1.1), the BSDEs (1.4) can be used to characterize nearly optimal values as well as approximately optimal controls. Due to the constraint q, it might happen that the minimal supersolution of (1.3) jumps at the final time T . In our last result, we show how this jump can be removed by replacing h with ˆ of h that is consistent with q – the so-called face-lift of h. the smallest majorant h The rest of the paper is structured as follows. In Section 2, we introduce the notation and our main results. All proofs are given in Section 3. 1 Independently, Elie, Moreau and Possama¨ı have been working on a similar idea in [10]. But the exact class of control problems studied in [10] is different. Moreover, they employ analytic methods, while we use purely probabilistic arguments.

2

2

Results

We consider a combined regular and singular stochastic control problem of the form  Z T Z T α,β α,β α,β g(t, X , αt )dβt + h(X ) f (t, X , αt )dt + I := sup E (α,β)∈A

(2.1)

0

0

for a constant time horizon T ∈ R+ and a d-dimensional controlled process evolving according to dXtα,β = µ(t, X α,β , αt )dt + ν(t, X α,β , αt )dβt + σ(t, X α,β )dWt ,

X0 = x ∈ Rd ,

(2.2)

where (Wt ) is an n-dimensional Brownian motion on a probability space (Ω, F, P) with corresponding augmented filtration F = (Ft ). The set of controls A consists of pairs (α, β), where (αt )0≤t≤T is an F-predictable process with values in a compact subset A ⊆ Rk (the regular control) and an l-dimensional F-adapted continuous process (βt ) with nondecreasing components such that β0 = 0 and βT ∈ L2 (P) (the singular control). The coefficients µ, ν, σ and the performance functions f, g, h can depend in a non-anticipative way on the paths of X α,β . Depending on their exact specification, there might exist an optimal control in A, or an optimal control might require (βt ) to jump and can only be approximated with controls in A. Let us denote by C d the space of all continuous functions from [0, T ] to Rd and set kxkt := sup |xs |,

x ∈ C d,

0≤s≤t

where |.| is the Euclidean norm on Rd . We make the following Assumption 2.1 (i) σ : [0, T ] × C d → Rd×n is a measurable function such that Z

T

|σ(t, 0)|2 dt < ∞

and

|σ(t, x) − σ(t, y)| ≤ Lkx − ykt for some constant L ∈ R+ .

0

(ii) µ is of the form µ(t, x, a) = η(t, x)+σ(t, x)˜ µ(t, x, a) for measurable functions η : [0, T ]×C d → Rd and µ ˜ : [0, T ] × C d × A → Rn such that Z

T

|η(t, 0)|2 dt < ∞

and |η(t, x) − η(t, y)| ≤ Lkx − ykt for some constant L ∈ R+ ,

0

µ ˜(t, x, a) is bounded and continuous in a and Z

0

T

sup |µ(t, 0, αt )|2 dt < ∞, a∈A

sup |µ(t, x, a) − µ(t, y, a)| ≤ Lkx − ykt for some L ∈ R+ .

a∈A

3

(iii) ν is of the form ν(t, x, a) = σ(t, x)˜ ν (t, x, a) for a measurable function ν˜ : [0, T ] × C d × A → Rd×n such that ν˜(t, x, a) is bounded and continuous in a and Z T sup |ν(t, 0, αt )|2 dt < ∞, sup |ν(t, x, a) − ν(t, y, a)| ≤ Lkx − ykt for some L ∈ R+ . 0

a∈A

a∈A

(iv) The functions f, g : [0, T ] × C d × A → R are measurable; f (t, x, a) and g(t, x, a) are nonanticipative in x and upper semicontinuous in (x, a); h : C d → R is upper semicontinuous in x; and the supremum in (1.1) is finite. Under these assumptions, equation (2.2) has for every pair (α, β) ∈ A a unique strong solution and the SDE dXt = η(t, X)dt + σ(t, X)dWt (2.3)

(Xtα,β ),

has a unique strong solution (Xt ); see e.g. Protter (2004). For our main representation result, Theorem 2.2, we need the mappings p : [0, T ]×C d ×Rn → R and q : [0, T ] × C d × Rn → Rl given by   n   X zj ν˜ji (t, x, a) . p(t, x, z) := sup {f (t, x, a) + z µ ˜(t, x, a)} and qi (t, x, z) := sup gi (t, x, a) +  a∈A a∈A  j=1

In this paper, a supersolution of the BSDE Z T Z T Zs dWs p(s, X, Zs )ds − Yt = h(X) + t

with constraint

t

consists of a triplet (Y, Z, K) ∈ S 2 × H2 × K2 such that Z T Z T Zs dWs p(s, X, Zs )ds + (KT − Kt )− Yt = h(X)+ t

t

q(t, X, Zt ) ∈ Rl−

and q(t, X, Zt ) ∈ Rl−

for all t, (2.4)

where • S 2 is the space of d-dimensional RCLL F-adapted processes (Yt ) such that E sup0≤t≤T |Yt |2 < ∞, RT • H2 the space of Rd×n -valued F-predictable processes (Zt ) such that E 0 |Zt |2 dt < ∞, and

• K2 the set of processes (Kt ) in S 2 with nondecreasing components starting at 0.

Moreover, we call (Y, Z, K) a minimal supersolution of (2.4) if Yt ≤ Yt′ , 0 ≤ t ≤ T , for any other supersolution (Y ′ , Z ′ , K ′ ); see e.g. Peng (1999). Our main result is the following: 4

Theorem 2.2 The constrained BSDE (2.4) has a minimal supersolution (Y, Z, K), and Y0 = I. The next result shows that problem (2.1) can be approximated by restricting the controls to regular piecewise j ∈ N, denote by Aj the set of all pairs (α, β) ∈ A of the Pm−1 constant controls: Rfor Pm−1 t form α = i=0 ai 1(ti ,ti+1 ] and βt = 0 bs ds, where b = i=0 bi 1(ti ,ti+1 ] , the bi are Fti -measurable with values in [0, j]l and 0 = t0 < t1 < t2 · · · < tm = T is a deterministic partition of [0, T ]. The corresponding control problem is  Z T Z T g(t, X α,β , αt )dβt + h(X α,β ) , (2.5) f (t, X α,β , αt )dt + I j := sup E 0

0

(α,β)∈Aj

and the following holds: Proposition 2.3 One has I j ↑ I for j → ∞. Moreover, since (2.5) is a regular control problem, it admits a representation through a standard BSDE Z T Z T j Zs dWs (2.6) p (s, X, Zs )ds − Yt = h(X) + t

t

with a driver given by pj (t, x, z) :=

sup

{f (t, x, a) + z µ ˜(t, x, a) + [g(t, x, a) + z˜ ν (t, x, a)]b} .

a∈A,b∈[0,j]m

Compared to the constrained BSDE (2.4), which gives the optimal value of the control problem (2.1), the BSDE (2.6) provides a characterization of the optimal value of (2.5) as well as corresponding optimal controls. Theorem 2.4 For every j ∈ N, BSDE (2.6) has a unique solution (Y j , Z j ) in S 2 ×H2 . Moreover, Y0j = I j , and for any pair of progressively measurable functionals α ˆ : [0, T ] × C → A, ˆb : [0, T ] × l C → [0, j] satisfying f (t, X, α ˆt (X))+Ztj µ ˜(t, X, α ˆt (X))+[g(t, X, α ˆt (X))+Ztj ν˜(t, X, α ˆt )]ˆbt (X) = pj (t, X, Ztj ) dt × dP-a.e., Rt αt = α ˆ t (X α,β ) and βt = 0 ˆbs (X α,β )ds defines a pair in A such that  Z T Z T α,β α,β α,β j g(t, X , αt )dβt + h(X ) . f (t, X , αt )dt + I =E 0

0

Our last result concerns the continuity of the minimal supersolution of (2.4) at the final time T . Due to the constraint q, Y might jump downwards at T . This can be avoided by modifying ˆ : C → R as follows h. Define the face-lift h ˆ h(x) := inf{h(x + ν(T, x)l1{T } ) + g(T, x)l, Then the following holds follwing an argument from [6]. 5

l ∈ Rd+ }.

Proposition 2.5 The BSDE Z Z T ˆ p(s, X, Zs )ds − Yt = h(X) + t

T

Zs dWs

with constraint

t

q(t, X, Zt ) ∈ Rl−

(2.7)

ˆ K), ˆ and one has ∆YˆT = 0 as well as (Yˆt , Zˆt , K ˆ t) = admits a minimal supersolution (Yˆ , Z, (Yt , Zt , Kt ) for t ∈ [0, T ), where (Y, Z, K) is the minimal supersolution of (2.4).

3

Proofs

We start with the Proof of Proposition 2.3. It is straightforward to see that I j is nondecreasing and I j ≤ I. By a density argument, we can prove that lim I j = I. j

 Next, we show that the approximate problems (2.5) admit a weak formulation. To do that, we note that by Girsanov’s theorem, the process Z t α,β [˜ µ(s, X, αs ) + ν˜(s, X, αs )bs ] ds. Wt := Wt − 0

is for every pair (α, β) ∈ Aj , a Brownian motion under the measure Pα,β given by Z .  dPα,β [˜ µ(s, X, αs ) + ν˜(s, X, αs )bs ] dWs . =E dP 0 Moreover, the following holds: Lemma 3.1 For all (α, β) ∈ Aj , the augmented filtration generated by W α,β equals F. Proof. Denote the augmented filtration of W α,β by Fα,β = (Ftα,β ). Since X is a strong solution of the SDE (2.3), it is F-adapted. So it follows from the definition of W α,β that Fα,β is contained in F. P Pm−1 On the other hand, one has α = m−1 i=0 ai 1(ti ,ti+1 ] and b = i=0 bi 1(ti ,ti+1 ] for ai and bi Fti measurable. In particular a0 and b0 are deterministic. So it follows from Assumption 2.1 that on [0, t1 ], (Xt ) is the unique strong solution of dXt = µ(t, X, αt )dt + ν(t, X, αt )bt dt + σ(t, X)dWtα,β , 6

X0 = x.

Hence, (Xt )t∈[t0 ,t1 ] is (Ftα,β )t∈[t0 ,t1 ] -adapted, from which it follows that Wt =

Wtα,β

+

Z

t

[˜ µ(s, X, αs ) + ν˜(s, X, αs )bs ] ds,

t ∈ [0, t1 ],

0

is (Ftα,β )t∈[0,t1 ] -adapted. This shows that a1 and b1 are Ftα,β -measurable. Now the lemma follows 1 by induction over i.  Using Lemma 3.1, one can derive the following weak formulation of problem (2.5): Lemma 3.2 One has j

I =

sup

E

α,β

(α,β)∈Aj

Z

T

f (t, X, αt )dt + 0

Z

T 0

 g(t, X, αt )dβt + h(X) ,

(3.1)

where Eα,β denotes the expectation under Pα,β . Proof. For all (α, β) ∈ Aj , X α,β is the unique strong solution of dXtα,β = µ(t, X α,β , αt )dt + ν(s, X α,β , αt )bt dt + σ(t, X α,β )dWt ,

X0 = x,

and X the unique strong solution of dXt = µ(t, X, αt )dt + ν(s, X, αt )bt dt + σ(t, X)dWtα,β ,

X0 = x.

Since a0 and b0 are deterministic, (αt , βt , Xt )t∈[0,t1 ] has the same distribution under the measure Pα,β as (αt , βt , Xtα,β )t∈[0,t1 ] under P. Moreover, a1 and b1 are functions of (Wt )t∈[0,t1 ] . So if one defines a ˜1 and ˜b1 to be the same functions of (Wtα,β )t∈[0,t1 ] , then (˜ αt , β˜t , Xt )t∈[t1 ,t2 ] has the same ˜

˜ β as (α , β , X α,β ) distribution under Pα, t t t∈[t1 ,t2 ] under P. Continuing like this, one sees that for t j ˜ ∈ Aj such that (˜ ˜ X) has the same distribution every pair (α, β) ∈ A , there exists a pair (˜ α, β) α, β, ˜ α, ˜ β α,β under P as (α, β, X ) under P. Conversely, it can be deduced from Lemma 3.1 with the same ˜ β˜) ˜ ∈ Aj such that (˜ ˜ X α, argument that for every pair (α, β) ∈ Aj , there exists a pair (˜ α, β) α, β, has the same distribution under P as (α, β, X) under Pα,β . This proves the lemma. 

Now, we are ready to give the Proof of Theorem 2.4 It follows from our assumptions that the BSDE (2.6) satisfies the standard conditions. So it has a unique solution (Y j , Z j ) in S 2 × H2 ; see e.g. ... For each pair (α, β) ∈ Aj , we set  Z T Z T α,β α,β g(t, X, αs )dβs + h(X) | Ft . Yt := E f (s, X, αs )ds + t

t

7

By Lemma 3.1 and the predictable representation theorem that there exists an Rn -valued Fpredictable process Z α,β such that Z T  Z T Z t α,β α,β g(t, X, αs )dβs + h(X) | Ft = Y0 + f (s, X, αs )ds + E Zsα,β dWsα,β . 0

0

0

Hence, Ytα,β

= h(X) +

Z

T

Z

T

f (s, X, αs )ds +

T

Z

g(s, X, αs )dβs −

t

t

t

Z

T

Zsα,β dWsα,β

n

o f (s, X, αs ) + Zsα,β µ ˜(s, X, αs ) ds = h(X) + t Z T Z Tn o α,β g(s, X, αs ) + Zs ν˜(s, X, αs ) dβs − Zsα,β dWs . + t

t

j

By a comparison result for BSDEs (see e.g. ...), one has Y ≥ Y α,β . On the other hand, it can be deduced from a measurable selection argument that there exist progressively measurable functions α ˜ : [0, T ] × C × Rn → A and ˜b : [0, T ] × C × Rn → [0, j]l such that f (t, x, α ˜ (t, x, z)) + z µ ˜(t, x, α(t, ˜ x, z)) + [g(t, x, α(t, ˜ x, z)) + z˜ ν (t, x, α(t, ˜ x, z))]˜b(t, x, z) = pj (t, x, z). R t˜ b(s, X, Zsj )ds defines a pair in A that can be approximated by a αt = α ˜ (t, X, Z j ) and βt = 0

sequence of pairs (αn , β n ) ∈ Aj in L2 . Then Z T Z αn ,β n n E f (s, X, αs )ds + 0

T 0

g(t, X, αns )dβsn

 + h(X)

converges to Eα,β

Z

T

f (s, X, αs )ds + 0

Z

T 0

 g(t, X, αs )dβs + h(X) ,

and Y0j

= h(X) +

Z

T

p 0

j

(s, X, Zsj )ds



Z

T 0

Zsj dWs

T  = h(X) + f (s, X, αs ) + Zsj µ ˜(s, X, αs ) ds 0 Z T Z T  Zsj dWs g(s, X, αs ) + Zsj ν˜(s, X, αs ) bs ds − + 0 0 Z T Z T Zsj dWsα,β {f (s, X, αs ) + g(s, X, αs )bs } ds − = h(X) + 0 0  Z T Z T α,β g(t, X, αs )dβs + h(X) . = E f (s, X, αs )ds +

Z

0

0

8

This shows that Y0j = I j . Finally, if α ˆ : [0, T ]×C → A and ˆb : [0, T ]×C → [0, j]l are progressively measurable functionals such that f (t, X, α ˆt (X))+Ztj µ ˜(t, X, α ˆt (X))+[g(t, X, α ˆt (X))+Ztj ν˜(t, X, α ˆt )]ˆbt (X) = pj (t, X, Ztj ) dt × dP-a.e., it follows as above that Y0j

=E

Z

ˆ α(X), ˆ β(X)

T

f (s, X, α(X) ˆ s )ds +

Z

0

0

T

 ˆ g(t, X, α(X) ˆ s )dβ(X)s + h(X) .

Rt

ˆbs (X α,β )ds defines a pair in A such that (α, β, X α,β ) has ˆ ˆ β(X) ˆ the same distribution under P as (ˆ α(X), β(X), X) under Pα(X), . As a consequence,  Z T Z T g(t, X α,β , αt )dβt + h(X α,β ) , f (t, X α,β , αt )dt + Ij = E Moreover, αt = α ˆ t (X α,β ) and βt =

0

0

0

and the proof is complete.



Proof of Theorem 2.2 We know that I j ↑ I and I j = Y0j , where (Y j , Z j ) is the solution of (2.6). On the other hand, it follows from Peng (1999) that Y j increases to Y , where (Y, Z) is the maximal subsolution of (2.4). 

References [1] F. Baldursson (1987). Singular stochastic control and optimal stopping. Stochastics 21(1), 1–40. [2] J.A. Bather and H. Chernoff (1967). Sequential decisions in the control of a spaceship. Proc. Fifth Berkeley Symposium on Mathematical Statistics and Probability 3, 181–207. [3] J.A. Bather and H. Chernoff (1967). Sequential decisions in the control of a spaceship (finite fuel). J. Appl. Prob. 4, 584–604. [4] V.E. Benes, L.A. Shepp and H.S. Witsenhausen (1980). Some solvable stochastic control problems. Stochastics 4(1), 39–83. [5] D. P. Bertsekas and S. E. Shreve (1978). Stochastic Optimal Control: The Discrete Time Case. New York: Academic Press, 1978. [6] B. Bouchard, R. Elie and L. Moreau (1914). Regularity of BSDEs with a convex constraint on the gains-process. Preprint arXiv:1409.5369. 9

[7] B. Bouchard, D. Possama and X. Tan. A general Doob-Meyer-Mertens decomposition for g-supermartingale systems. Preprint arXiv:1505.00597. [8] R. Buckdahn and Y. Hu (1998). Hedging contingent claims for a large investor in an incomplete market. Adv. in Appl. Probab. 30(1), 239–255. [9] P.L. Chow, J.L. Menaldi and M. Robin (1985). Additive control of stochastic linear systems with finite horizon. SIAM J. Control and Optimization 6, 858–899. [10] R. Elie, L. Moreau and D. Possama¨ı (2017). On a class of path-dependent singular stochastic control problems. Arxiv Preprint. [11] N. El Karoui and I. Karatzas (1988). Probabilistic aspects of finite-fuel, reflected follower problems. Acta Appl. Math. 11, 223–258. [12] N. El Karoui, S. Peng and M. C. Quenez (1997). Backward stochastic differential equations in finance. Math. Finance 7(1), 1--71. [13] W. Fleming and R. Rishel (1975). Deterministic and Stochastic Optimal Control. SpringerVerlag. [14] W. Fleming and M. Soner (1993). Controlled Markov Processes and Viscosity Solutions. Springer-Verlag. [15] U.G. Haussmann and W. Suo (1995). Singular optimal stochastic controls II. Dynamic programming. SIAM J. Control Optim. 33(3), 937–959. [16] I. Karatzas (1981). The monotone follower problem in stochastic decision theory. Applied Mathematics and Optimization 7, 175–189. [17] I. Karatzas (1983). A class of singular stochastic control problems. Adv. in Appl. Probab. 15(2), 225–254. [18] I. Karatzas (1985). Probabilistic aspects of finite-fuel stochastic control. Proc Nat. Acad. Sci. USA 82(17), 5579–81. [19] I. Karatzas and S.E. Shreve (1984). Connection between optimal stopping and singular stochastic control I. Monotone follower problems. SIAM J. on Control and Optimization 22, 856–877. [20] I. Karatzas and S.E. Shreve (1985). Connection between optimal stopping and singular stochastic control II. Reflected follower problems. SIAM J. on Control and Optimization 23, 433–451.

10

[21] T.O. Kobila (1993). A class of solvable stochastic investment problems involving singular controls. Stoch. Stoch. Rep. 43, 29–63. [22] N. Krylov (1980). Controlled Diffusion Processes. Springer-Verlag. [23] J.L. Menaldi and M.I. Taksar (1987). Singular control of multidimensional Brownian motion. IFAC Proceedings Volume 20(5), 221–224. [24] S. Peng (1999). Monotonic limit theorem of BSDE and nonlinear decomposition theorem of Doob–Meyer’s type. Probab. Theory Related Fields 113(4), 473–499. [25] S. Peng and M. Xu (2013). Constrained BSDEs, viscosity solutions of variational inequalities and their applications. Math. Control Relat. Fields 3(2), 233–244. [26] N. Touzi (2013). Optimal Stochastic Control, Stochastic Target Problems, and Backward SDE. Fields Institute Monographs. Springer. [27] J. Yong and X.Y. Zhou (2000). Stochastic Controls, Hamiltonian Systems and HJB Equations. Springer-Verlag.

11