Sep 29, 2010 - This article was downloaded by: [Zhu, Yuanguo]. On: 7 October 2010 ... Full terms and conditions of use: http://www.informaworld.com/terms-and-conditions-of-access.pdf ... optimal control is based on dynamic programming.
This article was downloaded by: [Zhu, Yuanguo] On: 7 October 2010 Access details: Access Details: [subscription number 927411823] Publisher Taylor & Francis Informa Ltd Registered in England and Wales Registered Number: 1072954 Registered office: Mortimer House, 3741 Mortimer Street, London W1T 3JH, UK
Cybernetics and Systems
Publication details, including instructions for authors and subscription information: http://www.informaworld.com/smpp/title~content=t713722751
UNCERTAIN OPTIMAL CONTROL WITH APPLICATION TO A PORTFOLIO SELECTION MODEL
Yuanguo Zhua a Department of Applied Mathematics, Nanjing University of Science and Technology, Nanjing, Jiangsu, China Online publication date: 29 September 2010
To cite this Article Zhu, Yuanguo(2010) 'UNCERTAIN OPTIMAL CONTROL WITH APPLICATION TO A PORTFOLIO
SELECTION MODEL', Cybernetics and Systems, 41: 7, 535 — 547 To link to this Article: DOI: 10.1080/01969722.2010.511552 URL: http://dx.doi.org/10.1080/01969722.2010.511552
PLEASE SCROLL DOWN FOR ARTICLE Full terms and conditions of use: http://www.informaworld.com/terms-and-conditions-of-access.pdf This article may be used for research, teaching and private study purposes. Any substantial or systematic reproduction, re-distribution, re-selling, loan or sub-licensing, systematic supply or distribution in any form to anyone is expressly forbidden. The publisher does not give any warranty express or implied or make any representation that the contents will be complete or accurate or up to date. The accuracy of any instructions, formulae and drug doses should be independently verified with primary sources. The publisher shall not be liable for any loss, actions, claims, proceedings, demand or costs or damages whatsoever or howsoever caused arising directly or indirectly in connection with or arising out of the use of this material.
Cybernetics and Systems: An International Journal, 41:535–547 Copyright # 2010 Taylor & Francis Group, LLC ISSN: 0196-9722 print=1087-6553 online DOI: 10.1080/01969722.2010.511552
Uncertain Optimal Control with Application to a Portfolio Selection Model YUANGUO ZHU
Downloaded By: [Zhu, Yuanguo] At: 11:37 7 October 2010
Department of Applied Mathematics, Nanjing University of Science and Technology, Nanjing, Jiangsu, China
Optimal control is a very important field of study not only in theory but in applications, and stochastic optimal control is also a significant branch of research in theory and applications. Based on the concept of uncertain process, an uncertain optimal control problem is dealt with. Applying Bellman’s principle of optimality, the principle of optimality for uncertain optimal control is obtained, and then a fundamental result called the equation of optimality in uncertain optimal control is given. Finally, as an application, the equation of optimality is used to solve a portfolio selection model. KEYWORDS equation of optimality, optimal control, portfolio selection, principle of optimality, uncertain process
INTRODUCTION Since the 1950s, optimal control theory has been an important branch of modern control theory. The study of optimal control greatly attracted the attention of many mathematicians because of the necessity of strict expression form in optimal control theory. With the greater use of methods and results in mathematics and computer science, optimal control theory has achieved great developments and has been applied to many fields such as production engineering, programming, economy, and management. The study of stochastic optimal control was initiated in the 1970s, such as in Merton (1971) for finance. Some investigations on optimal control of This work is supported by the National Natural Science Foundation of China (No. 60874038). Address correspondence to Yuanguo Zhu, Department of Applied Mathematics, Nanjing University of Science and Technology, Nanjing 210094, Jiangsu, China. E-mail: ygzhu@mail. njust.edu.cn 535
Downloaded By: [Zhu, Yuanguo] At: 11:37 7 October 2010
536
Y. Zhu
Brownian motion or stochastic differential equations and applications in finance may be found in some books such as Fleming and Rishel (1986), Harrison (1985), and Karatzas (1989). One of the main methods to study optimal control is based on dynamic programming. The use of dynamic programming in optimization over Ito’s process is discussed in Dixit and Pindyck (1994). The complexity of the world makes the events we face uncertain in various forms. Recently, Liu (2007) introduced an uncertain measure to measure the truth degree of an uncertain event. Based on the uncertain measure, uncertainty theory was founded by Liu in 2007 and refined by Liu (2010) as a branch of mathematics for studying the behavior of uncertain phenomena. Meanwhile, as counterpart of stochastic process and Brownian motion, uncertain process and canonical process were introduced in Liu (2008). In order to handle an optimal control problem with uncertain process, in this article we will introduce and deal with an uncertain optimal control problem by using dynamic programming. In the next section, we will review some concepts such as uncertain measure, uncertainty space, expected value of uncertain variable, uncertain process, canonical process, and uncertain differential equation. In the following section, we will introduce an uncertain optimal control problem and present the principle of optimality for uncertain optimal control based on Bellman’s principle of optimality in dynamic programming. Then we will obtain a fundamental result called the equation of optimality in uncertain optimal control. In the last section, we will solve a portfolio selection model by using the equation of optimality.
PRELIMINARY For convenience, we give some useful concepts at first. Let C be a nonempty set and L a r-algebra over C. Each element K 2 L is called an event. Definition 1 (Liu 2007). A set function M defined on the r-algebra over L is called an uncertain measure if it satisfies the following four axioms:
Axiom Axiom Axiom Axiom
1. 2. 3. 4.
(Normality) M{C} ¼ 1; (Monotonicity) M{K1} M{K2} whenever K1 K2 for K1, K2 2 L; 1 for any (Self-Duality) M{K} þ M{Kc} ¼ S event P1 K; K (Countable Subadditivity) M 1 i¼1 i i¼1 MfKi g.
Gao (2009) studied some properties of an uncertain measure. Definition 2 (Liu 2007). Let C be a nonempty set, L the r-algebra over C, and M be an uncertain measure. Then the triplet (C, L, M) is said to be an uncertainty space.
Uncertain Optimal Control
537
An uncertain variable is a measurable function n from an uncertainty space (C, L, M) to the set of real numbers; that is, for any Borel set of real numbers, the set fn 2 Bg ¼ fc 2 C j nðcÞ 2 Bg is an event. Definition 3 (Liu 2007). The uncertainty distribution U : < ! [0, 1] of an
uncertain variable n is defined by UðxÞ ¼ Mfn xg: Definition 4 (Liu 2007). Let n be an uncertain variable. Then the expected Downloaded By: [Zhu, Yuanguo] At: 11:37 7 October 2010
value of n is defined by Z
E½n ¼
þ1
Mfn rgdr
Z
0
Mfn rgdr
ð1Þ
1
0
provided that at least one of the two integrals is finite. Definition 5. Let n be an uncertain variable with finite expected value e.
Then the variance of n is defined by V [n] ¼ E[(n e)2]. Definition 6. The fuzzy variables n and g are said to be identically distribu-
ted if M{n 2 B} ¼ M{g 2 B} for any set B 0, the increments Xsþt Xs are identically distributed fuzzy variables for all s > 0. Definition 10 (Liu 2009a). An uncertain process Ct is said to be canonical
process if (i) C0 ¼ 0 and almost all sample paths are Lipschitz continuous, (ii) Ct has stationary and independent increments, (iii) every increment Csþt Cs is a normal uncertain variable with expected value 0 and variance t2, whose uncertainty distribution is
px UðxÞ ¼ 1 þ exp pffiffiffiffi 3t
1 ;
x 2 ð4Þ < J ð0; x0 Þ sup E 0 f ðXs ; D; sÞds þ GðXT ; T Þ D
> : subject to dXs ¼ nðXs ; D; sÞds þ rðXs ; D; sÞdCs
and X0 ¼ x0 :
ð5Þ
In the above problem, Xs is the state variable, D is the decision variable (represents the function D(Xt, t) of the state Xt and time t), f is the objective function, and G is the function of terminal reward. For a given D, dXs is defined by the uncertain differential equation (5), where n and r are two functions of Xs, D, and time s. The function J(0, x0) is the expected optimal reward obtainable in [0, T ] with the initial condition that at time 0 we are in state x0. For any 0 < t < T, J(t, x) is the expected optimal reward obtainable in [t, T ] with the condition that at time t we are in state Xt ¼ x. That is, we have hR i 8 T > < J ðt; xÞ sup E t f ðXs ; D; sÞds þ GðXT ; T Þ D ð6Þ subject to > : dXs ¼ nðXs ; D; sÞds þ rðXs ; D; sÞdCs and Xt ¼ x: Now we present the following principle of optimality for uncertain optimal control. Theorem 2 (Principle of Optimality). For any (t, x) 2 [0, T) R, and Dt > 0
with t þ Dt < T, we have J ðt; xÞ ¼ sup E D
where x þ DXt ¼ XtþDt.
Z
tþDt t
f ðXs ; D; sÞds þ J ðt þ Dt; x þ DXt Þ ;
ð7Þ
540
Y. Zhu
Proof. We denote the right side of (7) by Je(t, x). It follows from the defi-
nition of J(t, x) that J ðt; xÞ E
Z
tþDt
f ðXs ; Dj½t;tþDtÞ ; sÞds
t
þ
Z
T tþDt
f ðXs ; Dj½tþDt;T ; sÞds þ GðXT ; T Þ
ð8Þ
for any D, where Dj[t,tþDt) and Dj[tþDt,T ] are the values of decision variable D restricted on [t, t þ Dt) and [t þ Dt, T ], respectively. Because the uncertain processes dCs (s 2 [t, t þ Dt)) and dCs (s 2 [t þ Dt, T ]) are independent, we know that Z tþDt Z T f ðXs ; Dj½t;tþDtÞ ; sÞds and f ðXs ; Dj½tþDt;T ; sÞds Downloaded By: [Zhu, Yuanguo] At: 11:37 7 October 2010
t
tþDt
are independent. Thus, J ðt; xÞ E
Z
tþDt t
þE
f ðXs ; Dj½t;tþDtÞ ; sÞds Z
T tþDt
f ðXs ; Dj½tþDt;T ; sÞds þ GðXT ; T Þ
ð9Þ
by Theorem 1. Taking the supremum with respect to Dj[tþDt,T ] first, and then Dj[t,tþDt) in (9), we get J(t, x) Je(t, x). On the other hand, for all D, we have Z E
T
f ðXs ; D; sÞds þ GðXT ; T Þ
t
¼E E
Z
tþDt
f ðXs ; D; sÞds þ E
t
Z
Z
T tþDt
tþDt
f ðXs ; Dj½tþDt;T ; sÞds þ GðXT ; T Þ
f ðXs ; D; sÞds þ J ðt þ Dt; x þ DXt Þ
t
Jeðt; xÞ: Hence, J(t, x) (t, x), and then J(t, x) ¼ Je(t, x). The theorem is proved.
EQUATION OF OPTIMALITY Also, we assume that Ct is a canonical process. Consider the uncertain optimal control problem (6). Now let us give a fundamental result called equation of optimality in uncertain optimal control.
Uncertain Optimal Control
541
Theorem 3 (Equation of Optimality). Let J(t, x) be twice differentiable on
[0, T] R. Then we have Jt ðt; xÞ ¼ sup½ f ðx; D; tÞ þ Jx ðt; xÞnðx; D; tÞ;
ð10Þ
D
where Jt(t, x) and Jx(t, x) are the partial derivatives of the function J(t, x) in t and x, respectively. Proof. For any Dt > 0, we have
Z
tþDt
f ðXs ; D; sÞds ¼ f ðx; Dðt; xÞ; tÞDt þ oðDtÞ:
ð11Þ
t
Downloaded By: [Zhu, Yuanguo] At: 11:37 7 October 2010
By using Taylor series expansion, we get 1 J ðt þ Dt; x þ DXt Þ ¼ J ðt; xÞ þ Jt ðt; xÞDt þ Jx ðt; xÞDXt þ Jtt ðt; xÞDt 2 2 1 þ Jxx ðt; xÞDXt2 þ Jtx ðt; xÞDtDXt þ oðDtÞ: 2 Substituting Eqs. (11) and (12) into Eq. (7) yields 1 0 ¼ sup f ðx; D; tÞDt þ Jt ðt; xÞDt þ E Jx ðt; xÞDXt þ Jtt ðt; xÞDt 2 2 D 1 þ Jxx ðt; xÞDXt2 þ Jtx ðt; xÞDtDXt þ oðDtÞ : 2
ð12Þ
ð13Þ
Let n be an uncertain variable such that DXt ¼ n þ n(x, D, t)Dt. It follows from (13) that 0 ¼ supðf ðx; D; tÞDt þ Jt ðt; xÞDt þ Jx ðt; xÞnðx; D; tÞDt þ E ½ðJx ðt; xÞ D 1 2 þ Jxx ðt; xÞnðx; D; tÞDt þ Jtx ðt; xÞDtn þ Jxx ðt; xÞn þ oðDtÞÞ 2 ¼ supff ðx; D; tÞDt þ Jt ðt; xÞDt þ Jx ðt; xÞnðx; D; tÞDt þ E½an þ bn2 þ oðDtÞg; D
ð14Þ where a Jx(t, x) þ Jxx(t, x)n(x, D, t)Dt þ Jtx(t, x)Dt, and b 12 Jxx ðt; xÞ. It follows from the uncertain differential equation, the constraint in (6), that n ¼ DXt n(x, D, t)Dt is a normally distributed uncertain variable with expected value 0 and variance r2(x, D, t)Dt2. If b ¼ 0, then E[an þ bn2] ¼ aE[n] ¼ 0. Otherwise, Theorem A1 in the Appendix implies that ha i E½an þ bn2 ¼ bE n þ n2 ¼ oðDtÞ: ð15Þ b
542
Y. Zhu
Substituting Eq. (15) into Eq. (14) yields Jt ðt; xÞDt ¼ sup½f ðx; D; tÞDt þ Jx ðt; xÞnðx; D; tÞDt þ oðDtÞ:
ð16Þ
D
Dividing Eq. (16) by Dt, and letting Dt ! 0, we can obtain Eq. (10). Remark 1. The equation of optimality in uncertain optimal control gives a
necessary condition for an extremum. If the equation has solutions, then the optimal decision and optimal expected value of objective function are determined. If function f is convex in its arguments, then the equation will produce a minimum, and if f is concave in its arguments, then it will produce a maximum. We note that the boundary condition for the equation is J(T, XT) ¼ G(XT, T). Remark 2. We note that in the equation of optimality for stochastic optimal
Downloaded By: [Zhu, Yuanguo] At: 11:37 7 October 2010
control (called the Hamilton-Jacobi-Bellman equation), there is an extra term 1 2 2 Jxx ðt; xÞr ðx; D; tÞ.
A PORTFOLIO SELECTION MODEL A Portfolio selection problem is a classical problem in financial economics of allocating personal wealth between investment in a risk-free security and investment in a single risk asset. Under the assumption that the risk asset earns a random return, Merton (1971) studied a portfolio selection model by stochastic optimal control, and Kao (1997) considered a generalized Merton’s model. If we assume that the risk asset earns an uncertain return, this generalized Merton’s model may be solved by uncertain optimal control. Let Xt be the wealth of an investor at time t. The investor allocates a fraction w of the wealth in a sure asset and the remainder in a risk asset. The sure asset produces a rate of return b. The risk asset is assumed to earn an uncertain return and yields a mean rate of return l (l > b) along with a variance of r2 per unit time. That is, the risk asset earns a return drt in time interval (t, t þ dt), where drt ¼ ldt þ rdCt, and Ct is a canonical process. Thus, Xtþdt ¼ Xt þ bwXt dt þ drt ð1 wÞXt ¼ Xt þ bwXt dt þ ðldt þ rdCt Þð1 wÞXt ¼ Xt þ ½bw þ lð1 wÞXt dt þ rð1 wÞXt dCt : Assume that an investor is interested in maximizing the expected utility over an infinite time horizon. Then a portfolio selection model is provided by 8 hR i k 1 > < J ðt; xÞ max E 0 e bt ðwXk t Þ dt w
subject to > : dXt ¼ ½bwXt þ lð1 wÞXt dt þ rð1 wÞXt dCt ;
543
Uncertain Optimal Control
where b > 0, 0 < k < 1. By the equation of optimality (10), we have ( ) k ðwxÞ Jt ¼ max ebt þ ðb lÞwxJx þ lxJx ¼ max LðwÞ; w w k where L(w) represents the term in the braces. The optimal w satisfies @LðwÞ ¼ e bt ðwxÞk1 x þ ðb lÞxJx ¼ 0; @w or 1 1
ðl bÞJx e bt k1 : x
w¼
Downloaded By: [Zhu, Yuanguo] At: 11:37 7 October 2010
Hence,
k
1 1 Jt ¼ ebt ðl bÞJx ebt k1 þðb lÞ ðl bÞJx ebt k1 Jx þ lxJx ; k or Jt e bt ¼
k 1 1 ðl bÞJx e bt k1 þ lxJx e bt : k
ð17Þ
We conjecture that J(t, x) ¼ kxkebt. Then Jt ¼ kbx k ebt ;
Jx ¼ kkx k1 ebt :
Substituting them into Eq. (17) yields k k 1 k 1 ðl bÞk1 ðkkÞk1 x k þ lkkx k ; kbx ¼ k or b lk
1
ðkkÞk1 ¼
k
:
ð1 kÞðl bÞk1
So we get kk ¼
b lk k1 1 : 1k ðl bÞk
Therefore, the optimal fraction of investment on sure asset is determined by 1
1
w ¼ ðl bÞk1 ðkkÞk1 ¼
b lk : ð1 kÞðl bÞ
544
Y. Zhu
Remark 3. Note that the optimal fraction of investment on sure asset or risk asset is independent of total wealth. This conclusion is similar to that in the case of randomness (Kao 1997).
CONCLUSION
Downloaded By: [Zhu, Yuanguo] At: 11:37 7 October 2010
Based on the concept of canonical process, we studied an uncertain optimal control problem: optimizing the expected value of an objective function subject to an uncertain differential equation. By using Bellman’s principle of optimality in dynamic programming, we presented the principle of optimality and a fundamental result called the equation of optimality for uncertain optimal control. As an application of the equation of optimality, we solved a portfolio selection model.
REFERENCES Chen, X. and Liu, B. 2010. Existence and uniqueness theorem for uncertain differential equations. Fuzzy Optimization and Decision Making 9(1): 69–81. Dixit, A. K. and Pindyck, R. S. 1994. Investment under uncertainty. Princeton, NJ: Princeton University Press. Fleming, W. H. and Rishel, R. W. 1986. Deterministic and stochastic optimal control. New York: Springer-Verlag. Gao, X. 2009. Some properties of continuous uncertain measure. International Journal of Uncertainty, Fuzziness, & Knowledge-Based Systems 17(3): 419–426. Harrison, J. M. 1985. Brownian motion and stochastic flow systems. New York: John Wiley & Sons. Kao, E. P. C. 1997. An introduction to stochastic processes. Wadsworth Publishing Company, Belmont, CA. Karatzas, I. 1989. Optimization problems in the theory of continuous trading. SIAM Journal on Control and Optimization 27(6): 1221–1259. Liu, B. 2007. Uncertainty theory, 2nd ed. Berlin: Springer-Verlag. Liu, B. 2008. Fuzzy process, hybrid process, and uncertain process. Journal of Uncertain Systems 2(1): 3–16. Liu, B. 2009a. Some research problems in uncertainty theory. Journal of Uncertain Systems 3(1): 3–10. Liu, B. 2009b. Theory and practice of uncertain programming, 2nd ed. Berlin: Springer-Verlag. Liu, B. 2010. Uncertainty theory: A branch of mathematics for modeling human uncertainty. Berlin: Springer-Verlag. Merton, R. C. 1971. Optimal consumption and portfolio rules in a continuous time model. Journal of Economic Theory 3: 373–413.
545
Uncertain Optimal Control
APPENDIX Let us give an estimation for the expected value of an þ n2 if n is a normal uncertain variable. Theorem A1. Let n be a normal uncertain variable with expected value 0
and variance r2(r > 0), whose uncertainty distribution is px 1 ; UðxÞ ¼ 1 þ exp pffiffiffi 3r
x 2 0
because the similar method is suitable to the case that a 0. Let
x1 ¼
a
pffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi a2 þ 4r ; 2
x2 ¼
a þ
pffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi a2 þ 4r ; 2
which is derived from the solutions of the equation ax þ x2 ¼ r for any real number r. Denote y0 ¼ a2=4. Then
2
E½an þ n ¼
Z
þ1
Mfan þ n rgdr
0
¼
Z
0
2
Z
0
Mfan þ n2 rgdr
y0 þ1
Mfðn x1 Þ [ ðn x2 Þgdr
Z
0
Mfðn x1 Þ \ ðn x2 Þgdr:
y0
ðA2Þ Because Mfn x2 g ¼ Mf½ðn x1 Þ \ ðn x2 Þ [ ðn x1 Þg Mfðn x1 Þ \ ðn x2 Þg þ Mfn x1 g;
546
Y. Zhu
we have Mfðn x1 Þ \ ðn x2 Þg Mfn x2 g Mfn x1 g ¼ Uðx2 Þ Uðx1 Þ: We note that Mfðn x1 Þ [ ðn x2 Þg Mfn x1 g þ Mfn x2 g ¼ Uðx1 Þ þ 1 Uðx2 Þ: Hence, it follows from (A2) that
2
E½an þ n
Z
þ1
Uðx1 Þdr þ
Downloaded By: [Zhu, Yuanguo] At: 11:37 7 October 2010
0
Z
Z
þ1
0
½1 Uðx2 Þdr
Z
0
½Uðx2 Þ Uðx1 Þdr y0
Z þ1 1 1
dr þ dr ¼ 0 0 1 þ exp ppxffiffi31r 1 þ exp ppxffiffi32r Z 0 Z 0 1 1
dr dr þ px pxffiffi1 y0 1 þ exp pffiffi2 y0 1 þ exp p 3r 3r Z 1 Z þ1 a þ 2x a þ 2x
dx þ dx ¼ px a 1 þ exp pffiffi 0 ppx ffiffi 1 þ exp 3r 3r Z 0 Z a a þ 2x a þ 2x
dx þ
dx a=2 1 þ exp ppx a=2 1 þ exp ppx ffiffi ffiffi 3r 3r Z þ1 Z þ1 a 2x a þ 2x ðdxÞ þ dx ¼ a 0 1 þ exp ppxffiffi3r 1 þ exp ppxffiffi3r Z a Z 0 a 2x a 2x ðdxÞ þ
ðdxÞ a=2 1 þ exp ppx a=2 1 þ exp ppx ffiffi ffiffi 3r 3r Z a Z þ1 1 x dx þ 2 dx ¼a 0 1 þ exp ppx a ffiffi 1 þ exp ppxffiffi3r 3r Z þ1 Z a x a 2x dx dx þ2 px 0 0 1 þ exp ppx ffiffi 1 þ exp pffiffi3r 3r Z þ1 x dx ¼4 0 1 þ exp ppxffiffi þ1
3r
2
¼r :
ðA3Þ
547
Uncertain Optimal Control
On the other hand, becasue Mfðn x1 Þ [ ðn x2 Þg Mfn x2 g ¼ 1 Uðx2 Þ; and Mfðn x1 Þ \ ðn x2 Þg Mfn x2 g ¼ Uðx2 Þ; we have from (A2) that Z þ1 Z 0 2 ð1 Uðx2 ÞÞdr Uðx2 Þdr E½an þ n ¼
Z
0
y0 þ1
1 þ exp
Downloaded By: [Zhu, Yuanguo] At: 11:37 7 October 2010
0
Z
1
þ1
pxffiffi2 p 3r
dr
0
y0
Z
1
dr 1 þ exp ppxffiffi32r
0
a þ 2x
dx 0 a=2 1 þ exp ppx ffiffi 3r Z þ1 Z a=2 x x dx þ 2 dx ¼2 0 0 1 þ exp ppxffiffi3r 1 þ exp ppxffiffi3r Z þ1 1 dx þa a=2 1 þ exp ppx ffiffi 3r pffiffiffi Z þ1 2 2 Z 2ppaffi3r r 6r z 1 3ar dz þ dz ¼ þ 2 z paffi 1 þ ez p 2 p 0 1þe p
¼
a þ 2x dx 1 þ exp ppxffiffi3r
Z
2 3r
2
r : 2
Combining (A3) and (A4) yields the conclusion.
ðA4Þ
