c 2003 Society for Industrial and Applied Mathematics
SIAM J. APPL. MATH. Vol. 63, No. 6, pp. 1954–1971
OPTIMAL CONTROL APPLIED TO COMPETING CHEMOTHERAPEUTIC CELL-KILL STRATEGIES∗ K. RENEE FISTER† AND JOHN CARL PANETTA‡ Abstract. Optimal control techniques are used to develop optimal strategies for chemotherapy. In particular, we investigate the qualitative differences between three different cell-kill models: logkill hypothesis (cell-kill is proportional to mass); Norton–Simon hypothesis (cell-kill is proportional to growth rate); and, Emax hypothesis (cell-kill is proportional to a saturable function of mass). For each hypothesis, an optimal drug strategy is characterized that minimizes the cancer mass and the cost (in terms of total amount of drug). The cost of the drug is nonlinearly defined in one objective functional and linearly defined in the other. Existence and uniqueness for the optimal control problems are analyzed. Each of the optimality systems, which consists of the state system coupled with the adjoint system, is characterized. Finally, numerical results show that there are qualitatively different treatment schemes for each model studied. In particular, the log-kill hypothesis requires less drug compared to the Norton–Simon hypothesis to reduce the cancer an equivalent amount over the treatment interval. Therefore, understanding the dynamics of cell-kill for specific treatments is of great importance when developing optimal treatment strategies. Key words. optimal control, cancer, cell-kill AMS subject classifications. 49K20, 35F20 DOI. 10.1137/S0036139902413489
1. Introduction. When developing effective treatment strategies, understanding the effects of chemotherapeutic drugs on tumors is of primary importance. Several approaches to modeling chemotherapeutic induced cell-kill (killing of tumor cells) have been developed. One of the early approaches was by Schabel, Skipper, and Wilcox [1] who proposed that cell-kill due to a chemotherapeutic drug was proportional to the tumor population. This hypothesis is based on in vitro studies in the murine leukemia cell-line L1210. It states that for a fixed dose, the reduction of large tumors occurred more rapidly than for smaller tumors. Skipper’s concept is referred to as the log-kill mechanism. Norton and Simon [2, 3] find this model to be inconsistent with clinical observations of Hodgkin’s disease and acute lymphoblastic leukemia which showed that, in some cases, reduction in large tumors was slower than in histologically similar smaller tumors. Therefore, Norton and Simon hypothesize that the cell-kill is proportional to the growth rate (e.g., exponential, logistic, or Gompertz) of the tumor. A third hypothesis notes that some chemotherapeutic drugs must be metabolized by an enzyme before being activated. This reaction is saturable due to the fixed amount of enzyme. Thus, Holford and Sheiner [4] develop the Emax model which describes cell-kill in terms of a saturable function of Michaelis–Menton form. In this study, we use optimal control theory to evaluate and compare effective treatment strategies for each of these models by developing formal mathematical ∗ Received by the editors August 28, 2002; accepted for publication (in revised form) February 21, 2003; published electronically September 4, 2003. http://www.siam.org/journals/siap/63-6/41348.html † Department of Mathematics and Statistics, Murray State University, Murray, KY 42071 (renee. fi
[email protected]). The research of this author was supported by a KY NSF EPSCoR Research Enhancement grant. ‡ Department of Pharmaceutical Sciences, St. Jude Children’s Research Hospital, 332 North Lauderdale St., Memphis, TN 38105-2794 (
[email protected]). The research of this author was supported by Cancer Center CORE grant CA21765, a Center of Excellence grant from the State of Tennessee, and American Lebanese Syrian Associated Charities (ALSAC).
1954
OPTIMAL CONTROL
1955
criteria to be minimized. These include tumor mass and dose of drug. We give a mathematically detailed development of optimal control forms for the various growth and drug terms that are subject to different objective functionals. We also show therapeutically significant differences between the cell-kill hypotheses and their effect on treatment schedules. We have previously developed a treatment strategy using optimal control techniques for the use of cell-cycle specific drugs such as Taxol for the reduction of breast and ovarian cancers [5]. The model included a resting phase which made it more realistic in the clinical setting. Among other things, the model showed that treating with repeated shorter periods allows more drug to be given without excess damage to the bone marrow. Similar results were also observed in [6]. Several other models where optimal control methods have been utilized in analyzing effective chemotherapeutic treatments include Swan [7, 8] and Murray [9]. Swan [7, 8] obtained feedback treatment control drug characterizations for cancer models under a quadratic performance criterion. Murray [9] has considered systems of normal and tumor cells under the hypotheses of Gompertzian and logistic growth in which he controls the rate of administration of drugs. Murray has minimized the tumor burden at the end of treatment and, in another application, the toxicity level, defined as the area under the drug concentration curve. 2. The model. Mathematically, the general form of the model under investigation is depicted by the differential equation: (2.1)
dN = rN F (N ) − G(N, t), dt
where N is the tumor volume, r is the growth rate of the tumor, F (N ) is the generalized growth function. For the proposed model, we allow for Gompertzian growth: Θ (2.2) . F (N ) = ln N The function G(N, t) describes the pharmacokinetic and pharmacodynamic effects of the drug on the system. In this study, we compare three cell-kill strategies. These include the following: • G(N ) = δu(t)N : Skipper’s log-kill (i.e., percentage kill) hypothesis, • G(N ) = δu(t)N/(K + N ): Emax model, and • G(N ) = δu(t)F (N ): Norton–Simon hypothesis, where δ is the magnitude of the dose and the control, u(t), describes the time dependent pharmacokinetics of the drug; i.e., u(t) = 0 implies no drug effect is present and u(t) > 0 implies the amount or strength of the drug effect. We investigated the differences and similarities among the three drug effects via optimal control techniques for ordinary differential equations. We considered two objective functionals when determining the minimum amount of drug needed to reduce or eliminate the tumor mass. One criterion considered is (2.3)
Jα (u) =
0
T
a(N − Nd )2 + bu2 dt,
where the measure of the “closeness” of the tumor mass to the desired tumor density, Nd , and the cost of the control, u(t), are minimized over the class of measurable,
1956
K. RENEE FISTER AND JOHN CARL PANETTA
nonnegative controls. Here, a and b are positive weight parameters. The second criterion we considered (previously used by Boldrini and Costa [10] with variations by Murray [11] and Martin and Teo [12]) is (2.4)
Jβ (u) = aN (T ) + b
0
T
u(t) dt,
in which the tumor burden at the end of treatment (the first term in (2.4)) and the toxicity (the second term in (2.4)), in terms of area under the drug concentration curve, are minimized over the class of measurable, nonnegative controls. After scaling the models using N = N/Θ, k = k/Θ, and δ = δ/Θ and dropping the bars, we have the following three state equations (which will henceforth be referred to as P1, P2, and P3, respectively), all with the same initial condition of N (0) = N0 , where 0 < N0 < 1 since the tumor cells have been normalized via the above change of variables: dN 1 − u(t)δN, P1 = rN ln dt N dN = rN ln dt
P2
dN = rN ln dt
P3
1 N
1 N
− u(t)
δN , k+N
[1 − δu(t)] .
Ultimately, we determine the unique characterization of the optimal control u(t) in the admissible control class, (2.5)
U = {u measurable |0 ≤ u(t), t ∈ [0, T ]}
or (2.6)
V = {u measurable |0 ≤ u(t) ≤ M, t ∈ [0, T ]} ,
such that the objective functionals Jα and Jβ are minimized over the class of controls, U and V , respectively. In sections 3.1–3.3, we consider the existence issues, the characterization of the optimal control, and the uniqueness concept in association with problems P1–P3 such that the objective functional (2.3) involving the nonlinear control term is minimized over the class of controls, U . In sections 4.1–4.2, we discuss the existence of an optimal control and its characterization such that it minimizes the second objective functional (2.4) subject to each of the differential equations represented in P1–P3. Also, in section 5, numerical simulations representing the control situations in relation to the two objective functionals as well as the different cell-kill hypotheses depicted in the differential equations are analyzed. 3. Nonlinear control. 3.1. Existence. First, the existence of the state solution to each of problems P1– P3 given an optimal control in the admissible set, U , is shown. Also, the existence of the optimal control for the state system is analyzed.
1957
OPTIMAL CONTROL
Theorem 3.1. Given u ∈ U , there exists a bounded solution solving each of the problems (P1)–(P3). Proof. We consider the following differential equations in relation to P1, P2, P3, respectively. The state variables N1 (t), N2 (t), and N3 (t) represent supersolutions for problems P1, P2, and P3. (3.1)
dN1 = r, dt
(3.2)
dN2 = r + u(t)δN2 , dt
(3.3)
dN3 = −rN3 ln N3 (1 − u(t)δ). dt
Since N (t) > 0 and ln N1 ≤ N1 , then equation (3.3) follows from P1. Using that 0 ≤ t ≤ T , we can show that (3.4) (3.5) (3.6)
N1 (t) ≤ rT + N0 , δ
N2 (t) ≤ (N0 + rT )e (e
N3 (t) ≤ N0
−rt+rδ
T 0
T 0
u(s) ds
u(s) ds
,
)
.
Since u(t) ∈ U , then, along with N1 (t), N2 (t) and N3 (t) are bounded above. Via a maximum principle [13] and standard existence theory for first-order nonlinear differential equations, we obtain the existence of a solution to each of the problems P1–P3. Next, the existence of an optimal control for the state system is analyzed. Using the fact that the solution to each state equation is bounded, the existence of an optimal control for each problem can be determined using the theory developed by Fleming and Rishel [14]. T Theorem 3.2. Given the objective functional, Jα (u) = 0 a(N − Nd )2 + bu2 dt, where (3.7)
U = {u measurable |0 ≤ u(t), t ∈ [0, T ]}
and each of the problems P1–P3 with N (0) = N0 , then there exists an optimal control u∗ associated with each problem P1–P3 such that minu∈U Jα (u) = Jα (u∗ ) if the following conditions are met: (i) The class of all initial conditions with a control u in the admissible control set along with each state equation being satisfied is not empty. (ii) The admissible control set U is closed and convex. (iii) Each right-hand side of P1–P3 is continuous, is bounded above by a sum of the bounded control and the state, and can be written as a linear function of u with coefficients depending on time and the state. (iv) The integrand of (2.3) is convex on U and is bounded below by −c2 + c1 |u|η with c1 > 0 and η > 1. Proof. Since each problem has a bounded solution for the initial condition, given an optimal control, by Theorem 3.1, then part (i) is established. By definition, U is
1958
K. RENEE FISTER AND JOHN CARL PANETTA
closed and convex. To complete part (iii) we first reconsider the right-hand sides of P1–P3 below: 1 f (t, N (t), u(t)) = rN ln − δN u(t), N 1 δN g(t, N (t), u(t)) = rN ln − u(t), N k+N 1 1 h(t, N (t), u(t)) = rN ln − δrN ln u(t). N N We see by the representations of f, g, and h that they are continuous in t, u, and N since N (t) > 0. Also, they are each written as a linear function of the control with coefficients depending on time and the state. For the boundedness requirement, we use the bounds in the proof of Theorem 3.1 to obtain the result. Consequently, 1 |f (t, N (t), u(t))| ≤ rN ln + |δN (t)u(t)| N ≤ r + δ|u(t)|(N0 + rT ) ≤ C1 (1 + |u(t)| + |N (t)|), where C1 depends on r, d, N0 , and T ,
1 δN (t) |g(t, N (t), u(t))| ≤ rN ln + u(t) N k + N (t) ≤ r + δ|u(t)| ≤ r + δ|u(t)| + |N (t)|,
and
1 1 |h(t, N (t), u(t))| ≤ rN ln + δrN (t) ln u(t) N N ≤ r + δr|u(t)| ≤ C2 (1 + |u(t)| + |N (t)|),
where C2 depends on r and d. Hence, the right-hand side of each state equation is bounded above by a sum of the control and the state. Lastly, the integrand of the objective functional is convex on U. One can consider the second partial of the integrand of the objective functional with respect to the control and find that it is positive. To obtain the necessary lower bound for the integrand, we see the a(N − Nd )2 + bu2 ≥ bu2 ≥ −c + bu2 for any c > 0. Therefore, part (iv) is complete and so is the proof. 3.2. Characterization of optimal control. Since an optimal control exists for minimizing the objective functional (2.3) subject to each of the three equations P1–P3 with the initial conditions, the necessary conditions for an optimal control for each problem are determined. For brevity, we derive the conditions using a version of Pontryagin’s maximum principle for P3 [15, 16]. Then we give the optimality system, which is the state system coupled with the adjoint system, for each problem. In order to derive the necessary conditions, we define the Lagrangian associated with Jα (u) subject to P3 as 1 (3.8) L(N, u, λ3 , w1 ) = a(N − Nd )2 + bu2 + λ3 rN ln (1 − δru(t)) − w1 (t)u(t), N where w1 (t) ≥ 0 is a penalty multiplier satisfying w1 (t)u(t) = 0 at the optimal u∗ .
OPTIMAL CONTROL
1959
Similar definitions for hold for Jα (u) subject to P1 and P2. Theorem 3.3. Given an optimal control u∗ and solution of the corresponding state equation (P3), there exists an adjoint variable λ3 satisfying the following:
dλ3 ∂L 1 (3.9) =− = − 2a(N − Nd ) + λ3 r(1 − uδ) ln −1 dt ∂N N with λ3 (T ) = 0. Further, u∗ (t) can be represented by −λ δrN ln N + 3 . u∗ (t) = 2b Proof. The existence of the adjoint solution is found via a maximum principle satisfying [13]. Using the Lagrangian (3.8), we complete the representation for u∗ by analyzing the optimality condition ∂L ∂u = 0. Upon some algebraic manipulation, the representation of u∗ becomes −λ3 δrN ln N + w1 . 2b To determine an explicit expression for the optimal control, without w1 , a standard optimality technique is utilized. The optimal control is characterized as −λ δrN ln N + 3 u∗ (t) = (3.10) , 2b where r if r > 0, + r = (3.11) 0 if r ≤ 0. u∗ (t) =
Similarly, we can find the representations for the controls associated with problems δ P1 and P2 that are subject to Jα . The associated control for P1 is u∗ (t) = 2b (λ1 N )+ δ λ2 N + ∗ and the control for P2 is u (t) = 2b ( k+N ) . Using this explicit representation for the control, the adjoint equation coupled with the state equation and the initial and transversality conditions form the optimality system. The optimality systems associated with each of the state equations and their associated adjoint equations are given below. We note that Optimality System 1 is associated with P1 and its adjoint, Optimality System 2 is associated with P2 and its adjoint, and Optimality System 3 is associated with P3 and its adjoint. Optimality System 1 (OS1). dN δ2 1 − N (λ1 N )+ , = rN ln dt 2b
N
δ2 1 dλ1 + = − 2a(N − Nd ) + λ1 r ln − r − (λ1 N ) dt N 2b with N (0) = N0 and λ1 (T ) = 0. Optimality System 2 (OS2). dN 1 δ 2 N λ 2 N + = rN ln − , dt N 2b k + N k + N
λ N + kδ 2 1 dλ2 2 = − 2a(N − Nd ) + λ2 r ln −r− dt N 2b(k + N )2 k + N with N (0) = N0 and λ2 (T ) = 0.
1960
K. RENEE FISTER AND JOHN CARL PANETTA
Optimality System 3 (OS3). 1 δ2 r dN = rN ln 1− (−λ3 N ln N )+ , dt N 2b
1 dλ3 δ2 r + −1 ln = − 2a(N − Nd ) + λ3 r 1 − (−λ3 N ln N ) dt 2b N with N (0) = N0 and λ3 (T ) = 0. 3.3. Uniqueness. Here, we focus our attention on OS1 and note that similar analysis gives the uniqueness of the OS2 and OS3. The optimal control depends on the adjoint and the state variables. By proving the optimality system has a unique solution, we will argue that the optimal control is unique as well. We recognize that since the tumor mass, N (t), is bounded, then the adjoint equation (3.12) has a bounded right-hand side that is dependent on the final time T . Hence, there exists a D > 0, depending on the coefficients of the state equation and the uniform bound for N (t) such that |λ1 (t)| < DT on [0,T]. Theorem 3.4. For T sufficiently small, the solution to Optimality System 1 is unique. Proof. We suppose that (N, λ1 ) and (M, ψ) are two distinct solutions to OS1. Let m > 0 be chosen such that N = emt v, M = emt w, λ1 = e−mt y, and ψ = e−mt z. Also, d δ we have that u = 2b (yv)+ and f = 2b (wz)+ . Upon substitution of the representations for N , M , λ1 , and ψ into the state and adjoint equations followed by simplification, we obtain the following equations: dv δ2 + mv = rv(−mt − ln v) − v(yv)+ , dt 2b dw δ2 + mw = rw(−mt − ln w) − w(wz)+ , dt 2b dy δ2 − my = −2av + 2aemt Nd − yr(−mt − ln v) + ry + y(yv)+ , dt 2b δ2 dz − mz = −2aw + 2aemt Nd − zr(−mt − ln w) + rz + z(wz)+ dt 2b with v(0) = N0 , w(0) = N0 , y(T ) = 0, and z(T ) = 0. The next step is to subtract the equations corresponding to v, w, y, z. Then each of these differences are multiplied by an appropriate function and are integrated from 0 to T . We obtain the following two equations for the modified state and the adjoint: T T 1 2 2 [v(T ) − w(T )] + m (v − w) dt = mr t(v − w)2 dt 2 0 0 T [v ln v − w ln w](v − w) dt −r 0
−
2 T
δ 2b
0
(v(yv)+ − w(wz)+ )(v − w) dt
1961
OPTIMAL CONTROL
and
T T 1 2 2 [y(0) − z(0)] + m (y − z) dt = 2a (v − w)(y − z) dt 2 0 0 T 2 t(y − z) dt + r + mr 0
+r +
T
0
δ2 2b
0
T
(y − z)2 dt
[y ln v − z ln w](y − z) dt
0
T
(y(yv)+ − z(wz)+ )(y − z) dt.
We need to estimate several terms in order to obtain the uniqueness result. For explanation, we include one estimate below where the boundedness of the state variables and Cauchy’s inequality are used: T T (v ln v − w ln w)(v − w) dt = [v(ln v − ln w) + (v − w) ln w](v − w) dt 0 0 T v (v − w) + (v − w)2 ln w dt = v ln 0 w T 2 v ≤ |(v − w)| + |w|(v − w)2 dt w 0 T (v − w)2 dt. ≤ T C1 0
In the estimate above, C1 depends on the bounds of the state variables and the coefficients. To complete this uniqueness proof, we add the two integral equations together and bounds the terms to obtain T 1 1 [v(T ) − w(T )]2 + [y(0) − z(0)]2 + m [(v − w)2 + (y − z)2 ] dt 2 2 0 T [(v − w)2 + (y − z)2 ] dt, ≤ ((mr + C2 )T ) 0
where C2 depends on the coefficients and the bounds of the state and the adjoint variables. Since the variable expressions evaluated at the initial and the terminal times are nonnegative, the inequality reduces to T (3.12) (v − w)2 + (y − z)2 dt ≤ 0. (m − mrT − C2 T ) 0
For the optimality system to be unique, we must choose m such that m > thus,
C2 1−r
and,
m − mrT − C2 T > 0. m . Moreover, OS1 has a unique solution. For this choice of m we have that T < mr+C 2 Since the characterization of the optimal control directly depends on the state and the adjoint solutions, which are unique, then the optimal control corresponding to OS1 is unique. Similar results give uniqueness for Optimality Systems 2 and 3.
1962
K. RENEE FISTER AND JOHN CARL PANETTA
4. Linear control. In this case we still consider the same three differential equations, P1–P3, subject to their initial conditions. However, in this case, we determine the existence and the characterization of an optimal control in the admissible control class, V , such that the objective functional (2.4), T Jβ (u) = aN (T ) + b (4.1) u(t) dt, 0
is minimized over this class of controls. The goal is to find an optimal control, u∗ , such that min J(u) = J(u∗ ).
0≤u≤M
4.1. Existence. In subsection 3.1, we obtain the existence of the state solution for each problem P1–P3 given an optimal control in U . This work can be extended directly because the only change is that u(t) is bounded above by a maximum amount of drug M. For simpler discussions, we transform the original problems P1–P3 via x = ln N . Consequently, we minimize T x(T ) J1 (u) = ae (4.2) +b u(t) dt 0
over the class of admissible controls, V , subject to each of the three differential equations that correspond to P1, P2, and P3, respectively. We note that k = N0 and that the initial condition is x(0) = ln N0 and is negative since 0 < N0 < 1. dx = −rx − u(t)δ, dt dx δ = −rx − u(t) (4.4) , dt k + ex dx (4.5) = rx(u(t)δ − 1). dt The theorem for the existence of an optimal control for the appropriate objective functional is stated below. Since the proof involves standard arguments, it is omitted. For further information, see [14]. Theorem 4.1. There exists an optimal control in V that minimizes the objective functional J1 (u) subject to (4.3), (4.4), and (4.5), respectively. (4.3)
4.2. Characterization of optimal control. Since an optimal control exists, we determine the characterization for each optimal control u(t) associated with each problem (4.3), (4.4), and (4.5) that minimizes J1 (u). We use Pontryagin’s maximum principle [17] to obtain the necessary conditions for optimality for each problem. Theorem 4.2. Given an optimal control u∗ (t) and a solution, x(t) to (4.3), there exists an adjoint ψ1 satisfying dψ1 = rψ1 (t), dt ψ1 (T ) = aex(T ) . Furthermore, (4.6)
∗
u (t) =
M
if
0
if
ψ1 (t) > δb , ψ1 (t) < δb .
1963
OPTIMAL CONTROL
Note that this is a problem similar to Swierniak and Duda [18]. We note that ψ1 (t) = aex(T )−r(T −t) and a singular control could not exist. (A singular control exists when the Hamiltonian is linear in the control and the coefficient of the control is zero for some time interval.) If one were to exist, then ψ1 (t) must equal δb on some interval inclusive to [0,T]. This cannot occur since ψ1 (t) would be constant only for one instant in time. Furthermore, Swierniak and Duda provide conditions for the representation of the control u∗ (t) in terms of the model parameters. Please see [18, 19] for complete details. For problem (4.4), we have the following result. Theorem 4.3. Given an optimal control u∗ (t) and corresponding solution x∗ (t) to (4.4), there exists an adjoint ψ2 satisfying u(t)δex(t) dψ2 = ψ2 (t) r − , dt (k + ex(t) )2
(4.7)
∗
ψ2 (T ) = aex
(4.8) In addition, (4.9)
∗
u (t) =
(T )
.
0
if
b−
M
if
b−
ψ2 (t)δ (k+ex∗ (t) ) ψ2 (t)δ (k+ex∗ (t) )
> 0, < 0.
Proof. To determine the representation for u∗ (t) and the differential equation associated with ψ2 (t), we form the Hamiltonian. We note via Pontryagin’s maximum principle [17] that if u∗ is an optimal control associated with a corresponding trajectory on [0, T ], then there exists λ0 ≥ 0 and an absolutely continuous function λ : [0, T ] → R such that (λ0 , λ(t) = (0, 0)) for all t ∈ [0, T ] and λ(t) satisfies (4.7) and u(t)δ λ(T ) = λ0 aex(T ) . This optimal control minimizes H = λ0 bu(t)+λ(−rx(t)− (k+e x(t) )2 ) over V . Yet, λ0 cannot vanish for this problem because, if it did, then λ(T ) = 0 and hence λ(t) = 0 on [0, T ]. This contradicts the nontriviality of the multipliers. Therefore, without loss of generality, we let λ0 = 1. Consequently, we consider the following Hamiltonian, where we omit the asterisks for simplicity: u(t)δ H(t, x(t), u(t), ψ2 (t)) = bu(t) + ψ2 (t) −rx(t) − . (k + ex(t) ) We note that from standard existence theory we obtain the existence of ψ2 (t) solving (4.7) given that x(t) is bounded. The necessary conditions of optimality give that ψ2 (t)δ 0 if b − (k+e x∗ (t) ) > 0, ψ2 (t)δ M if b − (k+e u(t) = (4.10) x∗ (t) ) < 0, ψ (t)δ 2 singular if b − = 0. (k+ex∗ (t) ) We note that
T
x(T )−
ψ2 (t) = ae
is always positive on [0,T] since a > 0.
t
r−
u(t)δex(t) ds (k + ex(t) )2
1964
K. RENEE FISTER AND JOHN CARL PANETTA
We suppose that the control is singular on (t0 , t1 ) ⊂ [0, T ], i.e., b−
(4.11)
ψ2 (t)d =0 (k + ex(t) )
on that interval. We take the derivative with respect to time of (4.11) and obtain, after simplification, (4.12)
(k + ex(t) )
dx dψ2 (t) − ψ2 (t)ex(t) = 0. dt dt
Next, we substitute the right-hand sides of the differential equations for ψ2 (t) and for x(t) associated with problem (4.4) and find that u(t)δex(t) u(t)δ x (k + ex(t) ) ψ2 (t) r − − e = 0, ψ (t) −rx(t) − 2 (k + ex(t) )2 (k + ex(t) ) ψ2 (t)r(k + ex(t) + x(t)ex(t) ) = 0. Since ψ2 (t) > 0 on [0,T] and r > 0, then k + ex(t) + x(t)ex(t) = 0.
(4.13)
This immediately gives that x(t) is constant. Since x(t) is constant, then u(t) is constant here. We make note that with the fixed final time that H(t, x, u, ψ2 ) is constant [20]. Since H(t, x, u, ψ2 ) = bu + ψ2 dx dt = bu in this case and we are minimizing H, then u(t) = 0. However, for this to occur ψ2 δ b − (k+e x ) > 0, in (4.10), which contradicts our original assumption for the control to be singular. Thus, a singular control does not exist and our control is ψ2 (t)δ 0 if b − (k+e x∗ (t) ) > 0, (4.14) u∗ (t) = ψ (t)δ 2 M if b − < 0. (k+ex∗ (t) ) We now determine the characterization of the optimal control to minimize J1 (u) subject to (4.5). Theorem 4.4. Given an optimal control u∗ and a corresponding solution x∗ (t) to (4.5), there exists an adjoint ψ3 (t) satisfying (4.15) (4.16)
dψ3 = −ψ3 (t)r(u∗ (t)δ − 1), dt ∗ ψ3 (T ) = aex (T )
with (4.17)
u∗ (t) =
0
if
x(T )ex(T ) >
M
if
x(T )ex(T )
0, then ψ3 (t)x(t) < 0 on [0,T]. The necessary conditions for optimality give that if b + ψ3 (t)δrx(t) > 0, 0 M if b + ψ3 (t)δrx(t) < 0, u(t) = (4.20) singular if b + ψ3 (t)δrx(t) = 0. We see that x(T )−r
ψ3 (t)x(t) = (ae
T t
(1−u(s)δ) ds
x(T )−r(T −t)+rδ
= ax(0)e
x(T )−rT +rδ
= ax(0)e
T 0
−rt+rδ
)(x(0)e
T t
u(s) ds−rt+rδ
t 0
t 0
u(s) ds
)
u(s) ds
u(s) ds
= ax(T )ex(T ) . Hence, ψ3 (t)x(t) is constant on [0,T]. This means that u∗ (t) must be either zero, its maximum value—M , or its singular representation on the entire interval [0,T]. Using that ψ3 (t)x(t) is constant and that the Hamiltonian is to be minimized, we can exclude the singular case. If the control is singular, then the Hamiltonian is equal to −ψ3 (t)x(t)r. We can see for the case, u = M , that H(t, x(t), u(t), ψ3 (t)) < −ψ3 (t)x(t)r. Moreover, for the case u = 0 we see that the Hamiltonian is bounded strictly above by δb , which is the value of the Hamiltonian if the control is singular. Consequently, a singular control will not generate the minimum value for the Hamiltonian. Therefore, the necessary conditions for optimality are −b , 0 if x(T )ex(T ) > aδr ∗ u (t) = (4.21) −b x(T ) M if x(T )e < aδr . −b . We simply need to check if the expression x(T )ex(T ) is smaller or larger than arδ Then this determines the constant control on the interval [0,T]. Using the representation of the control in terms of the state and adjoint solutions to the transformed problems (4.3), (4.4), and (4.5), we have the associated Optimality Systems 4, 5, and 6. Optimality System 4 (OS4).
dx = −rx(t) − δu∗ (t), dt dψ1 = rψ1 (t), dt x(0) = ln N0 , and ψ1 (T ) = aex(T ) ,
1966
K. RENEE FISTER AND JOHN CARL PANETTA
where
∗
u (t) =
M
if
0
if
ψ1 (t) > δb ,
ψ1 (t) < δb .
Optimality System 5 (OS5). δu∗ (t) dx = −rx(t) − , dt (k + ex(t) ) dψ2 u∗ (t)δex(t) = ψ2 r − , dt (k + ex(t) )2 x(0) = ln N0 , and ψ2 (T ) = aex(T ) , where u∗ (t) =
0
if
b−
M
if
b−
ψ2 (t)δ (k+ex∗ (t) ) ψ2 (t)δ (k+ex∗ (t) )
> 0, < 0.
Optimality System 6 (OS6). dx = rx(t)(u∗ (t)δ − 1), dt dψ3 = −rψ3 (t)(u∗ (t)δ − 1), dt x(0) = ln N0 , and ψ3 (T ) = aex(T ) , where
u∗ (t) =
0
if
x(T )ex(T ) >
M
if
x(T )ex(T )
