Hindawi Publishing Corporation Discrete Dynamics in Nature and Society Volume 2016, Article ID 8676709, 10 pages http://dx.doi.org/10.1155/2016/8676709
Research Article A Fractional Trust Region Method for Linear Equality Constrained Optimization Honglan Zhu,1,2 Qin Ni,1 Liwei Zhang,3 and Weiwei Yang1 1
College of Science, Nanjing University of Aeronautics and Astronautics, Nanjing 210016, China Business School, Huaiyin Institute of Technology, Huaiβan 223003, China 3 Jiangsu Cable, Nanjing 210000, China 2
Correspondence should be addressed to Honglan Zhu;
[email protected] Received 21 April 2016; Accepted 29 May 2016 Academic Editor: Josef DiblΒ΄Δ±k Copyright Β© 2016 Honglan Zhu et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. A quasi-Newton trust region method with a new fractional model for linearly constrained optimization problems is proposed. We delete linear equality constraints by using null space technique. The fractional trust region subproblem is solved by a simple dogleg method. The global convergence of the proposed algorithm is established and proved. Numerical results for test problems show the efficiency of the trust region method with new fractional model. These results give the base of further research on nonlinear optimization.
1. Introduction In this paper, we consider the linear equality constrained optimization problem: minπ π (π₯) , π₯βπ
s.t.
π
π΄ π₯ = b,
(1)
min ππ (π ) ,
(2)
s.t.
πΓπ
where π(π₯) are continuously differentiable, π΄ β π
, b β π
π , and rank(π΄) = π. Trust region methods have the advantage in the theoretical analysis of convergence properties and the practice. Besides, Davidon proposed a conic model which makes use of more incorporate information at each iteration (see [1]). It is believed that combing trust region techniques with a conic model would be appealing. And it has attracted more and more attention of the researchers (see [2β8]). For unconstrained optimization problems, we proposed a new fractional model (see [9]): ππ (π ) =
(1 + πππ π ) (1 β πππ π ) (1 β πππ π )
πππ π 2
(1 + πππ π ) 1 π π π΅π π , + 2 (1 β ππ π )2 (1 β ππ π )2 π π
where ππ , ππ , ππ β π
π are horizontal vectors, ππ = βπ(π₯π ), π β π
π , π΅π β π
πΓπ is symmetric and an approximate Hessian of π(π₯) at π₯π . Then, the trust region subproblem of the unconstrained optimization problems is
(3)
βπ β β©½ Ξ π ,
(4) σ΅¨σ΅¨ σ΅¨ σ΅¨σ΅¨(1 β πππ π ) (1 β πππ π )σ΅¨σ΅¨σ΅¨ β©Ύ π0 , σ΅¨ σ΅¨ where π0 (0 < π0 < 1) is a sufficiently small positive number, β β
β refers to the Euclidean norm, and Ξ π > 0 is a trust region radius. If ππ = ππ = 0, then ππ (π ) is reduced to the conic model. If ππ = ππ = ππ = 0, then ππ (π ) is the quadratic model. In order to ensure that the fractional model function ππ (π ) is bounded over the trust region {π | βπ β β©½ Ξ π }, we assume βπ1 β (0, 1/3): σ΅©σ΅© σ΅©σ΅© σ΅©σ΅©ππ σ΅©σ΅© Ξ π β©½ π1 , σ΅©σ΅© σ΅©σ΅© (5) σ΅©σ΅©ππ σ΅©σ΅© Ξ π β©½ π1 , σ΅©σ΅© σ΅©σ΅© σ΅©σ΅©ππ σ΅©σ΅© Ξ π β©½ π1 . We denote Μ π = min {Ξ π , σ΅© π1 σ΅© , σ΅©π1 σ΅© , σ΅©π1σ΅© } ; Ξ σ΅©σ΅©ππ σ΅©σ΅© σ΅©σ΅©ππ σ΅©σ΅© σ΅©σ΅©ππ σ΅©σ΅© σ΅© σ΅© σ΅© σ΅© σ΅© σ΅©
(6)
2
Discrete Dynamics in Nature and Society
then, (4) reduces to a simplified fractional trust region subproblem: min π (π ) , s.t.
Μ π. βπ β β©½ Ξ
(7)
However, the subproblem (7) is solved only in quasi-Newton direction in [9]. In this paper, we made a further research, where the subproblem (7) is solved by a generalized dogleg algorithm. For linear equality constraints problem, we focus on the problems which are solved by a trust region method with new fractional model. If the constraints are linear inequality constraints or some constraint functions are nonlinear, then a few difficulties may arise. However, for these cases, the linear equality constraints problem may be as the subproblems of them. For example, the inequality constraints can be removed by an active set technique or a barrier transformation, and then the nonlinear constraints are linearized. In [10], Sun et al. established the algorithm for the problem (1)-(2) and prove the convergence. However, they do not consider the model computation. In [11], Lu and Ni proposed a trust region method with new conic model for solving (1)-(2) and carried out the numerical experiments. In this paper, we use a simple dogleg method to solve fractional model subproblems and present a quasi-Newton trust region algorithm for solving linear equality constrained optimization problems. This is a continuing work of the fractional model (see [9]), where the linear equality constraints (2) are deleted by using null space techniques. This paper is organized as follows. In Section 2, we give a description of the fractional trust region subproblem. In Section 3, we give a generalized dogleg algorithm for solving the fractional subproblem. In Section 4, we propose a new quasi-Newton method based on the fractional model for solving linearly constrained optimization problems and prove the global convergence of the proposed method under the reasonable assumptions. The numerical results are presented in Section 5.
In order to solve (8)β(11), firstly we consider removing the constraint (10) by the same assumption as in [9]. That is, we assume the parameters ππ , ππ , and ππ satisfied (5); then, subproblem (8)β(11) can be rewritten as the following reduced subproblem: min ππ (π ) s.t.
π΄π π = 0,
(13)
Μ π, βπ β β©½ Ξ
(14)
π
1
π΄ = ππ
= [π1 π2 ] [
0
] = π1 π
1 ,
(15)
where π1 β π
πΓπ , π2 β π
πΓ(πβπ) , and π
1 β π
πΓπ . Then, (13) can be rewritten as π
1π π1π π = 0.
(16)
Therefore, the feasible point for (13) can be presented by π = π2 π’,
(17)
for any π’ β π
πβπ , where π2 π’ lies in the null space of π΄. Then, the subproblem (12)β(14) becomes Μ π (π’) min π =
Μ ππ π’ (1 + Μπππ π’) π π (1 β πΜππ π’) (1 β Μππ π’)
(18)
2
In order to solve the problem (1)-(2), we assume that π΄ is column full rank and constraints are consistent. That is, the current point π₯π always satisfies π΄π π₯π = b. Obviously, the constrained condition is equivalent to π΄π π = 0 if π₯ = π₯π + π . Therefore, combing with (1)β(4), we can obtain that the trial step π π is computed by the following subproblem:
s.t.
π΄π π = 0,
Μ π is defined as (6). where Ξ The null space technology (see [4, 12, 13]) is an important technique for solving equality constraints programming problems. In the following, we use this technology to eliminate constraint (13). Since π΄ has full column rank, then there exist an orthogonal matrix π β π
πΓπ and a nonsingular upper triangular matrix π
β π
πΓπ such that
2. The Fractional Trust Region Subproblem
min ππ (π ) ,
(12)
(8) (9)
+
s.t.
Μ ππ’ (1 + Μπππ π’) π’π π΅ 2 (1 β
2 πΜππ π’)
Μ π, βπ’β β©½ Ξ
,
(19)
Μ π is the where πΜπ , Μππ , and Μππ are reduced horizontal vectors, π Μ π is Hessian approximation, and reduced gradient, π΅ Μ π = π2π ππ , π
σ΅¨σ΅¨ σ΅¨ σ΅¨σ΅¨(1 β πππ π ) (1 β πππ π )σ΅¨σ΅¨σ΅¨ β©Ύ π0 , σ΅¨ σ΅¨
(10)
Μ π = ππ π΅π π2 , π΅ 2
βπ β β©½ Ξ π .
(11)
πΜπ = π2π ππ ,
It can be found that our trust region subproblem (8)β(11) is the minimization of a fractional function subject to the trust region constraint and the linear constraints.
2
π (1 β Μππ π’)
Μπ = ππ π , π 2 π Μππ = π2π ππ .
(20)
Discrete Dynamics in Nature and Society
3
It can be seen that this subproblem has the same form as the subproblem (7) of the unconstrained optimization problems and it can be considered as the subproblem of the unconstrained minimization over π’. Therefore, we can find a solution of (18)-(19) by the dogleg method. Besides, it is easy Μ π is equivalent to βπ’β β©½ Ξ Μ π due to find that βπ β = βπ2 π’β β©½ Ξ to π2π π2 = πΌ.
3. The Dogleg Method of Fractional Trust Region Subproblem Now we consider calculating the trial step π’π of the new subproblem (18)-(19) by a simple dogleg method. Firstly, we need to recall the choice of the Newton point π’π as the following subalgorithm (see [9]). Subalgorithm 3.1. Given π΅π > 0, (21)
Μ π as defined in (20) and Μ π , and π΅ Step 1. Calculate πΜπ , Μππ , Μππ , π π β1 Μ β1 π Μ π Μ π Μππ π΅ πΜππ π΅ π Μπ,
ππ =
Μ β1 π Μπππ π΅ π Μπ
+
ππ =
Μ β1 π πΜππ π΅ π Μπ
π β1 Μ π + Μππ π΅ π Μ π β 1.
(22)
β1
Μ π Step 2. If ππ < 0, then π’π = βπππ΅ π Μ π , where ππ + βππ2 + (β4ππ )
π
π
β1
β2ππ
π
π
(23)
.
π
π
π
π
In the following, we consider determining the steepest descent point of (18)-(19), where the steepest descent point ππ =ΜΈ 0) and from is defined by Definition 1. Let π’ = βπΜ ππ (Μ (18) we have Μ (π) = π Μ π (βπΜ ππ ) ; π
(24)
then, (18)-(19) becomes Μ (π) , min π s.t. Μ (π) π Μ ππ Μ ππ π΅ Μ π ) (β2Vπ π Μ ππ π Μ π + π (1 β πΜπππ π Μπ) π Μπ) π (1 β πΜπππ π 2Vπ2
,
(26)
π Μ π ) (1 + πΜππ π Μπ) , Vπ = (1 + πΜπππ π
Μ Ξ πΞ = σ΅©σ΅© πσ΅©σ΅© . Μ π σ΅©σ΅© σ΅©σ΅©π
Μπ ππ π , 3 Vπ
(28)
where Μ ππ π Μπ, ππ = ππ π2 + ππ π + π
(29)
Μ π = πΜπ π2 + 2Μπππ π Μ π π β 1, π
(30)
Μ ππ Μππ Μ ππ π΅ Μ π + πΜππ π Μ π Μππ π Μππ Μ ππ π Μπ, ππ = Μπππ π
(31)
π
Μ ππ Μ ππ π΅ Μππ Μ ππ π Μ π + Μππ π Μππ Μ ππ π Μπ β π Μπ, ππ = πΜππ π π
π
Μ π πΜππ π Μ π + Μπππ π Μ π Μππ π Μ π + πΜππ π Μ π Μππ π Μπ. πΜπ = Μπππ π
(32) (33)
By direct calculating, we have (34)
Μ π β©½ 0, During the analysis, we find that if ππ β©Ύ 0 and Μπππ π then (29) and (30) may have no positive real zero points in most of the cases, respectively. Thus, in order to simplify the discussion, we assume ππ < 0,
(35)
Μπππ π Μ π > 0.
In order to discuss the feasible stationary points of ππ and Μ π , we first define the sets ππ and ππΜ which contain all the π Μ π , respectively, and these positive extremum points of ππ and π extremum points should be inside the feasible region [0, πΞ ]. It is easy to obtain that
(25)
0 β©½ π β©½ πΞ ,
where
=
Μ σΈ (π) = π
Μ ππ π Μ π β ππΜπππ π Μπ. ππ = πΜπ π
Μ β1 π Step 3. If ππ β©Ύ 0, then set Μππ = Μππ = 0. If |Μπππ π΅ π Μ π | β©½ π1 , β1 π Μ β1 π Μ β1 Μ Μ /(1 β πΜ π΅ π Μ ). If |Μπ π΅ π Μ | > π1 , then set then π’π = βπ΅ π Μ π πΜπ = 0 and π’π = βπ΅ π Μπ.
Μ (π), by comIn order to discuss the stationary points of π Μ (π) is putation we have that the derivative of π
π
1 π1 β (0, ) . 3
ππ =
Definition 1. Let πcp be the solution of (25). Then, π’cp = βπcp π is called a steepest descent point of (18)-(19).
(27)
0, if ππ > 0, { { { { ππ = {{Μπ1 } , if ππ = 0, { { { {{π2 } , otherwise, {π3 } , { { { { { {{Μπ3 } , { ππΜ = { { { {π3 , π4 } , { { { { {0,
if πΜπ > 0,
(36)
if πΜπ = 0, 2
Μ π ) < πΜπ < 0, if β (Μπππ π otherwise,
4
Discrete Dynamics in Nature and Society Theorem 3. Suppose that (5) and (35) hold. Then, the solution of (25) is
where πΜ1 = π1 = π2 = πΜ3 =
π3 =
π4 =
Μπ Μ ππ π π βππ
,
βππ + βΞ π , 2ππ
(37)
βππ β βΞ π , 2ππ 1 Μπ 2Μπππ π
πΜπ Μπ Μ π β βΞ βΜπππ π πΜπ
,
(38)
(39)
2 Μ π = (Μππ π Μπ . Ξ π Μπ) + π
In order to propose a generalized dogleg method of (18)(19), we set π’ (π‘) = π‘π’π + (1 β π‘) π’cp
Remark 2. Suppose that (5) and (35) hold. (i) If πΜπ β©Ύ 0, then Μ π )2 β©½ πΜπ < 0, ππ = from (34) we know that ππ > 0. (ii) If β(Μπππ π Μ 0, then π1 > πΞ . From (37) and (33), we have =
Μπ) πΜπ + (Μπππ π πΜπ
β©½ 0,
Μ π σ΅©σ΅©σ΅©σ΅© β©Ύ 1 β π1 > 0, Μ π β©Ύ 1 β πΞ σ΅©σ΅©σ΅©σ΅©Μππ σ΅©σ΅©σ΅©σ΅© σ΅©σ΅©σ΅©σ΅©π 1 β πΜ1Μπππ π
σ΅©2 σ΅© π = σ΅©σ΅©σ΅©σ΅©π’π β π’cp σ΅©σ΅©σ΅©σ΅© ,
if πΜπ = 0, 2 2
Μ π ) , πΜπ < 0, if πΜπ < β (Μπππ π otherwise,
where πΜ1 , π2 , πΜ3 , π3 , and π4 are determined by (37)β(39). Hence, it is easy to get the following theorems.
Μ π , then there exist two real roots π‘1 and π‘2 : If βπ’cp β < Ξ π‘1 = π‘2 =
βπ + βπ2 β ππ π βπ β βπ2 β ππ π
, (47) ,
where 0 < π‘1 < 1 and π‘2 < 0 (see [14, 15]). Based on the preceding theorems and analysis, we now give a generalized dogleg algorithm for solving (18)-(19). Algorithm 5.
2
Μ π ) , πΜπ = 0, if πΜπ < β (Μπππ π
(46)
σ΅© σ΅©2 Μ 2 π = σ΅©σ΅©σ΅©σ΅©π’cp σ΅©σ΅©σ΅©σ΅© β Ξ π.
(41)
if πΜπ > 0, Μ π ) < πΜπ < 0, if β (Μπππ π
π
π = (π’π β π’cp ) π’cp ,
(40)
where the last inequality is obtained by (21) and this conflicts with (40). Therefore, πΜ1 > πΞ . Μ π )2 β©½ πΜπ < 0, ππ < 0, Similarly, we can prove that if β(Μπππ π then π2 > πΞ . Then, combining with (36) and Remark 2., we define a set
(45)
Μ π . Denote and calculate π‘β such that βπ’(π‘β )β = Ξ
2
where πΜ1 is defined in (37). However, from (37) and (35) we have πΜ1 > 0. We assume πΜ1 β (0, πΞ ]; then, from (5) and (27) we have
{π3 } , { { { { { { {Μπ3 } , { { { { { { { {{π3 , π4 } , π={ { { {Μπ1 } , { { { { { { { {π2 } , { { { { {0,
(44)
Therefore, the steepest descent point π’cp = βπcp π is an approximate solution of the fractional trust region subproblem (18)-(19), where πcp is defined by (43). Similarly, the fractional trust region subproblem (18)(19) has the following property. The proof of this theorem is similar to Theorem 3.1 in [14], so we omit its proof.
Now, we have the the following conclusions.
1β
πβ = (π β© {[0, πΞ ]}) βͺ {0, πΞ }
where
Theorem 4. Suppose that (5) and (35) hold, where π1 β Μ π , then the optimal solution of (18)-(19) (0, 1/3). If βπ’πβ > Ξ must be on the boundary of trust region, where π’π is defined in Subalgorithm 3.1.
,
Μ ππ π Μπ, Ξ π = ππ2 β 4ππ π
Μπ πΜ1Μπππ π
(43)
and π is defined by (36).
,
Μπ Μ π + βΞ βΜπππ π
Μ (π) , π β πβ } , πcp = arg min {π
(42)
Step 1. Compute π’π by Subalgorithm 3.1. Μ π , then π’π = π’π, and stop. Step 2. If βπ’πβ β©½ Ξ Step 3. If ππ < 0, then compute ππ as defined in (32), and go to Step 4. Otherwise, go to Step 5. Μ π β©½ 0, go to Step 5. Otherwise, compute Step 4. If ππ β©Ύ 0 or Μπππ π Μ π , where πcp is defined by (43). If πcp = πΞ , where π’cp = βπcp π
Discrete Dynamics in Nature and Society
5
πΞ is defined by (27), then π’π = π’cp , and stop. If πcp = 0, go to Step 5. Otherwise, go to Step 6. Step 5. Set Μππ = Μππ = 0 and compute Μππ Μ ππ π΅ Μ π β πΜππ π Μππ Μ ππ π Μπ, πππ = π πΜcp =
Μπ Μ ππ π π , πππ
(48)
Μ π (π’) is defined where π₯ = π₯π +π = π₯π +π2 π’, ππ = π(π₯π ), and π Μ π are the corresponding Μ π , and π΅ as (18). Thus, ππ (0) = ππ , π gradient and Hessian approximations of the function at the πth iteration. We choose π’π to minimize ππ (π’). There is a Μ π is positive definite. In the unique minimizer if and only if π΅ following, we give our algorithm. If the current iteration is the feasible point π₯π , then an equivalent form of (1)-(2) is to solve the reduced unconstrained problem min
π’βπ
πβπ
Μπ, π’π = βΜπβ π
if πππ β©½ 0, if πππ > 0.
(49)
Μ , ππ (0) = π π Μ , ππ (βπ’πβ1 ) = π πβ1
Step 6. Calculate π‘1 and π‘2 as defined in (47); then,
Μ πβ1 , βππ (βπ’πβ1 ) = π (50)
π₯π = π₯πβ1 + π πβ1 = π₯πβ1 + π2 π’πβ1 , Μ =π Μ (0) = π (π₯ ) = π , π π π π
Then, we give the predicted decent bound in each iteration, which is the lower bound of the predicted reduction in each iteration:
Μ πβ1 = π2π ππβ1 . π
ππβ1 = ππ β
Μ ππ π’πβ1 (1 β Μπππ π’πβ1 ) π ΜVπβ1
(58)
2
+
(52) Μ πβ1 = π
2
where π = (1 β π1 )/(1 + π1 ) . This theorem is similar to that in [9] and its proof is omitted.
π Μ (1 β Μπππ π’πβ1 ) π’πβ1 π΅π π’πβ1
2ΜV2πβ1
,
1 Μ Μ π π’πβ1 ] , Μ π β (1 β Μπππ π’πβ1 ) π΅ ππβ1 [ΜVπβ1 π
ΜV3πβ1
(59)
π
where ΜVπβ1 = (1 + πΜππ π’πβ1 )(1 + Μππ π’πβ1 ) and Μ πβ1 = (1 β Μππ π’πβ1 ) ΜVπβ1 πΌ β [Μππ ΜVπβ1 + (1 β Μππ π’πβ1 ) π π π
4. New Quasi-Newton Algorithm and Its Global Convergence
π π . β
(Μππ (1 + Μππ π’πβ1 ) + Μππ (1 + πΜππ π’πβ1 ))] π’πβ1
In this section, we propose a quasi-Newton method with a fractional model for linearly equality constrained optimization and prove its convergence under some reasonable conditions. In order to solve problem (1)-(2), we consider the fractional model approximation for π(π₯) about π’ = 0; that is Μ π (π’) , ππ (π’) = ππ + π
(57)
Obviously, (55) holds. Then, from (56), we have
Theorem 6. Suppose that (5) and (35) hold, where π1 β (0, 1/3). If π’π is obtained by Algorithm 5, then σ΅© Μ σ΅©σ΅© Μπ } 1 σ΅©σ΅©σ΅©π σ΅©σ΅© πΞ πσ΅© σ΅© min {σ΅©σ΅©σ΅©π Μ , , σ΅© σ΅© σ΅© σ΅© π σ΅© σ΅© { σ΅©σ΅© Μ β1 σ΅©σ΅©σ΅© } Μ σ΅©σ΅© 2 σ΅©σ΅©σ΅©π΅ σ΅©σ΅©π΅π σ΅©σ΅© } σ΅© π σ΅©σ΅© { σ΅© σ΅©
Μ Μ π πβ1 = π (βπ’πβ1 ) = π (π₯πβ1 ) = ππβ1 ,
(51)
Μ π (π’) is defined by (18). where π
pred (π’π ) β©Ύ
(56)
where
where π’(π‘) is defined by (45). π’π = π’(π‘β ), and stop.
Μ π (0) β π Μ π (π’) , predπ (π’) = π
(55)
Μπ, βππ (0) = π
If πΜβ = πΞ , then π’π = π’π , and stop. Otherwise, calculate π’cp = Μ π , go to Step 6. βΜπcp π
Μ π (π’ (0)) , π Μ π (π’ (π‘2 ))} , Μ π (π’ (π‘1 )) , π π‘β = arg min {π
(54)
In the following, we consider the choice of the parameter vectors πΜπ , Μππ , and Μππ . We choose these vectors such that (53) satisfies the following conditions:
where {πΞ , πΜβ = { min {πΞ , πΜcp } , {
Μ (π’) = π (π₯ + π π’) . π π 2
(53)
(60)
If we choose Μ πβ1 , πΜπ = π1 π Μπ = π π΅ Μ π 2 πβ1 π’πβ1 , Μπ, Μππ = π3 π
(61)
6
Discrete Dynamics in Nature and Society
then these unknown parameters π1 , π2 , and π3 can be obtained from (58)-(59). In the following, we give the derivation process of π1 , π2 , and π3 . First, we define some notations: Μ ππβ1 π’πβ1 , πΌπ = π
where Μ β ππ½Μπ β π½πΎ. Μ πΎΜ = ππ½π
Similarly, by left multiplying π2π on (59), from (63) and (66) we have
Μ ππβ1 π1 , Μπ = π πΌ Μ ππβ1 π2 , Μπ = π πΌ
π1 =
π Μ π΅πβ1 π’πβ1 , π½π = π’πβ1
Μ = π’π π΅ Μ π½ πβ1 πβ1 π1 , π Μ = π½ π
Μ π +πΌ Μ) β ((Μ πΌπ½ + Μπππ½) 2 , Μ π + πΌΜ ΜΜππ (πΌΜ πΌπ½ + Μππ (Μ πΌπ½ + πΌπ½)) πΌ+πΌ 2
(62)
Μπ = π’π π΅ Μ π πβ1 π π1 ,
πΜ π1 = β , πΎΜ
Μ , Μ β π2 Μππ½) π Μ = Μπ (Μ πΌπ½
Μ ππ π’πβ1 , π
Μ π2 . πΎΜ = πΌΜπ + πΜ πΌπ½Μ
Μ ππ π1 , πΎΜπ = π
where the vectors π1 and π2 are chosen to satisfy Μ π = πΎΜπ = Μππ = πΎΜπ = 0. πΌ
π3 = (63)
Μπ, πΌ Μ π , π½π , . . . , πΎΜπ For convenience, we omit the index π of πΌπ , πΌ and πΎΜπ . On one hand, from (58) we have = πΜVπβ1 ,
(64)
where π=
(72)
And then from (66), we have
Μ ππ π2 , πΎΜπ = π
1
(71)
where
Μπ = π’π π΅ Μ π πβ1 π π2 ,
β Μπππ π’πβ1
(70)
where Μπ = πΎ β ππ. Substituting (68) into the above equation, we have
π Μ π΅πβ1 π2 , π’πβ1
π Μ π΅π π’πβ1 , ππ = π’πβ1
πΎπ =
(69)
Μ βπ Μ ) πΎ + βπΎ2 + 2π (π πβ1 π π
.
(65)
Μ } is monotonically decreasing and π΅ Μ π is If the sequence {π π positive definite, then we know that π > 0 and (64) becomes 1 β π3 πΎ = π (1 + π1 πΌ) (1 + π2 π½) .
(66)
Μ = 0. β (1 + π1 πΌ) (1 + π2 π½) π2 π½πΎ
(73)
Now we give the new quasi-Newton algorithm based on the fractional model (53). Algorithm 7. Step 0. Choose π₯0 β π
π , π1 β (0, 1/3), π > 0, Ξ max > 0, 0 < Μ 0 = πΌ, and the initial π1 < π2 < 1, 0 < πΏ1 < 1 < πΏ2 , 0 < π < 1, π΅ trust region radius Ξ 0 β (0, Ξ max ]. Compute π2 as defined in (15). Set π = 0. Step 1 (stopping criterion). Compute ππ = π(π₯π ), ππ = Μ π = π2π ππ . If βΜ βπ(π₯π ), and π ππ β β©½ π, then π₯β = π₯π , and stop. If π = 0, go to Step 3. Μ π by Μπ β π Μ πβ1 . Update π΅ Step 2. Compute π¦πβ1 = π Μπ = π΅ Μ πβ1 β π΅
On the other hand, by left-multiplying π1π on (59) and combining with (63), we have Μ β π½Μπ) β Μπ) (1 β π3 πΎ) (π2 (π½π
1 β π (1 + π1 πΌ) (1 + π2 π½) . πΎ
π Μ πβ1 π’πβ1 π’π π΅ Μ π΅ π§πβ1 π§πβ1 πβ1 πβ1 + , π π’ π π΅ Μ πβ1 π’πβ1 π§πβ1 π’πβ1 πβ1
(74)
where (67)
Μ πβ1 π’πβ1 , π β [0, 1] , π§πβ1 = ππ¦πβ1 + (1 β π) π΅ π
Then, from (66), we have πΜπ π2 = , πΎΜ
(68)
1, { { { π Μ ={ π΅πβ1 π’πβ1 0.8π’πβ1 { , { π π Μ { π’πβ1 π΅πβ1 π’πβ1 β π¦πβ1 π’πβ1
π π Μ π΅πβ1 π’πβ1 , (75) π’πβ1 β©Ύ 0.2π’πβ1 if π¦πβ1
otherwise.
Discrete Dynamics in Nature and Society
7
Μ π = ππ π΅π π2 , and Step 3. If π β©½ 1, then set πΜπ = Μππ = Μππ = 0, π΅ 2 β1 Μ π Μ , compute πΌ such that Wolfe-Powell conditions ππ = βπ΅ π π π are satisfied, and set π₯π+1 = π₯π +π π = π₯π +πΌπ ππ and π’π = π2π π π , π = π + 1, and go to Step 1. Μ π π’πβ1 get π1 and π2 . Μ π , and π΅ Μ πβ1 , π Step 4. By the parameters π
Next we present the global convergence theorem which says the reduced gradients converge to zero. Theorem 8. Assume that (5) and (35) hold, where π1 β (0, 1/3). If π is continuously differentiable and bounded below in some set containing all iterations generated by Algorithm 7, the Μ π β} are uniformly bounded. Then, sequences {βΜ ππ β} and {βπ΅
Μ, πΌ Μ , . . . , πΎ, πΎΜ and πΎΜ as defined in (62). If Step 5. Compute πΌ, πΌ πΎ = 0, then set Μππ = Μππ = 0. Calculate Μ βπ Μ )2 β (Μ ππ = (π πππβ1 π’πβ1 ) (Μ πππ π’πβ1 ) , πβ1 π Μ ) + βπ Μ βπ (π π π { { πβ1 , Μ π π½={ βΜ ππβ1 π’πβ1 { {1,
if ππ β©Ύ 0,
(76)
σ΅© Μ σ΅©σ΅© lim inf σ΅©σ΅©σ΅©π πσ΅© σ΅© = 0. πββ
Proof. Assume that the theorem is false and there is π > 0 such Μ π β©Ύ π for all π. From the assumption, we can assume that that π σ΅©σ΅©Μ σ΅©σ΅© σ΅©σ΅©ππ σ΅©σ΅© β©½ π, σ΅©σ΅©Μ σ΅©σ΅© σ΅©σ΅©ππ σ΅©σ΅© β©½ π, σ΅© σ΅© σ΅©σ΅©Μ σ΅©σ΅© σ΅©σ΅©ππ σ΅©σ΅© β©½ π, σ΅©σ΅© Μ σ΅©σ΅© σ΅©σ΅©ππ σ΅©σ΅© β©½ π, σ΅© Μ σ΅©σ΅© 0 < σ΅©σ΅©σ΅©σ΅©π΅ σ΅©σ΅© β©½ π΅, πσ΅© σ΅©σ΅© β1 σ΅©σ΅© Μ Μ σ΅©σ΅© β©½ π΅ 0 < σ΅©σ΅©σ΅©π΅ σ΅© π σ΅©σ΅©
otherwise
and set πΜπ =
1 β π½Μ Μ ππβ1 π’πβ1 π
Μ πβ1 . π
(77)
Otherwise, compute πΎΜ and πΎΜ as defined in (72) and (69). If πΎΜ = 0 or πΎΜ = 0, then set πΜπ = Μππ = Μππ = 0. Otherwise, calculate πΜπ , Μππ , and Μππ , where π1 , π2 , and π3 in (61) are determined by (71), (68), and (73). Μ π , then πΜπ = π1 ππ /Ξ Μ π βππ β. Update Μππ and Step 6. If βΜππ β > π1 /Ξ Μππ with the same way such that (5) are satisfied. Μ π , solve the Μ π , and Ξ Step 7. By the parameters πΜπ , Μππ , Μππ , π subproblem (18)-(19) by Algorithm 5 to get π’π . Set π π = π2 π’π .
(82)
(83)
hold for all π. From (52) and the assumptions in the theorem, we have pred (π π ) β©Ύ
Μ πΞ π Μ π , π2 } , min {π, π } β©Ύ π1 min {Ξ Μ 2π΅ π΅
(84)
where π1 and π2 are some positive constants. Then, from Step 10 of Algorithm 7, we have
Step 8. Compute Ared (π’π ) ππ = , Pred (π’π )
Μ π , π2 } . ππ β ππ+1 β©Ύ π1 pred (π π ) β©Ύ π1 π1 min {Ξ (78)
where Ared (π’π ) = Ared (π π ) = π (π₯π ) β π (π₯π + π π ) ,
(79)
Μ π (π’π ) . Pred (π’π ) = ππ (0) β ππ (π’π ) = βπ
(80)
Step 9. Update the trust region radius:
Μ π+1 Ξ
Μ π, πΏ1 Ξ { { { { Μ π , Ξ max } , = {min {πΏ2 Ξ { { {Μ {Ξπ ,
(85)
if ππ β©½ π1 ,
σ΅© σ΅© Μ if ππ β©Ύ π2 , σ΅©σ΅©σ΅©π’π σ΅©σ΅©σ΅© = Ξ π , (81)
otherwise.
Step 10. If ππ β©Ύ π1 , then π₯π+1 = π₯π + π π . Set π = π + 1, and go to Step 1. Otherwise, π₯π+1 = π₯π , π = π + 1, and go to Step 6.
Since π(π₯) is bounded from below and ππ+1 < ππ for all π, Μ Μ we have that ββ π=1 min{Ξπ , π2 } is convergent, and Ξπ β 0 as π β β. Μ π β 0, βπ’π β β 0. From Step 6 On the other hand, when Ξ of Algorithm 7, we have σ΅¨σ΅¨ π σ΅¨σ΅¨ σ΅©σ΅© σ΅©σ΅© Μ σ΅¨σ΅¨πΜπ π’π σ΅¨σ΅¨ β©½ σ΅©σ΅©πΜπ σ΅©σ΅© Ξπ β©½ π1 , σ΅¨ σ΅¨ σ΅¨σ΅¨ π σ΅¨σ΅¨ σ΅¨σ΅¨Μπ π’ σ΅¨σ΅¨ β©½ π , σ΅¨σ΅¨ π π σ΅¨σ΅¨ 1 σ΅¨ σ΅¨
(86)
where π1 β (0, 1/3). Thus, we have 1 σ΅© σ΅© = 1 + πΜππ π’π + π (σ΅©σ΅©σ΅©π’π σ΅©σ΅©σ΅©) , 1 β πΜππ π’π π 1 σ΅© σ΅© = 1 + Μππ π’π + π (σ΅©σ΅©σ΅©π’π σ΅©σ΅©σ΅©) . π Μ 1βπ π’ π π
(87)
8
Discrete Dynamics in Nature and Society and Chen 3.3.2. Moreover, in order to test Algorithm 7 more generally, we designed some problems where the objective functions are Pro. 7β18 (see [14, 18]) and the linear equality constraints are Pro. 1β6. If Μππ = Μππ = 0 in Algorithm 7, we can obtain the conic model algorithm and call this algorithm CTR. We solve the following 18 test problems by FTR and CTR and compare their results. All the computations are carried out in Matlab R2012b on a microcomputer in double precision arithmetic. These tests use the same stopping criterion βΜ ππ β β©½ 10β4 . The columns in the tables have the following meanings: Pro. denotes the numbers of the test problems; π is the dimension of the test problems; Iter is the number of iterations; nf and ng are the numbers of function and gradient evaluations, respectively; βΜ πβ is the Euclidean norm of the final reduced gradient; CPU(s) denotes the total iteration time of the algorithm in seconds. The parameters in these algorithms are
Table 1: Test functions. Pro.
Function name
1
HS 9
2
HS 48
3
HS 49
4
HS 50
5
Chen 3.3.1
6
Chen 3.3.2
7
Cube
8
Penalty-I
9
Beale
10
Conic
11
Extended Powell
12
Broyden Tridiagonal
13
Rosenbrock
14
Chained Freudenstein and Roth
15
Extended Trigonometric
16
Penalty-III
17
Troesch
18
Cragg and Levy
π΅0 = πΌ, π1 = 0.33, Ξ 0 = 1, Ξ max = 10, π = 0.15, π = 0.85,
By computing, we obtain Μ ππ π’π (1 + Μπππ π’π ) π
2
Μ π π’π (1 + Μπππ π’π ) π’ππ π΅ (1 β
π1 = 0.25,
σ΅© σ΅©2 = πππ π’π + π (σ΅©σ΅©σ΅©π’π σ΅©σ΅©σ΅© ) ,
π (1 β πΜππ π’π ) (1 β Μππ π’π )
2 πΜππ π’π )
(91)
2
π (1 β Μππ π’π )
=
Μ π π’π π’ππ π΅
σ΅© σ΅©2 + π (σ΅©σ΅©σ΅©π’π σ΅©σ΅©σ΅© ) ,
π2 = 0.75, (88)
Μ π π’π + π (σ΅©σ΅©σ΅©π’π σ΅©σ΅©σ΅©2 ) . Ared (π’π ) = βπππ π’π β π’ππ π΅ σ΅© σ΅© Then, from (80), we have σ΅© Μ σ΅©σ΅©2 σ΅© ), Pred (π’π ) = Ared (π’π ) + π (σ΅©σ΅©σ΅©σ΅©Ξ πσ΅© σ΅©
(89)
which indicates that lim ππ = 1.
πββ
(90)
Μ π+1 β©Ύ Ξ Μ π, By the updating in Step 9 of Algorithm 7, we have Ξ Μ π β 0. The theorem is proved. which is a contradiction to Ξ
5. Numerical Tests In this section, Algorithm 7 (abbreviated as FTR) is tested with some test problems which are chosen from [16, 17]. These test problems are listed in Table 1. We choose linear constrained problems HS9, HS48, HS49, HS50, Chen 3.3.1,
πΏ2 = 4. The numerical comparison for 18 small-scale test problems is listed in Table 2. We can see that FTR is better than CTR for 15 tests in the number of iterations and the remaining 3 tests are similar. Because FTR needs some extra algebra computation for some parameters, FTR takes more time than CTR for small problems. The numerical results of some large-scale problems are presented in Table 3. From Table 3, we find that for large-scale problems the CPU time of FTR is approximately the same as that of CTR but it has fewer number of iterations. From the above comparison, we see that FTR is slightly more effective and robust for these large-scale test problems. The fractional model in Algorithm 7 is the extension of conic model. By using more information of function and gradient from the previous iterations and choosing parameters flexibly, the fractional model can be more approximate to the original problem. And the global convergence of the proposed quasi-Newton trust region algorithm is also proved. Numerical experiment shows the algorithm is effective and robust, including for large-scale test problems. The theoretical results and the numerical results lead us to believe that the method is worthy of further study. For example, we can consider using fractional model to solve the nonlinear equality constrained optimization problem.
Discrete Dynamics in Nature and Society
9
Table 2: The numerical results of Algorithm 7 for some test problems. Pro.
π
Starting point
1
6
(0, 0, 0, 0, 0, 0)
2
5
(3, 5, β3, 2, β2)
3
5
(2, 2, β9, 3, 3)
4
10
5
8
6
8
7
6
(3, 4, 3, 4, 3, 4)
8
6
(3, 4, 3, 4, 3, 4)
9
6
(6, 8, 6, 8, 6, 8)
10
6
β(1.5, 2, 1.5, 2, 1.5, 2)
11
8
(0.3, 0.4, . . ., 0.3, 0.4)
12
6
(0.3, 0.4, . . ., 0.3, 0.4)
13
6
(1.5, 2, 1.5, 2, 1.5, 2)
14
6
(6, 8, 6, 8, 6, 8)
15
8
(0.3, 0.4, . . ., 0.3, 0.4)
16
6
(0.3, 0.4, . . ., 0.3, 0.4)
17
6
(0.3, 0.4, . . ., 0.3, 0.4)
18
8
(0.3, 0.4, . . ., 0.3, 0.4)
(0, β6, 6, 0, 0, 0, β6, 6, 0, 0) (0, 3, β1, β2 0, 3, β1, β2) (7/3, 11/3, β1, β1 7/3, 11/3, β1, β1)
Algorithm CTR FTR CTR FTR CTR FTR CTR FTR CTR FTR CTR FTR CTR FTR CTR FTR CTR FTR CTR FTR CTR FTR CTR FTR CTR FTR CTR FTR CTR FTR CTR FTR CTR FTR CTR FTR
Iter 6 5 13 8 27 25 12 8 16 16 22 21 8 8 14 12 21 19 13 9 15 7 26 24 6 6 34 22 10 9 13 12 29 19 12 9
nf/ng 7/8 6/6 14/9 9/9 28/21 26/17 13/13 9/9 17/14 17/17 23/19 22/19 9/9 9/9 15/11 13/10 22/18 20/18 14/10 10/7 16/13 8/8 27/16 25/14 7/7 7/7 35/20 23/23 11/9 10/8 14/12 13/12 30/17 20/17 13/9 10/9
βπβ 7.559746(β5) 1.693174(β5) 1.637781(β5) 2.148387(β6) 5.815109(β5) 4.996278(β5) 3.889891(β5) 2.397640(β5) 7.255223(β5) 1.249467(β5) 3.128858(β6) 2.865070(β7) 2.208537(β7) 1.765690(β8) 8.493218(β7) 1.155176(β5) 1.208648(β6) 3.705893(β5) 1.789828(β6) 4.017095(β6) 3.013621(β5) 7.252083(β5) 5.317180(β5) 5.917188(β5) 8.707409(β5) 5.998420(β5) 3.185767(β5) 4.845919(β6) 1.738762(β5) 5.005195(β5) 3.409181(β5) 9.332787(β5) 1.845528(β6) 5.983264(β6) 7.722466(β7) 4.353866(β7)
CPU (s) 0.084639 0.161438 0.094287 0.098268 0.110429 0.149000 0.100359 0.152117 0.099120 0.129272 0.101571 0.167233 0.093060 0.139521 0.095873 0.166608 0.094048 0.170549 0.104145 0.158877 0.087370 0.121116 0.1160001 0.187447 0.081667 0.115520 0.106515 0.158603 0.092062 0.146342 0.083445 0.164178 0.088997 0.159851 0.087020 0.149733
Table 3: The numerical results of Algorithm 7 for some test problems (π = 3000). Pro.
Starting point
7
(6, 8, . . ., 6, 8)
8
(6, 8, . . ., 6, 8)
9
(6, 8, . . ., 6, 8)
10
(6, 8, . . ., 6, 8)
11
(6, 8, . . ., 6, 8)
Algorithm CTR FTR CTR FTR CTR FTR CTR FTR CTR FTR
Iter 25 21 12 12 25 24 12 10 18 4
nf/ng 26/26 22/22 13/12 13/11 26/26 25/25 13/13 11/11 19/19 5/5
βπβ 3.737202(β9) 3.849243(β9) 8.930053(β6) 3.113077(β7) 3.388081(β6) 1.176732(β6) 1.226943(β7) 6.842114(β6) 2.718913(β8) 5.738665(β5)
CPU (s) 28.5976 26.2795 13.8295 14.7506 28.4904 30.7385 14.9229 14.9369 21.4796 8.48251
10
Competing Interests The authors have no competing interests regarding this paper.
Acknowledgments This work was supported by the National Natural Science Foundation of China (Grant no. 11071117 and 71301060), the Natural Science Foundation of Jiangsu Province (BK20141409), Funding of Jiangsu Innovation Program for Graduate Education (KYZZ 0089) (βthe Fundamental Research Funds for the Central Universitiesβ), and the Humanistic and Social Science Foundation of Ministry of Education of China (12YJA630122).
References [1] W. C. Davidon, βConic approximations and collinear scalings for optimizers,β SIAM Journal on Numerical Analysis, vol. 17, no. 2, pp. 268β281, 1980. [2] R. Schnabel, βConic methods for unconstrained minimization and tensor methods for nonlinear equations,β in Mathematical Programming: The State of the Art, A. Bachem, M. GrΒ¨otschel, and B. Korte, Eds., pp. 417β438, Springer, Heidelberg, Germany, 1982. [3] D. C. Sorensen, βNewtonβs method with a model trust region modification,β SIAM Journal on Numerical Analysis, vol. 19, no. 2, pp. 409β426, 1982. [4] W. Sun and Y. X. Yuan, βA conic trust-region method for nonlinearly constrained optimization,β Annals of Operations Research, vol. 103, pp. 175β191, 2001. [5] C. X. Xu and X. Y. Yang, βConvergence of conic quasiNewton trust region methods for unconstrained minimization,β Mathematical Application, vol. 11, no. 2, pp. 71β76, 1998. [6] Y. X. Yuan, βA review of trust region algorithms for optimization,β in Proceedings of the International Congress on Industrial and Applied Mathematics (ICIAM β00), vol. 99, pp. 271β282, 2000. [7] D. M. Gay, βComputing optimal locally constrained steps,β SIAM Journal on Scientific and Statistical Computing, vol. 2, no. 2, pp. 186β197, 1981. [8] J.-M. Peng and Y.-X. Yuan, βOptimality conditions for the minimization of a quadratic with two quadratic constraints,β SIAM Journal on Optimization, vol. 7, no. 3, pp. 579β594, 1997. [9] H. L. Zhu, Q. Ni, and M. L. Zeng, βA quasi-Newton trust region method based on a new fractional model,β Numerical Algebra, Control and Optimization, vol. 5, no. 3, pp. 237β249, 2015. [10] W. Y. Sun, J. Y. Yuan, and Y. X. Yuan, βConic trust region method for linearly constrained optimization,β Journal of Computational Mathematics, vol. 21, no. 3, pp. 295β304, 2003. [11] X. P. Lu and Q. Ni, βA trust region method with new conic model for linearly constrained optimization,β Or Transactions, vol. 12, pp. 32β42, 2008. [12] Q. Ni, Optimization Method and Program Design, Science Press, Beijing, China, 2009. [13] L. W. Zhang and Q. Ni, βTrust region algorithm of new conic model for nonlinearly equality constrained optimization,β Journal on Numerical Methods and Computer Applications, vol. 31, no. 4, pp. 279β289, 2010.
Discrete Dynamics in Nature and Society [14] M. F. Zhu, Y. Xue, and F. S. Zhang, βA quasi-Newton type trust region method based on the conic model,β Numerical Mathematics, vol. 17, no. 1, pp. 36β47, 1995 (Chinese). [15] X. P. Lu and Q. Ni, βA quasi-Newton trust region method with a new conic model for the unconstrained optimization,β Applied Mathematics and Computation, vol. 204, no. 1, pp. 373β384, 2008. [16] W. Hock and K. Schittkowski, Test Examples for Nonlinear Programming Codes, Springer, Berlin, Germany, 1981. [17] X. Y. Chen, Research on the geometric algorithms for programs with constraints of linear equalities [M.S. thesis], Fujian Normal University, 2012. [18] J. J. More, B. S. Garbow, and K. E. Hillstrom, βTesting unconstrained optimization software,β ACM Transactions on Mathematical Software, vol. 7, no. 1, pp. 17β41, 1981.
Advances in
Operations Research Hindawi Publishing Corporation http://www.hindawi.com
Volume 2014
Advances in
Decision Sciences Hindawi Publishing Corporation http://www.hindawi.com
Volume 2014
Journal of
Applied Mathematics
Algebra
Hindawi Publishing Corporation http://www.hindawi.com
Hindawi Publishing Corporation http://www.hindawi.com
Volume 2014
Journal of
Probability and Statistics Volume 2014
The Scientific World Journal Hindawi Publishing Corporation http://www.hindawi.com
Hindawi Publishing Corporation http://www.hindawi.com
Volume 2014
International Journal of
Differential Equations Hindawi Publishing Corporation http://www.hindawi.com
Volume 2014
Volume 2014
Submit your manuscripts at http://www.hindawi.com International Journal of
Advances in
Combinatorics Hindawi Publishing Corporation http://www.hindawi.com
Mathematical Physics Hindawi Publishing Corporation http://www.hindawi.com
Volume 2014
Journal of
Complex Analysis Hindawi Publishing Corporation http://www.hindawi.com
Volume 2014
International Journal of Mathematics and Mathematical Sciences
Mathematical Problems in Engineering
Journal of
Mathematics Hindawi Publishing Corporation http://www.hindawi.com
Volume 2014
Hindawi Publishing Corporation http://www.hindawi.com
Volume 2014
Volume 2014
Hindawi Publishing Corporation http://www.hindawi.com
Volume 2014
Discrete Mathematics
Journal of
Volume 2014
Hindawi Publishing Corporation http://www.hindawi.com
Discrete Dynamics in Nature and Society
Journal of
Function Spaces Hindawi Publishing Corporation http://www.hindawi.com
Abstract and Applied Analysis
Volume 2014
Hindawi Publishing Corporation http://www.hindawi.com
Volume 2014
Hindawi Publishing Corporation http://www.hindawi.com
Volume 2014
International Journal of
Journal of
Stochastic Analysis
Optimization
Hindawi Publishing Corporation http://www.hindawi.com
Hindawi Publishing Corporation http://www.hindawi.com
Volume 2014
Volume 2014