A Fractional Trust Region Method for Linear Equality Constrained ...

Hindawi Publishing Corporation Discrete Dynamics in Nature and Society Volume 2016, Article ID 8676709, 10 pages http://dx.doi.org/10.1155/2016/8676709

Research Article A Fractional Trust Region Method for Linear Equality Constrained Optimization Honglan Zhu,1,2 Qin Ni,1 Liwei Zhang,3 and Weiwei Yang1 1

College of Science, Nanjing University of Aeronautics and Astronautics, Nanjing 210016, China Business School, Huaiyin Institute of Technology, Huai’an 223003, China 3 Jiangsu Cable, Nanjing 210000, China 2

Correspondence should be addressed to Honglan Zhu; [email protected] Received 21 April 2016; Accepted 29 May 2016 Academic Editor: Josef Dibl´ık Copyright © 2016 Honglan Zhu et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. A quasi-Newton trust region method with a new fractional model for linearly constrained optimization problems is proposed. We delete linear equality constraints by using null space technique. The fractional trust region subproblem is solved by a simple dogleg method. The global convergence of the proposed algorithm is established and proved. Numerical results for test problems show the efficiency of the trust region method with new fractional model. These results give the base of further research on nonlinear optimization.

1. Introduction In this paper, we consider the linear equality constrained optimization problem: min𝑛 𝑓 (𝑥) , 𝑥∈𝑅

s.t.

𝑇

𝐴 𝑥 = b,

(1)

min 𝜓𝑘 (𝑠) ,

(2)

s.t.

𝑛×𝑚

where 𝑓(𝑥) are continuously differentiable, 𝐴 ∈ 𝑅 , b ∈ 𝑅𝑚 , and rank(𝐴) = 𝑚. Trust region methods have the advantage in the theoretical analysis of convergence properties and the practice. Besides, Davidon proposed a conic model which makes use of more incorporate information at each iteration (see [1]). It is believed that combing trust region techniques with a conic model would be appealing. And it has attracted more and more attention of the researchers (see [2–8]). For unconstrained optimization problems, we proposed a new fractional model (see [9]): 𝜓𝑘 (𝑠) =

(1 + 𝑐𝑘𝑇 𝑠) (1 − 𝑎𝑘𝑇 𝑠) (1 − 𝑏𝑘𝑇 𝑠)

𝑔𝑘𝑇 𝑠 2

(1 + 𝑐𝑘𝑇 𝑠) 1 𝑠𝑇 𝐵𝑘 𝑠, + 2 (1 − 𝑎𝑇 𝑠)2 (1 − 𝑏𝑇 𝑠)2 𝑘 𝑘

where 𝑎𝑘 , 𝑏𝑘 , 𝑐𝑘 ∈ 𝑅𝑛 are horizontal vectors, 𝑔𝑘 = ∇𝑓(𝑥𝑘 ), 𝑠 ∈ 𝑅𝑛 , 𝐵𝑘 ∈ 𝑅𝑛×𝑛 is symmetric and an approximate Hessian of 𝑓(𝑥) at 𝑥𝑘 . Then, the trust region subproblem of the unconstrained optimization problems is

(3)

‖𝑠‖ ⩽ Δ 𝑘 ,

(4) 󵄨󵄨 󵄨 󵄨󵄨(1 − 𝑎𝑘𝑇 𝑠) (1 − 𝑏𝑘𝑇 𝑠)󵄨󵄨󵄨 ⩾ 𝜖0 , 󵄨 󵄨 where 𝜖0 (0 < 𝜖0 < 1) is a sufficiently small positive number, ‖ ⋅ ‖ refers to the Euclidean norm, and Δ 𝑘 > 0 is a trust region radius. If 𝑏𝑘 = 𝑐𝑘 = 0, then 𝜓𝑘 (𝑠) is reduced to the conic model. If 𝑎𝑘 = 𝑏𝑘 = 𝑐𝑘 = 0, then 𝜓𝑘 (𝑠) is the quadratic model. In order to ensure that the fractional model function 𝜓𝑘 (𝑠) is bounded over the trust region {𝑠 | ‖𝑠‖ ⩽ Δ 𝑘 }, we assume ∃𝜖1 ∈ (0, 1/3): 󵄩󵄩 󵄩󵄩 󵄩󵄩𝑎𝑘 󵄩󵄩 Δ 𝑘 ⩽ 𝜖1 , 󵄩󵄩 󵄩󵄩 (5) 󵄩󵄩𝑏𝑘 󵄩󵄩 Δ 𝑘 ⩽ 𝜖1 , 󵄩󵄩 󵄩󵄩 󵄩󵄩𝑐𝑘 󵄩󵄩 Δ 𝑘 ⩽ 𝜖1 . We denote ̃ 𝑘 = min {Δ 𝑘 , 󵄩 𝜖1 󵄩 , 󵄩𝜖1 󵄩 , 󵄩𝜖1󵄩 } ; Δ 󵄩󵄩𝑎𝑘 󵄩󵄩 󵄩󵄩𝑏𝑘 󵄩󵄩 󵄩󵄩𝑐𝑘 󵄩󵄩 󵄩 󵄩 󵄩 󵄩 󵄩 󵄩

(6)

2

Discrete Dynamics in Nature and Society

then, (4) reduces to a simplified fractional trust region subproblem: min 𝜓 (𝑠) , s.t.

̃ 𝑘. ‖𝑠‖ ⩽ Δ

(7)

However, the subproblem (7) is solved only in quasi-Newton direction in [9]. In this paper, we made a further research, where the subproblem (7) is solved by a generalized dogleg algorithm. For linear equality constraints problem, we focus on the problems which are solved by a trust region method with new fractional model. If the constraints are linear inequality constraints or some constraint functions are nonlinear, then a few difficulties may arise. However, for these cases, the linear equality constraints problem may be as the subproblems of them. For example, the inequality constraints can be removed by an active set technique or a barrier transformation, and then the nonlinear constraints are linearized. In [10], Sun et al. established the algorithm for the problem (1)-(2) and prove the convergence. However, they do not consider the model computation. In [11], Lu and Ni proposed a trust region method with new conic model for solving (1)-(2) and carried out the numerical experiments. In this paper, we use a simple dogleg method to solve fractional model subproblems and present a quasi-Newton trust region algorithm for solving linear equality constrained optimization problems. This is a continuing work of the fractional model (see [9]), where the linear equality constraints (2) are deleted by using null space techniques. This paper is organized as follows. In Section 2, we give a description of the fractional trust region subproblem. In Section 3, we give a generalized dogleg algorithm for solving the fractional subproblem. In Section 4, we propose a new quasi-Newton method based on the fractional model for solving linearly constrained optimization problems and prove the global convergence of the proposed method under the reasonable assumptions. The numerical results are presented in Section 5.

In order to solve (8)–(11), firstly we consider removing the constraint (10) by the same assumption as in [9]. That is, we assume the parameters 𝑎𝑘 , 𝑏𝑘 , and 𝑐𝑘 satisfied (5); then, subproblem (8)–(11) can be rewritten as the following reduced subproblem: min 𝜓𝑘 (𝑠) s.t.

𝐴𝑇 𝑠 = 0,

(13)

̃ 𝑘, ‖𝑠‖ ⩽ Δ

(14)

𝑅1

𝐴 = 𝑄𝑅 = [𝑄1 𝑄2 ] [

0

] = 𝑄1 𝑅1 ,

(15)

where 𝑄1 ∈ 𝑅𝑛×𝑚 , 𝑄2 ∈ 𝑅𝑛×(𝑛−𝑚) , and 𝑅1 ∈ 𝑅𝑚×𝑚 . Then, (13) can be rewritten as 𝑅1𝑇 𝑄1𝑇 𝑠 = 0.

(16)

Therefore, the feasible point for (13) can be presented by 𝑠 = 𝑄2 𝑢,

(17)

for any 𝑢 ∈ 𝑅𝑛−𝑚 , where 𝑄2 𝑢 lies in the null space of 𝐴. Then, the subproblem (12)–(14) becomes ̃ 𝑘 (𝑢) min 𝜓 =

̃ 𝑇𝑘 𝑢 (1 + ̃𝑐𝑇𝑘 𝑢) 𝑔 𝑇 (1 − 𝑎̃𝑇𝑘 𝑢) (1 − ̃𝑏𝑘 𝑢)

(18)

2

In order to solve the problem (1)-(2), we assume that 𝐴 is column full rank and constraints are consistent. That is, the current point 𝑥𝑘 always satisfies 𝐴𝑇 𝑥𝑘 = b. Obviously, the constrained condition is equivalent to 𝐴𝑇 𝑠 = 0 if 𝑥 = 𝑥𝑘 + 𝑠. Therefore, combing with (1)–(4), we can obtain that the trial step 𝑠𝑘 is computed by the following subproblem:

s.t.

𝐴𝑇 𝑠 = 0,

̃ 𝑘 is defined as (6). where Δ The null space technology (see [4, 12, 13]) is an important technique for solving equality constraints programming problems. In the following, we use this technology to eliminate constraint (13). Since 𝐴 has full column rank, then there exist an orthogonal matrix 𝑄 ∈ 𝑅𝑛×𝑛 and a nonsingular upper triangular matrix 𝑅 ∈ 𝑅𝑚×𝑚 such that

2. The Fractional Trust Region Subproblem

min 𝜓𝑘 (𝑠) ,

(12)

(8) (9)

+

s.t.

̃ 𝑘𝑢 (1 + ̃𝑐𝑇𝑘 𝑢) 𝑢𝑇 𝐵 2 (1 −

2 𝑎̃𝑇𝑘 𝑢)

̃ 𝑘, ‖𝑢‖ ⩽ Δ

,

(19)

̃ 𝑘 is the where 𝑎̃𝑘 , ̃𝑏𝑘 , and ̃𝑐𝑘 are reduced horizontal vectors, 𝑔 ̃ 𝑘 is Hessian approximation, and reduced gradient, 𝐵 ̃ 𝑘 = 𝑄2𝑇 𝑔𝑘 , 𝑔

󵄨󵄨 󵄨 󵄨󵄨(1 − 𝑎𝑘𝑇 𝑠) (1 − 𝑏𝑘𝑇 𝑠)󵄨󵄨󵄨 ⩾ 𝜖0 , 󵄨 󵄨

(10)

̃ 𝑘 = 𝑄𝑇 𝐵𝑘 𝑄2 , 𝐵 2

‖𝑠‖ ⩽ Δ 𝑘 .

(11)

𝑎̃𝑘 = 𝑄2𝑇 𝑎𝑘 ,

It can be found that our trust region subproblem (8)–(11) is the minimization of a fractional function subject to the trust region constraint and the linear constraints.

2

𝑇 (1 − ̃𝑏𝑘 𝑢)

̃𝑏 = 𝑄𝑇 𝑏 , 𝑘 2 𝑘 ̃𝑐𝑘 = 𝑄2𝑇 𝑐𝑘 .

(20)

Discrete Dynamics in Nature and Society

3

It can be seen that this subproblem has the same form as the subproblem (7) of the unconstrained optimization problems and it can be considered as the subproblem of the unconstrained minimization over 𝑢. Therefore, we can find a solution of (18)-(19) by the dogleg method. Besides, it is easy ̃ 𝑘 is equivalent to ‖𝑢‖ ⩽ Δ ̃ 𝑘 due to find that ‖𝑠‖ = ‖𝑄2 𝑢‖ ⩽ Δ to 𝑄2𝑇 𝑄2 = 𝐼.

3. The Dogleg Method of Fractional Trust Region Subproblem Now we consider calculating the trial step 𝑢𝑘 of the new subproblem (18)-(19) by a simple dogleg method. Firstly, we need to recall the choice of the Newton point 𝑢𝑁 as the following subalgorithm (see [9]). Subalgorithm 3.1. Given 𝐵𝑘 > 0, (21)

̃ 𝑘 as defined in (20) and ̃ 𝑘 , and 𝐵 Step 1. Calculate 𝑎̃𝑘 , ̃𝑏𝑘 , ̃𝑐𝑘 , 𝑔 𝑇 −1 ̃ −1 𝑔 ̃ 𝑔 ̃ 𝑘 ̃𝑏𝑘 𝐵 𝑎̃𝑇𝑘 𝐵 𝑘 ̃𝑘,

𝜉𝑎 =

̃ −1 𝑔 ̃𝑐𝑇𝑘 𝐵 𝑘 ̃𝑘

+

𝜉𝑏 =

̃ −1 𝑔 𝑎̃𝑇𝑘 𝐵 𝑘 ̃𝑘

𝑇 −1 ̃ 𝑔 + ̃𝑏𝑘 𝐵 𝑘 ̃ 𝑘 − 1.

(22)

−1

̃ 𝑔 Step 2. If 𝜉𝑎 < 0, then 𝑢𝑁 = −𝜏𝑁𝐵 𝑘 ̃ 𝑘 , where 𝜉𝑏 + √𝜉𝑏2 + (−4𝜉𝑎 )

𝑘

𝑘

−1

−2𝜉𝑎

𝑘

𝑘

(23)

.

𝑘

𝑘

𝑘

𝑘

In the following, we consider determining the steepest descent point of (18)-(19), where the steepest descent point 𝑔𝑘 ≠ 0) and from is defined by Definition 1. Let 𝑢 = −𝜏̃ 𝑔𝑘 (̃ (18) we have ̂ (𝜏) = 𝜓 ̃ 𝑘 (−𝜏̃ 𝑔𝑘 ) ; 𝜓

(24)

then, (18)-(19) becomes ̂ (𝜏) , min 𝜓 s.t. ̂ (𝜏) 𝜓 ̃ 𝑘𝑔 ̃ 𝑇𝑘 𝐵 ̃ 𝑘 ) (−2V𝜏 𝑔 ̃ 𝑇𝑘 𝑔 ̃ 𝑘 + 𝜏 (1 − 𝜏̃𝑐𝑇𝑘 𝑔 ̃𝑘) 𝑔 ̃𝑘) 𝜏 (1 − 𝜏̃𝑐𝑇𝑘 𝑔 2V𝜏2

,

(26)

𝑇 ̃ 𝑘 ) (1 + 𝜏̃𝑏𝑘 𝑔 ̃𝑘) , V𝜏 = (1 + 𝜏̃𝑎𝑇𝑘 𝑔

̃ Δ 𝜏Δ = 󵄩󵄩 𝑘󵄩󵄩 . ̃ 𝑘 󵄩󵄩 󵄩󵄩𝑔

̃𝜏 𝜑𝜏 𝜑 , 3 V𝜏

(28)

where ̃ 𝑇𝑘 𝑔 ̃𝑘, 𝜑𝜏 = 𝑎𝜏 𝜏2 + 𝑏𝜏 𝜏 + 𝑔

(29)

̃ 𝜏 = 𝑎̃𝜏 𝜏2 + 2̃𝑐𝑇𝑘 𝑔 ̃ 𝑘 𝜏 − 1, 𝜑

(30)

̃ 𝑘𝑔 ̃𝑘𝑔 ̃ 𝑇𝑘 𝐵 ̃ 𝑘 + 𝑎̃𝑇𝑘 𝑔 ̃ 𝑘 ̃𝑏𝑘 𝑔 ̃𝑘𝑔 ̃ 𝑇𝑘 𝑔 ̃𝑘, 𝑎𝜏 = ̃𝑐𝑇𝑘 𝑔

(31)

𝑇

̃ 𝑘𝑔 ̃ 𝑇𝑘 𝐵 ̃𝑘𝑔 ̃ 𝑇𝑘 𝑔 ̃ 𝑘 + ̃𝑏𝑘 𝑔 ̃𝑘𝑔 ̃ 𝑇𝑘 𝑔 ̃𝑘 − 𝑔 ̃𝑘, 𝑏𝜏 = 𝑎̃𝑇𝑘 𝑔 𝑇

𝑇

̃ 𝑘 𝑎̃𝑇𝑘 𝑔 ̃ 𝑘 + ̃𝑐𝑇𝑘 𝑔 ̃ 𝑘 ̃𝑏𝑘 𝑔 ̃ 𝑘 + 𝑎̃𝑇𝑘 𝑔 ̃ 𝑘 ̃𝑏𝑘 𝑔 ̃𝑘. 𝑎̃𝜏 = ̃𝑐𝑇𝑘 𝑔

(32) (33)

By direct calculating, we have (34)

̃ 𝑘 ⩽ 0, During the analysis, we find that if 𝑏𝜏 ⩾ 0 and ̃𝑐𝑇𝑘 𝑔 then (29) and (30) may have no positive real zero points in most of the cases, respectively. Thus, in order to simplify the discussion, we assume 𝑏𝜏 < 0,

(35)

̃𝑐𝑇𝑘 𝑔 ̃ 𝑘 > 0.

In order to discuss the feasible stationary points of 𝜑𝜏 and ̃ 𝜏 , we first define the sets 𝑇𝜑 and 𝑇𝜑̃ which contain all the 𝜑 ̃ 𝜏 , respectively, and these positive extremum points of 𝜑𝜏 and 𝜑 extremum points should be inside the feasible region [0, 𝜏Δ ]. It is easy to obtain that

(25)

0 ⩽ 𝜏 ⩽ 𝜏Δ ,

where

=

̂ 󸀠 (𝜏) = 𝜓

̃ 𝑇𝑘 𝑔 ̃ 𝑘 − 𝑏𝜏̃𝑐𝑇𝑘 𝑔 ̃𝑘. 𝑎𝜏 = 𝑎̃𝜏 𝑔

̃ −1 𝑔 Step 3. If 𝜉𝑎 ⩾ 0, then set ̃𝑏𝑘 = ̃𝑐𝑘 = 0. If |̃𝑎𝑇𝑘 𝐵 𝑘 ̃ 𝑘 | ⩽ 𝜖1 , −1 𝑇 ̃ −1 𝑇 ̃ −1 ̃ ̃ /(1 − 𝑎̃ 𝐵 𝑔 ̃ ). If |̃𝑎 𝐵 𝑔 ̃ | > 𝜖1 , then set then 𝑢𝑁 = −𝐵 𝑔 ̃ 𝑔 𝑎̃𝑘 = 0 and 𝑢𝑁 = −𝐵 𝑘 ̃𝑘.

̂ (𝜏), by comIn order to discuss the stationary points of 𝜓 ̂ (𝜏) is putation we have that the derivative of 𝜓

𝑇

1 𝜖1 ∈ (0, ) . 3

𝜏𝑁 =

Definition 1. Let 𝜏cp be the solution of (25). Then, 𝑢cp = −𝜏cp 𝑔 is called a steepest descent point of (18)-(19).

(27)

0, if 𝑎𝜏 > 0, { { { { 𝑇𝜑 = {{̂𝜏1 } , if 𝑎𝜏 = 0, { { { {{𝜏2 } , otherwise, {𝜏3 } , { { { { { {{̂𝜏3 } , { 𝑇𝜑̃ = { { { {𝜏3 , 𝜏4 } , { { { { {0,

if 𝑎̃𝜏 > 0,

(36)

if 𝑎̃𝜏 = 0, 2

̃ 𝑘 ) < 𝑎̃𝜏 < 0, if − (̃𝑐𝑇𝑘 𝑔 otherwise,

4

Discrete Dynamics in Nature and Society Theorem 3. Suppose that (5) and (35) hold. Then, the solution of (25) is

where 𝜏̂1 = 𝜏1 = 𝜏2 = 𝜏̂3 =

𝜏3 =

𝜏4 =

̃𝑘 ̃ 𝑇𝑘 𝑔 𝑔 −𝑏𝜏

,

−𝑏𝜏 + √Δ 𝜏 , 2𝑎𝜏

(37)

−𝑏𝜏 − √Δ 𝜏 , 2𝑎𝜏 1 ̃𝑘 2̃𝑐𝑇𝑘 𝑔

𝑎̃𝜏 ̃𝜏 ̃ 𝑘 − √Δ −̃𝑐𝑇𝑘 𝑔 𝑎̃𝜏

,

(38)

(39)

2 ̃ 𝜏 = (̃𝑐𝑇 𝑔 ̃𝜏 . Δ 𝑘 ̃𝑘) + 𝑎

In order to propose a generalized dogleg method of (18)(19), we set 𝑢 (𝑡) = 𝑡𝑢𝑁 + (1 − 𝑡) 𝑢cp

Remark 2. Suppose that (5) and (35) hold. (i) If 𝑎̃𝜏 ⩾ 0, then ̃ 𝑘 )2 ⩽ 𝑎̃𝜏 < 0, 𝑎𝜏 = from (34) we know that 𝑎𝜏 > 0. (ii) If −(̃𝑐𝑇𝑘 𝑔 ̂ 0, then 𝜏1 > 𝜏Δ . From (37) and (33), we have =

̃𝑘) 𝑎̃𝜏 + (̃𝑐𝑇𝑘 𝑔 𝑎̃𝜏

⩽ 0,

̃ 𝑘 󵄩󵄩󵄩󵄩 ⩾ 1 − 𝜖1 > 0, ̃ 𝑘 ⩾ 1 − 𝜏Δ 󵄩󵄩󵄩󵄩̃𝑐𝑘 󵄩󵄩󵄩󵄩 󵄩󵄩󵄩󵄩𝑔 1 − 𝜏̂1̃𝑐𝑇𝑘 𝑔

󵄩2 󵄩 𝑑 = 󵄩󵄩󵄩󵄩𝑢𝑁 − 𝑢cp 󵄩󵄩󵄩󵄩 ,

if 𝑎̃𝜏 = 0, 2 2

̃ 𝑘 ) , 𝑎̃𝜏 < 0, if 𝑎̃𝜏 < − (̃𝑐𝑇𝑘 𝑔 otherwise,

where 𝜏̂1 , 𝜏2 , 𝜏̂3 , 𝜏3 , and 𝜏4 are determined by (37)–(39). Hence, it is easy to get the following theorems.

̃ 𝑘 , then there exist two real roots 𝑡1 and 𝑡2 : If ‖𝑢cp ‖ < Δ 𝑡1 = 𝑡2 =

−𝑒 + √𝑒2 − 𝑑𝑓 𝑑 −𝑒 − √𝑒2 − 𝑑𝑓 𝑑

, (47) ,

where 0 < 𝑡1 < 1 and 𝑡2 < 0 (see [14, 15]). Based on the preceding theorems and analysis, we now give a generalized dogleg algorithm for solving (18)-(19). Algorithm 5.

2

̃ 𝑘 ) , 𝑎̃𝜏 = 0, if 𝑎̃𝜏 < − (̃𝑐𝑇𝑘 𝑔

(46)

󵄩 󵄩2 ̃ 2 𝑓 = 󵄩󵄩󵄩󵄩𝑢cp 󵄩󵄩󵄩󵄩 − Δ 𝑘.

(41)

if 𝑎̃𝜏 > 0, ̃ 𝑘 ) < 𝑎̃𝜏 < 0, if − (̃𝑐𝑇𝑘 𝑔

𝑇

𝑒 = (𝑢𝑁 − 𝑢cp ) 𝑢cp ,

(40)

where the last inequality is obtained by (21) and this conflicts with (40). Therefore, 𝜏̂1 > 𝜏Δ . ̃ 𝑘 )2 ⩽ 𝑎̃𝜏 < 0, 𝑎𝜏 < 0, Similarly, we can prove that if −(̃𝑐𝑇𝑘 𝑔 then 𝜏2 > 𝜏Δ . Then, combining with (36) and Remark 2., we define a set

(45)

̃ 𝑘 . Denote and calculate 𝑡∗ such that ‖𝑢(𝑡∗ )‖ = Δ

2

where 𝜏̂1 is defined in (37). However, from (37) and (35) we have 𝜏̂1 > 0. We assume 𝜏̂1 ∈ (0, 𝜏Δ ]; then, from (5) and (27) we have

{𝜏3 } , { { { { { { {̂𝜏3 } , { { { { { { { {{𝜏3 , 𝜏4 } , 𝑇={ { { {̂𝜏1 } , { { { { { { { {𝜏2 } , { { { { {0,

(44)

Therefore, the steepest descent point 𝑢cp = −𝜏cp 𝑔 is an approximate solution of the fractional trust region subproblem (18)-(19), where 𝜏cp is defined by (43). Similarly, the fractional trust region subproblem (18)(19) has the following property. The proof of this theorem is similar to Theorem 3.1 in [14], so we omit its proof.

Now, we have the the following conclusions.

1−

𝑇∗ = (𝑇 ∩ {[0, 𝜏Δ ]}) ∪ {0, 𝜏Δ }

where

Theorem 4. Suppose that (5) and (35) hold, where 𝜖1 ∈ ̃ 𝑘 , then the optimal solution of (18)-(19) (0, 1/3). If ‖𝑢𝑁‖ > Δ must be on the boundary of trust region, where 𝑢𝑁 is defined in Subalgorithm 3.1.

,

̃ 𝑇𝑘 𝑔 ̃𝑘, Δ 𝜏 = 𝑏𝜏2 − 4𝑎𝜏 𝑔

̃𝑘 𝜏̂1̃𝑐𝑇𝑘 𝑔

(43)

and 𝑇 is defined by (36).

,

̃𝜏 ̃ 𝑘 + √Δ −̃𝑐𝑇𝑘 𝑔

̂ (𝜏) , 𝜏 ∈ 𝑇∗ } , 𝜏cp = arg min {𝜓

(42)

Step 1. Compute 𝑢𝑁 by Subalgorithm 3.1. ̃ 𝑘 , then 𝑢𝑘 = 𝑢𝑁, and stop. Step 2. If ‖𝑢𝑁‖ ⩽ Δ Step 3. If 𝜉𝑎 < 0, then compute 𝑏𝜏 as defined in (32), and go to Step 4. Otherwise, go to Step 5. ̃ 𝑘 ⩽ 0, go to Step 5. Otherwise, compute Step 4. If 𝑏𝜏 ⩾ 0 or ̃𝑐𝑇𝑘 𝑔 ̃ 𝑘 , where 𝜏cp is defined by (43). If 𝜏cp = 𝜏Δ , where 𝑢cp = −𝜏cp 𝑔

Discrete Dynamics in Nature and Society

5

𝜏Δ is defined by (27), then 𝑢𝑘 = 𝑢cp , and stop. If 𝜏cp = 0, go to Step 5. Otherwise, go to Step 6. Step 5. Set ̃𝑏𝑘 = ̃𝑐𝑘 = 0 and compute ̃𝑘𝑔 ̃ 𝑇𝑘 𝐵 ̃ 𝑘 − 𝑎̃𝑇𝑘 𝑔 ̃𝑘𝑔 ̃ 𝑇𝑘 𝑔 ̃𝑘, 𝑐𝑎𝑔 = 𝑔 𝜏̃cp =

̃𝑘 ̃ 𝑇𝑘 𝑔 𝑔 , 𝑐𝑎𝑔

(48)

̃ 𝑘 (𝑢) is defined where 𝑥 = 𝑥𝑘 +𝑠 = 𝑥𝑘 +𝑄2 𝑢, 𝑓𝑘 = 𝑓(𝑥𝑘 ), and 𝜓 ̃ 𝑘 are the corresponding ̃ 𝑘 , and 𝐵 as (18). Thus, 𝑚𝑘 (0) = 𝑓𝑘 , 𝑔 gradient and Hessian approximations of the function at the 𝑘th iteration. We choose 𝑢𝑘 to minimize 𝑚𝑘 (𝑢). There is a ̃ 𝑘 is positive definite. In the unique minimizer if and only if 𝐵 following, we give our algorithm. If the current iteration is the feasible point 𝑥𝑘 , then an equivalent form of (1)-(2) is to solve the reduced unconstrained problem min

𝑢∈𝑅𝑛−𝑚

̃𝑘, 𝑢𝑑 = −̃𝜏∗ 𝑔

if 𝑐𝑎𝑔 ⩽ 0, if 𝑐𝑎𝑔 > 0.

(49)

̃ , 𝑚𝑘 (0) = 𝑓 𝑘 ̃ , 𝑚𝑘 (−𝑢𝑘−1 ) = 𝑓 𝑘−1

Step 6. Calculate 𝑡1 and 𝑡2 as defined in (47); then,

̃ 𝑘−1 , ∇𝑚𝑘 (−𝑢𝑘−1 ) = 𝑔 (50)

𝑥𝑘 = 𝑥𝑘−1 + 𝑠𝑘−1 = 𝑥𝑘−1 + 𝑄2 𝑢𝑘−1 , ̃ =𝑓 ̃ (0) = 𝑓 (𝑥 ) = 𝑓 , 𝑓 𝑘 𝑘 𝑘

Then, we give the predicted decent bound in each iteration, which is the lower bound of the predicted reduction in each iteration:

̃ 𝑘−1 = 𝑄2𝑇 𝑔𝑘−1 . 𝑔

𝑓𝑘−1 = 𝑓𝑘 −

̃ 𝑇𝑘 𝑢𝑘−1 (1 − ̃𝑐𝑇𝑘 𝑢𝑘−1 ) 𝑔 ̃V𝑘−1

(58)

2

+

(52) ̃ 𝑘−1 = 𝑔

2

where 𝜉 = (1 − 𝜖1 )/(1 + 𝜖1 ) . This theorem is similar to that in [9] and its proof is omitted.

𝑇 ̃ (1 − ̃𝑐𝑇𝑘 𝑢𝑘−1 ) 𝑢𝑘−1 𝐵𝑘 𝑢𝑘−1

2̃V2𝑘−1

,

1 ̃ ̃ 𝑘 𝑢𝑘−1 ] , ̃ 𝑘 − (1 − ̃𝑐𝑇𝑘 𝑢𝑘−1 ) 𝐵 𝑄𝑘−1 [̃V𝑘−1 𝑔

̃V3𝑘−1

(59)

𝑇

where ̃V𝑘−1 = (1 + 𝑎̃𝑇𝑘 𝑢𝑘−1 )(1 + ̃𝑏𝑘 𝑢𝑘−1 ) and ̃ 𝑘−1 = (1 − ̃𝑐𝑇 𝑢𝑘−1 ) ̃V𝑘−1 𝐼 − [̃𝑐𝑘 ̃V𝑘−1 + (1 − ̃𝑐𝑇 𝑢𝑘−1 ) 𝑄 𝑘 𝑘

4. New Quasi-Newton Algorithm and Its Global Convergence

𝑇 𝑇 . ⋅ (̃𝑎𝑘 (1 + ̃𝑏𝑘 𝑢𝑘−1 ) + ̃𝑏𝑘 (1 + 𝑎̃𝑇𝑘 𝑢𝑘−1 ))] 𝑢𝑘−1

In this section, we propose a quasi-Newton method with a fractional model for linearly equality constrained optimization and prove its convergence under some reasonable conditions. In order to solve problem (1)-(2), we consider the fractional model approximation for 𝑓(𝑥) about 𝑢 = 0; that is ̃ 𝑘 (𝑢) , 𝑚𝑘 (𝑢) = 𝑓𝑘 + 𝜓

(57)

Obviously, (55) holds. Then, from (56), we have

Theorem 6. Suppose that (5) and (35) hold, where 𝜖1 ∈ (0, 1/3). If 𝑢𝑘 is obtained by Algorithm 5, then 󵄩 ̃ 󵄩󵄩 ̃𝑘 } 1 󵄩󵄩󵄩𝑔 󵄩󵄩 𝜉Δ 𝑘󵄩 󵄩 min {󵄩󵄩󵄩𝑔 ̃ , , 󵄩 󵄩 󵄩 󵄩 𝑘 󵄩 󵄩 { 󵄩󵄩 ̃ −1 󵄩󵄩󵄩 } ̃ 󵄩󵄩 2 󵄩󵄩󵄩𝐵 󵄩󵄩𝐵𝑘 󵄩󵄩 } 󵄩 𝑘 󵄩󵄩 { 󵄩 󵄩

̃ ̃ 𝑓 𝑘−1 = 𝑓 (−𝑢𝑘−1 ) = 𝑓 (𝑥𝑘−1 ) = 𝑓𝑘−1 ,

(51)

̃ 𝑘 (𝑢) is defined by (18). where 𝜓

pred (𝑢𝑘 ) ⩾

(56)

where

where 𝑢(𝑡) is defined by (45). 𝑢𝑘 = 𝑢(𝑡∗ ), and stop.

̃ 𝑘 (0) − 𝜓 ̃ 𝑘 (𝑢) , pred𝑘 (𝑢) = 𝜓

(55)

̃𝑘, ∇𝑚𝑘 (0) = 𝑔

If 𝜏̃∗ = 𝜏Δ , then 𝑢𝑘 = 𝑢𝑑 , and stop. Otherwise, calculate 𝑢cp = ̃ 𝑘 , go to Step 6. −̃𝜏cp 𝑔

̃ 𝑘 (𝑢 (0)) , 𝜓 ̃ 𝑘 (𝑢 (𝑡2 ))} , ̃ 𝑘 (𝑢 (𝑡1 )) , 𝜓 𝑡∗ = arg min {𝜓

(54)

In the following, we consider the choice of the parameter vectors 𝑎̃𝑘 , ̃𝑏𝑘 , and ̃𝑐𝑘 . We choose these vectors such that (53) satisfies the following conditions:

where {𝜏Δ , 𝜏̃∗ = { min {𝜏Δ , 𝜏̃cp } , {

̃ (𝑢) = 𝑓 (𝑥 + 𝑄 𝑢) . 𝑓 𝑘 2

(53)

(60)

If we choose ̃ 𝑘−1 , 𝑎̃𝑘 = 𝑘1 𝑔 ̃𝑏 = 𝑘 𝐵 ̃ 𝑘 2 𝑘−1 𝑢𝑘−1 , ̃𝑘, ̃𝑐𝑘 = 𝑘3 𝑔

(61)

6

Discrete Dynamics in Nature and Society

then these unknown parameters 𝑘1 , 𝑘2 , and 𝑘3 can be obtained from (58)-(59). In the following, we give the derivation process of 𝑘1 , 𝑘2 , and 𝑘3 . First, we define some notations: ̃ 𝑇𝑘−1 𝑢𝑘−1 , 𝛼𝑘 = 𝑔

where ̃ − 𝜂𝛽̃𝜁 − 𝛽𝛾. ̃ 𝛾̈ = 𝜂𝛽𝜁

Similarly, by left multiplying 𝜉2𝑇 on (59), from (63) and (66) we have

̃ 𝑇𝑘−1 𝜉1 , ̃𝑘 = 𝑔 𝛼 ̃ 𝑇𝑘−1 𝜉2 , ̂𝑘 = 𝑔 𝛼

𝑘1 =

𝑇 ̃ 𝐵𝑘−1 𝑢𝑘−1 , 𝛽𝑘 = 𝑢𝑘−1

̃ = 𝑢𝑇 𝐵 ̃ 𝛽 𝑘−1 𝑘−1 𝜉1 , 𝑘 ̂ = 𝛽 𝑘

̂ 𝑘 +𝛼 ̂) − ((̂ 𝛼𝛽 + ̃𝜄𝜂𝛽) 2 , ̂ 𝑘 + 𝛼̂ ̂̃𝜄𝜂 (𝛼̂ 𝛼𝛽 + ̃𝜄𝜂 (̂ 𝛼𝛽 + 𝛼𝛽)) 𝛼+𝛼 2

(62)

̃𝜁 = 𝑢𝑇 𝐵 ̃ 𝑘 𝑘−1 𝑘 𝜉1 ,

𝜄̇ 𝑘1 = − , 𝛾̇

̂ , ̃ − 𝜂2 ̃𝜁𝛽) 𝜄 ̇ = ̃𝜄 (̂ 𝛼𝛽

̃ 𝑇𝑘 𝑢𝑘−1 , 𝑔

̃ 𝜄2 . 𝛾̇ = 𝛼̂𝜄 + 𝜂̂ 𝛼𝛽̃

̃ 𝑇𝑘 𝜉1 , 𝛾̃𝑘 = 𝑔

where the vectors 𝜉1 and 𝜉2 are chosen to satisfy ̃ 𝑘 = 𝛾̃𝑘 = ̂𝜁𝑘 = 𝛾̂𝑘 = 0. 𝛼

𝑘3 = (63)

̃𝑘, 𝛼 ̂ 𝑘 , 𝛽𝑘 , . . . , 𝛾̃𝑘 For convenience, we omit the index 𝑘 of 𝛼𝑘 , 𝛼 and 𝛾̂𝑘 . On one hand, from (58) we have = 𝜂̃V𝑘−1 ,

(64)

where 𝜂=

(72)

And then from (66), we have

̃ 𝑇𝑘 𝜉2 , 𝛾̂𝑘 = 𝑔

1

(71)

where

̂𝜁 = 𝑢𝑇 𝐵 ̃ 𝑘 𝑘−1 𝑘 𝜉2 ,

− ̃𝑐𝑇𝑘 𝑢𝑘−1

(70)

where ̃𝜄 = 𝛾 − 𝜂𝜁. Substituting (68) into the above equation, we have

𝑇 ̃ 𝐵𝑘−1 𝜉2 , 𝑢𝑘−1

𝑇 ̃ 𝐵𝑘 𝑢𝑘−1 , 𝜁𝑘 = 𝑢𝑘−1

𝛾𝑘 =

(69)

̃ −𝑓 ̃ ) 𝛾 + √𝛾2 + 2𝜁 (𝑓 𝑘−1 𝑘 𝜁

.

(65)

̃ } is monotonically decreasing and 𝐵 ̃ 𝑘 is If the sequence {𝑓 𝑘 positive definite, then we know that 𝜂 > 0 and (64) becomes 1 − 𝑘3 𝛾 = 𝜂 (1 + 𝑘1 𝛼) (1 + 𝑘2 𝛽) .

(66)

̃ = 0. − (1 + 𝑘1 𝛼) (1 + 𝑘2 𝛽) 𝑘2 𝛽𝛾

(73)

Now we give the new quasi-Newton algorithm based on the fractional model (53). Algorithm 7. Step 0. Choose 𝑥0 ∈ 𝑅𝑛 , 𝜖1 ∈ (0, 1/3), 𝜀 > 0, Δ max > 0, 0 < ̃ 0 = 𝐼, and the initial 𝜄1 < 𝜄2 < 1, 0 < 𝛿1 < 1 < 𝛿2 , 0 < 𝜄 < 1, 𝐵 trust region radius Δ 0 ∈ (0, Δ max ]. Compute 𝑄2 as defined in (15). Set 𝑘 = 0. Step 1 (stopping criterion). Compute 𝑓𝑘 = 𝑓(𝑥𝑘 ), 𝑔𝑘 = ̃ 𝑘 = 𝑄2𝑇 𝑔𝑘 . If ‖̃ ∇𝑓(𝑥𝑘 ), and 𝑔 𝑔𝑘 ‖ ⩽ 𝜀, then 𝑥∗ = 𝑥𝑘 , and stop. If 𝑘 = 0, go to Step 3. ̃ 𝑘 by ̃𝑘 − 𝑔 ̃ 𝑘−1 . Update 𝐵 Step 2. Compute 𝑦𝑘−1 = 𝑔 ̃𝑘 = 𝐵 ̃ 𝑘−1 − 𝐵

On the other hand, by left-multiplying 𝜉1𝑇 on (59) and combining with (63), we have ̃ − 𝛽̃𝜁) − ̃𝜁) (1 − 𝑘3 𝛾) (𝑘2 (𝛽𝜁

1 − 𝜂 (1 + 𝑘1 𝛼) (1 + 𝑘2 𝛽) . 𝛾

𝑇 ̃ 𝑘−1 𝑢𝑘−1 𝑢𝑇 𝐵 ̃ 𝐵 𝑧𝑘−1 𝑧𝑘−1 𝑘−1 𝑘−1 + , 𝑇 𝑢 𝑇 𝐵 ̃ 𝑘−1 𝑢𝑘−1 𝑧𝑘−1 𝑢𝑘−1 𝑘−1

(74)

where (67)

̃ 𝑘−1 𝑢𝑘−1 , 𝜃 ∈ [0, 1] , 𝑧𝑘−1 = 𝜃𝑦𝑘−1 + (1 − 𝜃) 𝐵 𝜃

Then, from (66), we have 𝜂̃𝜁 𝑘2 = , 𝛾̈

(68)

1, { { { 𝑇 ̃ ={ 𝐵𝑘−1 𝑢𝑘−1 0.8𝑢𝑘−1 { , { 𝑇 𝑇 ̃ { 𝑢𝑘−1 𝐵𝑘−1 𝑢𝑘−1 − 𝑦𝑘−1 𝑢𝑘−1

𝑇 𝑇 ̃ 𝐵𝑘−1 𝑢𝑘−1 , (75) 𝑢𝑘−1 ⩾ 0.2𝑢𝑘−1 if 𝑦𝑘−1

otherwise.

Discrete Dynamics in Nature and Society

7

̃ 𝑘 = 𝑄𝑇 𝐵𝑘 𝑄2 , and Step 3. If 𝑘 ⩽ 1, then set 𝑎̃𝑘 = ̃𝑏𝑘 = ̃𝑐𝑘 = 0, 𝐵 2 −1 ̃ 𝑔 ̃ , compute 𝛼 such that Wolfe-Powell conditions 𝑑𝑘 = −𝐵 𝑘 𝑘 𝑘 are satisfied, and set 𝑥𝑘+1 = 𝑥𝑘 +𝑠𝑘 = 𝑥𝑘 +𝛼𝑘 𝑑𝑘 and 𝑢𝑘 = 𝑄2𝑇 𝑠𝑘 , 𝑘 = 𝑘 + 1, and go to Step 1. ̃ 𝑘 𝑢𝑘−1 get 𝜉1 and 𝜉2 . ̃ 𝑘 , and 𝐵 ̃ 𝑘−1 , 𝑔 Step 4. By the parameters 𝑔

Next we present the global convergence theorem which says the reduced gradients converge to zero. Theorem 8. Assume that (5) and (35) hold, where 𝜖1 ∈ (0, 1/3). If 𝑓 is continuously differentiable and bounded below in some set containing all iterations generated by Algorithm 7, the ̃ 𝑘 ‖} are uniformly bounded. Then, sequences {‖̃ 𝑔𝑘 ‖} and {‖𝐵

̃, 𝛼 ̂ , . . . , 𝛾, 𝛾̃ and 𝛾̂ as defined in (62). If Step 5. Compute 𝛼, 𝛼 𝛾 = 0, then set ̃𝑏𝑘 = ̃𝑐𝑘 = 0. Calculate ̃ −𝑓 ̃ )2 − (̃ 𝜌𝑘 = (𝑓 𝑔𝑇𝑘−1 𝑢𝑘−1 ) (̃ 𝑔𝑇𝑘 𝑢𝑘−1 ) , 𝑘−1 𝑘 ̃ ) + √𝜌 ̃ −𝑓 (𝑓 𝑘 𝑘 { { 𝑘−1 , ̇ 𝑇 𝛽={ −̃ 𝑔𝑘−1 𝑢𝑘−1 { {1,

if 𝜌𝑘 ⩾ 0,

(76)

󵄩 ̃ 󵄩󵄩 lim inf 󵄩󵄩󵄩𝑔 𝑘󵄩 󵄩 = 0. 𝑘→∞

Proof. Assume that the theorem is false and there is 𝜀 > 0 such ̃ 𝑘 ⩾ 𝜀 for all 𝑘. From the assumption, we can assume that that 𝑔 󵄩󵄩̃ 󵄩󵄩 󵄩󵄩𝑎𝑘 󵄩󵄩 ⩽ 𝑎, 󵄩󵄩̃ 󵄩󵄩 󵄩󵄩𝑏𝑘 󵄩󵄩 ⩽ 𝑏, 󵄩 󵄩 󵄩󵄩̃ 󵄩󵄩 󵄩󵄩𝑐𝑘 󵄩󵄩 ⩽ 𝑐, 󵄩󵄩 ̃ 󵄩󵄩 󵄩󵄩𝑔𝑘 󵄩󵄩 ⩽ 𝑔, 󵄩 ̃ 󵄩󵄩 0 < 󵄩󵄩󵄩󵄩𝐵 󵄩󵄩 ⩽ 𝐵, 𝑘󵄩 󵄩󵄩 −1 󵄩󵄩 ̃ ̃ 󵄩󵄩 ⩽ 𝐵 0 < 󵄩󵄩󵄩𝐵 󵄩 𝑘 󵄩󵄩

otherwise

and set 𝑎̃𝑘 =

1 − 𝛽̇ ̃ 𝑇𝑘−1 𝑢𝑘−1 𝑔

̃ 𝑘−1 . 𝑔

(77)

Otherwise, compute 𝛾̇ and 𝛾̈ as defined in (72) and (69). If 𝛾̇ = 0 or 𝛾̈ = 0, then set 𝑎̃𝑘 = ̃𝑏𝑘 = ̃𝑐𝑘 = 0. Otherwise, calculate 𝑎̃𝑘 , ̃𝑏𝑘 , and ̃𝑐𝑘 , where 𝑘1 , 𝑘2 , and 𝑘3 in (61) are determined by (71), (68), and (73). ̃ 𝑘 , then 𝑎̃𝑘 = 𝜖1 𝑎𝑘 /Δ ̃ 𝑘 ‖𝑎𝑘 ‖. Update ̃𝑏𝑘 and Step 6. If ‖̃𝑎𝑘 ‖ > 𝜖1 /Δ ̃𝑐𝑘 with the same way such that (5) are satisfied. ̃ 𝑘 , solve the ̃ 𝑘 , and Δ Step 7. By the parameters 𝑎̃𝑘 , ̃𝑏𝑘 , ̃𝑐𝑘 , 𝑔 subproblem (18)-(19) by Algorithm 5 to get 𝑢𝑘 . Set 𝑠𝑘 = 𝑄2 𝑢𝑘 .

(82)

(83)

hold for all 𝑘. From (52) and the assumptions in the theorem, we have pred (𝑠𝑘 ) ⩾

̃ 𝜉Δ 𝜀 ̃ 𝑘 , 𝑐2 } , min {𝜀, 𝑘 } ⩾ 𝑐1 min {Δ ̃ 2𝐵 𝐵

(84)

where 𝑐1 and 𝑐2 are some positive constants. Then, from Step 10 of Algorithm 7, we have

Step 8. Compute Ared (𝑢𝑘 ) 𝜌𝑘 = , Pred (𝑢𝑘 )

̃ 𝑘 , 𝑐2 } . 𝑓𝑘 − 𝑓𝑘+1 ⩾ 𝜄1 pred (𝑠𝑘 ) ⩾ 𝜄1 𝑐1 min {Δ (78)

where Ared (𝑢𝑘 ) = Ared (𝑠𝑘 ) = 𝑓 (𝑥𝑘 ) − 𝑓 (𝑥𝑘 + 𝑠𝑘 ) ,

(79)

̃ 𝑘 (𝑢𝑘 ) . Pred (𝑢𝑘 ) = 𝑚𝑘 (0) − 𝑚𝑘 (𝑢𝑘 ) = −𝜓

(80)

Step 9. Update the trust region radius:

̃ 𝑘+1 Δ

̃ 𝑘, 𝛿1 Δ { { { { ̃ 𝑘 , Δ max } , = {min {𝛿2 Δ { { {̃ {Δ𝑘 ,

(85)

if 𝜌𝑘 ⩽ 𝜄1 ,

󵄩 󵄩 ̃ if 𝜌𝑘 ⩾ 𝜄2 , 󵄩󵄩󵄩𝑢𝑘 󵄩󵄩󵄩 = Δ 𝑘 , (81)

otherwise.

Step 10. If 𝜌𝑘 ⩾ 𝜄1 , then 𝑥𝑘+1 = 𝑥𝑘 + 𝑠𝑘 . Set 𝑘 = 𝑘 + 1, and go to Step 1. Otherwise, 𝑥𝑘+1 = 𝑥𝑘 , 𝑘 = 𝑘 + 1, and go to Step 6.

Since 𝑓(𝑥) is bounded from below and 𝑓𝑘+1 < 𝑓𝑘 for all 𝑘, ̃ ̃ we have that ∑∞ 𝑘=1 min{Δ𝑘 , 𝑐2 } is convergent, and Δ𝑘 → 0 as 𝑘 → ∞. ̃ 𝑘 → 0, ‖𝑢𝑘 ‖ → 0. From Step 6 On the other hand, when Δ of Algorithm 7, we have 󵄨󵄨 𝑇 󵄨󵄨 󵄩󵄩 󵄩󵄩 ̃ 󵄨󵄨𝑎̃𝑘 𝑢𝑘 󵄨󵄨 ⩽ 󵄩󵄩𝑎̃𝑘 󵄩󵄩 Δ𝑘 ⩽ 𝜖1 , 󵄨 󵄨 󵄨󵄨 𝑇 󵄨󵄨 󵄨󵄨̃𝑏 𝑢 󵄨󵄨 ⩽ 𝜖 , 󵄨󵄨 𝑘 𝑘 󵄨󵄨 1 󵄨 󵄨

(86)

where 𝜖1 ∈ (0, 1/3). Thus, we have 1 󵄩 󵄩 = 1 + 𝑎̃𝑇𝑘 𝑢𝑘 + 𝑜 (󵄩󵄩󵄩𝑢𝑘 󵄩󵄩󵄩) , 1 − 𝑎̃𝑇𝑘 𝑢𝑘 𝑇 1 󵄩 󵄩 = 1 + ̃𝑏𝑘 𝑢𝑘 + 𝑜 (󵄩󵄩󵄩𝑢𝑘 󵄩󵄩󵄩) . 𝑇 ̃ 1−𝑏 𝑢 𝑘 𝑘

(87)

8

Discrete Dynamics in Nature and Society and Chen 3.3.2. Moreover, in order to test Algorithm 7 more generally, we designed some problems where the objective functions are Pro. 7–18 (see [14, 18]) and the linear equality constraints are Pro. 1–6. If ̃𝑏𝑘 = ̃𝑐𝑘 = 0 in Algorithm 7, we can obtain the conic model algorithm and call this algorithm CTR. We solve the following 18 test problems by FTR and CTR and compare their results. All the computations are carried out in Matlab R2012b on a microcomputer in double precision arithmetic. These tests use the same stopping criterion ‖̃ 𝑔𝑘 ‖ ⩽ 10−4 . The columns in the tables have the following meanings: Pro. denotes the numbers of the test problems; 𝑛 is the dimension of the test problems; Iter is the number of iterations; nf and ng are the numbers of function and gradient evaluations, respectively; ‖̃ 𝑔‖ is the Euclidean norm of the final reduced gradient; CPU(s) denotes the total iteration time of the algorithm in seconds. The parameters in these algorithms are

Table 1: Test functions. Pro.

Function name

1

HS 9

2

HS 48

3

HS 49

4

HS 50

5

Chen 3.3.1

6

Chen 3.3.2

7

Cube

8

Penalty-I

9

Beale

10

Conic

11

Extended Powell

12

Broyden Tridiagonal

13

Rosenbrock

14

Chained Freudenstein and Roth

15

Extended Trigonometric

16

Penalty-III

17

Troesch

18

Cragg and Levy

𝐵0 = 𝐼, 𝜖1 = 0.33, Δ 0 = 1, Δ max = 10, 𝑙 = 0.15, 𝜇 = 0.85,

By computing, we obtain ̃ 𝑇𝑘 𝑢𝑘 (1 + ̃𝑐𝑇𝑘 𝑢𝑘 ) 𝑔

2

̃ 𝑘 𝑢𝑘 (1 + ̃𝑐𝑇𝑘 𝑢𝑘 ) 𝑢𝑘𝑇 𝐵 (1 −

𝜄1 = 0.25,

󵄩 󵄩2 = 𝑔𝑘𝑇 𝑢𝑘 + 𝑂 (󵄩󵄩󵄩𝑢𝑘 󵄩󵄩󵄩 ) ,

𝑇 (1 − 𝑎̃𝑇𝑘 𝑢𝑘 ) (1 − ̃𝑏𝑘 𝑢𝑘 )

2 𝑎̃𝑇𝑘 𝑢𝑘 )

(91)

2

𝑇 (1 − ̃𝑏𝑘 𝑢𝑘 )

=

̃ 𝑘 𝑢𝑘 𝑢𝑘𝑇 𝐵

󵄩 󵄩2 + 𝑜 (󵄩󵄩󵄩𝑢𝑘 󵄩󵄩󵄩 ) ,

𝜄2 = 0.75, (88)

̃ 𝑘 𝑢𝑘 + 𝑜 (󵄩󵄩󵄩𝑢𝑘 󵄩󵄩󵄩2 ) . Ared (𝑢𝑘 ) = −𝑔𝑘𝑇 𝑢𝑘 − 𝑢𝑘𝑇 𝐵 󵄩 󵄩 Then, from (80), we have 󵄩 ̃ 󵄩󵄩2 󵄩 ), Pred (𝑢𝑘 ) = Ared (𝑢𝑘 ) + 𝑂 (󵄩󵄩󵄩󵄩Δ 𝑘󵄩 󵄩

(89)

which indicates that lim 𝜌𝑘 = 1.

𝑘→∞

(90)

̃ 𝑘+1 ⩾ Δ ̃ 𝑘, By the updating in Step 9 of Algorithm 7, we have Δ ̃ 𝑘 → 0. The theorem is proved. which is a contradiction to Δ

5. Numerical Tests In this section, Algorithm 7 (abbreviated as FTR) is tested with some test problems which are chosen from [16, 17]. These test problems are listed in Table 1. We choose linear constrained problems HS9, HS48, HS49, HS50, Chen 3.3.1,

𝛿2 = 4. The numerical comparison for 18 small-scale test problems is listed in Table 2. We can see that FTR is better than CTR for 15 tests in the number of iterations and the remaining 3 tests are similar. Because FTR needs some extra algebra computation for some parameters, FTR takes more time than CTR for small problems. The numerical results of some large-scale problems are presented in Table 3. From Table 3, we find that for large-scale problems the CPU time of FTR is approximately the same as that of CTR but it has fewer number of iterations. From the above comparison, we see that FTR is slightly more effective and robust for these large-scale test problems. The fractional model in Algorithm 7 is the extension of conic model. By using more information of function and gradient from the previous iterations and choosing parameters flexibly, the fractional model can be more approximate to the original problem. And the global convergence of the proposed quasi-Newton trust region algorithm is also proved. Numerical experiment shows the algorithm is effective and robust, including for large-scale test problems. The theoretical results and the numerical results lead us to believe that the method is worthy of further study. For example, we can consider using fractional model to solve the nonlinear equality constrained optimization problem.

Discrete Dynamics in Nature and Society

9

Table 2: The numerical results of Algorithm 7 for some test problems. Pro.

𝑛

Starting point

1

6

(0, 0, 0, 0, 0, 0)

2

5

(3, 5, −3, 2, −2)

3

5

(2, 2, −9, 3, 3)

4

10

5

8

6

8

7

6

(3, 4, 3, 4, 3, 4)

8

6

(3, 4, 3, 4, 3, 4)

9

6

(6, 8, 6, 8, 6, 8)

10

6

−(1.5, 2, 1.5, 2, 1.5, 2)

11

8

(0.3, 0.4, . . ., 0.3, 0.4)

12

6

(0.3, 0.4, . . ., 0.3, 0.4)

13

6

(1.5, 2, 1.5, 2, 1.5, 2)

14

6

(6, 8, 6, 8, 6, 8)

15

8

(0.3, 0.4, . . ., 0.3, 0.4)

16

6

(0.3, 0.4, . . ., 0.3, 0.4)

17

6

(0.3, 0.4, . . ., 0.3, 0.4)

18

8

(0.3, 0.4, . . ., 0.3, 0.4)

(0, −6, 6, 0, 0, 0, −6, 6, 0, 0) (0, 3, −1, −2 0, 3, −1, −2) (7/3, 11/3, −1, −1 7/3, 11/3, −1, −1)

Algorithm CTR FTR CTR FTR CTR FTR CTR FTR CTR FTR CTR FTR CTR FTR CTR FTR CTR FTR CTR FTR CTR FTR CTR FTR CTR FTR CTR FTR CTR FTR CTR FTR CTR FTR CTR FTR

Iter 6 5 13 8 27 25 12 8 16 16 22 21 8 8 14 12 21 19 13 9 15 7 26 24 6 6 34 22 10 9 13 12 29 19 12 9

nf/ng 7/8 6/6 14/9 9/9 28/21 26/17 13/13 9/9 17/14 17/17 23/19 22/19 9/9 9/9 15/11 13/10 22/18 20/18 14/10 10/7 16/13 8/8 27/16 25/14 7/7 7/7 35/20 23/23 11/9 10/8 14/12 13/12 30/17 20/17 13/9 10/9

‖𝑔‖ 7.559746(−5) 1.693174(−5) 1.637781(−5) 2.148387(−6) 5.815109(−5) 4.996278(−5) 3.889891(−5) 2.397640(−5) 7.255223(−5) 1.249467(−5) 3.128858(−6) 2.865070(−7) 2.208537(−7) 1.765690(−8) 8.493218(−7) 1.155176(−5) 1.208648(−6) 3.705893(−5) 1.789828(−6) 4.017095(−6) 3.013621(−5) 7.252083(−5) 5.317180(−5) 5.917188(−5) 8.707409(−5) 5.998420(−5) 3.185767(−5) 4.845919(−6) 1.738762(−5) 5.005195(−5) 3.409181(−5) 9.332787(−5) 1.845528(−6) 5.983264(−6) 7.722466(−7) 4.353866(−7)

CPU (s) 0.084639 0.161438 0.094287 0.098268 0.110429 0.149000 0.100359 0.152117 0.099120 0.129272 0.101571 0.167233 0.093060 0.139521 0.095873 0.166608 0.094048 0.170549 0.104145 0.158877 0.087370 0.121116 0.1160001 0.187447 0.081667 0.115520 0.106515 0.158603 0.092062 0.146342 0.083445 0.164178 0.088997 0.159851 0.087020 0.149733

Table 3: The numerical results of Algorithm 7 for some test problems (𝑛 = 3000). Pro.

Starting point

7

(6, 8, . . ., 6, 8)

8

(6, 8, . . ., 6, 8)

9

(6, 8, . . ., 6, 8)

10

(6, 8, . . ., 6, 8)

11

(6, 8, . . ., 6, 8)

Algorithm CTR FTR CTR FTR CTR FTR CTR FTR CTR FTR

Iter 25 21 12 12 25 24 12 10 18 4

nf/ng 26/26 22/22 13/12 13/11 26/26 25/25 13/13 11/11 19/19 5/5

‖𝑔‖ 3.737202(−9) 3.849243(−9) 8.930053(−6) 3.113077(−7) 3.388081(−6) 1.176732(−6) 1.226943(−7) 6.842114(−6) 2.718913(−8) 5.738665(−5)

CPU (s) 28.5976 26.2795 13.8295 14.7506 28.4904 30.7385 14.9229 14.9369 21.4796 8.48251

10

Competing Interests The authors have no competing interests regarding this paper.

Acknowledgments This work was supported by the National Natural Science Foundation of China (Grant no. 11071117 and 71301060), the Natural Science Foundation of Jiangsu Province (BK20141409), Funding of Jiangsu Innovation Program for Graduate Education (KYZZ 0089) (“the Fundamental Research Funds for the Central Universities”), and the Humanistic and Social Science Foundation of Ministry of Education of China (12YJA630122).

References [1] W. C. Davidon, “Conic approximations and collinear scalings for optimizers,” SIAM Journal on Numerical Analysis, vol. 17, no. 2, pp. 268–281, 1980. [2] R. Schnabel, “Conic methods for unconstrained minimization and tensor methods for nonlinear equations,” in Mathematical Programming: The State of the Art, A. Bachem, M. Gr¨otschel, and B. Korte, Eds., pp. 417–438, Springer, Heidelberg, Germany, 1982. [3] D. C. Sorensen, “Newton’s method with a model trust region modification,” SIAM Journal on Numerical Analysis, vol. 19, no. 2, pp. 409–426, 1982. [4] W. Sun and Y. X. Yuan, “A conic trust-region method for nonlinearly constrained optimization,” Annals of Operations Research, vol. 103, pp. 175–191, 2001. [5] C. X. Xu and X. Y. Yang, “Convergence of conic quasiNewton trust region methods for unconstrained minimization,” Mathematical Application, vol. 11, no. 2, pp. 71–76, 1998. [6] Y. X. Yuan, “A review of trust region algorithms for optimization,” in Proceedings of the International Congress on Industrial and Applied Mathematics (ICIAM ’00), vol. 99, pp. 271–282, 2000. [7] D. M. Gay, “Computing optimal locally constrained steps,” SIAM Journal on Scientific and Statistical Computing, vol. 2, no. 2, pp. 186–197, 1981. [8] J.-M. Peng and Y.-X. Yuan, “Optimality conditions for the minimization of a quadratic with two quadratic constraints,” SIAM Journal on Optimization, vol. 7, no. 3, pp. 579–594, 1997. [9] H. L. Zhu, Q. Ni, and M. L. Zeng, “A quasi-Newton trust region method based on a new fractional model,” Numerical Algebra, Control and Optimization, vol. 5, no. 3, pp. 237–249, 2015. [10] W. Y. Sun, J. Y. Yuan, and Y. X. Yuan, “Conic trust region method for linearly constrained optimization,” Journal of Computational Mathematics, vol. 21, no. 3, pp. 295–304, 2003. [11] X. P. Lu and Q. Ni, “A trust region method with new conic model for linearly constrained optimization,” Or Transactions, vol. 12, pp. 32–42, 2008. [12] Q. Ni, Optimization Method and Program Design, Science Press, Beijing, China, 2009. [13] L. W. Zhang and Q. Ni, “Trust region algorithm of new conic model for nonlinearly equality constrained optimization,” Journal on Numerical Methods and Computer Applications, vol. 31, no. 4, pp. 279–289, 2010.

Discrete Dynamics in Nature and Society [14] M. F. Zhu, Y. Xue, and F. S. Zhang, “A quasi-Newton type trust region method based on the conic model,” Numerical Mathematics, vol. 17, no. 1, pp. 36–47, 1995 (Chinese). [15] X. P. Lu and Q. Ni, “A quasi-Newton trust region method with a new conic model for the unconstrained optimization,” Applied Mathematics and Computation, vol. 204, no. 1, pp. 373–384, 2008. [16] W. Hock and K. Schittkowski, Test Examples for Nonlinear Programming Codes, Springer, Berlin, Germany, 1981. [17] X. Y. Chen, Research on the geometric algorithms for programs with constraints of linear equalities [M.S. thesis], Fujian Normal University, 2012. [18] J. J. More, B. S. Garbow, and K. E. Hillstrom, “Testing unconstrained optimization software,” ACM Transactions on Mathematical Software, vol. 7, no. 1, pp. 17–41, 1981.

Advances in

Operations Research Hindawi Publishing Corporation http://www.hindawi.com

Volume 2014

Advances in

Decision Sciences Hindawi Publishing Corporation http://www.hindawi.com

Volume 2014

Journal of

Applied Mathematics

Algebra

Hindawi Publishing Corporation http://www.hindawi.com

Hindawi Publishing Corporation http://www.hindawi.com

Volume 2014

Journal of

Probability and Statistics Volume 2014

The Scientific World Journal Hindawi Publishing Corporation http://www.hindawi.com

Hindawi Publishing Corporation http://www.hindawi.com

Volume 2014

International Journal of

Differential Equations Hindawi Publishing Corporation http://www.hindawi.com

Volume 2014

Volume 2014

Submit your manuscripts at http://www.hindawi.com International Journal of

Advances in

Combinatorics Hindawi Publishing Corporation http://www.hindawi.com

Mathematical Physics Hindawi Publishing Corporation http://www.hindawi.com

Volume 2014

Journal of

Complex Analysis Hindawi Publishing Corporation http://www.hindawi.com

Volume 2014

International Journal of Mathematics and Mathematical Sciences

Mathematical Problems in Engineering

Journal of

Mathematics Hindawi Publishing Corporation http://www.hindawi.com

Volume 2014

Hindawi Publishing Corporation http://www.hindawi.com

Volume 2014

Volume 2014

Hindawi Publishing Corporation http://www.hindawi.com

Volume 2014

Discrete Mathematics

Journal of

Volume 2014

Hindawi Publishing Corporation http://www.hindawi.com

Discrete Dynamics in Nature and Society

Journal of

Function Spaces Hindawi Publishing Corporation http://www.hindawi.com

Abstract and Applied Analysis

Volume 2014

Hindawi Publishing Corporation http://www.hindawi.com

Volume 2014

Hindawi Publishing Corporation http://www.hindawi.com

Volume 2014

International Journal of

Journal of

Stochastic Analysis

Optimization

Hindawi Publishing Corporation http://www.hindawi.com

Hindawi Publishing Corporation http://www.hindawi.com

Volume 2014

Volume 2014