A Fractional Trust Region Method for Linear Equality Constrained ...

7 downloads 0 Views 2MB Size Report
May 29, 2016 - delete linear equality constraints by using null space technique. The fractional trust region subproblem is solved by a simple dogleg method.
Hindawi Publishing Corporation Discrete Dynamics in Nature and Society Volume 2016, Article ID 8676709, 10 pages http://dx.doi.org/10.1155/2016/8676709

Research Article A Fractional Trust Region Method for Linear Equality Constrained Optimization Honglan Zhu,1,2 Qin Ni,1 Liwei Zhang,3 and Weiwei Yang1 1

College of Science, Nanjing University of Aeronautics and Astronautics, Nanjing 210016, China Business School, Huaiyin Institute of Technology, Huai’an 223003, China 3 Jiangsu Cable, Nanjing 210000, China 2

Correspondence should be addressed to Honglan Zhu; [email protected] Received 21 April 2016; Accepted 29 May 2016 Academic Editor: Josef DiblΒ΄Δ±k Copyright Β© 2016 Honglan Zhu et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. A quasi-Newton trust region method with a new fractional model for linearly constrained optimization problems is proposed. We delete linear equality constraints by using null space technique. The fractional trust region subproblem is solved by a simple dogleg method. The global convergence of the proposed algorithm is established and proved. Numerical results for test problems show the efficiency of the trust region method with new fractional model. These results give the base of further research on nonlinear optimization.

1. Introduction In this paper, we consider the linear equality constrained optimization problem: min𝑛 𝑓 (π‘₯) , π‘₯βˆˆπ‘…

s.t.

𝑇

𝐴 π‘₯ = b,

(1)

min πœ“π‘˜ (𝑠) ,

(2)

s.t.

π‘›Γ—π‘š

where 𝑓(π‘₯) are continuously differentiable, 𝐴 ∈ 𝑅 , b ∈ π‘…π‘š , and rank(𝐴) = π‘š. Trust region methods have the advantage in the theoretical analysis of convergence properties and the practice. Besides, Davidon proposed a conic model which makes use of more incorporate information at each iteration (see [1]). It is believed that combing trust region techniques with a conic model would be appealing. And it has attracted more and more attention of the researchers (see [2–8]). For unconstrained optimization problems, we proposed a new fractional model (see [9]): πœ“π‘˜ (𝑠) =

(1 + π‘π‘˜π‘‡ 𝑠) (1 βˆ’ π‘Žπ‘˜π‘‡ 𝑠) (1 βˆ’ π‘π‘˜π‘‡ 𝑠)

π‘”π‘˜π‘‡ 𝑠 2

(1 + π‘π‘˜π‘‡ 𝑠) 1 𝑠𝑇 π΅π‘˜ 𝑠, + 2 (1 βˆ’ π‘Žπ‘‡ 𝑠)2 (1 βˆ’ 𝑏𝑇 𝑠)2 π‘˜ π‘˜

where π‘Žπ‘˜ , π‘π‘˜ , π‘π‘˜ ∈ 𝑅𝑛 are horizontal vectors, π‘”π‘˜ = βˆ‡π‘“(π‘₯π‘˜ ), 𝑠 ∈ 𝑅𝑛 , π΅π‘˜ ∈ 𝑅𝑛×𝑛 is symmetric and an approximate Hessian of 𝑓(π‘₯) at π‘₯π‘˜ . Then, the trust region subproblem of the unconstrained optimization problems is

(3)

‖𝑠‖ β©½ Ξ” π‘˜ ,

(4) 󡄨󡄨 󡄨 󡄨󡄨(1 βˆ’ π‘Žπ‘˜π‘‡ 𝑠) (1 βˆ’ π‘π‘˜π‘‡ 𝑠)󡄨󡄨󡄨 β©Ύ πœ–0 , 󡄨 󡄨 where πœ–0 (0 < πœ–0 < 1) is a sufficiently small positive number, β€– β‹… β€– refers to the Euclidean norm, and Ξ” π‘˜ > 0 is a trust region radius. If π‘π‘˜ = π‘π‘˜ = 0, then πœ“π‘˜ (𝑠) is reduced to the conic model. If π‘Žπ‘˜ = π‘π‘˜ = π‘π‘˜ = 0, then πœ“π‘˜ (𝑠) is the quadratic model. In order to ensure that the fractional model function πœ“π‘˜ (𝑠) is bounded over the trust region {𝑠 | ‖𝑠‖ β©½ Ξ” π‘˜ }, we assume βˆƒπœ–1 ∈ (0, 1/3): σ΅„©σ΅„© σ΅„©σ΅„© σ΅„©σ΅„©π‘Žπ‘˜ σ΅„©σ΅„© Ξ” π‘˜ β©½ πœ–1 , σ΅„©σ΅„© σ΅„©σ΅„© (5) σ΅„©σ΅„©π‘π‘˜ σ΅„©σ΅„© Ξ” π‘˜ β©½ πœ–1 , σ΅„©σ΅„© σ΅„©σ΅„© σ΅„©σ΅„©π‘π‘˜ σ΅„©σ΅„© Ξ” π‘˜ β©½ πœ–1 . We denote Μƒ π‘˜ = min {Ξ” π‘˜ , σ΅„© πœ–1 σ΅„© , σ΅„©πœ–1 σ΅„© , σ΅„©πœ–1σ΅„© } ; Ξ” σ΅„©σ΅„©π‘Žπ‘˜ σ΅„©σ΅„© σ΅„©σ΅„©π‘π‘˜ σ΅„©σ΅„© σ΅„©σ΅„©π‘π‘˜ σ΅„©σ΅„© σ΅„© σ΅„© σ΅„© σ΅„© σ΅„© σ΅„©

(6)

2

Discrete Dynamics in Nature and Society

then, (4) reduces to a simplified fractional trust region subproblem: min πœ“ (𝑠) , s.t.

Μƒ π‘˜. ‖𝑠‖ β©½ Ξ”

(7)

However, the subproblem (7) is solved only in quasi-Newton direction in [9]. In this paper, we made a further research, where the subproblem (7) is solved by a generalized dogleg algorithm. For linear equality constraints problem, we focus on the problems which are solved by a trust region method with new fractional model. If the constraints are linear inequality constraints or some constraint functions are nonlinear, then a few difficulties may arise. However, for these cases, the linear equality constraints problem may be as the subproblems of them. For example, the inequality constraints can be removed by an active set technique or a barrier transformation, and then the nonlinear constraints are linearized. In [10], Sun et al. established the algorithm for the problem (1)-(2) and prove the convergence. However, they do not consider the model computation. In [11], Lu and Ni proposed a trust region method with new conic model for solving (1)-(2) and carried out the numerical experiments. In this paper, we use a simple dogleg method to solve fractional model subproblems and present a quasi-Newton trust region algorithm for solving linear equality constrained optimization problems. This is a continuing work of the fractional model (see [9]), where the linear equality constraints (2) are deleted by using null space techniques. This paper is organized as follows. In Section 2, we give a description of the fractional trust region subproblem. In Section 3, we give a generalized dogleg algorithm for solving the fractional subproblem. In Section 4, we propose a new quasi-Newton method based on the fractional model for solving linearly constrained optimization problems and prove the global convergence of the proposed method under the reasonable assumptions. The numerical results are presented in Section 5.

In order to solve (8)–(11), firstly we consider removing the constraint (10) by the same assumption as in [9]. That is, we assume the parameters π‘Žπ‘˜ , π‘π‘˜ , and π‘π‘˜ satisfied (5); then, subproblem (8)–(11) can be rewritten as the following reduced subproblem: min πœ“π‘˜ (𝑠) s.t.

𝐴𝑇 𝑠 = 0,

(13)

Μƒ π‘˜, ‖𝑠‖ β©½ Ξ”

(14)

𝑅1

𝐴 = 𝑄𝑅 = [𝑄1 𝑄2 ] [

0

] = 𝑄1 𝑅1 ,

(15)

where 𝑄1 ∈ π‘…π‘›Γ—π‘š , 𝑄2 ∈ 𝑅𝑛×(π‘›βˆ’π‘š) , and 𝑅1 ∈ π‘…π‘šΓ—π‘š . Then, (13) can be rewritten as 𝑅1𝑇 𝑄1𝑇 𝑠 = 0.

(16)

Therefore, the feasible point for (13) can be presented by 𝑠 = 𝑄2 𝑒,

(17)

for any 𝑒 ∈ π‘…π‘›βˆ’π‘š , where 𝑄2 𝑒 lies in the null space of 𝐴. Then, the subproblem (12)–(14) becomes Μƒ π‘˜ (𝑒) min πœ“ =

Μƒ π‘‡π‘˜ 𝑒 (1 + Μƒπ‘π‘‡π‘˜ 𝑒) 𝑔 𝑇 (1 βˆ’ π‘ŽΜƒπ‘‡π‘˜ 𝑒) (1 βˆ’ Μƒπ‘π‘˜ 𝑒)

(18)

2

In order to solve the problem (1)-(2), we assume that 𝐴 is column full rank and constraints are consistent. That is, the current point π‘₯π‘˜ always satisfies 𝐴𝑇 π‘₯π‘˜ = b. Obviously, the constrained condition is equivalent to 𝐴𝑇 𝑠 = 0 if π‘₯ = π‘₯π‘˜ + 𝑠. Therefore, combing with (1)–(4), we can obtain that the trial step π‘ π‘˜ is computed by the following subproblem:

s.t.

𝐴𝑇 𝑠 = 0,

Μƒ π‘˜ is defined as (6). where Ξ” The null space technology (see [4, 12, 13]) is an important technique for solving equality constraints programming problems. In the following, we use this technology to eliminate constraint (13). Since 𝐴 has full column rank, then there exist an orthogonal matrix 𝑄 ∈ 𝑅𝑛×𝑛 and a nonsingular upper triangular matrix 𝑅 ∈ π‘…π‘šΓ—π‘š such that

2. The Fractional Trust Region Subproblem

min πœ“π‘˜ (𝑠) ,

(12)

(8) (9)

+

s.t.

Μƒ π‘˜π‘’ (1 + Μƒπ‘π‘‡π‘˜ 𝑒) 𝑒𝑇 𝐡 2 (1 βˆ’

2 π‘ŽΜƒπ‘‡π‘˜ 𝑒)

Μƒ π‘˜, ‖𝑒‖ β©½ Ξ”

,

(19)

Μƒ π‘˜ is the where π‘ŽΜƒπ‘˜ , Μƒπ‘π‘˜ , and Μƒπ‘π‘˜ are reduced horizontal vectors, 𝑔 Μƒ π‘˜ is Hessian approximation, and reduced gradient, 𝐡 Μƒ π‘˜ = 𝑄2𝑇 π‘”π‘˜ , 𝑔

󡄨󡄨 󡄨 󡄨󡄨(1 βˆ’ π‘Žπ‘˜π‘‡ 𝑠) (1 βˆ’ π‘π‘˜π‘‡ 𝑠)󡄨󡄨󡄨 β©Ύ πœ–0 , 󡄨 󡄨

(10)

Μƒ π‘˜ = 𝑄𝑇 π΅π‘˜ 𝑄2 , 𝐡 2

‖𝑠‖ β©½ Ξ” π‘˜ .

(11)

π‘ŽΜƒπ‘˜ = 𝑄2𝑇 π‘Žπ‘˜ ,

It can be found that our trust region subproblem (8)–(11) is the minimization of a fractional function subject to the trust region constraint and the linear constraints.

2

𝑇 (1 βˆ’ Μƒπ‘π‘˜ 𝑒)

̃𝑏 = 𝑄𝑇 𝑏 , π‘˜ 2 π‘˜ Μƒπ‘π‘˜ = 𝑄2𝑇 π‘π‘˜ .

(20)

Discrete Dynamics in Nature and Society

3

It can be seen that this subproblem has the same form as the subproblem (7) of the unconstrained optimization problems and it can be considered as the subproblem of the unconstrained minimization over 𝑒. Therefore, we can find a solution of (18)-(19) by the dogleg method. Besides, it is easy Μƒ π‘˜ is equivalent to ‖𝑒‖ β©½ Ξ” Μƒ π‘˜ due to find that ‖𝑠‖ = ‖𝑄2 𝑒‖ β©½ Ξ” to 𝑄2𝑇 𝑄2 = 𝐼.

3. The Dogleg Method of Fractional Trust Region Subproblem Now we consider calculating the trial step π‘’π‘˜ of the new subproblem (18)-(19) by a simple dogleg method. Firstly, we need to recall the choice of the Newton point 𝑒𝑁 as the following subalgorithm (see [9]). Subalgorithm 3.1. Given π΅π‘˜ > 0, (21)

Μƒ π‘˜ as defined in (20) and Μƒ π‘˜ , and 𝐡 Step 1. Calculate π‘ŽΜƒπ‘˜ , Μƒπ‘π‘˜ , Μƒπ‘π‘˜ , 𝑔 𝑇 βˆ’1 Μƒ βˆ’1 𝑔 Μƒ 𝑔 Μƒ π‘˜ Μƒπ‘π‘˜ 𝐡 π‘ŽΜƒπ‘‡π‘˜ 𝐡 π‘˜ Μƒπ‘˜,

πœ‰π‘Ž =

Μƒ βˆ’1 𝑔 Μƒπ‘π‘‡π‘˜ 𝐡 π‘˜ Μƒπ‘˜

+

πœ‰π‘ =

Μƒ βˆ’1 𝑔 π‘ŽΜƒπ‘‡π‘˜ 𝐡 π‘˜ Μƒπ‘˜

𝑇 βˆ’1 Μƒ 𝑔 + Μƒπ‘π‘˜ 𝐡 π‘˜ Μƒ π‘˜ βˆ’ 1.

(22)

βˆ’1

Μƒ 𝑔 Step 2. If πœ‰π‘Ž < 0, then 𝑒𝑁 = βˆ’πœπ‘π΅ π‘˜ Μƒ π‘˜ , where πœ‰π‘ + βˆšπœ‰π‘2 + (βˆ’4πœ‰π‘Ž )

π‘˜

π‘˜

βˆ’1

βˆ’2πœ‰π‘Ž

π‘˜

π‘˜

(23)

.

π‘˜

π‘˜

π‘˜

π‘˜

In the following, we consider determining the steepest descent point of (18)-(19), where the steepest descent point π‘”π‘˜ =ΜΈ 0) and from is defined by Definition 1. Let 𝑒 = βˆ’πœΜƒ π‘”π‘˜ (Μƒ (18) we have Μ‚ (𝜏) = πœ“ Μƒ π‘˜ (βˆ’πœΜƒ π‘”π‘˜ ) ; πœ“

(24)

then, (18)-(19) becomes Μ‚ (𝜏) , min πœ“ s.t. Μ‚ (𝜏) πœ“ Μƒ π‘˜π‘” Μƒ π‘‡π‘˜ 𝐡 Μƒ π‘˜ ) (βˆ’2V𝜏 𝑔 Μƒ π‘‡π‘˜ 𝑔 Μƒ π‘˜ + 𝜏 (1 βˆ’ πœΜƒπ‘π‘‡π‘˜ 𝑔 Μƒπ‘˜) 𝑔 Μƒπ‘˜) 𝜏 (1 βˆ’ πœΜƒπ‘π‘‡π‘˜ 𝑔 2V𝜏2

,

(26)

𝑇 Μƒ π‘˜ ) (1 + πœΜƒπ‘π‘˜ 𝑔 Μƒπ‘˜) , V𝜏 = (1 + πœΜƒπ‘Žπ‘‡π‘˜ 𝑔

Μƒ Ξ” πœΞ” = σ΅„©σ΅„© π‘˜σ΅„©σ΅„© . Μƒ π‘˜ σ΅„©σ΅„© 󡄩󡄩𝑔

Μƒπœ πœ‘πœ πœ‘ , 3 V𝜏

(28)

where Μƒ π‘‡π‘˜ 𝑔 Μƒπ‘˜, πœ‘πœ = π‘Žπœ 𝜏2 + π‘πœ 𝜏 + 𝑔

(29)

Μƒ 𝜏 = π‘ŽΜƒπœ 𝜏2 + 2Μƒπ‘π‘‡π‘˜ 𝑔 Μƒ π‘˜ 𝜏 βˆ’ 1, πœ‘

(30)

Μƒ π‘˜π‘” Μƒπ‘˜π‘” Μƒ π‘‡π‘˜ 𝐡 Μƒ π‘˜ + π‘ŽΜƒπ‘‡π‘˜ 𝑔 Μƒ π‘˜ Μƒπ‘π‘˜ 𝑔 Μƒπ‘˜π‘” Μƒ π‘‡π‘˜ 𝑔 Μƒπ‘˜, π‘Žπœ = Μƒπ‘π‘‡π‘˜ 𝑔

(31)

𝑇

Μƒ π‘˜π‘” Μƒ π‘‡π‘˜ 𝐡 Μƒπ‘˜π‘” Μƒ π‘‡π‘˜ 𝑔 Μƒ π‘˜ + Μƒπ‘π‘˜ 𝑔 Μƒπ‘˜π‘” Μƒ π‘‡π‘˜ 𝑔 Μƒπ‘˜ βˆ’ 𝑔 Μƒπ‘˜, π‘πœ = π‘ŽΜƒπ‘‡π‘˜ 𝑔 𝑇

𝑇

Μƒ π‘˜ π‘ŽΜƒπ‘‡π‘˜ 𝑔 Μƒ π‘˜ + Μƒπ‘π‘‡π‘˜ 𝑔 Μƒ π‘˜ Μƒπ‘π‘˜ 𝑔 Μƒ π‘˜ + π‘ŽΜƒπ‘‡π‘˜ 𝑔 Μƒ π‘˜ Μƒπ‘π‘˜ 𝑔 Μƒπ‘˜. π‘ŽΜƒπœ = Μƒπ‘π‘‡π‘˜ 𝑔

(32) (33)

By direct calculating, we have (34)

Μƒ π‘˜ β©½ 0, During the analysis, we find that if π‘πœ β©Ύ 0 and Μƒπ‘π‘‡π‘˜ 𝑔 then (29) and (30) may have no positive real zero points in most of the cases, respectively. Thus, in order to simplify the discussion, we assume π‘πœ < 0,

(35)

Μƒπ‘π‘‡π‘˜ 𝑔 Μƒ π‘˜ > 0.

In order to discuss the feasible stationary points of πœ‘πœ and Μƒ 𝜏 , we first define the sets π‘‡πœ‘ and π‘‡πœ‘Μƒ which contain all the πœ‘ Μƒ 𝜏 , respectively, and these positive extremum points of πœ‘πœ and πœ‘ extremum points should be inside the feasible region [0, πœΞ” ]. It is easy to obtain that

(25)

0 β©½ 𝜏 β©½ πœΞ” ,

where

=

Μ‚ σΈ€  (𝜏) = πœ“

Μƒ π‘‡π‘˜ 𝑔 Μƒ π‘˜ βˆ’ π‘πœΜƒπ‘π‘‡π‘˜ 𝑔 Μƒπ‘˜. π‘Žπœ = π‘ŽΜƒπœ 𝑔

Μƒ βˆ’1 𝑔 Step 3. If πœ‰π‘Ž β©Ύ 0, then set Μƒπ‘π‘˜ = Μƒπ‘π‘˜ = 0. If |Μƒπ‘Žπ‘‡π‘˜ 𝐡 π‘˜ Μƒ π‘˜ | β©½ πœ–1 , βˆ’1 𝑇 Μƒ βˆ’1 𝑇 Μƒ βˆ’1 Μƒ Μƒ /(1 βˆ’ π‘ŽΜƒ 𝐡 𝑔 Μƒ ). If |Μƒπ‘Ž 𝐡 𝑔 Μƒ | > πœ–1 , then set then 𝑒𝑁 = βˆ’π΅ 𝑔 Μƒ 𝑔 π‘ŽΜƒπ‘˜ = 0 and 𝑒𝑁 = βˆ’π΅ π‘˜ Μƒπ‘˜.

Μ‚ (𝜏), by comIn order to discuss the stationary points of πœ“ Μ‚ (𝜏) is putation we have that the derivative of πœ“

𝑇

1 πœ–1 ∈ (0, ) . 3

πœπ‘ =

Definition 1. Let 𝜏cp be the solution of (25). Then, 𝑒cp = βˆ’πœcp 𝑔 is called a steepest descent point of (18)-(19).

(27)

0, if π‘Žπœ > 0, { { { { π‘‡πœ‘ = {{Μ‚πœ1 } , if π‘Žπœ = 0, { { { {{𝜏2 } , otherwise, {𝜏3 } , { { { { { {{Μ‚πœ3 } , { π‘‡πœ‘Μƒ = { { { {𝜏3 , 𝜏4 } , { { { { {0,

if π‘ŽΜƒπœ > 0,

(36)

if π‘ŽΜƒπœ = 0, 2

Μƒ π‘˜ ) < π‘ŽΜƒπœ < 0, if βˆ’ (Μƒπ‘π‘‡π‘˜ 𝑔 otherwise,

4

Discrete Dynamics in Nature and Society Theorem 3. Suppose that (5) and (35) hold. Then, the solution of (25) is

where πœΜ‚1 = 𝜏1 = 𝜏2 = πœΜ‚3 =

𝜏3 =

𝜏4 =

Μƒπ‘˜ Μƒ π‘‡π‘˜ 𝑔 𝑔 βˆ’π‘πœ

,

βˆ’π‘πœ + βˆšΞ” 𝜏 , 2π‘Žπœ

(37)

βˆ’π‘πœ βˆ’ βˆšΞ” 𝜏 , 2π‘Žπœ 1 Μƒπ‘˜ 2Μƒπ‘π‘‡π‘˜ 𝑔

π‘ŽΜƒπœ Μƒπœ Μƒ π‘˜ βˆ’ βˆšΞ” βˆ’Μƒπ‘π‘‡π‘˜ 𝑔 π‘ŽΜƒπœ

,

(38)

(39)

2 Μƒ 𝜏 = (̃𝑐𝑇 𝑔 Μƒπœ . Ξ” π‘˜ Μƒπ‘˜) + π‘Ž

In order to propose a generalized dogleg method of (18)(19), we set 𝑒 (𝑑) = 𝑑𝑒𝑁 + (1 βˆ’ 𝑑) 𝑒cp

Remark 2. Suppose that (5) and (35) hold. (i) If π‘ŽΜƒπœ β©Ύ 0, then Μƒ π‘˜ )2 β©½ π‘ŽΜƒπœ < 0, π‘Žπœ = from (34) we know that π‘Žπœ > 0. (ii) If βˆ’(Μƒπ‘π‘‡π‘˜ 𝑔 Μ‚ 0, then 𝜏1 > πœΞ” . From (37) and (33), we have =

Μƒπ‘˜) π‘ŽΜƒπœ + (Μƒπ‘π‘‡π‘˜ 𝑔 π‘ŽΜƒπœ

β©½ 0,

Μƒ π‘˜ σ΅„©σ΅„©σ΅„©σ΅„© β©Ύ 1 βˆ’ πœ–1 > 0, Μƒ π‘˜ β©Ύ 1 βˆ’ πœΞ” σ΅„©σ΅„©σ΅„©σ΅„©Μƒπ‘π‘˜ σ΅„©σ΅„©σ΅„©σ΅„© 󡄩󡄩󡄩󡄩𝑔 1 βˆ’ πœΜ‚1Μƒπ‘π‘‡π‘˜ 𝑔

σ΅„©2 σ΅„© 𝑑 = 󡄩󡄩󡄩󡄩𝑒𝑁 βˆ’ 𝑒cp σ΅„©σ΅„©σ΅„©σ΅„© ,

if π‘ŽΜƒπœ = 0, 2 2

Μƒ π‘˜ ) , π‘ŽΜƒπœ < 0, if π‘ŽΜƒπœ < βˆ’ (Μƒπ‘π‘‡π‘˜ 𝑔 otherwise,

where πœΜ‚1 , 𝜏2 , πœΜ‚3 , 𝜏3 , and 𝜏4 are determined by (37)–(39). Hence, it is easy to get the following theorems.

Μƒ π‘˜ , then there exist two real roots 𝑑1 and 𝑑2 : If ‖𝑒cp β€– < Ξ” 𝑑1 = 𝑑2 =

βˆ’π‘’ + βˆšπ‘’2 βˆ’ 𝑑𝑓 𝑑 βˆ’π‘’ βˆ’ βˆšπ‘’2 βˆ’ 𝑑𝑓 𝑑

, (47) ,

where 0 < 𝑑1 < 1 and 𝑑2 < 0 (see [14, 15]). Based on the preceding theorems and analysis, we now give a generalized dogleg algorithm for solving (18)-(19). Algorithm 5.

2

Μƒ π‘˜ ) , π‘ŽΜƒπœ = 0, if π‘ŽΜƒπœ < βˆ’ (Μƒπ‘π‘‡π‘˜ 𝑔

(46)

σ΅„© σ΅„©2 Μƒ 2 𝑓 = 󡄩󡄩󡄩󡄩𝑒cp σ΅„©σ΅„©σ΅„©σ΅„© βˆ’ Ξ” π‘˜.

(41)

if π‘ŽΜƒπœ > 0, Μƒ π‘˜ ) < π‘ŽΜƒπœ < 0, if βˆ’ (Μƒπ‘π‘‡π‘˜ 𝑔

𝑇

𝑒 = (𝑒𝑁 βˆ’ 𝑒cp ) 𝑒cp ,

(40)

where the last inequality is obtained by (21) and this conflicts with (40). Therefore, πœΜ‚1 > πœΞ” . Μƒ π‘˜ )2 β©½ π‘ŽΜƒπœ < 0, π‘Žπœ < 0, Similarly, we can prove that if βˆ’(Μƒπ‘π‘‡π‘˜ 𝑔 then 𝜏2 > πœΞ” . Then, combining with (36) and Remark 2., we define a set

(45)

Μƒ π‘˜ . Denote and calculate π‘‘βˆ— such that ‖𝑒(π‘‘βˆ— )β€– = Ξ”

2

where πœΜ‚1 is defined in (37). However, from (37) and (35) we have πœΜ‚1 > 0. We assume πœΜ‚1 ∈ (0, πœΞ” ]; then, from (5) and (27) we have

{𝜏3 } , { { { { { { {Μ‚πœ3 } , { { { { { { { {{𝜏3 , 𝜏4 } , 𝑇={ { { {Μ‚πœ1 } , { { { { { { { {𝜏2 } , { { { { {0,

(44)

Therefore, the steepest descent point 𝑒cp = βˆ’πœcp 𝑔 is an approximate solution of the fractional trust region subproblem (18)-(19), where 𝜏cp is defined by (43). Similarly, the fractional trust region subproblem (18)(19) has the following property. The proof of this theorem is similar to Theorem 3.1 in [14], so we omit its proof.

Now, we have the the following conclusions.

1βˆ’

π‘‡βˆ— = (𝑇 ∩ {[0, πœΞ” ]}) βˆͺ {0, πœΞ” }

where

Theorem 4. Suppose that (5) and (35) hold, where πœ–1 ∈ Μƒ π‘˜ , then the optimal solution of (18)-(19) (0, 1/3). If ‖𝑒𝑁‖ > Ξ” must be on the boundary of trust region, where 𝑒𝑁 is defined in Subalgorithm 3.1.

,

Μƒ π‘‡π‘˜ 𝑔 Μƒπ‘˜, Ξ” 𝜏 = π‘πœ2 βˆ’ 4π‘Žπœ 𝑔

Μƒπ‘˜ πœΜ‚1Μƒπ‘π‘‡π‘˜ 𝑔

(43)

and 𝑇 is defined by (36).

,

Μƒπœ Μƒ π‘˜ + βˆšΞ” βˆ’Μƒπ‘π‘‡π‘˜ 𝑔

Μ‚ (𝜏) , 𝜏 ∈ π‘‡βˆ— } , 𝜏cp = arg min {πœ“

(42)

Step 1. Compute 𝑒𝑁 by Subalgorithm 3.1. Μƒ π‘˜ , then π‘’π‘˜ = 𝑒𝑁, and stop. Step 2. If ‖𝑒𝑁‖ β©½ Ξ” Step 3. If πœ‰π‘Ž < 0, then compute π‘πœ as defined in (32), and go to Step 4. Otherwise, go to Step 5. Μƒ π‘˜ β©½ 0, go to Step 5. Otherwise, compute Step 4. If π‘πœ β©Ύ 0 or Μƒπ‘π‘‡π‘˜ 𝑔 Μƒ π‘˜ , where 𝜏cp is defined by (43). If 𝜏cp = πœΞ” , where 𝑒cp = βˆ’πœcp 𝑔

Discrete Dynamics in Nature and Society

5

πœΞ” is defined by (27), then π‘’π‘˜ = 𝑒cp , and stop. If 𝜏cp = 0, go to Step 5. Otherwise, go to Step 6. Step 5. Set Μƒπ‘π‘˜ = Μƒπ‘π‘˜ = 0 and compute Μƒπ‘˜π‘” Μƒ π‘‡π‘˜ 𝐡 Μƒ π‘˜ βˆ’ π‘ŽΜƒπ‘‡π‘˜ 𝑔 Μƒπ‘˜π‘” Μƒ π‘‡π‘˜ 𝑔 Μƒπ‘˜, π‘π‘Žπ‘” = 𝑔 πœΜƒcp =

Μƒπ‘˜ Μƒ π‘‡π‘˜ 𝑔 𝑔 , π‘π‘Žπ‘”

(48)

Μƒ π‘˜ (𝑒) is defined where π‘₯ = π‘₯π‘˜ +𝑠 = π‘₯π‘˜ +𝑄2 𝑒, π‘“π‘˜ = 𝑓(π‘₯π‘˜ ), and πœ“ Μƒ π‘˜ are the corresponding Μƒ π‘˜ , and 𝐡 as (18). Thus, π‘šπ‘˜ (0) = π‘“π‘˜ , 𝑔 gradient and Hessian approximations of the function at the π‘˜th iteration. We choose π‘’π‘˜ to minimize π‘šπ‘˜ (𝑒). There is a Μƒ π‘˜ is positive definite. In the unique minimizer if and only if 𝐡 following, we give our algorithm. If the current iteration is the feasible point π‘₯π‘˜ , then an equivalent form of (1)-(2) is to solve the reduced unconstrained problem min

π‘’βˆˆπ‘…π‘›βˆ’π‘š

Μƒπ‘˜, 𝑒𝑑 = βˆ’Μƒπœβˆ— 𝑔

if π‘π‘Žπ‘” β©½ 0, if π‘π‘Žπ‘” > 0.

(49)

Μƒ , π‘šπ‘˜ (0) = 𝑓 π‘˜ Μƒ , π‘šπ‘˜ (βˆ’π‘’π‘˜βˆ’1 ) = 𝑓 π‘˜βˆ’1

Step 6. Calculate 𝑑1 and 𝑑2 as defined in (47); then,

Μƒ π‘˜βˆ’1 , βˆ‡π‘šπ‘˜ (βˆ’π‘’π‘˜βˆ’1 ) = 𝑔 (50)

π‘₯π‘˜ = π‘₯π‘˜βˆ’1 + π‘ π‘˜βˆ’1 = π‘₯π‘˜βˆ’1 + 𝑄2 π‘’π‘˜βˆ’1 , Μƒ =𝑓 Μƒ (0) = 𝑓 (π‘₯ ) = 𝑓 , 𝑓 π‘˜ π‘˜ π‘˜

Then, we give the predicted decent bound in each iteration, which is the lower bound of the predicted reduction in each iteration:

Μƒ π‘˜βˆ’1 = 𝑄2𝑇 π‘”π‘˜βˆ’1 . 𝑔

π‘“π‘˜βˆ’1 = π‘“π‘˜ βˆ’

Μƒ π‘‡π‘˜ π‘’π‘˜βˆ’1 (1 βˆ’ Μƒπ‘π‘‡π‘˜ π‘’π‘˜βˆ’1 ) 𝑔 ΜƒVπ‘˜βˆ’1

(58)

2

+

(52) Μƒ π‘˜βˆ’1 = 𝑔

2

where πœ‰ = (1 βˆ’ πœ–1 )/(1 + πœ–1 ) . This theorem is similar to that in [9] and its proof is omitted.

𝑇 Μƒ (1 βˆ’ Μƒπ‘π‘‡π‘˜ π‘’π‘˜βˆ’1 ) π‘’π‘˜βˆ’1 π΅π‘˜ π‘’π‘˜βˆ’1

2ΜƒV2π‘˜βˆ’1

,

1 Μƒ Μƒ π‘˜ π‘’π‘˜βˆ’1 ] , Μƒ π‘˜ βˆ’ (1 βˆ’ Μƒπ‘π‘‡π‘˜ π‘’π‘˜βˆ’1 ) 𝐡 π‘„π‘˜βˆ’1 [ΜƒVπ‘˜βˆ’1 𝑔

ΜƒV3π‘˜βˆ’1

(59)

𝑇

where ΜƒVπ‘˜βˆ’1 = (1 + π‘ŽΜƒπ‘‡π‘˜ π‘’π‘˜βˆ’1 )(1 + Μƒπ‘π‘˜ π‘’π‘˜βˆ’1 ) and Μƒ π‘˜βˆ’1 = (1 βˆ’ ̃𝑐𝑇 π‘’π‘˜βˆ’1 ) ΜƒVπ‘˜βˆ’1 𝐼 βˆ’ [Μƒπ‘π‘˜ ΜƒVπ‘˜βˆ’1 + (1 βˆ’ ̃𝑐𝑇 π‘’π‘˜βˆ’1 ) 𝑄 π‘˜ π‘˜

4. New Quasi-Newton Algorithm and Its Global Convergence

𝑇 𝑇 . β‹… (Μƒπ‘Žπ‘˜ (1 + Μƒπ‘π‘˜ π‘’π‘˜βˆ’1 ) + Μƒπ‘π‘˜ (1 + π‘ŽΜƒπ‘‡π‘˜ π‘’π‘˜βˆ’1 ))] π‘’π‘˜βˆ’1

In this section, we propose a quasi-Newton method with a fractional model for linearly equality constrained optimization and prove its convergence under some reasonable conditions. In order to solve problem (1)-(2), we consider the fractional model approximation for 𝑓(π‘₯) about 𝑒 = 0; that is Μƒ π‘˜ (𝑒) , π‘šπ‘˜ (𝑒) = π‘“π‘˜ + πœ“

(57)

Obviously, (55) holds. Then, from (56), we have

Theorem 6. Suppose that (5) and (35) hold, where πœ–1 ∈ (0, 1/3). If π‘’π‘˜ is obtained by Algorithm 5, then σ΅„© Μƒ σ΅„©σ΅„© Μƒπ‘˜ } 1 󡄩󡄩󡄩𝑔 σ΅„©σ΅„© πœ‰Ξ” π‘˜σ΅„© σ΅„© min {󡄩󡄩󡄩𝑔 Μƒ , , σ΅„© σ΅„© σ΅„© σ΅„© π‘˜ σ΅„© σ΅„© { σ΅„©σ΅„© Μƒ βˆ’1 σ΅„©σ΅„©σ΅„© } Μƒ σ΅„©σ΅„© 2 󡄩󡄩󡄩𝐡 σ΅„©σ΅„©π΅π‘˜ σ΅„©σ΅„© } σ΅„© π‘˜ σ΅„©σ΅„© { σ΅„© σ΅„©

Μƒ Μƒ 𝑓 π‘˜βˆ’1 = 𝑓 (βˆ’π‘’π‘˜βˆ’1 ) = 𝑓 (π‘₯π‘˜βˆ’1 ) = π‘“π‘˜βˆ’1 ,

(51)

Μƒ π‘˜ (𝑒) is defined by (18). where πœ“

pred (π‘’π‘˜ ) β©Ύ

(56)

where

where 𝑒(𝑑) is defined by (45). π‘’π‘˜ = 𝑒(π‘‘βˆ— ), and stop.

Μƒ π‘˜ (0) βˆ’ πœ“ Μƒ π‘˜ (𝑒) , predπ‘˜ (𝑒) = πœ“

(55)

Μƒπ‘˜, βˆ‡π‘šπ‘˜ (0) = 𝑔

If πœΜƒβˆ— = πœΞ” , then π‘’π‘˜ = 𝑒𝑑 , and stop. Otherwise, calculate 𝑒cp = Μƒ π‘˜ , go to Step 6. βˆ’Μƒπœcp 𝑔

Μƒ π‘˜ (𝑒 (0)) , πœ“ Μƒ π‘˜ (𝑒 (𝑑2 ))} , Μƒ π‘˜ (𝑒 (𝑑1 )) , πœ“ π‘‘βˆ— = arg min {πœ“

(54)

In the following, we consider the choice of the parameter vectors π‘ŽΜƒπ‘˜ , Μƒπ‘π‘˜ , and Μƒπ‘π‘˜ . We choose these vectors such that (53) satisfies the following conditions:

where {πœΞ” , πœΜƒβˆ— = { min {πœΞ” , πœΜƒcp } , {

Μƒ (𝑒) = 𝑓 (π‘₯ + 𝑄 𝑒) . 𝑓 π‘˜ 2

(53)

(60)

If we choose Μƒ π‘˜βˆ’1 , π‘ŽΜƒπ‘˜ = π‘˜1 𝑔 ̃𝑏 = π‘˜ 𝐡 Μƒ π‘˜ 2 π‘˜βˆ’1 π‘’π‘˜βˆ’1 , Μƒπ‘˜, Μƒπ‘π‘˜ = π‘˜3 𝑔

(61)

6

Discrete Dynamics in Nature and Society

then these unknown parameters π‘˜1 , π‘˜2 , and π‘˜3 can be obtained from (58)-(59). In the following, we give the derivation process of π‘˜1 , π‘˜2 , and π‘˜3 . First, we define some notations: Μƒ π‘‡π‘˜βˆ’1 π‘’π‘˜βˆ’1 , π›Όπ‘˜ = 𝑔

where Μƒ βˆ’ πœ‚π›½Μƒπœ βˆ’ 𝛽𝛾. Μƒ π›ΎΜˆ = πœ‚π›½πœ

Similarly, by left multiplying πœ‰2𝑇 on (59), from (63) and (66) we have

Μƒ π‘‡π‘˜βˆ’1 πœ‰1 , Μƒπ‘˜ = 𝑔 𝛼 Μƒ π‘‡π‘˜βˆ’1 πœ‰2 , Μ‚π‘˜ = 𝑔 𝛼

π‘˜1 =

𝑇 Μƒ π΅π‘˜βˆ’1 π‘’π‘˜βˆ’1 , π›½π‘˜ = π‘’π‘˜βˆ’1

Μƒ = 𝑒𝑇 𝐡 Μƒ 𝛽 π‘˜βˆ’1 π‘˜βˆ’1 πœ‰1 , π‘˜ Μ‚ = 𝛽 π‘˜

Μ‚ π‘˜ +𝛼 Μ‚) βˆ’ ((Μ‚ 𝛼𝛽 + Μƒπœ„πœ‚π›½) 2 , Μ‚ π‘˜ + 𝛼̂ Μ‚Μƒπœ„πœ‚ (𝛼̂ 𝛼𝛽 + Μƒπœ„πœ‚ (Μ‚ 𝛼𝛽 + 𝛼𝛽)) 𝛼+𝛼 2

(62)

Μƒπœ = 𝑒𝑇 𝐡 Μƒ π‘˜ π‘˜βˆ’1 π‘˜ πœ‰1 ,

πœ„Μ‡ π‘˜1 = βˆ’ , 𝛾̇

Μ‚ , Μƒ βˆ’ πœ‚2 Μƒπœπ›½) πœ„ Μ‡ = Μƒπœ„ (Μ‚ 𝛼𝛽

Μƒ π‘‡π‘˜ π‘’π‘˜βˆ’1 , 𝑔

Μƒ πœ„2 . 𝛾̇ = π›ΌΜ‚πœ„ + πœ‚Μ‚ 𝛼𝛽̃

Μƒ π‘‡π‘˜ πœ‰1 , π›ΎΜƒπ‘˜ = 𝑔

where the vectors πœ‰1 and πœ‰2 are chosen to satisfy Μƒ π‘˜ = π›ΎΜƒπ‘˜ = Μ‚πœπ‘˜ = π›ΎΜ‚π‘˜ = 0. 𝛼

π‘˜3 = (63)

Μƒπ‘˜, 𝛼 Μ‚ π‘˜ , π›½π‘˜ , . . . , π›ΎΜƒπ‘˜ For convenience, we omit the index π‘˜ of π›Όπ‘˜ , 𝛼 and π›ΎΜ‚π‘˜ . On one hand, from (58) we have = πœ‚ΜƒVπ‘˜βˆ’1 ,

(64)

where πœ‚=

(72)

And then from (66), we have

Μƒ π‘‡π‘˜ πœ‰2 , π›ΎΜ‚π‘˜ = 𝑔

1

(71)

where

Μ‚πœ = 𝑒𝑇 𝐡 Μƒ π‘˜ π‘˜βˆ’1 π‘˜ πœ‰2 ,

βˆ’ Μƒπ‘π‘‡π‘˜ π‘’π‘˜βˆ’1

(70)

where Μƒπœ„ = 𝛾 βˆ’ πœ‚πœ. Substituting (68) into the above equation, we have

𝑇 Μƒ π΅π‘˜βˆ’1 πœ‰2 , π‘’π‘˜βˆ’1

𝑇 Μƒ π΅π‘˜ π‘’π‘˜βˆ’1 , πœπ‘˜ = π‘’π‘˜βˆ’1

π›Ύπ‘˜ =

(69)

Μƒ βˆ’π‘“ Μƒ ) 𝛾 + βˆšπ›Ύ2 + 2𝜁 (𝑓 π‘˜βˆ’1 π‘˜ 𝜁

.

(65)

Μƒ } is monotonically decreasing and 𝐡 Μƒ π‘˜ is If the sequence {𝑓 π‘˜ positive definite, then we know that πœ‚ > 0 and (64) becomes 1 βˆ’ π‘˜3 𝛾 = πœ‚ (1 + π‘˜1 𝛼) (1 + π‘˜2 𝛽) .

(66)

Μƒ = 0. βˆ’ (1 + π‘˜1 𝛼) (1 + π‘˜2 𝛽) π‘˜2 𝛽𝛾

(73)

Now we give the new quasi-Newton algorithm based on the fractional model (53). Algorithm 7. Step 0. Choose π‘₯0 ∈ 𝑅𝑛 , πœ–1 ∈ (0, 1/3), πœ€ > 0, Ξ” max > 0, 0 < Μƒ 0 = 𝐼, and the initial πœ„1 < πœ„2 < 1, 0 < 𝛿1 < 1 < 𝛿2 , 0 < πœ„ < 1, 𝐡 trust region radius Ξ” 0 ∈ (0, Ξ” max ]. Compute 𝑄2 as defined in (15). Set π‘˜ = 0. Step 1 (stopping criterion). Compute π‘“π‘˜ = 𝑓(π‘₯π‘˜ ), π‘”π‘˜ = Μƒ π‘˜ = 𝑄2𝑇 π‘”π‘˜ . If β€–Μƒ βˆ‡π‘“(π‘₯π‘˜ ), and 𝑔 π‘”π‘˜ β€– β©½ πœ€, then π‘₯βˆ— = π‘₯π‘˜ , and stop. If π‘˜ = 0, go to Step 3. Μƒ π‘˜ by Μƒπ‘˜ βˆ’ 𝑔 Μƒ π‘˜βˆ’1 . Update 𝐡 Step 2. Compute π‘¦π‘˜βˆ’1 = 𝑔 Μƒπ‘˜ = 𝐡 Μƒ π‘˜βˆ’1 βˆ’ 𝐡

On the other hand, by left-multiplying πœ‰1𝑇 on (59) and combining with (63), we have Μƒ βˆ’ π›½Μƒπœ) βˆ’ Μƒπœ) (1 βˆ’ π‘˜3 𝛾) (π‘˜2 (π›½πœ

1 βˆ’ πœ‚ (1 + π‘˜1 𝛼) (1 + π‘˜2 𝛽) . 𝛾

𝑇 Μƒ π‘˜βˆ’1 π‘’π‘˜βˆ’1 𝑒𝑇 𝐡 Μƒ 𝐡 π‘§π‘˜βˆ’1 π‘§π‘˜βˆ’1 π‘˜βˆ’1 π‘˜βˆ’1 + , 𝑇 𝑒 𝑇 𝐡 Μƒ π‘˜βˆ’1 π‘’π‘˜βˆ’1 π‘§π‘˜βˆ’1 π‘’π‘˜βˆ’1 π‘˜βˆ’1

(74)

where (67)

Μƒ π‘˜βˆ’1 π‘’π‘˜βˆ’1 , πœƒ ∈ [0, 1] , π‘§π‘˜βˆ’1 = πœƒπ‘¦π‘˜βˆ’1 + (1 βˆ’ πœƒ) 𝐡 πœƒ

Then, from (66), we have πœ‚Μƒπœ π‘˜2 = , π›ΎΜˆ

(68)

1, { { { 𝑇 Μƒ ={ π΅π‘˜βˆ’1 π‘’π‘˜βˆ’1 0.8π‘’π‘˜βˆ’1 { , { 𝑇 𝑇 Μƒ { π‘’π‘˜βˆ’1 π΅π‘˜βˆ’1 π‘’π‘˜βˆ’1 βˆ’ π‘¦π‘˜βˆ’1 π‘’π‘˜βˆ’1

𝑇 𝑇 Μƒ π΅π‘˜βˆ’1 π‘’π‘˜βˆ’1 , (75) π‘’π‘˜βˆ’1 β©Ύ 0.2π‘’π‘˜βˆ’1 if π‘¦π‘˜βˆ’1

otherwise.

Discrete Dynamics in Nature and Society

7

Μƒ π‘˜ = 𝑄𝑇 π΅π‘˜ 𝑄2 , and Step 3. If π‘˜ β©½ 1, then set π‘ŽΜƒπ‘˜ = Μƒπ‘π‘˜ = Μƒπ‘π‘˜ = 0, 𝐡 2 βˆ’1 Μƒ 𝑔 Μƒ , compute 𝛼 such that Wolfe-Powell conditions π‘‘π‘˜ = βˆ’π΅ π‘˜ π‘˜ π‘˜ are satisfied, and set π‘₯π‘˜+1 = π‘₯π‘˜ +π‘ π‘˜ = π‘₯π‘˜ +π›Όπ‘˜ π‘‘π‘˜ and π‘’π‘˜ = 𝑄2𝑇 π‘ π‘˜ , π‘˜ = π‘˜ + 1, and go to Step 1. Μƒ π‘˜ π‘’π‘˜βˆ’1 get πœ‰1 and πœ‰2 . Μƒ π‘˜ , and 𝐡 Μƒ π‘˜βˆ’1 , 𝑔 Step 4. By the parameters 𝑔

Next we present the global convergence theorem which says the reduced gradients converge to zero. Theorem 8. Assume that (5) and (35) hold, where πœ–1 ∈ (0, 1/3). If 𝑓 is continuously differentiable and bounded below in some set containing all iterations generated by Algorithm 7, the Μƒ π‘˜ β€–} are uniformly bounded. Then, sequences {β€–Μƒ π‘”π‘˜ β€–} and {‖𝐡

Μƒ, 𝛼 Μ‚ , . . . , 𝛾, 𝛾̃ and 𝛾̂ as defined in (62). If Step 5. Compute 𝛼, 𝛼 𝛾 = 0, then set Μƒπ‘π‘˜ = Μƒπ‘π‘˜ = 0. Calculate Μƒ βˆ’π‘“ Μƒ )2 βˆ’ (Μƒ πœŒπ‘˜ = (𝑓 π‘”π‘‡π‘˜βˆ’1 π‘’π‘˜βˆ’1 ) (Μƒ π‘”π‘‡π‘˜ π‘’π‘˜βˆ’1 ) , π‘˜βˆ’1 π‘˜ Μƒ ) + √𝜌 Μƒ βˆ’π‘“ (𝑓 π‘˜ π‘˜ { { π‘˜βˆ’1 , Μ‡ 𝑇 𝛽={ βˆ’Μƒ π‘”π‘˜βˆ’1 π‘’π‘˜βˆ’1 { {1,

if πœŒπ‘˜ β©Ύ 0,

(76)

σ΅„© Μƒ σ΅„©σ΅„© lim inf 󡄩󡄩󡄩𝑔 π‘˜σ΅„© σ΅„© = 0. π‘˜β†’βˆž

Proof. Assume that the theorem is false and there is πœ€ > 0 such Μƒ π‘˜ β©Ύ πœ€ for all π‘˜. From the assumption, we can assume that that 𝑔 σ΅„©σ΅„©Μƒ σ΅„©σ΅„© σ΅„©σ΅„©π‘Žπ‘˜ σ΅„©σ΅„© β©½ π‘Ž, σ΅„©σ΅„©Μƒ σ΅„©σ΅„© σ΅„©σ΅„©π‘π‘˜ σ΅„©σ΅„© β©½ 𝑏, σ΅„© σ΅„© σ΅„©σ΅„©Μƒ σ΅„©σ΅„© σ΅„©σ΅„©π‘π‘˜ σ΅„©σ΅„© β©½ 𝑐, σ΅„©σ΅„© Μƒ σ΅„©σ΅„© σ΅„©σ΅„©π‘”π‘˜ σ΅„©σ΅„© β©½ 𝑔, σ΅„© Μƒ σ΅„©σ΅„© 0 < 󡄩󡄩󡄩󡄩𝐡 σ΅„©σ΅„© β©½ 𝐡, π‘˜σ΅„© σ΅„©σ΅„© βˆ’1 σ΅„©σ΅„© Μƒ Μƒ σ΅„©σ΅„© β©½ 𝐡 0 < 󡄩󡄩󡄩𝐡 σ΅„© π‘˜ σ΅„©σ΅„©

otherwise

and set π‘ŽΜƒπ‘˜ =

1 βˆ’ 𝛽̇ Μƒ π‘‡π‘˜βˆ’1 π‘’π‘˜βˆ’1 𝑔

Μƒ π‘˜βˆ’1 . 𝑔

(77)

Otherwise, compute 𝛾̇ and π›ΎΜˆ as defined in (72) and (69). If 𝛾̇ = 0 or π›ΎΜˆ = 0, then set π‘ŽΜƒπ‘˜ = Μƒπ‘π‘˜ = Μƒπ‘π‘˜ = 0. Otherwise, calculate π‘ŽΜƒπ‘˜ , Μƒπ‘π‘˜ , and Μƒπ‘π‘˜ , where π‘˜1 , π‘˜2 , and π‘˜3 in (61) are determined by (71), (68), and (73). Μƒ π‘˜ , then π‘ŽΜƒπ‘˜ = πœ–1 π‘Žπ‘˜ /Ξ” Μƒ π‘˜ β€–π‘Žπ‘˜ β€–. Update Μƒπ‘π‘˜ and Step 6. If β€–Μƒπ‘Žπ‘˜ β€– > πœ–1 /Ξ” Μƒπ‘π‘˜ with the same way such that (5) are satisfied. Μƒ π‘˜ , solve the Μƒ π‘˜ , and Ξ” Step 7. By the parameters π‘ŽΜƒπ‘˜ , Μƒπ‘π‘˜ , Μƒπ‘π‘˜ , 𝑔 subproblem (18)-(19) by Algorithm 5 to get π‘’π‘˜ . Set π‘ π‘˜ = 𝑄2 π‘’π‘˜ .

(82)

(83)

hold for all π‘˜. From (52) and the assumptions in the theorem, we have pred (π‘ π‘˜ ) β©Ύ

Μƒ πœ‰Ξ” πœ€ Μƒ π‘˜ , 𝑐2 } , min {πœ€, π‘˜ } β©Ύ 𝑐1 min {Ξ” Μƒ 2𝐡 𝐡

(84)

where 𝑐1 and 𝑐2 are some positive constants. Then, from Step 10 of Algorithm 7, we have

Step 8. Compute Ared (π‘’π‘˜ ) πœŒπ‘˜ = , Pred (π‘’π‘˜ )

Μƒ π‘˜ , 𝑐2 } . π‘“π‘˜ βˆ’ π‘“π‘˜+1 β©Ύ πœ„1 pred (π‘ π‘˜ ) β©Ύ πœ„1 𝑐1 min {Ξ” (78)

where Ared (π‘’π‘˜ ) = Ared (π‘ π‘˜ ) = 𝑓 (π‘₯π‘˜ ) βˆ’ 𝑓 (π‘₯π‘˜ + π‘ π‘˜ ) ,

(79)

Μƒ π‘˜ (π‘’π‘˜ ) . Pred (π‘’π‘˜ ) = π‘šπ‘˜ (0) βˆ’ π‘šπ‘˜ (π‘’π‘˜ ) = βˆ’πœ“

(80)

Step 9. Update the trust region radius:

Μƒ π‘˜+1 Ξ”

Μƒ π‘˜, 𝛿1 Ξ” { { { { Μƒ π‘˜ , Ξ” max } , = {min {𝛿2 Ξ” { { {Μƒ {Ξ”π‘˜ ,

(85)

if πœŒπ‘˜ β©½ πœ„1 ,

σ΅„© σ΅„© Μƒ if πœŒπ‘˜ β©Ύ πœ„2 , σ΅„©σ΅„©σ΅„©π‘’π‘˜ σ΅„©σ΅„©σ΅„© = Ξ” π‘˜ , (81)

otherwise.

Step 10. If πœŒπ‘˜ β©Ύ πœ„1 , then π‘₯π‘˜+1 = π‘₯π‘˜ + π‘ π‘˜ . Set π‘˜ = π‘˜ + 1, and go to Step 1. Otherwise, π‘₯π‘˜+1 = π‘₯π‘˜ , π‘˜ = π‘˜ + 1, and go to Step 6.

Since 𝑓(π‘₯) is bounded from below and π‘“π‘˜+1 < π‘“π‘˜ for all π‘˜, Μƒ Μƒ we have that βˆ‘βˆž π‘˜=1 min{Ξ”π‘˜ , 𝑐2 } is convergent, and Ξ”π‘˜ β†’ 0 as π‘˜ β†’ ∞. Μƒ π‘˜ β†’ 0, β€–π‘’π‘˜ β€– β†’ 0. From Step 6 On the other hand, when Ξ” of Algorithm 7, we have 󡄨󡄨 𝑇 󡄨󡄨 σ΅„©σ΅„© σ΅„©σ΅„© Μƒ σ΅„¨σ΅„¨π‘ŽΜƒπ‘˜ π‘’π‘˜ 󡄨󡄨 β©½ σ΅„©σ΅„©π‘ŽΜƒπ‘˜ σ΅„©σ΅„© Ξ”π‘˜ β©½ πœ–1 , 󡄨 󡄨 󡄨󡄨 𝑇 󡄨󡄨 󡄨󡄨̃𝑏 𝑒 󡄨󡄨 β©½ πœ– , 󡄨󡄨 π‘˜ π‘˜ 󡄨󡄨 1 󡄨 󡄨

(86)

where πœ–1 ∈ (0, 1/3). Thus, we have 1 σ΅„© σ΅„© = 1 + π‘ŽΜƒπ‘‡π‘˜ π‘’π‘˜ + π‘œ (σ΅„©σ΅„©σ΅„©π‘’π‘˜ σ΅„©σ΅„©σ΅„©) , 1 βˆ’ π‘ŽΜƒπ‘‡π‘˜ π‘’π‘˜ 𝑇 1 σ΅„© σ΅„© = 1 + Μƒπ‘π‘˜ π‘’π‘˜ + π‘œ (σ΅„©σ΅„©σ΅„©π‘’π‘˜ σ΅„©σ΅„©σ΅„©) . 𝑇 Μƒ 1βˆ’π‘ 𝑒 π‘˜ π‘˜

(87)

8

Discrete Dynamics in Nature and Society and Chen 3.3.2. Moreover, in order to test Algorithm 7 more generally, we designed some problems where the objective functions are Pro. 7–18 (see [14, 18]) and the linear equality constraints are Pro. 1–6. If Μƒπ‘π‘˜ = Μƒπ‘π‘˜ = 0 in Algorithm 7, we can obtain the conic model algorithm and call this algorithm CTR. We solve the following 18 test problems by FTR and CTR and compare their results. All the computations are carried out in Matlab R2012b on a microcomputer in double precision arithmetic. These tests use the same stopping criterion β€–Μƒ π‘”π‘˜ β€– β©½ 10βˆ’4 . The columns in the tables have the following meanings: Pro. denotes the numbers of the test problems; 𝑛 is the dimension of the test problems; Iter is the number of iterations; nf and ng are the numbers of function and gradient evaluations, respectively; β€–Μƒ 𝑔‖ is the Euclidean norm of the final reduced gradient; CPU(s) denotes the total iteration time of the algorithm in seconds. The parameters in these algorithms are

Table 1: Test functions. Pro.

Function name

1

HS 9

2

HS 48

3

HS 49

4

HS 50

5

Chen 3.3.1

6

Chen 3.3.2

7

Cube

8

Penalty-I

9

Beale

10

Conic

11

Extended Powell

12

Broyden Tridiagonal

13

Rosenbrock

14

Chained Freudenstein and Roth

15

Extended Trigonometric

16

Penalty-III

17

Troesch

18

Cragg and Levy

𝐡0 = 𝐼, πœ–1 = 0.33, Ξ” 0 = 1, Ξ” max = 10, 𝑙 = 0.15, πœ‡ = 0.85,

By computing, we obtain Μƒ π‘‡π‘˜ π‘’π‘˜ (1 + Μƒπ‘π‘‡π‘˜ π‘’π‘˜ ) 𝑔

2

Μƒ π‘˜ π‘’π‘˜ (1 + Μƒπ‘π‘‡π‘˜ π‘’π‘˜ ) π‘’π‘˜π‘‡ 𝐡 (1 βˆ’

πœ„1 = 0.25,

σ΅„© σ΅„©2 = π‘”π‘˜π‘‡ π‘’π‘˜ + 𝑂 (σ΅„©σ΅„©σ΅„©π‘’π‘˜ σ΅„©σ΅„©σ΅„© ) ,

𝑇 (1 βˆ’ π‘ŽΜƒπ‘‡π‘˜ π‘’π‘˜ ) (1 βˆ’ Μƒπ‘π‘˜ π‘’π‘˜ )

2 π‘ŽΜƒπ‘‡π‘˜ π‘’π‘˜ )

(91)

2

𝑇 (1 βˆ’ Μƒπ‘π‘˜ π‘’π‘˜ )

=

Μƒ π‘˜ π‘’π‘˜ π‘’π‘˜π‘‡ 𝐡

σ΅„© σ΅„©2 + π‘œ (σ΅„©σ΅„©σ΅„©π‘’π‘˜ σ΅„©σ΅„©σ΅„© ) ,

πœ„2 = 0.75, (88)

Μƒ π‘˜ π‘’π‘˜ + π‘œ (σ΅„©σ΅„©σ΅„©π‘’π‘˜ σ΅„©σ΅„©σ΅„©2 ) . Ared (π‘’π‘˜ ) = βˆ’π‘”π‘˜π‘‡ π‘’π‘˜ βˆ’ π‘’π‘˜π‘‡ 𝐡 σ΅„© σ΅„© Then, from (80), we have σ΅„© Μƒ σ΅„©σ΅„©2 σ΅„© ), Pred (π‘’π‘˜ ) = Ared (π‘’π‘˜ ) + 𝑂 (σ΅„©σ΅„©σ΅„©σ΅„©Ξ” π‘˜σ΅„© σ΅„©

(89)

which indicates that lim πœŒπ‘˜ = 1.

π‘˜β†’βˆž

(90)

Μƒ π‘˜+1 β©Ύ Ξ” Μƒ π‘˜, By the updating in Step 9 of Algorithm 7, we have Ξ” Μƒ π‘˜ β†’ 0. The theorem is proved. which is a contradiction to Ξ”

5. Numerical Tests In this section, Algorithm 7 (abbreviated as FTR) is tested with some test problems which are chosen from [16, 17]. These test problems are listed in Table 1. We choose linear constrained problems HS9, HS48, HS49, HS50, Chen 3.3.1,

𝛿2 = 4. The numerical comparison for 18 small-scale test problems is listed in Table 2. We can see that FTR is better than CTR for 15 tests in the number of iterations and the remaining 3 tests are similar. Because FTR needs some extra algebra computation for some parameters, FTR takes more time than CTR for small problems. The numerical results of some large-scale problems are presented in Table 3. From Table 3, we find that for large-scale problems the CPU time of FTR is approximately the same as that of CTR but it has fewer number of iterations. From the above comparison, we see that FTR is slightly more effective and robust for these large-scale test problems. The fractional model in Algorithm 7 is the extension of conic model. By using more information of function and gradient from the previous iterations and choosing parameters flexibly, the fractional model can be more approximate to the original problem. And the global convergence of the proposed quasi-Newton trust region algorithm is also proved. Numerical experiment shows the algorithm is effective and robust, including for large-scale test problems. The theoretical results and the numerical results lead us to believe that the method is worthy of further study. For example, we can consider using fractional model to solve the nonlinear equality constrained optimization problem.

Discrete Dynamics in Nature and Society

9

Table 2: The numerical results of Algorithm 7 for some test problems. Pro.

𝑛

Starting point

1

6

(0, 0, 0, 0, 0, 0)

2

5

(3, 5, βˆ’3, 2, βˆ’2)

3

5

(2, 2, βˆ’9, 3, 3)

4

10

5

8

6

8

7

6

(3, 4, 3, 4, 3, 4)

8

6

(3, 4, 3, 4, 3, 4)

9

6

(6, 8, 6, 8, 6, 8)

10

6

βˆ’(1.5, 2, 1.5, 2, 1.5, 2)

11

8

(0.3, 0.4, . . ., 0.3, 0.4)

12

6

(0.3, 0.4, . . ., 0.3, 0.4)

13

6

(1.5, 2, 1.5, 2, 1.5, 2)

14

6

(6, 8, 6, 8, 6, 8)

15

8

(0.3, 0.4, . . ., 0.3, 0.4)

16

6

(0.3, 0.4, . . ., 0.3, 0.4)

17

6

(0.3, 0.4, . . ., 0.3, 0.4)

18

8

(0.3, 0.4, . . ., 0.3, 0.4)

(0, βˆ’6, 6, 0, 0, 0, βˆ’6, 6, 0, 0) (0, 3, βˆ’1, βˆ’2 0, 3, βˆ’1, βˆ’2) (7/3, 11/3, βˆ’1, βˆ’1 7/3, 11/3, βˆ’1, βˆ’1)

Algorithm CTR FTR CTR FTR CTR FTR CTR FTR CTR FTR CTR FTR CTR FTR CTR FTR CTR FTR CTR FTR CTR FTR CTR FTR CTR FTR CTR FTR CTR FTR CTR FTR CTR FTR CTR FTR

Iter 6 5 13 8 27 25 12 8 16 16 22 21 8 8 14 12 21 19 13 9 15 7 26 24 6 6 34 22 10 9 13 12 29 19 12 9

nf/ng 7/8 6/6 14/9 9/9 28/21 26/17 13/13 9/9 17/14 17/17 23/19 22/19 9/9 9/9 15/11 13/10 22/18 20/18 14/10 10/7 16/13 8/8 27/16 25/14 7/7 7/7 35/20 23/23 11/9 10/8 14/12 13/12 30/17 20/17 13/9 10/9

‖𝑔‖ 7.559746(βˆ’5) 1.693174(βˆ’5) 1.637781(βˆ’5) 2.148387(βˆ’6) 5.815109(βˆ’5) 4.996278(βˆ’5) 3.889891(βˆ’5) 2.397640(βˆ’5) 7.255223(βˆ’5) 1.249467(βˆ’5) 3.128858(βˆ’6) 2.865070(βˆ’7) 2.208537(βˆ’7) 1.765690(βˆ’8) 8.493218(βˆ’7) 1.155176(βˆ’5) 1.208648(βˆ’6) 3.705893(βˆ’5) 1.789828(βˆ’6) 4.017095(βˆ’6) 3.013621(βˆ’5) 7.252083(βˆ’5) 5.317180(βˆ’5) 5.917188(βˆ’5) 8.707409(βˆ’5) 5.998420(βˆ’5) 3.185767(βˆ’5) 4.845919(βˆ’6) 1.738762(βˆ’5) 5.005195(βˆ’5) 3.409181(βˆ’5) 9.332787(βˆ’5) 1.845528(βˆ’6) 5.983264(βˆ’6) 7.722466(βˆ’7) 4.353866(βˆ’7)

CPU (s) 0.084639 0.161438 0.094287 0.098268 0.110429 0.149000 0.100359 0.152117 0.099120 0.129272 0.101571 0.167233 0.093060 0.139521 0.095873 0.166608 0.094048 0.170549 0.104145 0.158877 0.087370 0.121116 0.1160001 0.187447 0.081667 0.115520 0.106515 0.158603 0.092062 0.146342 0.083445 0.164178 0.088997 0.159851 0.087020 0.149733

Table 3: The numerical results of Algorithm 7 for some test problems (𝑛 = 3000). Pro.

Starting point

7

(6, 8, . . ., 6, 8)

8

(6, 8, . . ., 6, 8)

9

(6, 8, . . ., 6, 8)

10

(6, 8, . . ., 6, 8)

11

(6, 8, . . ., 6, 8)

Algorithm CTR FTR CTR FTR CTR FTR CTR FTR CTR FTR

Iter 25 21 12 12 25 24 12 10 18 4

nf/ng 26/26 22/22 13/12 13/11 26/26 25/25 13/13 11/11 19/19 5/5

‖𝑔‖ 3.737202(βˆ’9) 3.849243(βˆ’9) 8.930053(βˆ’6) 3.113077(βˆ’7) 3.388081(βˆ’6) 1.176732(βˆ’6) 1.226943(βˆ’7) 6.842114(βˆ’6) 2.718913(βˆ’8) 5.738665(βˆ’5)

CPU (s) 28.5976 26.2795 13.8295 14.7506 28.4904 30.7385 14.9229 14.9369 21.4796 8.48251

10

Competing Interests The authors have no competing interests regarding this paper.

Acknowledgments This work was supported by the National Natural Science Foundation of China (Grant no. 11071117 and 71301060), the Natural Science Foundation of Jiangsu Province (BK20141409), Funding of Jiangsu Innovation Program for Graduate Education (KYZZ 0089) (β€œthe Fundamental Research Funds for the Central Universities”), and the Humanistic and Social Science Foundation of Ministry of Education of China (12YJA630122).

References [1] W. C. Davidon, β€œConic approximations and collinear scalings for optimizers,” SIAM Journal on Numerical Analysis, vol. 17, no. 2, pp. 268–281, 1980. [2] R. Schnabel, β€œConic methods for unconstrained minimization and tensor methods for nonlinear equations,” in Mathematical Programming: The State of the Art, A. Bachem, M. GrΒ¨otschel, and B. Korte, Eds., pp. 417–438, Springer, Heidelberg, Germany, 1982. [3] D. C. Sorensen, β€œNewton’s method with a model trust region modification,” SIAM Journal on Numerical Analysis, vol. 19, no. 2, pp. 409–426, 1982. [4] W. Sun and Y. X. Yuan, β€œA conic trust-region method for nonlinearly constrained optimization,” Annals of Operations Research, vol. 103, pp. 175–191, 2001. [5] C. X. Xu and X. Y. Yang, β€œConvergence of conic quasiNewton trust region methods for unconstrained minimization,” Mathematical Application, vol. 11, no. 2, pp. 71–76, 1998. [6] Y. X. Yuan, β€œA review of trust region algorithms for optimization,” in Proceedings of the International Congress on Industrial and Applied Mathematics (ICIAM ’00), vol. 99, pp. 271–282, 2000. [7] D. M. Gay, β€œComputing optimal locally constrained steps,” SIAM Journal on Scientific and Statistical Computing, vol. 2, no. 2, pp. 186–197, 1981. [8] J.-M. Peng and Y.-X. Yuan, β€œOptimality conditions for the minimization of a quadratic with two quadratic constraints,” SIAM Journal on Optimization, vol. 7, no. 3, pp. 579–594, 1997. [9] H. L. Zhu, Q. Ni, and M. L. Zeng, β€œA quasi-Newton trust region method based on a new fractional model,” Numerical Algebra, Control and Optimization, vol. 5, no. 3, pp. 237–249, 2015. [10] W. Y. Sun, J. Y. Yuan, and Y. X. Yuan, β€œConic trust region method for linearly constrained optimization,” Journal of Computational Mathematics, vol. 21, no. 3, pp. 295–304, 2003. [11] X. P. Lu and Q. Ni, β€œA trust region method with new conic model for linearly constrained optimization,” Or Transactions, vol. 12, pp. 32–42, 2008. [12] Q. Ni, Optimization Method and Program Design, Science Press, Beijing, China, 2009. [13] L. W. Zhang and Q. Ni, β€œTrust region algorithm of new conic model for nonlinearly equality constrained optimization,” Journal on Numerical Methods and Computer Applications, vol. 31, no. 4, pp. 279–289, 2010.

Discrete Dynamics in Nature and Society [14] M. F. Zhu, Y. Xue, and F. S. Zhang, β€œA quasi-Newton type trust region method based on the conic model,” Numerical Mathematics, vol. 17, no. 1, pp. 36–47, 1995 (Chinese). [15] X. P. Lu and Q. Ni, β€œA quasi-Newton trust region method with a new conic model for the unconstrained optimization,” Applied Mathematics and Computation, vol. 204, no. 1, pp. 373–384, 2008. [16] W. Hock and K. Schittkowski, Test Examples for Nonlinear Programming Codes, Springer, Berlin, Germany, 1981. [17] X. Y. Chen, Research on the geometric algorithms for programs with constraints of linear equalities [M.S. thesis], Fujian Normal University, 2012. [18] J. J. More, B. S. Garbow, and K. E. Hillstrom, β€œTesting unconstrained optimization software,” ACM Transactions on Mathematical Software, vol. 7, no. 1, pp. 17–41, 1981.

Advances in

Operations Research Hindawi Publishing Corporation http://www.hindawi.com

Volume 2014

Advances in

Decision Sciences Hindawi Publishing Corporation http://www.hindawi.com

Volume 2014

Journal of

Applied Mathematics

Algebra

Hindawi Publishing Corporation http://www.hindawi.com

Hindawi Publishing Corporation http://www.hindawi.com

Volume 2014

Journal of

Probability and Statistics Volume 2014

The Scientific World Journal Hindawi Publishing Corporation http://www.hindawi.com

Hindawi Publishing Corporation http://www.hindawi.com

Volume 2014

International Journal of

Differential Equations Hindawi Publishing Corporation http://www.hindawi.com

Volume 2014

Volume 2014

Submit your manuscripts at http://www.hindawi.com International Journal of

Advances in

Combinatorics Hindawi Publishing Corporation http://www.hindawi.com

Mathematical Physics Hindawi Publishing Corporation http://www.hindawi.com

Volume 2014

Journal of

Complex Analysis Hindawi Publishing Corporation http://www.hindawi.com

Volume 2014

International Journal of Mathematics and Mathematical Sciences

Mathematical Problems in Engineering

Journal of

Mathematics Hindawi Publishing Corporation http://www.hindawi.com

Volume 2014

Hindawi Publishing Corporation http://www.hindawi.com

Volume 2014

Volume 2014

Hindawi Publishing Corporation http://www.hindawi.com

Volume 2014

Discrete Mathematics

Journal of

Volume 2014

Hindawi Publishing Corporation http://www.hindawi.com

Discrete Dynamics in Nature and Society

Journal of

Function Spaces Hindawi Publishing Corporation http://www.hindawi.com

Abstract and Applied Analysis

Volume 2014

Hindawi Publishing Corporation http://www.hindawi.com

Volume 2014

Hindawi Publishing Corporation http://www.hindawi.com

Volume 2014

International Journal of

Journal of

Stochastic Analysis

Optimization

Hindawi Publishing Corporation http://www.hindawi.com

Hindawi Publishing Corporation http://www.hindawi.com

Volume 2014

Volume 2014