A Variable Metric Algorithm with Broyden Rank One Modifications for ...

Open Journal of Optimization, 2013, 2, 33-37 http://dx.doi.org/10.4236/ojop.2013.21005 Published Online March 2013 (http://www.scirp.org/journal/ojop)

A Variable Metric Algorithm with Broyden Rank One Modifications for Nonlinear Equality Constraints Optimization Chunyan Hu1, Zhibin Zhu2 1

School of Electronic Engineering and Automation, Guilin University of Electronic Technology, Guilin, China 2 School of Mathematics and Computing Science, Guilin University of Electronic Technology, Guilin, China Email: [email protected] Received January 17, 2013; revised February 18, 2013; accepted March 8, 2013

ABSTRACT In this paper, a variable metric algorithm is proposed with Broyden rank one modifications for the equality constrained optimization. This method is viewed expansion in constrained optimization as the quasi-Newton method to unconstrained optimization. The theoretical analysis shows that local convergence can be induced under some suitable conditions. In the end, it is established an equivalent condition of superlinear convergence. Keywords: Equality Constrained Optimization; Variable Metric Algorithm; Broyden Rank One Modification; Superlinear Convergence

1. Introduction & Algorithm This In this paper, it is proposed to consider the following nonlinear mathematical programming problem: min f  x  s.t. g j  x   0, j  I  1, , m ,

(1)

where f : R  R, g j  j  I  : R  R are continuously differentiable functions. Denote the feasible set as follows: n

n





X  x  R n g j  x   0, j  I . m

L  x,    f  x     j g j  x  . j 1

If  x ,   is a KKT point pair of Equation (1), then Lx,  x  x* ,   *  0, i.e.,

g  x   0,

(2)

where g  x    g  x  , j  I  , g  x    g  x  , j  I  . At the point pair  xk , k  , the Newton’s iteration of (2) is defined as follows:   2xx L  xk , k  g  xk    xk     g  x  T    0 k   k   f  xk   g  xk  k      g  xk    Copyright © 2013 SciRes.

of variable metric methods, such as sequential quadratic programming (SQP) methods [1-5], sequential systems of linear equations (SSLE) algorithms [6-8]. In general, the computational cost of those methods is large. In this paper, a new variable metric method is presented, in which the following fact is based on: a positive definite matrix Bk is replaced for the matrix   2xx L  xk , k  g  xk    .  g  x  T  0 k  

Let L  x,   be Lagrangian function of (1), and

f  x   g  x    0,

Later, a positive definite matrix H k1 is replaced for  2xx L  xk , k  by a lot of authors to develop some kinds

In the sequel, we describe the algorithm for the solution of (1). Denote   2xx L  xk , k  g  xk   x   x , z    , zk   k  , Bk    g  x  T  0    k  k    f  xk   g  xk  k   xk  v  zk      , pk  zk 1  zk   , g  xk   k    qk  v  zk 1   v  zk  , F  z   f  x   g  x    g  x  . 2

(3)

2

F  z  , and from It is obvious that min f  x   min nm x X

zR

Equation (3), we have Bk pk  v  zk  .

(4)

OJOp

C. Y. HU, Z. B. ZHU

34

To the system of linear Equations (4), like unconstrained optimization, Bk is dealt with using Broyden rank one modifications as follows: Bk 1  Bk 

 qk  Bk pk  pkT pkT pk

is invertible for some x  D , then there exist some   0,     0 , such that for all u , v  D , the fact  u  x , v  x    implies that

 u  v  F u   F v   u  v .

(5)

.

Now, the algorithm for the solution of Equation (1) can be stated as follows: Algorithm A: Step 1: Initialization: Given a starting point z0  R n  m (i.e., x0  R n , 0  R m ), and a initial positive definite n m  n m matrix B0  R     .   0, k  0; Step 2: Compute v  zk  . If v  zk    , stop; Step 3: Compute d k   Bk1v  zk  ; Step 4: Let zk 1  zk  d k , and obtain Bk 1 according to (5). Set k  k  1 . Go back to step 2.

Lemma 3 [9] For operator F : R n  m  R n  m , which satisfies: R1 F is continuously differentiable on D ; R2 There exists a point z  D , such that F  z   0 , and F   z  is reversible; R3 F  is Lipschitz continuous at z , i.e., there exists a constant  , such that F   z   F   z    z  z* , z  D. Let zk 1  zk  H k1 F  zk  , where H k1  R  and it holds that H k 1  F   z   H k  F   z  

2. Convergence of Algorithm x  If the algorithm stops at zk   k  , then xk is a KKT  k  point of (1). In the sequel, we suppose that algorithm generates an infinite sequence  zk  .

Four basic assumptions are given as follows: A1 The feasible set X is nonempty; The functions f , g j  j  I  are two-times continuously differentiable; A2 For all x  R , the vectors n

g  x  , j  I  j

KKT point pair

 x ,   ,

such that  2xx L  x ,   is

positive definite; A4 There exists a ball N  x ,   of radius   0 about

x , where

 2 f  x  ,  2 g j  x  , g j  x  j  I 

satisfy the Lipschitz condition on N  x ,   . Lemma 1 [9] Let F : R n  m  R n  m be continuously differentiable in some open and convex set D , and F  is Lipschitz continuous in D , then x  d  D , it holds that F  x  d   F  x  F  x d 

 2

2

d ,

where  is the Lipschitz constant. Moreover, u , , x  D , it follows that F u   F v  F u  v   v

ux  vx 2

u v .

Lemma 2 [9] Let F : R n  m  R n  m be continuously differentiable in some open and convex set D , and F  is Lipschitz continuous in D . Moreover, assume F   x  Copyright © 2013 SciRes.

 zk 1  z 2

 zk  z  , k ,

,

(6)

then there exist   0 and   0 , when z0  z   , H 0  F   z    ,

it is true that zk 1 is meaning, and zk is linearly convergent to z . Thereby, we can conclude that zk is superlinearly convergent to z , if and only if lim

k 

are

linearly independent; A3  xk  and k  are bounded. There exists a



n  m  n  m 

 H k  F   z    zk 1  zk  zk 1  zk

 0.

(7)

In the sequel, we prove the convergence Theorem as follows: Theorem 1 If there exist constants   0 and   0, such that z0  z   , B0  v  z    , then

 zk  is

meaning, and zk is linearly convergent to

z , thereby xk and k are linearly convergent to x and  respectively. Proof: From Lemma 3, we only prove that v and Bk satisfy conditions R1, R2, R3 and the inequality (6). From assumption A1, it is obvious that v is continuously differentiable. From A3, it holds that v  z   v  x ,    0 , and   2 f  x    2 g  x  * v  z     T  g  x  

g  x   . 0 

While  2xx L  x ,     2 f  x    2 g  x  * is positive

definite, and g j  x  , j  I  are linearly independent. So, it is easy to see that v  z  is reversible. In addition,   2 f  xk    2 g  xk  k v  zk     T  g  xk  

g  xk   . 0  OJOp

C. Y. HU, Z. B. ZHU

From assumption A4, it holds that v is Lipschitz continuous at z . In a word, v satisfies the conditions R1, R2, R3. Now, we prove that, for v  z  and Bk generated according to (5), (6) is satisfied. In fact, from (5), we have Bk 1  v  z   Bk  v  z    Bk  v  z   

35

satisfy (7). Denote Dk  Bk  v  z  ,

Dk 1

F

 p pT   Dk  I  kT k  pk pk  

pkT pk

 qk  v  z  pk  p pkT pk

2

Dk

 p p   qk  v  z  pk  p .  Bk  v  z   I  kT   pk pk  pkT pk 

p p 

F

T k



2

Dk pk



qk  v  z  pk pk p  . T pk pk pk T k

pk

2

2

F

2

,

2

 p pT   Dk  I  kT k  pk pk  

2

F

2

 p pT   Dk  I  kT k  . pk pk  F 

 p pT  Dk  I  kT k  pk pk  

While pk pkT pkT pk   1, pkT pk pkT pk

 Dk

and from Lemma 1, we have qk  v  z  pk  v  zk 1   v  zk   v  z   zk 1  zk  2

. (11)

So,

 Bk  v  z  I 



F

   

Dk pk



2



T k

pk





k

p pT  tr D Dk  Dk kT k pk pk

So,

I

F



F

T k

Bk 1  v  z 

F

pkT pk

 Dk pk   Dk pk  pkT pk



T k

T k

 qk  v  z  pk  pkT

T

(8)

p pk



 p pT DT p pT  tr  k k k k2 k  pkT pk 

2

p pT Dk kT k pk pk

T k



Since

 qk  Bk pk  pkT

 v  z  pk  Bk pk  pkT

 l2 norm,

Frobenius norm. From (8),

 zk 1  z

 zk  z



(9)

zk 1  zk .



F



  zk 1  z*  zk  z*  , 2 i.e., (6) is true, thereby, from Lemma 3, we get

Dk pk pk

The claim holds. Theorem 2 Under above-mentioned assumptions, if  ,  in Theorem 1 satisfy that  1, 3  2 ,

i



k 0

2

(12)

2

  . 



 x   x  then  k   is superlinearly convergent to  k  , i.e.,  k   k    . 

Proof. From Lemma 3, we only prove that v and Bk

F

 2 ,  zk  z*  2 , k .

(14)

k 0

2

 2 Dk

2

2

Dk pk pk

2

F

  Dk 

  4  D0 

F

F

 Dk 1

 Di 1

F

F

3    zk  z  , 4 

3 i     zk  z  4 k 0 

 4 2  6 , i,

(10) i

Copyright © 2013 SciRes.

pk

2   

Thus, from (9), (12), (13), we have

zk  z , i.e., xk  x , k   .

 x  x  o k k 1    k  

F



2



Bk 1  v  z   Bk  v  z  

xk 1  x

 Dk pk   pk

F

Dk pk

thereby,

So,

1

F

1

2

In addition, from (6), (10), using the method of mathematical induction, it is not difficult to prove that 1 Bk  v   z   2  2 k  , zk 1  z*  zk  z* , k , (13) 2

Dk

6  v  z  

1 2 Dk

   Dk  

i.e.



k 0

Dk pk pk

2

2

is convergent, thereby, lim

k 

From Theorem 2, we don’t conclude that

Dk pk pk

2

 xk 

2

 0.

is su-

perlinearly convergent to x , i.e.,

xk 1  x    xk  x  . In the sequel, we discuss one

condition which assures that xk is superlinearly converOJOp

C. Y. HU, Z. B. ZHU

36

gent to x . Lemma 4 Let R : R n  R n be a function defined by R  x   A  x   f  x   g  x     g  x  g  x  ,



where A  x   I n  g  x  g  x 1 g  x  Then,



1

From (4), we have

T

So, the fact A  xk  g  xk   0 implies that

R  xk 1     xk 1  xk  .

 A  xk 

Proof. Obviously, using the triangle inequality, it holds that



xk 1  x*    xk 1  x*  .

In addition, from A1, we know that R  x  is continuously differentiable, and R  x   0,

(15)

R  x   A  x   2xx L  x ,    g  x  g  x  . T

It holds that R  x  is nonsingular. In fact, let d  0 , and

 A x   

2 xx

L  x ,    g  x  g  x 

 g  xk  k     v  xk ,    Bk pk   , 0  

g  x  .

xk 1  x*    xk  x*  , if and only if

xk 1  x*    xk  x*

Proof. The sufficiency: Suppose that (16) holds. We only prove that R  xk 1     xk 1  xk  .

T

 d  0,

0nm  Bk pk   A  xk  0nm  v  xk ,    0,

thereby,  R  xk 1    A  xk  0nm  Bk pk   R  xk 1   R  xk    g  x  g  xk  . From (15), we have R  x   xk 1  xk 



 A  x   2xx L  x ,    g  x  g  x 



 g  x  g  xk   g  x 

While

A  x  d  d , d T  2xx L  x ,   d  0,

 A x  k



 A  x   2 f  x    2 g  x  *

 xk 1  x  R  xk 1    xk 1  x* . R  xk 1     xk 1  xk   xk 1  x    xk 1  xk

.

 xk 



  2 f  x    2 g  x  *

k 1

 xk 



g  xk   g  x 

T

 xk 1  xk    

   xk 1  xk  .

Copyright © 2013 SciRes.

k 1

(16)



 x

k 1

 xk  .

R  xk 1   Rk  R  x   xk 1  xk     xk 1  xk  ,

R  xk 1     xk 1  xk  .

The necessity: Suppose that

xk 1  xk  , i.e.,

 xk 

 xk 

In addition, according to Lemma 1, we have

i.e.,

 A  xk  0nm     v  zk   vx  z   xk 1  xk    0mn I mm 

 x



so, from (16), it holds that

   xk 1  xk  ,

and

k 1

  A  xk   A  x    2 f  x    2 g  x  *

is superlinearly convergent to x* if and only if

 x

 x

  A  xk   f  xk   g  xk  k

So,

The claim holds. Theorem 3 Under above-mentioned conditions,

0nm  v  zk    A  xk  0nm  v  zk  pk

  A  xk   f  xk   g  xk  k 

which is a contradiction. Thereby, from A4 and Lemma 2, there exist     0 , such that



 xk 1  xk  

  A  xk  0nm  v  zk  pk .

So,

  f  x    g  x  

T

  A  xk  0nm  v  zk 

A  x   L  x ,   d  0, g  x  d  0.

2

 xk 1  xk 

 R  xk 1    R  xk 1   Rk  R  x   xk 1  xk  

T

2

T

 xk 

So,

that

A  x   f  xk   g  xk  k

k 1

   A  xk  0nm  v  z  pk  g  x  g  x 

the facts that g  x  has full rank, A  x  is semi-positive definite and  2xx L  x ,   is positive definite imply 2 xx

 x

T

xk 1  x    xk  x



xk 1  x    xk 1  xk  .

On one hand,

g  xk   g  x 

 xk 1  xk  T  g  xk   g  xk   xk 1  xk 



T

 g  x   g  xk  T

T

 x

k 1

 xk  . OJOp

C. Y. HU, Z. B. ZHU

3. Acknowledgements

So, in order to prove that g  xk   g  xk 

 xk 1  xk    

T

xk 1  xk  .

This work was supported in part by NNSF (No. 11061 011) of China and Guangxi Fund for Distinguished Young Scholars (2012GXSFFA060003).

While g  xk   g  xk 

 xk 1  xk 

T



  g  xk 1   g  xk   g  xk 

T

 xk 1  xk  

REFERENCES

  g  xk 1   g  x   .

[1]

M. J. D. Powell, “A Fast Algorithm for Nonlinearly Constrained Optimization Calculations,” In: G. A. Watson, Ed., Numerical Analysis, Springer-Verlag, Berlin, 1978, pp. 144-157. doi:10.1007/BFb0067703

[2]

S.-P. Han, “Superlinearly Convergent Variable Metric Algorithm for General Nolinear Programming Problem,” Mathematical Programming, Vol. 11, No. 1, 1976, pp. 263-282. doi:10.1007/BF01580395

[3]

D. Q. Mayne and E. Polak, “A Superlinearly Convergent Algorithm for Constrained Optimization Problems,” Mathematical Programming Studies, Vol. 16, 1982, pp. 4561. doi:10.1007/BFb0120947

[4]

J.-B. Jian, C.-M. Tang, Q.-J. Hu and H.-Y. Zheng, “A Feasible Descent SQP Algorithm for General Constrained Optimization without Strict Complementarity,” Journal of Computational and Applied Mathematics, Vol. 180, No. 2, 2005, pp. 391-412. doi:10.1016/j.cam.2004.11.008

[5]

Z. Jin and Y. Q. Wang, “A Simple Feasible SQP Method for Inequality Constrained Optimization with Global and Superlinear Convergence,” Journal of Computational and Applied Mathematics, Vol. 233, No. 1, 2010, pp. 30603073. doi:10.1016/j.cam.2009.11.061

[6]

Y.-F. Yang, D.-H. Li and L. Q. Qi, “A Feasible Sequential Linear Equation Method for Inequality Constrained Optimization,” SIAM Journal on Optimization, Vol. 13, No. 4, 2003, pp. 1222-1244. doi:10.1137/S1052623401383881

[7]

Z. B. Zhu, “An Interior Point Type QP-Free Algorithm with Superlinear Convergence for Inequality Constrained Optimization,” Applied Mathematical Modelling, Vol. 31, No. 6, 2007, pp. 1201-1212. doi:10.1016/j.apm.2006.04.019

[8]

J. B. Jian, D. L. Han and Q. J. Xu, “A New Sequential Systems of Linear Equations Algorithm of Feasible Descent for Inequality Constrained Optimization,” Acta Mathematica Sinica, English Series, Vol. 26, No. 12, 2010, pp. 2399-2420. doi:10.1007/s10114-010-7432-0

[9]

Y. Yuan and W. Sun, “Theory and Methods of Optimization,” Science Press, Beijing, 1997.

According to Lemma 1, we have g  xk 1   g  xk   g  xk 

 xk 1  xk 

T

   xk 1  xk  ,

 g  x   g  x  k 1

and

k

 k1 xk 1  x*    xk 1  xk

So,

  k1  0  . T g  xk   g  xk   xk 1  xk     xk 1  xk  .

On the other hand, the fact A  xk  g  xk   0 implies that A  xk   f  xk   g  xk  k



  2 f  x    2 g  x  

 x

k 1

 xk 

 A  xk   f  xk   g  xk  



  2 f  x    2 g  x  

 x

k 1



f  xk   g  xk  k   f  x    g  x  *  h  xk   h  x 



T

2

2

 xk 1  xk 

  h  xk 1   h  xk   h  x 



T



 xk  .

Let h  x   g  x   g  x   , then



 x

k 1

 xk 

 xk 1  xk  



 h  xk 1   h  x  .

It is easy to see that h  xk   h  x 

T

 xk 1  xk    

xk 1  xk  .

Thereby, A  xk   f  xk   g  xk  k  f  xk   g  xk  k



  2 f  x    2 g  x  * The claim holds.

Copyright © 2013 SciRes.

37

 x

k 1



 xk     xk 1  xk  .

OJOp