A Merit Function Piecewise SQP Algorithm for Solving Mathematical ...

A Merit Function Piecewise SQP Algorithm for Solving Mathematical Programs with Equilibrium Constraints∗ G.S. Liu† and J.J. Ye‡

Abstract. In this paper we propose a merit function piecewise SQP algorithm for solving mathematical programs with equilibrium constraints (MPECs) formulated as mathematical programs with complementarity constraints. Under some mild conditions, the new algorithm is globally convergent to a piecewise stationary point. Moreover if the partial MPECLICQ is satisfied at the accumulation point then the accumulation point is a S-stationary point. Keywords: mathematical program with equilibrium constraints, mathematical program with complementarity constraints, partial MPEC-LICQ, global convergence, piecewise stationary point.

∗

The authors would like to thank the two anonymous referees for their careful reading of the

paper and the helpful suggestions. The research of the first author was supported by the National Natural Science Foundation of China under its grand # 10571177 and # 70271014. The research of the second author was partially supported by NSERC. † Professor, School of Business, Renmin University of China, Beijing 100872, China, email: [email protected]. ‡ Professor, Department of Mathematics and Statistics, University of Victoria, Victoria, B.C., Canada V8W 3P4, e-mail: [email protected].

1

1

Introduction.

In this paper we consider a mathematical program with equilibrium constraints (MPEC) in the form of: (MPEC)

min s.t.

f (x) g(x) ≤ 0, h(x) = 0, G(x) ≥ 0,

H(x) ≥ 0,

G(x)> H(x) = 0

where f : Rn → R, g : Rn → Rp , h : Rn → Rq , G : Rn → Rm , H : Rn → Rm are continuously differentiable and a> denotes the transpose of a vector a. An MPEC formulated as above is actually a mathematical program with complementarity constraints and hence is a special but most important case of an optimization program with variational inequality constraints. MPECs are closely related to bilevel programming problems and also called generalized bilevel programming problems. MPECs arise naturally in many areas such as economics, computational mechanics, neural networks training and traffic control. We refer the readers to Refs. 1, 2 for a survey about recent developments of MPECs. For MPECs, it is well-known that the usual nonlinear programming constraint qualification such as the Mangasarian-Fromovitz constraint qualification (MFCQ) does not hold (see Proposition 1.1 in Ref. 3) and hence Lagrange multipliers in the sense of nonlinear programming may not exist and the usual numerical algorithms may not be useful. Based on several different approaches to reformulate MPECs, various concepts of stationary points and multipliers arise in recent years (see e.g. Refs. 4, 5). Compared with the development in theory, the development of algorithms for MPECs has been slower. Up to now most algorithms for MPECs either do not ensure global convergence, or guarantee convergence to a stationary point only under some strong assumptions such as the MPEC-LICQ (see Definition (2.5)), or the nondegeneracy condition (in the sense that the index set β defined in the beginning of §2 is empty), which is also called lower-level strict complementarity condition. It is well-known that in general an MPEC is a nonsmooth and nonconvex problem. The nondegeneracy (lower-level strict complementarity) condition at a feasible point of an MPEC implies that the problem is locally smooth while the MPEC-LICQ at a locally optimal solution of an MPEC implies that the locally optimal solution of 2

the MPEC is also a locally optimal solution of the objective function restricted to a convex relaxed feasible region. Hence both the nondegeneracy condition and the MPEC-LICQ are rather restrictive in practice. This fact motivates us to design some algorithms for solving general MPEC which are globally convergent under mild conditions. Recently for the class of MPECs with linear complementarity constraints, using a new technique called δ-active search, the PSQP algorithm, which is proposed in Ref. 6 and extended in Refs. 1, 7, 8 has been adjusted to provide a global convergence without the MPEC-LICQ in Ref. 9. In this paper by using a partial exact penalty function as a merit function we extend the result in Ref. 9 to general MPECs. The resulting algorithm is globally convergent to a piecewise stationary point under mild assumptions. Moreover if the partial MPEC-LICQ (see Definition (2.6)) is satisfied at the accumulation point of an iterative sequence, then the algorithm will globally converge to a S-stationary point. Compared with most existing globally convergent algorithms for solving general MPECs (see e.g. Refs. 10-19), the merit function PSQP algorithm has the advantage that it uses quadratic programs instead of general nonlinear programs as subproblems and no MPEC-LICQ is required. The following notations are used throughout the paper. For a vector d ∈ Rn , di or (d)i indicate the ith component of d. kdk1 and kdk∞ indicate the 1-norm and ∞-norm of d respectively.

2

Stationarity conditions

Given a feasible vector x¯ of MPEC, we define the following index sets: Ig := {i : gi (¯ x) = 0} α := α(¯ x) := {i : Gi (¯ x) = 0, Hi (¯ x) > 0} β := β(¯ x) := {i : Gi (¯ x) = 0, Hi (¯ x) = 0} γ := γ(¯ x) := {i : Gi (¯ x) > 0, Hi (¯ x) = 0}.

3

Let x¯ be a local solution of an MPEC. Then for each partition of the index set β into sets P, Q, x¯ must be also locally optimal for the associated branch of subproblem: (M P EC)(P,Q)

min s.t.

f (x) g(x) ≤ 0, h(x) = 0, Hi (x) = 0, i ∈ γ,

Gi (x) = 0, i ∈ α,

Gi (x) = 0,

Hi (x) ≥ 0, i ∈ P,

Gi (x) ≥ 0,

Hi (x) = 0, i ∈ Q.

Definition 2.1 (piecewise MFCQ) We say that the piecewise Mangasarian Fromovitz constraint qualification (MFCQ) holds at a feasible solution x¯ of MPEC if the MFCQ holds for every branch (M P EC)(P,Q) associated with any partition P, Q. It is well-known that for a nonlinear programming problem, the MFCQ is equivalent to the no nonzero abnormal multiplier constraint qualification (NNAMCQ) and hence the piecewise MFCQ is equivalent to the piecewise NNAMCQ defined below. Definition 2.2 (piecewise NNAMCQ) We say that the piecewise NNAMCQ holds at x¯, a feasible solution of MPEC, if for each partition P, Q of β there exists no nonzero vector (η g , η h , η G , η H ) ∈ Rp+q+2m such that 0 = ∇g(¯ x)> η g + ∇h(¯ x)> η h − ∇G(¯ x)> η G − ∇H(¯ x)> η H , η g ≥ 0,

g(¯ x)> η g = 0,

ηiG = 0 ∀i ∈ γ,

ηiH = 0 ∀i ∈ α,

ηiH ≥ 0 ∀i ∈ P and ηiG ≥ 0 ∀i ∈ Q. Under the piecewise MFCQ or equivalently the piecewise NNAMCQ, it is clear that a locally optimal solution of an MPEC is also a piecewise stationary point (first introduced in Ref. 1) defined as follows. Note that a piecewise stationary point is equivalent to a Bouligrand stationary (B-stationary) point in the sense of Ref. 4 and is equivalent to a B-stationary point in the classical sense of Ref. 1 if a constraint qualification for each branch (M P EC)(P,Q) holds. Definition 2.3 (piecewise stationary point) We call a feasible solution x¯ of MPEC a piecewise stationary point for MPEC if for each partition of the index set β into 4

sets P, Q, there exists (η g , η h , η G , η H ) ∈ Rp+q+2m such that 0 = ∇f (¯ x) + ∇g(¯ x)> η g + ∇h(¯ x)> η h − ∇G(¯ x)> η G − ∇H(¯ x)> η H , η g ≥ 0,

g(¯ x)> η g = 0,

ηiG = 0 ∀i ∈ γ,

(1) (2)

ηiH = 0 ∀i ∈ α,

(3)

ηiH ≥ 0 ∀i ∈ P and ηiG ≥ 0 ∀i ∈ Q. We now state three commonly used stationarity concepts of decreasing strength (see e.g. Refs. 4, 5 for more details). Definition 2.4 (S-stationary, M-stationary and C-stationary points) A feasible point x¯ of MPEC is called strongly stationary, Mordukhovich stationary and Clarke Stationary (S-stationary, M-stationary and C-stationary) respectively if there exists η = (η g , η h , η G , η H ) ∈ Rp+q+2m such that (1)-(3) and the following conditions hold respectively: ∀i ∈ β, ηiG ≥ 0 ηiH ≥ 0,

(4)

∀i ∈ β, either ηiG > 0, ηiH > 0 or ηiG ηiH = 0, ∀i ∈ β, ηiG ηiH ≥ 0. It is well-known that under the following MPEC-LICQ, a locally optimal solution x¯ of an MPEC is a S-stationary point. It is clear that the MPEC-LICQ is a much stronger condition than the piecewise MFCQ. Definition 2.5 (MPEC-LICQ) Let x¯ be a feasible point of MPEC. We say that the MPEC-LICQ is satisfied at x¯ if ∇gi (¯ x) ∀i ∈ Ig ∇hi (¯ x) ∀i = 1, 2, . . . , q ∇Gi (¯ x) ∀i ∈ α ∪ β ∇Hi (¯ x) ∀i ∈ γ ∪ β are linearly independent.

5

Definition 2.6 (partial MPEC-LICQ) Let x¯ be a feasible point of MPEC. The partial MPEC-LICQ holds at x¯ if for any vectors (η g , η h , η G , η H ) ∈ Rp+q+2m , 0 = ∇g(¯ x)> η g + ∇h(¯ x)> η h − ∇G(¯ x)> η G − ∇H(¯ x)> η H , ηiG = 0 ∀i ∈ γ, ηiH = 0 ∀i ∈ α implies that ηiG = 0 ηiH = 0 ∀i ∈ β. The concept of the partial MPEC-LICQ was introduced in Theorem 3.2 of Ref. 20. It is obvious that the MPEC-LICQ implies the partial MPEC-LICQ. Unlike the MPECLICQ under which there exists a unique multiplier (η g , η h , η G , η H ), under the partial MPEC-LICQ only the β component of (η G , η H ) is unique but there may exist multimultipliers. An example in which the partial MPEC-LICQ holds but the MPECLICQ does not hold is illustrated in Example 3.1 of Ref. 20. It is known that under certain piecewise MPEC constraint qualifications, a local solution of an MPEC is a piecewise stationary point and under MPEC-LICQ, a local solution of an MPEC is a S-stationary point. For detailed discussion about various MPEC stationarity concepts and MPEC constraint qualifications, the reader is referred to Ref. 5. In the case when the partial MPEC-LICQ is satisfied at a feasible solution x¯, the following proposition states that a piecewise stationary point must be a S-stationary point. We omit the proof of the proposition since the proof is easy and is similar to the proof of Theorem 3.2 of Ref. 20. Proposition 2.1 Let x¯ be a feasible point of MPEC and suppose that the partial MPEC-LICQ holds at x¯. Then x¯ is S-stationary if and only if it is piecewise stationary.

3

Merit function PSQP methods

In order to introduce the merit function PSQP method, we reformulate MPEC in the following equivalent form : (FMPEC)

min

f (x)

s.t.

g(x) ≤ 0, h(x) = 0, 6

G(x) − y = 0, H(x) − z = 0, (x, y, z) ∈ WF, where WF is the weak feasible region defined by WF = {(x, y, z) : y ≥ 0, z ≥ 0, y > z = 0}. We call a point (¯ x, y¯, z¯) ∈ WF a weak feasible solution, and we call a feasible solution (¯ x, y¯, z¯) of FMPEC a piecewise stationary point of FMPEC if x¯ is a piecewise stationary point of the corresponding MPEC with y¯ = G(¯ x) and z¯ = H(¯ x). Obviously the weak feasible region is the union of weak faces [

WF =

WF (A,B) ,

(A,B)∈P

where WF (A,B) =

 

yi = 0, zi ≥ 0

∀i ∈ A, 



yi ≥ 0, zi = 0

∀i ∈ B 

(x, y, z) :



and P is the set of all pairs (A, B) such that A and B are subsets of I := {1, 2, . . . , m}, A ∩ B = ∅ and A ∪ B = {1, 2, · · · , m}. Definition 3.1 Let (¯ x, y¯, z¯) be any point in the weak feasible region. We call a pair (A, B) ∈ P the adjacent pair at (¯ x, y¯, z¯) if A ⊆ {i : y¯i = 0}, and B ⊆ {i : z¯i = 0} and denote by P(¯x,¯y,¯z) the set of all adjacent pairs at (¯ x, y¯, z¯). Given an iteration point (¯ x, y¯, z¯) ∈ WF and a pair (A, B) ∈ P(¯x,¯y,¯z) , consider the following quadratic subproblem: QP(A,B) (¯ x, y¯, z¯)

min

dx ,dy ,dz

s.t.

1 ∇f (¯ x)> dx + d> Dd 2 ∇G(¯ x)dx + G(¯ x) − dy − y¯ = 0,

(5)

∇H(¯ x)dx + H(¯ x) − dz − z¯ = 0,

(6)

(dy )i + y¯i = 0,

(dz )i + z¯i ≥ 0 i ∈ A,

(dy )i + y¯i ≥ 0,

(dz )i + z¯i = 0 i ∈ B,

∇g(¯ x)dx + g(¯ x) ≤ 0,

(7)

∇h(¯ x)dx + h(¯ x) = 0,

(8)

where d := (dx , dy , dz ) ∈ Rn+2m and D is a positive definite matrix. 7

Remark 3.1 The above quadratic subproblem may not be feasible, i.e., the feasible region of the above problem may be empty. This is a common issue to all SQP methods. In order to focus on the main goal of the paper we assume that all quadratic subproblems QP(A,B) (¯ x, y¯, z¯) are feasible for any weak feasible solution (¯ x, y¯, z¯) ∈ WF and any feasible pair (A, B) ∈ P(¯x,¯y,¯z) throughout this paper. Since the objective function is strictly convex and the constraints are linear, d is a solution to problem QP(A,B) (¯ x, y¯, z¯) if and only if there exists a Lagrange multiplier (η g , η h , η G , η H , η y , η z ) ∈ Rp+q+2m+2m such that 

∇f (¯ x) + ∇g(¯ x)> η g + ∇h(¯ x)> η h − ∇G(¯ x)> η G − ∇H(¯ x)> η H



   

ηG − ηy

   

ηH − ηz   

dx

  

+D  dy  = 0,   dz η g ≥ 0, [∇g(¯ x)dx + g(¯ x)]> η g = 0, ηiy ≥ 0, ηiy ((dy )i + y¯i ) = 0, i ∈ B, ηiz ≥ 0, ηiz ((dz )i + z¯i ) = 0, i ∈ A. It is easy to see that (¯ x, y¯, z¯) is a piecewise stationary point if and only if for all adjacent pair (A, B), the solution to the quadratic problem QP(A,B) (¯ x, y¯, z¯) is zero. Hence if (¯ x, y¯, z¯) is not a piecewise stationary point then there exists an adjacent pair (A, B) such that the solution to problem QP(A,B) (¯ x, y¯, z¯) is nonzero. In the following result we show that any nonzero solution d to the quadratic problem QP(A,B) (¯ x, y¯, z¯) provides a feasible descent direction to the weak face WF (A,B) for the partial exact penalty function φµ (x, y, z) := f (x) + µ(kG(x) − yk1 + kH(x) − zk1 + k max{0, g(x)}k1 + kh(x)k1 ) at (¯ x, y¯, z¯) for an appropriate choice of µ, i.e., there exists α ¯ ∈ (0, 1) such that for all α ∈ (0, α ¯) φµ (¯ x, y¯, z¯) > φµ ((¯ x, y¯, z¯) + αd) and (¯ x, y¯, z¯) + αd ∈ WF (A,B) . Hence if a solution d to QP(A,B) (¯ x, y¯, z¯) is nonzero then we say that d is a descent direction and (A, B) is a descent pair. 8

Proposition 3.1 Given an (¯ x, y¯, z¯) ∈ WF and a pair (A, B) ∈ P(¯x,¯y,¯z) , suppose the quadratic problem QP(A,B) (¯ x, y¯, z¯) has a solution d 6= 0 and η G , η H , η g , η h are some Lagrange multipliers associated with constraints (5),(6), (7) and (8) respectively. Then d is a descent direction to the weak face WF (A,B) at (¯ x, y¯, z¯) for the penalty function φµ (x, y, z) at (¯ x, y¯, z¯) for all µ ≥ k(η G , η H , η g , η h ))k∞ . Proof. For simplicity in the proof we omit the constraints (7) and (8). Since d is a solution and the constraints are all linear, by KKT condition there exists η = (η y , η z , η G , η H ) such that 

∇f (¯ x)

   0 

0





dx

     + D  dy  

dz





    y − η  

η





0 z

∇G(¯ x)> η G

    +  

−η

G

0





∇H(¯ x)> η H

    +  

0 −η H

    = 0, 

(dy + y¯)> η y = 0, ηiy ≥ 0, i ∈ B, (dz + z¯)> η z = 0, ηiz ≥ 0, i ∈ A. Hence, we have >

>

∇f (¯ x)> dx = −d> Dd − η G ∇G(¯ x)dx + η G dy >

>

−η H ∇H(¯ x)dx + η H dz + η y> dy + η z > dz >

>

>

>

= −d> Dd + η G G(¯ x) − η G y¯ + η H H(¯ x) − η H z¯ − η y> y¯ − η z > z¯ ≤ −d> Dd + kη G k∞ kG(¯ x) − y¯k1 + kη H k∞ kH(¯ x) − z¯k1 ≤ −d> Dd + µ(kG(¯ x) − y¯k1 + kH(¯ x) − z¯k1 ), where µ ≥ k(η G , η H )k∞ . By differentiability of f, G, H, and by positive definiteness of D, for any 0 < σ < 1, there is an α ¯ > 0 such that for any 0 < α < α ¯ we have f (¯ x + αdx ) ≤ f (¯ x) + α∇f (¯ x)> dx + α(1 − σ) > d Dd, 3mµ α(1 − σ) > |Hi (¯ x + αdx ) − Hi (¯ x) − α∇Hi (¯ x)> dx | ≤ d Dd, 3mµ Therefore |Gi (¯ x + αdx ) − Gi (¯ x) − α∇Gi (¯ x)> dx | ≤

α(1 − σ) > d Dd, 3

i = 1, 2, · · · , m, i = 1, 2, · · · , m.

kG(¯ x + αdx ) − y¯ − αdy k1 ≤ kG(¯ x + αdx ) − G(¯ x) − α∇G(¯ x)dx k1 +kα∇G(¯ x)dx + G(¯ x) − y¯ − αdy k1 α(1 − σ) > ≤ d Dd + (1 − α)kG(¯ x) − y¯k1 , 3µ 9

and similarly kH(¯ x + αdx ) − z¯ − αdz k1 ≤

α(1 − σ) > d Dd + (1 − α)kH(¯ x) − y¯k1 . 3µ

Hence, φµ (¯ x, y¯, z¯) − φµ ((¯ x, y¯, z¯) + α(dx , dy , dz )) = (f (¯ x) − f (¯ x + αdx )) + µ(kG(¯ x) − y¯k1 − kG(¯ x + αdx ) − y¯ − αdy k1 +kH(¯ x) − z¯k1 − kH(¯ x + αdx ) − z¯ − αdz k1 ) ≥ α(−∇f (¯ x)> dx + µ(kG(¯ x) − y¯k1 + kH(¯ x) − z¯k1 ) − (1 − σ)d> Dd) ≥ ασd> Dd. Since (¯ x, y¯, z¯) ∈ WF (A,B) , (¯ x, y¯, z¯) + (dx , dy , dz ) ∈ WF (A,B) and α ∈ (0, 1), (¯ x, y¯, z¯) + α(dx , dy , dz ) ∈ WF (A,B) so the proof is completed. Denote the objective function of the problem QP(A,B) (¯ x, y¯, z¯) by 1 ϕ((¯ x, y¯, z¯), d, D) = ∇f (¯ x)> dx + d> Dd. 2 Observe that the function ϕ is independent of (A, B). If a quadratic subproblem associated with an adjacent pair generates a solution d with the smallest value of ϕ((¯ x, y¯, z¯), d, D) among all adjacent pairs (A, B) ∈ P(¯x,¯y,¯z) , then we call this descent direction d the steepest descent direction. Definition 3.2 (steepest descent direction) Let (¯ x, y¯, z¯) be a weak feasible solu¨ B) ¨ ∈ P(¯x,¯y,¯z) generates a solution d¨ to QP ¨ ¨ (¯ tion. If (A, x, y¯, z¯) such that (A,B)

¨ D) ≤ ϕ((¯ ϕ((¯ x, y¯, z¯), d, x, y¯, z¯), d, D) for all d, solutions of problem QP(A,B) (¯ x, y¯, z¯) and all pairs (A, B) ∈ P(¯x,¯y,¯z) . Then d¨ is called the steepest descent direction of FMPEC at (¯ x, y¯, z¯). Ideally along the steepest descent direction, the partial exact penalty function will descend the most. However in the algorithm proposed in this paper, we only need ˜ B) ˜ ∈ P(¯x,¯y,¯z) with the solution d˜ to the problem to obtain an adjacent pair (A, QP(A, x, y¯, z¯) which satisfies the following inequality ˜ B) ˜ (¯ ˜ D) ≤ c1 ϕ((¯ ¨ D), ϕ((¯ x, y¯, z¯), d, x, y¯, z¯), d, 10

(9)

˜ B) ˜ an acceptable descent for a given 0 < c1 ≤ 1. We call such an adjacent pair (A, pair and such a descent direction d˜ an acceptable descent direction. Therefore, if (¯ x, y¯, z¯) is not a piecewise stationary point of FMPEC, then we ˜ B) ˜ ∈ P(¯x,¯y,¯z) such that the solution d˜ to the can find an acceptable descent pair (A, quadratic program QP ˜ ˜ (¯ x, y¯, z¯) is not a zero vector. By Proposition 3.1, d˜ provides (A,B)

a descent direction at (¯ x, y¯, z¯) for the l1 penalty function φµ with µ sufficiently large. ˜ Let s be the smallest nonnegative Now use the following inexact line search along d: integer such that ˜ > 1 σcs (d˜> Dd), ˜ φµ (¯ x, y¯, z¯) − φµ ((¯ x, y¯, z¯) + cs2 d) 2 2

(10)

where 0 < c2 < 1 and 0 < σ < 1. Then the next iteration point defined by (˜ x, y˜, z˜) = (¯ x, y¯, z¯) + cs d˜ will still lie on the weak feasible region and the value of the penalty 2

function φµ will be reduced. ˜ B) ˜ and However given a current iterate (¯ x, y¯, z¯) if we always find a descent pair (A, ˜ move to the next iteration point on the piece WF ˜ ˜ along the descent direction d, (A,B)

the algorithm may converge to a stationary point of the piece WF (A, ˜ B) ˜ and not to a stationary point of FMPEC. For example, consider the following problem: (P )

min

f (y, z) := (y 2 − z + 1)2

s.t.

y ≥ 0, z ≥ 0, yz = 0.

y,z

This problem is in the form of FMPEC if we let x1 = y and x2 = z. The constraint region is a union of two faces: {(y, z) : y ≥ 0, z = 0}

(11)

{(y, z) : y = 0, z ≥ 0}.

(12)

and

The optimal solution restricted on the face (11) is (0, 0) with the optimal value 1. The optimal solution restricted on the face (12) is (0, 1) with the optimal value 0. Hence the optimal solution of problem (P) is (0, 1). Assume the kth iteration point is (yk , zk ) with yk ∈ (0, 1), zk = 0. At this point there is only one adjacent pair A = ∅, B = {1} associated with the face (11) and the quadratic subproblem with D

11

being the Hessian matrix of the objective function at this point is 

min

dy ,dz

s.t.





12yk2 + 4 −4yk d 1  y  4yk (yk2 + 1)dy − 2(yk2 + 1)dz + (dy , dz )  2 −4yk 2 dz dy + yk ≥ 0, dz + zk = 0.

Or equivalently min dy

s.t.

1 4yk (yk2 + 1)dy + (12yk2 + 4)d2y 2 dy + yk ≥ 0.

It is easy to see that the solution to the above problem is dy = −

yk (yk2 +1) . 3yk2 +1

Using the

Armijo-type inexact line search, we know that m = 0 satisfy (10) for a sufficient small σ > 0, and hence we can take (yk + dy , 0) as the next iteration point. Then we have 2y 3

k that (yk+1 , zk+1 ) = ( 3y2 +1 , 0) which implies that k

lim (yk , zk ) = (0, 0).

k→0

Hence starting from any point on the face (11) if we just move to the next iteration point along the direction found by using the quadratic subproblem then the iterative sequence will converge to the optimal solution of the problem restricted on the face (11) but not to the optimal solution (0, 1) which lies on the other face (12). To move to the other face from the face the iteration point (yk , zk ) lies, we search the nearby point with more active constraints from the iterative point. Notice that (0, 0) is the only point which has more active constraints than (yk , zk ). At (0, 0) there are two adjacent pairs A = ∅, B = {1} associated with the face (11) and A = {1}, B = ∅ associated with the face (12). The quadratic subproblem associated with the face (12) has a nonzero solution d = (0, 1). Hence (0, 1) is a descent direction of the function f (y, z) = (y 2 − z + 1)2 on the face (12). We compare the value of function f on the iteration point (yk , zk ) and the point obtained by moving along the direction (0, 1) from (0, 0) and using the Armijo-type inexact line search and move to the next iteration point which has the smallest value of f . Since the point (yk , zk ) is getting closer and closer to the point (0, 0), the value of f on (yk , zk ) will eventually be larger than the one on the point obtained by moving along the direction (0, 1) from (0, 0). Hence the iteration points will escape from (0, 0) and converge to the true optimal solution (0, 1). 12

It is this simple observation that leads to the δ-active search technique first proposed in Ref. 9 which is used in the inner iteration from Step 2 to Step 6 of the following algorithm. In this search, starting from an initial parameter δ0 > 0 and for several decreasing δ > 0, we perform the following operation. (i) Given a current iterate (¯ x, y¯, z¯). Let x˜ = x ¯  (˜ y )i =

(˜ z )i =

 0, (¯ y )i ≤ δ  (¯ y )i , otherwise   0, (¯ z) ≤ δ i

 (¯ z )i ,

otherwise

i = 1, 2, · · · , m

i = 1, 2, · · · , m .

(ii) If (˜ x, y˜, z˜) is not a piecewise stationary point, then we find an acceptable ˜ B) ˜ ∈ P(˜x,˜y,˜z) and solve the quadratic subproblem QP ˜ ˜ (˜ descent pair (A, ˜, z˜). (A,B) x, y ˜ (iii) We use the solution d to QP ˜ ˜ (˜ x, y˜, z˜) and its multipliers to update the (A,B)

exact penalty parameter µ. Perform an Armijo-type line search to determine a step size α. The point (˜ x, y˜, z˜) + αd˜ is a candidate for the next iteration point if it provides a certain amount of reduction for the partial exact penalty function φµ i.e., if ˜ φµ (¯ x, y¯, z¯) > φµ ((˜ x, y˜, z˜) + αd). (iv) We reduce δ until (˜ x, y˜, z˜) = (¯ x, y¯, z¯), repeat the above procedure (i)-(iv) for each reduced δ, and take the best candidate obtained as the next iteration point. In the algorithm stated below, if the penalty parameter µ does not change (which is the case when k is sufficiently large by the scheme of updating exact penalty parameter µ) then the final temporary point (xtemp , ytemp , ztemp ) in the following algorithm is indeed the best among all candidates obtained in iteration k. We now describe the merit function PSQP algorithm. Algorithm 3.1. Step 1. Given (x1 , y1 , z1 ) ∈ WF, σ, c1 , c2 , c3 ∈ (0, 1), δ0 , µ1 , 1 > 0 and a symmetric positive definite matrix D1 ∈ R(n+2m)×(n+2m) . Let k := 1. Step 2. Let δ = δ0 , (xtemp , ytemp , ztemp ) = (xk , yk , zk ) and µ = µk . Step 3. Let x˜ = xk 13

(˜ y )i

(˜ z )i

  0, (yk )i ≤ δ i = 1, 2, · · · , m =  (y ) , otherwise k i   0, (zk )i ≤ δ i = 1, 2, · · · , m . =  (z ) , otherwise k i

˜ B) ˜ ∈ P(˜x,˜y,˜z) with the given c1 such Step 4. Choose an acceptable descent pair (A, that inequality (9) holds or conclude that the algorithm terminate with x˜ as a piecewise stationary point of MPEC. Step 5. Find the solution d˜ to the problem QP(A, x, y˜, z˜) with the multipliers ˜ B) ˜ (˜ (η g , η h , η G , η H ). Update the penalty parameter by   µ, µ=  k(η G , η H , η g , η h )k

∞

if k(η G , η H , η g , η h )k∞ ≤ µ + 1 ,

otherwise

.

Let s be the smallest nonnegative integer such that ˜ > 1 σc2 s (d˜> Dk d). ˜ φµ (˜ x, y˜, z˜) − φµ ((˜ x, y˜, z˜) + c2 s d) 2 If ˜ φµ (xtemp , ytemp , ztemp ) > φµ ((˜ x, y˜, z˜) + c2 s d) set ˜ (xtemp , ytemp , ztemp ) = (˜ x, y˜, z˜) + c2 s d. Step 6. Let δ = c3 max{(yk )i , (yk )i ≤ δ; (zk )i , (zk )i ≤ δ}. If δ > 0, go to Step 3. Step 7. If δ = 0, let µk+1 = µ, (xk+1 , yk+1 , zk+1 ) = (xtemp , ytemp , ztemp ), adjust Dk+1 . Set k = k + 1 and go to Step 2. Remark 3.2 We may update Dk according to BFGS rule or other rules.

14

Remark 3.3 In Step 4, we need to find an acceptable descent pair and an acceptable descent direction such that inequality (9) holds. This step may be time consuming. However in implementing the algorithm, we only need to find an approximate piecewise stationary point and hence we can simplify the rule for finding the acceptable pair ˜ B) ˜ in Step 4 as follows: Introduce a parameter  and set  > 0. Find a descent (A, ˜ B) ˜ with solution d˜ satisfying pair (A, ˜ D) ≤ − ϕ((˜ x, y˜, z˜), d,

(13)

and then go to step 5; If there is no such descent pair with a descent direction such that (13) holds then stop and output (˜ x, y˜, z˜) as an approximate piecewise stationary point with accuracy . The reason for the simplification in numerical computing is that any computer can express numbers by only a finite number of digits, and hence ¨ D) cannot equal −∞. Suppose the computer can express amounts beϕ((˜ x, y˜, z˜), d, ¨ D) ≥ −1064 . If we choose c1 = 10log10 −64 , then tween ±1064 . Then ϕ((˜ x, y˜, z˜), d, ¨ D) ≥ − which implies by (13) that c1 ϕ((˜ x, y˜, z˜), d, ¨ D) ≥ ϕ((˜ ˜ D), c1 ϕ((˜ x, y˜, z˜), d, x, y˜, z˜), d, ˜ B) ˜ is an acceptable descent pair. On the other hand, suppose that for each i.e., (A, feasible pair (A, B) ∈ P(˜x,˜y,˜z) and the corresponding solution d, ϕ((˜ x, y˜, z˜), d, D) ≥ −.

(14)

as the absolute value of − is very small, we may regard the iteration point as an approximate piecewise stationary point and stop computation. Note that in practical computation what we can obtain is always an approximate solution with a specified accuracy which can be controlled in our case by adjusting the value of . The question that whether a small c1 affects convergence speed deserves future consideration, but in our practice the simplified rule (13) does not cause any problem. Remark 3.4 Theoretically at each iteration point (˜ x, y˜, z˜) one can randomly pick a pair (A, B) ∈ P(˜x,˜y,˜z) and solve the quadratic problem. If the solution is nonzero then it is a descent direction and a descent pair is found. When m, the number of complementarity constraints is not too large, the approach is effective but it becomes however impractical when m is large. Effectively finding a descent pair is actually a 15

common issue for all PSQP algorithms and we do not intend to study this issue in detail in this paper. The following two methods can be used to find a descent pair without searching through all possible pairs. Firstly in the case where the MPECLICQ is satisfied, if the solution to the quadratic subproblem QP(A,B) (˜ x, y˜, z˜) is zero and the multiplier satisfies the sign condition (4), then (˜ x, y˜, z˜) is a S-stationary point of FMPEC. Otherwise if the solution to the quadratic subproblem QP(A,B) (˜ x, y˜, z˜) is zero but the multiplier does not satisfy the sign condition (4), then by using the active set algorithm given in Ref. 21 (which has been further developed in Ref. 22) we ˜ B) ˜ for which the multiplier is no longer the multiplier for can find a new pair (A, the new quadratic problem QP(A, x, y˜, z˜). Consequently by the uniqueness of the ˜ B) ˜ (˜ ˜ B) ˜ is multiplier, d = 0 is not the solution to the new problem QP ˜ ˜ (˜ x, y˜, z˜) and (A, (A,B)

a descent pair. Secondly the active set algorithm may not work if the MPEC-LICQ does not hold. The reason is that if the MPEC-LICQ does not hold then there may exist more than one multiplier for quadratic problem QP(A, x, y˜, z˜) and hence one ˜ B) ˜ (˜ may not conclude that d = 0 is not the the solution to the new quadratic problem QP(A, x, y˜, z˜) even though the current multiplier is not a multiplier for the new ˜ B) ˜ (˜ problem. In such a case, we suggest to use the extreme point algorithm proposed in Ref. 23 to find a descent pair.

4

Convergence analysis

First we state the following assumptions in order to guarantee global convergence of Algorithm 3.1. (A1) There are constants β1 > 0 and β2 > β1 such that β1 d> d ≤ d> Dk d ≤ β2 d> d for any k and any d ∈ Rn+m+m . (A2) The sequence of iteration points {(xk , yk , zk )} is bounded. (A3) The sequence of penalty parameters {µk } is bounded. Note that the above assumptions are mild. Assumption (A2) is standard and can be ensured by imposing certain growth condition on the objective function or certain inf-compactness condition. Assumption (A3) can be ensured by certain constraint qualifications such as the piecewise MFCQ as shown in Proposition 4.2. 16

Proposition 4.1 If Algorithm 3.1 stops in a finite number of iterations, then it must stop at a piecewise stationary point of FMPEC. Moreover if the partial MPEC-LICQ is satisfied at this point then the point must be a S-stationary point. Proof. The proof follows from the termination rule in Step 4 of Algorithm 3.1 and Proposition 2.1. Theorem 4.1 Suppose that assumptions (A1)–(A3) hold, if the sequence of iteration points {(xk , yk , zk )} generated by Algorithm 3.1 is infinite, then each accumulation point of this sequence is a piecewise stationary point of FMPEC. Moreover if the partial MPEC-LICQ is satisfied at an accumulation point then the accumulation point must be a S-stationary point. Proof. The second statement of the theorem follows from the first statement and Proposition 2.1. Hence it suffices to prove the first statement. By assumptions (A1), (A2), (A3) and the scheme for updating the exact penalty parameter µ in step 5, for any accumulation point (¯ x, y¯, z¯) of the sequence of iteration points {(xk , yk , zk )} generated by Algorithm 3.1, there must be a subsequence {(xk , yk , zk )}K , a positive definite matrix D and a positive number µ ¯ such that lim

k→∞,k∈K

{(xk , yk , zk )} = (¯ x, y¯, z¯), lim

k→∞,k∈K

(15)

Dk = D

(16)

k ∈ K.

(17)

and µk = µ ¯, For any y, z ∈ Rm , let I1 (y) = {i|(y)i = 0}, I2 (z) = {i|(z)i = 0} and set x˜k (˜ yk )i (˜ zk )i

= xk   0, i∈ = (yk )i , i 6∈   0, i∈ =  (z ) , i 6∈ k i

17

I1 (¯ y) I1 (¯ y) I2 (¯ z) I2 (¯ z ).

Then it is obvious that lim

k→∞,k∈K

(˜ xk , y˜k , z˜k ) = (¯ x, y¯, z¯).

(18)

To continue the proof of the theorem we shall need the following lemma. Lemma 4.1 For sufficiently large k ∈ K, (˜ xk , y˜k , z˜k ) must be searched in the inner iteration of iteration k, i.e., there is a δ used in the inner iteration of the kth iteration such that I1 (¯ y ) = I1k (δ), I2 (¯ z ) = I2k (δ), where I1k (δ) = {i : (yk )i ≤ δ}, I2k (δ) = {i : (zk )i ≤ δ}. Proof. By virtue of (15) we have lim

max{(yk )i , i ∈ I1 (¯ y ); (zk )i , i ∈ I2 (¯ z )} = 0

lim

min{(yk )i , i 6∈ I1 (¯ y ); (zk )i , i 6∈ I2 (¯ z )} > 0.

k→∞, k∈K

and k→∞, k∈K

Hence there is k1 ≥ 0 such that max{(yk )i , i ∈ I1 (¯ y ); (zk )i , i ∈ I2 (¯ z )} < c3 min{(yk )i , i 6∈ I1 (¯ y ); (zk )i , i 6∈ I2 (¯ z )} and max{(yk )i , i ∈ I1 (¯ y ); (zk )i , i ∈ I2 (¯ z )} < c3 δ0 for any k ≥ k1 , k ∈ K. In Algorithm 3.1, we reduce δ until I1k (δ) = I1 (yk ), I2k (δ) = I2 (zk ) at each inner iteration. Assume the sequence of δ used in iteration k be t(k)

δk0 = δ0 > δk1 > · · · > δk

> 0.

By (18), without loss of generality, we may assume that I1 (¯ y ) ⊆ I1k (δ0 ), I2 (¯ z ) ⊆ I2k (δ0 ) 18

(19)

for every k ≥ k1 and k ∈ K. Notice that for any 0 ≤ δ 2 ≤ δ 1 one has I1k (δ 2 ) ⊆ I1k (δ 1 ), I2k (δ 2 ) ⊆ I2k (δ 1 ). Let t¯k = max{t|I1 (¯ y ) ⊆ I1k (δkt ), I2 (¯ z ) ⊆ I2k (δkt ), t = 0, 1, 2, · · · , t(k)} for each k ≥ k1 and k ∈ K. Due to (15), we may assume that for k ≥ k1 and k ∈ K, we have I1 (yk ) ⊆ I1 (¯ y ), I2 (zk ) ⊆ I2 (¯ z ). We now claim that ¯

¯

z ) = I2k (δktk ). I1 (¯ y ) = I1k (δktk ), I2 (¯

(20)

First, if t¯k = t(k), then, we have ¯

t(k)

¯

t(k)

I1 (¯ y ) ⊆ I1k (δktk ) = I1k (δk ) = I1 (yk ) ⊆ I1 (¯ y) and z ). I2 (¯ z ) ⊆ I2k (δktk ) = I2k (δk ) = I2 (zk ) ⊆ I2 (¯ Hence, (20) hold. Second, we consider the case that t¯k < t(k) and prove (20) by ¯ y) contradiction. Assume (20) is not true. Then there is an ¯i such that ¯i ∈ I1k (δktk ) \ I1 (¯ ¯

or ¯i ∈ I2k (δktk ) \ I2 (¯ z ). Therefore, ¯

¯

¯

δktk +1 = c3 max{(yk )i , i ∈ I1k (δktk ); (zk )i , i ∈ I2k (δktk )} ≥ c3 max{(yk )¯i , (zk )¯i } ≥ c3 min{(yk )i , i 6∈ I1 (¯ y ); (zk )i , i 6∈ I2 (¯ z )} ≥ max{(yk )i , i ∈ I1 (¯ y ); (zk )i , i ∈ I2 (¯ z )}

by virtue of (19).

It follows that ¯

¯

I1 (¯ y ) ⊆ I1k (δktk +1 ), I2 (¯ z ) ⊆ I2k (δktk +1 ), which contradicts the definition of t¯k . So, we have proved that for every k ≥ k1 and k ∈ K, (20) is always true. Hence, (˜ xk , y˜k , z˜k ) must have been searched for any k ≥ k2 , k ∈ K.

19

˜k ) be an acceptable descent Now we continue the proof of the theorem. Let (A˜k , B pair found in the algorithm at (˜ xk , y˜k , z˜k ) and d˜k be the solution to the problem QP(A˜k ,B˜k ) (˜ xk , y˜k , z˜k ) which satisfies (9), i.e. ϕ((˜ xk , y˜k , z˜k ), d˜k , Dk ) ≤ c1 ϕ((˜ xk , y˜k , z˜k ), d¨k , Dk ),

(21)

where d¨k is the steepest descent direction to FMPEC at (˜ xk , y˜k , z˜k ) as defined in Definition 3.2. Since P(˜xk ,˜yk ,˜zk ) has only a finite number of adjacent pairs and lim

k→∞,k∈K

(˜ xk , y˜k , z˜k ) = (¯ x, y¯, z¯),

˜ B) ˜ ∈ without loss of generality, we may assume that there is an adjacent pair (A, P(¯x,¯y,¯z) such that A˜k = A˜ ⊆ I1 (¯ y ), k ∈ K, ˜k = B ˜ ⊆ I2 (¯ B z ), k ∈ K. Without loss of generality, assume that lim

k→∞,k∈K

˜ d˜k = d,

(22)

˜ Since the quadratic subproblems have strictly convex objective function for some d. and the sequence of penalty parameters are assumed to be bounded, we can show that the limiting direction d˜ is the solution to the problem QP ˜ ˜ (¯ x, y¯, z¯). Indeed (A,B)

d˜k solves QP(A, xk , y˜k , z˜k ) if and only if there exists a multiplier (ηkg , ηkh , ηkG , ηkH ) sat˜ B) ˜ (˜ isfying the KKT condition. By the scheme of updating the penalty parameters µk in Algorithm 3.1, the sequence of multipliers (ηkg , ηkh , ηkG , ηkH ) must be bounded. Without loss of generality, assume that it converges to (η g , η h , η G , η H ). Taking the limits in the KKT condition for QP ˜ ˜ (¯ xk , y¯k , z¯k ) at d˜k as k → ∞, it is easy to see that the (A,B)

KKT condition for the problem QP(A, x, y¯, z¯) holds at d˜ with (η g , η h , η G , η H ) as a ˜ B) ˜ (¯ multiplier which implies that d˜ is indeed a solution to the problem QP(A, x, y¯, z¯). ¯ B) ¯ (¯ We now finish the proof of the theorem by contradiction. Assume that (¯ x, y¯, z¯) is not a piecewise stationary point of FMPEC. Then the following two cases may happen. In case 1, (¯ x, y¯, z¯) is not feasible to FMPEC and in case 2, (¯ x, y¯, z¯) is feasible to FMPEC but not a piecewise stationary point to FMPEC. In case one since (¯ x, y¯, z¯) is not feasible to FMPEC, d˜ 6= 0 and hence from Proposition 3.1, d˜ is a descent 20

direction of φµ at (¯ x, y¯, z¯) for appropriate choice of µ. By (15)-(17), it is obvious that d˜ is a descent direction of φµ¯ . In case 2 since (¯ x, y¯, z¯) is feasible to FMPEC but not ¯ B) ¯ ∈ P(¯x,¯y,¯z) such that a piecewise stationary point to FMPEC we can find a pair (A, (¯ x, y¯, z¯) is not a stationary point of QP(A, x, y¯, z¯). Consequently d = 0 is feasible ¯ B) ¯ (¯ for QP(A, x, y¯, z¯) and the solution d¯ to QP(A, x, y¯, z¯) satisfies ¯ B) ¯ (¯ ¯ B) ¯ (¯ ¯ D) < 0. ϕ((¯ x, y¯, z¯), d,

(23)

Let d¯k be the solution to the problem QP(A, xk , y˜k , z˜k ). Then by boundedness of the ¯ B) ¯ (˜ penalty parameters and the strict convexity of the objective function for the quadratic problems, we must have lim

k→∞,k∈K

¯ d¯k = d.

(24)

By (21), we have ϕ((˜ xk , y˜k , z˜k ), d˜k , Dk ) ≤ c1 ϕ((˜ xk , y˜k , z˜k ), d¨k , Dk ) ≤ c1 ϕ((˜ xk , y˜k , z˜k ), d¯k , Dk ),

(25)

for any k ≥ k1 , k ∈ K. By (21)–(24), taking k → ∞, k ∈ K in (25) we have ˜ D) ≤ c1 ϕ((¯ ¯ D), ϕ((¯ x, y¯, z¯), d, x, y¯, z¯), d, 1 σc2 s (d˜> Dd). ˜ φµ¯ (¯ x, y¯, z¯) − φµ¯ ((¯ x, y¯, z¯) + c2 s d) 2 Then there exists k2 ≥ k1 such that for all k ≥ k2 , 1 ˜ φµ¯ (˜ xk , y˜k , z˜k ) − φµ¯ ((˜ xk , y˜k , z˜k ) + c2 s¯d˜k ) > σc2 s¯(d˜> k D dk ) 2 21

which implies that sk ≤ s¯

∀k ≥ k2 , k ∈ K,

(26)

where sk is the smallest positive integer s satisfying 1 ˜ φµ¯ (˜ xk , y˜k , z˜k ) − φµ¯ ((˜ xk , y˜k , z˜k ) + c2 sk d˜k ) > σc2 sk (d˜> k Dk dk ) 2

(27)

for all k ∈ K. Therefore, there is a k3 ≥ k2 such that φµ¯ (˜ xk , y˜k , z˜k ) − φµ¯ ((˜ xk , y˜k , z˜k ) + c2 sk d˜k ) 1 ˜ > by (27) σc2 sk (d˜> k Dk dk ) 2 1 ˜ ≥ σc2 s¯(d˜> by(26) k Dk dk ) 2 1 ˜ ≥ σc2 s¯(d˜> Dd) 3

(28)

for any k ≥ k3 , k ∈ K. By (15)–(18), there is a k4 ≥ k3 such that for any k > k4 , k ∈ K, we have 1 ˜ |φµ¯ (xk , yk , zk ) − φµ¯ (˜ xk , y˜k , z˜k )| < σc2 s¯(d˜> Dd). 6

(29)

By (28) and (29), φµ¯ (xk , yk , zk ) − φµ¯ ((˜ xk , y˜k , z˜k ) + c2 sk d˜k ) = φµ¯ (xk , yk , zk ) − φµ¯ (˜ xk , y˜k , z˜k ) + φµ (˜ xk , y˜k , z˜k ) − φµ¯ ((˜ xk , y˜k , z˜k ) + c2 sk d˜k ) 1 ˜ + 1 σcs¯(d˜> Dd) ˜ > − σc2 s¯(d˜> Dd) 6 3 2 1 ˜ σc2 s¯(d˜> Dd) = 6 for any k ≥ k4 , k ∈ K. By Algorithm 3.1, we choose the next iteration point (xk+1 , yk+1 , zk+1 ) such that it provides the largest reduction on the exact penalty function, hence we have φµ¯ (xk , yk , zk ) − φµ¯ (xk+1 , yk+1 , zk+1 ) ≥ φµ¯ (xk , yk , zk ) − φµ¯ ((˜ xk , y˜k , z˜k ) + c2 sk d˜k ) 1 ˜ ≥ σc2 s¯(d˜> Dd) 6 > 0 ∀k ≥ k4 , k ∈ K. 22

(30)

However by (15), we have lim φµ¯ (xk , yk , zk ) = φµ¯ (¯ x, y¯, z¯)

k→∞

which contradicts (30) and the proof of the theorem is completed. Since any accumulation point of the algorithm may not be feasible for FMPEC, we extend the piecewise NNAMCQ to accomodate such a weak feasible solution. Note that the extended piecewise NNAMCQ is reduced to the piecewise NNAMCQ if (¯ x, , y¯, z¯) is actually feasible for FMPEC. Definition 4.1 (extended piecewise NNAMCQ) Let (¯ x, y¯, z¯) be a weak feasible solution. We say that the extended piecewise NNAMCQ is satisfied at (¯ x, y¯, z¯) if for any adjacent pair (A, B) ∈ P(¯x,¯y,¯z) there is no nonzero vector (η g , η h , η G , η H ) ∈ Rp+q+2m such that 0 = ∇g(¯ x)> η g + ∇h(¯ x)> η h − ∇G(¯ x)> η G − ∇H(¯ x)> η H , η g ≥ 0,

(g(¯ x) − max {gi (¯ x), 0})> η g = 0,

ηiG = 0, ∀i ∈ γ 0 ,

ηiH = 0, ∀i ∈ α0 ,

ηiH ≥ 0 ∀i ∈ B and ηiG ≥ 0 ∀i ∈ A, where α0 := {i : y¯i = 0, z¯i > 0}

γ 0 := {¯ yi > 0, z¯i = 0}.

Proposition 4.2 Assume that the extended piecewise NNAMCQ holds at any accumulation point (¯ x, y¯, z¯) of the sequence {(xk , yk , zk )} generated by Algorithm 3.1, then the exact penalty parameter sequence {µk } is bounded. Proof. As in the beginning of the proof of Theorem 4.1, using the same notations, without loss of generality we can assume that lim (˜ xk , y˜k , z˜k ) = (¯ x, y¯, z¯)

k→∞

lim Dk = D

k→∞

lim d˜k = 0.

k→∞

From the scheme of updating the penalty parameter in the algorithm, it suffices to prove that for the sequence of multipliers (ηkg , ηkh , ηkG , ηkH , ηky , ηkz ) for the quadratic 23

problem QP(A˜k ,B˜k ) (˜ xk , y˜k , z˜k ) at d˜k is bounded. Since by the KKT condition 

xk )> ηkH xk )> ηkG − ∇H(˜ ∇f (˜ xk ) + ∇g(˜ xk )> ηkg + ∇h(˜ xk )> ηkh − ∇G(˜

   

ηkG − ηky ηkH − ηkz

    + Dk d˜k = 0, 

ηkg ≥ 0, [∇g(˜ xk )(d˜k )x + g(˜ xk )]> ηkg = 0, ˜ (ηky )i ≥ 0, (ηky )i ((d˜k )y + y˜k )i = 0, i ∈ B, ˜ (ηkz )i ≥ 0, (ηkz )i ((d˜k )z ) + z˜k )i = 0, i ∈ A. Dividing the equation by the norm of the multiplier and passing to the limits as k → ∞, we find that the piecewise NNAMCQ is violated at (¯ x, y¯, z¯) and hence the proof of the proposition is complete.

5

Conclusion

In this paper a merit function piecewise SQP algorithm for solving general mathematical programs with nonlinear complementarity constraints is introduced. By using auxiliary variables, the nonlinear complementarity constraints in the MPEC was reformulated as some equality constraints and simple linear complementarity constraints. A partial exact penalty function which includes l1 norm of all constraint functions of the equivalent problem except the simple linear complementarity constraints are introduced as a merit function. A QP subproblem is defined within the branch of a given iteration point, whose solution provides an optimality check on the current point. If the current point is not piecewise stationary, then there exists an adjacent branch (possibly the current one) where a descent of the merit function is available. In order to prevent the method to converge to a stationary point of the current branch, the so called δ-active search technique proposed in Ref. 9 is adopted. Under some mild conditions, without the usual MPEC LICQ condition, the algorithm is proved to be globally convergent to a piecewise stationary point. If moreover the partial MPEC LICQ condition (weaker than the usual MPEC LICQ) holds, this stationary point is a S-stationary point.

24

References 1. Luo, Z.Q., Pang, J.S., and Ralph, D., Mathematical Programs with Equilibrium Constraints, Cambridge University Press, 1996. ˘vara, M. and Zowe, J., Nonsmooth Approach to Opti2. Outrata, J. Koc mization Problem with Equilibrium Constraints: Theory, Application and Numerical Results, Kluwer, Dordrecht, The Netherlands, 1998. 3. Ye, J.J., Zhu, D., and Zhu, Q., Exact penalization and necessary optimality conditions for generalized bilevel programming problems, SIAM Journal on Optimization, Vol. 2, pp. 481-507, 1997. 4. Scheel, H., and Scholtes, S., Mathematical programs with complementarity constraints: stationarity, optimality and sensitivity, Mathematics of Operations Research, Vol. 25, pp. 1-22, 2000. 5. Ye, J.J., Necessary and sufficient optimality conditions for mathematical programs with equilibrium constraints, Journal of Mathematical Analysis and Applications, Vol. 307, pp. 305-369, 2005. 6. Ralph, D., Sequential quadratic programming for mathematical programs with linear complementarity constraints, Computational Techniques and Applications: CTAC95(Melbourne, 1995), pp. 663-668, World Scientific, River Edge, NJ, 1996. 7. Jiang, H., and Ralph, D., Extension of Quasi-Newton methods to mathematical programs with complementarity constraints, Computational Optimization and Applications, Vol. 25, pp. 123-150, 2003. 8. Luo, Z.Q., Pang, J.S., and Ralph, D., Piece-wise sequential quadratic programming for mathematical programs with nonlinear complementarity constraints, in: M.C. Ferris and J.S. Pang, eds., Complementarity and Variational Problems: State of the Art, SIAM Publications, 1997. 9. Zhang, J.Z., Liu, G.S., and Wang, S.Y. A globally convergent approximately active search algorithm for solving mathematical programs with linear 25

complementarity constraints, Numerische Mathematik, Vol. 98, pp. 539-558, 2004. 10. Anitescu, M., Global convergence of an elastic mode approach for a class of mathematical programs with complementarity constraints, SIAM Journal on Optimization, Vol. 16, pp. 120-145, 2005. 11. Chen, X,J., and Fukushima, F., A smoothing method for a mathematical program with P-matrix linear complementarity constraints, Computational Optimization and Appllications, Vol. 27, pp. 223-246, 2004. 12. Facchinei, F., Jiang, H., and L. Qi, A smoothing method for mathematical programs with equilibrium constraints, Mathematical Programming, Vol. 85, pp. 107-135, 1999. 13. Fletcher, R., and Leyffer, S., Solving mathematical programs with complementarity constraints as nonlinear programs, Optimization Methods and Software, Vol. 19, pp. 15-40, 2004. 14. Fukushima, M., Luo, Z.Q., and Pang, J.S., A globally convergent sequential quadratic programming algorithm for mathematical programs with linear complementarity constraints, Computational Optimization and Applications, Vol. 10, pp. 5-34, 1998. 15. Fukushima, M., and Pang, J.S., Convergence of a smoothing continuation method for mathematical programs with complementarity constraints, Ill-posed Variational Problems and Regularization Techniques (Trier, 1998), 99–11, Lecture Notes in Economics and Mathematical Systems, Vol. 477, Springer-Verlag, New York, 1999, pp. 99-110. 16. Jiang, H., and Ralph, D., Smooth SQP methods for mathematical programs with nonlinear complementarity constraints, SIAM Journal on Optimization, Vol. 10, pp. 779-808, 2000. 17. Scholtes, S., Nonconvex structures in nonlinear programming, Operations Research, Vol. 52, pp. 368-383, 2004.

26

18. Scholtes, S., Convergence properties of a regularization scheme for mathematical programs with complementarity constraints, SIAM Journal on Optimization, Vol. 11, pp. 918-936, 2001. 19. Scholtes, S. and St¨ohr, M., Exact penalization of mathematical programs with equilibrium constraints, SIAM Journal on Control and Optimization, Vol. 37, pp. 617-652, 1999. 20. Ye, J.J., Optimality conditions for optimization problems with complementarity constraints, SIAM Journal on Optimization, Vol. 9, pp. 374-387, 1999. 21. Scholtes, S., Active set method for inverse linear complementarity problems, Technical Report, Department of Engineering and Judge Institute of Management Studies, University of Cambridge, Cambridge CB2 1AG, England. 22. Fukushima, M., and Tseng, P., An implementable active-set algorithm for computing a B-stationary point of a mathematical programs with linear complementarity constraints, SIAM Journal on Optimization, Vol. 12, pp. 724-739, 2002. 23. Zhang, J.Z., and Liu, G.S., A new extreme point algorithm and its application in PSQP algorithms for solving mathematical programs with linear complementarity constraints, Journal of Global Optimization, Vol. 19, pp. 345-361, 2001.

27