J Glob Optim DOI 10.1007/s10898-011-9693-2
Dual extragradient algorithms extended to equilibrium problems Tran D. Quoc · Pham N. Anh · Le D. Muu
Received: 7 June 2010 / Accepted: 7 February 2011 © Springer Science+Business Media, LLC. 2011
Abstract In this paper we propose two iterative schemes for solving equilibrium problems which are called dual extragradient algorithms. In contrast with the primal extragradient methods in Quoc et al. (Optimization 57(6):749–776, 2008) which require to solve two general strongly convex programs at each iteration, the dual extragradient algorithms proposed in this paper only need to solve, at each iteration, one general strongly convex program, one projection problem and one subgradient calculation. Moreover, we provide the worst case complexity bounds of these algorithms, which have not been done in the primal extragradient methods yet. An application to Nash-Cournot equilibrium models of electricity markets is presented and implemented to examine the performance of the proposed algorithms. Keywords Dual extragradient algorithm · Equilibrium problem · Gap function · Complexity · Nash-Cournot equilibria
T. D. Quoc (B) Hanoi University of Science, Hanoi, Vietnam e-mail:
[email protected] Present Address: T. D. Quoc Department of Electrical Engineering (ESAT/SCD) and OPTEC, K.U. Leuven, Leuven, Belgium P. N. Anh Posts and Telecommunications Institute of Technology, Hanoi, Vietnam e-mail:
[email protected] L. D. Muu Institute of Mathematics, Hanoi, Vietnam e-mail:
[email protected]
123
J Glob Optim
1 Introduction In recent years, equilibrium problems (EP) become an attractive field for many researchers both in theory and applications (see, e.g. [1,7–10,12,15,18,19,21,22,27–29,33,34] and the references quoted therein). It is well-known that equilibrium problems include many important problems in nonlinear analysis and optimization such as the Nash equilibrium problem, variational inequalities, complementarity problems, (vector) optimization problems, fixed point problems, saddle point problems and game theory [1,3,9,10,26]. Furthermore, they can represent rather general and suitable format for the formulation and investigation of various complex problems arising in economics, physics, transportation and network models (see, e.g. [7,10]). The typical form of equilibrium problems is formulated by means of Ky Fan’s inequality and is given as [1]:
Find x ∗ ∈ C
such that f (x ∗ , y) ≥ 0 for all y ∈ C,
(PEP)
where C is a nonempty closed convex subset in Rn x and f : C × C → R is a bifunction such that f (x, x) = 0 for all x ∈ C. Problems of the form (PEP) are referred as primal equilibrium problems. Associated with problem PEP), the dual form is presented as: Find y ∗ ∈ C
such that f (x, y ∗ ) ≤ 0 for all x ∈ C.
(DEP)
Let S ∗p and Sd∗ denote the solution sets of (PEP) and (DEP), respectively. The conditions for nonemptiness of S ∗p and Sd∗ and their characterizations can be found in many research papers and monographs (see, e.g. [10,12,14]). Methods for solving problems (PEP)-(DEP) have been studied extensively. They can be roughly categorized into three popular approaches. The first direction is using gap functions. Instead of solving problem (PEP) directly, the methods based on gap functions convert the original problem into a suitable optimization problem. Then local optimization methods are usually applied to solve the resulting problem. Gap function-based methods frequently appear in optimization and applied mathematics, and were used for variational inequalities by Zhu and Marcotte [36]. Mastroeni [19] further exploited them for equilibrium problems. The second approach is based on auxiliary problem principle. Problem (PEP) is reformulated equivalently to an auxiliary problem, which is usually easier to solve than the original one. This principle was first introduced by Cohen [4] for optimization problems and then applied to variational inequalities in [5]. Mastroeni [18] further extended the auxiliary problem principle to the equilibrium problems of the form (PEP) involving a strongly monotone bifunction and satisfying a certain Lipschitz-type condition. The third approach is proximal point method. Proximal point methods were first investigated by Martinet [17] for solving variational inequalities and then was deeply studied by Rockafellar [31] for finding a zero point of a maximal monotone operator. Recently, many researchers have exploited this method for equilibrium problems (see, e.g. [20,22]). One of the methods for solving equilibrium problems based on the auxiliary problem principle recently proposed in [8], which is called proximal-like methods. The authors in [29] further extended and investigated the convergence of this method under different assumptions. The methods in [29] are also called extragradient methods due to the results of Korpelevich in
123
J Glob Optim
[13]. The extragradient method for solving problem (PEP) generates two iterative sequences {x k } and {y k } as: " # $ # $ % ! k := argmin "ρ f #x k , y $ + G #x k , y $ : y ∈ C % , y (1) x k+1 := argmin ρ f y k , y + G x k , y : y ∈ C ,
where x 0 ∈ C is given, ρ > 0 is a regularization parameter, and G(x, y) is the Bregman distance function (see, e.g. [7,25]). Under mild conditions, the sequences {x k } and {y k } generated by scheme (1) simultaneously converge to a solution of problem (PEP). This method is further investigated in [33] combining with interior proximal point methods. Recently, Nesterov [24] introduced a dual extrapolation method1 for solving monotone variational inequalities. Instead of working on the primal space, this method performs the main step in the dual space. Motivated by this work and comparing to the primal extragradient methods in [8,29], in this paper, we extend the dual extrapolation method to convex monotone equilibrium problems. Note that, in the primal extragradient method (1), two general convex programs need to be solved at each iteration. In contrast to the primal methods, the dual extragradient algorithms developed here only require (i) to solve one general convex program, (ii) to compute one projection point on a convex set and (iii) to calculate one subgradient of a convex function. In practice, if the feasible set C is simple (e.g. box, ball, polytope) then the projection problem (ii) is usually cheap to compute. Moreover, if the bifunction f is convex and differentiable with respect to the second argument then problem (iii) collapses to calculate the gradient vector ∇2 f (x, ·) of f (x, ·) at x. The methods proposed in this paper look quite similar to the Ergodic iteration scheme in [2,23] among many other average schemes in fixed point theory and nonlinear analysis. However, the methods developed here use the new information computed at the current iteration in a different way. The new information w k enters into the next iteration with the same weight thanks to the guidance of the Lipschitz constants (see Algorithm 1 below). In contrast to the k method in [2], the new information w k is not thoroughly exploited. The subgradient & −w is used& to compute the next iteration with a decreasing weight (tk > 0 such that k tk = ∞ and k tk2 ∥w k ∥2 < ∞). The average schemes are usually very sensitive to the calculation errors and leads to slow convergence in practice. The main contribution of this paper is twofold: algorithms and convergence theory. We provide two algorithms for solving (PEP)-(DEP) and prove their convergence. The worst-case complexity bounds of these algorithms are also estimated. This task has not been done in the primal extragradient methods yet. An application to Nash-Cournot oligopolistic equilibrium models of electricity markets is presented. This problem is not monotone, which can not directly apply the algorithms developed here. However, we will show that this problem can be reformulated equivalently to a monotone equilibrium problem by means of the auxiliary problem principle. The rest of this paper is organized as follows. In Sect. 2 a restricted dual gap function of (DEP) is defined and its properties are considered. Then, a scheme to compute dual extragradient step is provided and its properties are investigated. The dual extragradient algorithms are presented in detail in Sect. 3. The convergence of these algorithms is proved and the complexity bound is estimated in this section. An application to Nash-Cournot equilibrium models of electricity markets is presented and implemented in the last section. Notation. Throughout this paper, we use the notation ∥ · ∥ for the Euclidean norm. The notation “:=” means “to be defined”. For a given real number x, [x] defines the largest integer 1 It is also called dual extragradient method in [32].
123
J Glob Optim
number which is less than or equal to x. A function f : C ⊆ Rn x → R is said to be strongly convex on C with parameter ρ > 0 if f (·) − ρ2 ∥ · ∥2 is convex on C. The notation ∂ f denotes the classical subdifferential of a convex function f and ∂2 f is the subdifferential of f with respect to the second argument. If f is differentiable with respect to the second argument then ∇2 f (x, ·) denotes the gradient vector of f (x, ·). 2 Dual extragradient scheme Let X be a subset of Rn x and f : X × X → R ∪ {+∞} be a bifunction such that f (x, x) = 0 for all x ∈ X . We first recall the following well-known definitions that will be used in the sequel (see [1,19,29]). Definition 1 A bifunction f is said to be a) b) c)
strongly monotone on X with a parameter ρ > 0 if f (x, y) + f (y, x) ≤ −ρ∥x − y∥2 for all x and y in X ; monotone on X if f (x, y) + f (y, x) ≤ 0 for all x and y in X ; pseudomonotone on X if f (x, y) ≥ 0 implies f (y, x) ≤ 0 for all x and y in X .
It is obvious from these definitions that: (a) ⇒ (b) ⇒ (c). The following concept is family in nonlinear analysis. A multivalued mapping F : X ⊆ nx Rn x ⇒ 2R is said to be Lipschitz continuous on X with a Lipschitz constant L > 0 if dist(F(x), F(y)) ≤ L∥x − y∥, ∀x, y ∈ X,
(2)
where dist(A, B) is the Hausdorff distance between two sets A and B. The multivalued mapping F is said to be uniformly bounded on X if there exists M > 0 such that sup {dist(0, F(x)) | x ∈ X } ≤ M. Throughout this paper, we will use the following assumptions: A. 1 The set of interior points int(C) of C is nonempty. A. 2 f (·, y) is upper semi-continuous on C for all y in C, and f (x, ·) is proper, closed, convex and subdifferentiable on C for all x in C. A. 3 f is monotone on C. Not that if f (x, ·) is convex for all x ∈ C then Sd∗ ⊆ S ∗p . If f is pseudomonotone on C then S ∗p ⊆ Sd∗ . Therefore, under Assumptions Assumptions A.1–A.3 one has S ∗p ≡ Sd∗ (see, e.g. [10]). Now, let us recall the dual gap function of problem (PEP) defined as follows [19]: g(x) := sup{ f (y, x) | y ∈ C}.
(3)
Under Assumption A.2, to compute one value of g, a general optimization problem needs to be solved. When f (·, x) is concave for all x ∈ C, it becomes a convex problem. The following lemma shows that (3) is indeed a gap function of (DEP) whose proof can be found, for instance, in [11,19]. Lemma 1 The function g defined by (3) is a gap function of (DEP), i.e.: a)
g(x) ≥ 0 for all x ∈ C;
123
J Glob Optim
b)
x ∗ ∈ C and g(x ∗ ) = 0 if and only if x ∗ is a solution of (DEP).
If f is pseudomonotone then x ∗ is a solution of (DEP) if and only if it solves (PEP). Under Assumptions A.1–A.3, the gap function g may not be well-defined due to the fact that problem sup{ f (y, x) | y ∈ C} may not be solvable. Instead of using gap function g, we consider a restricted dual gap function g R defined as follows. Definition 2 Suppose that x¯ ∈ int(C) is fixed and R > 0 is given. The restricted dual gap function of problem (DEP) is defined as: g R (x) := sup { f (y, x) | y ∈ C, ∥y − x∥ ¯ ≤ R} .
(4)
Let us denote by B R (x) ¯ := {y ∈ Rn x | ∥y − x∥ ¯ ≤ R} the closed ball in Rn x of radius R centered at x, ¯ and by C R (x) ¯ := C ∩ B R (x). ¯ Then the characterizations of the restricted dual gap function g R are indicated in the following lemma. Lemma 2 Suppose that Assumptions A.1–A.3 hold. Then: a) b) c)
The function g R defined by (4) is well-defined and convex on C. If x ∗ ∈ C R (x) ¯ is a solution of (DEP) then g R (x ∗ ) = 0. If there exists x˜ ∈ C such that g R (x) ˜ = 0, ∥x˜ − x∥ ¯ < R and f is pseudomonotone then x˜ is a solution of (DEP) (and, therefore, a solution of (PEP)).
Proof Since f (·, x) is upper semi-continuous on C for all x ∈ C and B R (x) ¯ is bounded, the supremum in (4) attains. Hence, g R is well-defined. Moreover, since f (x, ·) is convex for all x ∈ C and g R is the supremum of a family of convex functions depending on parameter x, then g R is convex (see [30]). The statement a) is proved. Now, we prove b). Since f (x, x) = 0 for all x ∈ C, it immediately follows from the definition of g R that g R (x) ≥ 0 for all x ∈ C. Let x ∗ ∈ B R (x) ¯ be a solution of (DEP), we have f (y, x ∗ ) ≤ 0 for all y ∈ C and, particularly, f (y, x ∗ ) ≤ 0 for all x ∈ C ∩ B R (x) ¯ ≡ C R (x). ¯ Hence, g R (x ∗ ) = sup{ f (y, x ∗ ) | y ∈ C ∩ B R (x)} ¯ ≤ 0. However, since g R (x) ≥ 0 for all x ∈ C, we conclude that g R (x ∗ ) = 0. By the definition of g R , it is obvious that g R is a gap function of (DEP) restricted to C ∩ B R (x). ¯ Therefore, if g(x) ˜ = 0 for some x˜ ∈ C and ∥x˜ − x∥ ¯ < R then x˜ is a solution of (DEP) restricted to C ∩ B R (x). ¯ On the other hand, since f is pseudomonotone, x˜ is also a solution of (PEP) restricted to C ∩ B R (x). ¯ Furthermore, since x˜ ∈ int(B R (x)), ¯ for any y ∈ C, we can choose t > 0 sufficiently small such that yt := x˜ + t (y − x) ˜ ∈ B R (x) ¯ and 0 ≤ f (x, ˜ yt ) = f (x, ˜ t y + (1 − t)x) ˜ ≤ t f (x, ˜ y) + (1 − t) f (x, ˜ x) ˜ = t f (x, ˜ y). Here, the middle inequality follows from the convexity of f (x, ˜ ·) and the last equality happens because f (x, x) = 0. Since t > 0, dividing this inequality by t > 0, we conclude that x˜ is a solution of (PEP) on C. Finally, since f is pseudomonotone, x˜ is also a solution of (DEP). The lemma is proved. 2 ⊓ For a given nonempty, closed, convex set C ⊆ Rn x and an arbitrary point x ∈ Rn x , let us denote by dC (x) the Euclidean distance from x to C and by πC (x) the point attained this distance, i.e. dC (x) := min ∥y − x∥, and πC (x) := argmin ∥y − x∥. y∈C
y∈C
(5)
123
J Glob Optim
It is well-known that πC is a nonexpansive and co-coercive operator on C [7]. For any x, y ∈ Rn x and β > 0, we define the following function: ( ' 1 (6) Q β (x, y) := ∥y∥2 − β 2 dC2 x + y . β
In this paper, the function Q β plays a role as a Lyapunov function in the investigation of the convergence of the algorithms [18,25]. Lemma 3 For given x, y ∈ Rn x , the function dC and the mapping πC defined by (5) satisfy: dC2 (x
[πC (x) − x]T [v − πC (x)] ≥ 0, ∀v ∈ C.
+ y) ≥
dC2 (x) + dC2 (πC (x) +
T
y) − 2y [πC (x) − x].
(7) (8)
Consequently, the function Q β defined by (6) possesses the following properties: For x, y ∈ Rn x , Q β (x, y) ≤ ∥y∥2 . If x ∈ C then Q β (x, y) ≥ β 2 ∥x − πC (x + β1 y)∥2 for any y ∈ Rn x . b) For x, y, z ∈ Rn x , it holds that ' ' ( ( ' ' ( ( 1 1 Q β (x, y + z) ≤ Q β (x, y) + Q β πC x + y , z + 2βz T πC x + y − x . β β (9) a)
Proof The inequality (7) is a well-known property of the projection mapping πC (see, e.g. [7,10]). Now, we prove the inequality (8). For any v ∈ C, from (7) we have ∥v − (x + y)∥2 = ∥v − [πC (x) + y] + [πC (x) − x]∥2
= ∥v − [πC (x) + y]∥2 + 2 {v − [πC (x) + y]}T [πC (x) − x] + ∥πC (x) − x∥2
= ∥v − [πC (x) + y]∥2 + 2[πC (x) − x]T [v − πC (x)] −2y T [πC (x) − x] + ∥πC (x) − x∥2
≥ ∥v − [πC (x) + y]∥2 − 2y T [πC (x) − x] + ∥πC (x) − x∥2 .
By the definition of dC (·) and noting that dC2 (x)
respect to v ∈ C in both sides of (10) we get
= ∥πC
(x) − x∥2 , taking the minimum
(10) with
dC2 (x + y) ≥ dC2 (πC (x) + y) + dC2 (x) − 2y T [πC (x) − x].
This inequality is indeed (8). The inequality Q β (x, y) ≤ ∥y∥2 directly follows from the definition (6) of Q β . Furthermore, if we denote by πCk := πC (x + β1 y) then, from (6), we have ' ( ' ( 1 1 1 Q β (x, y) = ∥y∥2 − β 2 dC2 x + y = ∥y∥2 − β 2 ∥πC x + y − x − y∥2 β β β ( ' 1 1 = β 2 ∥ x + y − πCk − [x − πCk )]∥2 − β 2 ∥πCk − x − y∥2 β β ) , ' ( (T * ' + 1 1 2 2 k k x − πC = β ∥x − πC x + y ∥ − 2 x + y − πC . (11) β β Since x ∈ C, applying (7) with πCk instead of πC (x) and v = x, it follows from (11) that Q β (x, y) ≥ β 2 ∥x − πC (x + β1 y)∥2 which proves the second part of a).
123
J Glob Optim
To prove (9), we substitute x by x + β1 y, and y by β1 z into (8) to obtain ' ( ' ' ( ( ' ( 1 1 1 1 dC2 x + (y + z) ≥ dC2 πC x + y + z + dC2 x + y β β β β - ' ( ' (. 1 1 2 T − z πC x + y − x + y . β β β
Then subtracting the identity ∥y + z∥2 = ∥y∥2 + ∥z∥2 + 2y T z to the last inequality after multiplying by β 2 and using the definition of Q β , we get (9). 2 ⊓ 1 Q β (x, y) then the function q(β) is Remark 1 For any x, y ∈ Rn x , if we define q(β) := 2β nonincreasing with respect to β > 0, i.e. q(β1 ) ≤ q(β2 ) for all β1 ≥ β2 > 0.
1 Indeed, consider the function ψ(v, β) := 2β ∥y∥2 − β2 ∥v−x− β1 y∥2 = y T (v−x)− β2 ∥v−x∥2 , which is convex with respect to (v, β). Since q(β) = minv∈C ψ(v, β), it is convex (see [30]). On the other hand, q ′ (β) := −∥πC (x + β1 y) − x∥2 ≤ 0. Thus q is nonincreasing. For a given integer number n ≥ 0, suppose that {x k }nk=0 is a finite sequence of arbitrary points in C and {λk }nk=0 ⊆ (0, +∞) is a finite sequence of positive numbers. Let us define
Sn := and
n / k=0
λk , x¯ n :=
n 1 / λk x k , Sn
(12)
k=0
0 1 ¯ , r R (w, x) := max w T (y − x) | y ∈ C R (x)
(13)
for given w ∈ Rn x and x ∈ C. Clearly, the point x¯ n is a convex combination of {x k }nk=0 with given coefficients { λSnk }nk=0 . Using the definition of g R and the convexity of f (x, ·), it is easy to show that ) 2 , 3 n " % 1 / n n k g R (x¯ ) = max f (y, x¯ )|y ∈ C R (x) ¯ = max f y, λk x |y ∈ C R (x) ¯ Sn k=0 ) , n + * 1 / k ≤ max λk f y, x |y ∈ C R (x) ¯ Sn k=0 ) n , + * / 1 1 n = max λk f y, x k |y ∈ C R (x) ¯ := ' . (14) Sn Sn R k=0
The following lemma provides an upper estimation for the quantity ' nR . Lemma 4 a) The function r R define by (13) satisfies ¯ ≤ r R (w, x)
1 β R2 ¯ w) + Q β (x, . 2β 2
(15)
b) Suppose that Assumptions A.1–A.3 hold and w k ∈ −∂2 f (x k , x k ). Then the quantity ' nR defined by (14) satisfies: ' nR ≤ where s n :=
&n
n / k=0
k=0 λk w
* +T * + # $ β R2 1 ¯ sn + x¯ − x k + λk w k Q β x, , 2β 2
(16)
k.
123
J Glob Optim
Proof Let us define L(x, ρ) := w T (y − x) ¯ + ρ(R 2 − ∥y − x∥ ¯ 2 ) as the Lagrange function of the minimizing problem in (13). Using duality theory in convex optimization, for some β > 0, we have 1 0 r R (w, x) ¯ = max w T (y − x) ¯ | y ∈ C, ∥y − x∥ ¯ 2 ≤ R2 0 $1 # = max min w T (y − x) ¯ + ρ R 2 − ∥y − x∥ ¯ 2 y∈C ρ≥0 ! ! 4 4 ρ′ ρ′ 2 T 2 = min (y − x) ¯ − max w ∥y − x∥ ¯ + R 2 2 ρ ′ ≥0 y∈C ! ! 4 4 1 ρ′ 2 1 2 ′2 2 = min max ∥w∥ − ρ ∥y − x¯ − ′ w∥ + R ρ 2 ρ ′ ≥0 2ρ ′ y∈C . 2 βR 1 1 ≤ + ∥w∥2 − β 2 min ∥y − x¯ − w∥2 . y∈C 2 2β β Thus the inequality (15) follows from this estimation by using the definition (6) of Q β . From (14), by the monotonicity of f , we have: ) n , ) n , + * * + / / n k k λk f y, x | y ∈ C R (x) ¯ ≤ max − λk f x , y | y ∈ C R (x) ¯ . ' R = max k=0
k=0
(17) $ # $ # T y − x k for all y ∈ C R (x) ¯ ⊆ C. Since w k ∈ −∂2 f x k , x , we have − f x k , y ≤ w k Multiplying this inequality by λk > 0 and then summing up from k = 0 to k = n we get −
n / k=0
$ k
#
#
$
n * +T * + * + / y − xk λk f x k , y ≤ λk w k k=0
=
n / k=0
n * +T * +T * + / ¯ + x¯ − x k λk w k (y − x) λk w k k=0
n * +T * + / # $T ¯ + x¯ − x k . = s n (y − x) λk w k
(18)
k=0
Combining (15), (17) and (18) we obtain
n * +T * + 0 1 / ' nR ≤ max (s n )T (y − x) x¯ − x k ¯ | y ∈ C R (x) ¯ + λk w k k=0
#
$
= r R s n , x¯ + ≤
n / k=0
which proves (16).
*
λk w k
n / k=0
+T *
* +T * + x¯ − x k λk w k
+ # $ β 1 ¯ sn + R2, x¯ − x k + Q β x, 2β 2
(19) 2 ⊓
For a given tolerance ε ≥ 0, we say that x¯ n is an ε-solution of (PEP) if g R (x¯ n ) ≤ ε and ∥x¯ n − x∥ ¯ < R. Note that if ε = 0 then an ε-solution x¯ n is indeed a solution of (PEP) due to Lemma 2.
123
J Glob Optim
Remark 2 If we denote by rn :=
n &
k=0
λk (w k )T (x¯ − x k ) then g R (x¯ n ) ≤
1 Sn
Hence, x¯ n is an ε-solution of (PEP), it requires that (s n , x) ¯
5 6 rn + r R (s n , x) ¯ .
rn + r R (s n , x) ¯ ≤ εSn .
(20)
x¯ n
≤ εSn then is an ε-solution of (PEP). This condition can be Therefore, if rn + r R used as a stopping criterion of the algorithms described in the next section. The main idea of designing our algorithms is to construct a sequence {x¯ n } such that the sequence {g R (x¯ n )} of the restricted dual gap function g R tends to 0 as n → ∞. By virtue of Lemma 2, we can check whether or not x¯ n being an ε-solution of (PEP). Let s −1 := 0. The dual extragradient step (u k , x k , s k , w k ) at iteration k(k ≥ 0) is computed as follows: * + ⎧ 1 k−1 k := π ⎪ u x ¯ + , s C ⎪ ⎨ 0 β# 1 $ βρk k k (21) x := argmin f u , y + 2 ∥y − u k ∥2 | y ∈ C , ⎪ ⎪ ⎩ k 1 k k−1 + ρk w , s := s where ρk > 0 and β > 0 are given parameters, and w k ∈ −∂2 f (x k , x k ).
Lemma 5 The sequence {(u k , x k , s k , w k )} generated by scheme (21) satisfies: + * + * + 2β * k +T * w x¯ − x k + Q β x, ¯ s k ≤ Q β x, ¯ s k−1 − β 2 ∥x k − u k ∥2 − β 2 ∥πCk − x k ∥2 ρk +* + 2β * k ξ + w k πCk − x k , (22) + ρk * # $+ where ξ k ∈ ∂2 f (u k , x k ) and πCk := πC x k + βρ1 k w k + ξ k . As a consequence, one has
1 2β k T (w ) (x¯ − x k ) + Q β (x, ¯ s k ) ≤ Q β (x, ¯ s k−1 ) − β 2 ∥x k − u k ∥2 + 2 ∥w k + ξ k ∥2 . (23) ρk ρk
Proof Applying the inequality (9) with x = x, ¯ y = s k−1 and z = uk
= πC (x¯ +
1 k−1 ), βs
1 k ρk w
and noting that
we get
' ' ( ( ( 1 1 1 Q β (x, ¯ s k ) = Q β x, ¯ s k−1 ) + Q πC x¯ + s k−1 , w k ¯ s k−1 + w k ≤ Q β (x, ρk β ρk ' ' ( ( 1 k−1 2β k T − x¯ + (w ) πC x¯ + s ρk β ( ' + 2β * k +T * k 1 w u − x¯ . = Q(x, ¯ s k−1 ) + Q u k , w k + (24) ρk ρk '
Since f (u k , ·) is subdifferentiable, using the first order necessary condition for optimality in convex optimization, it follows from the second line of (21) that ; * + 0 then: terminate. Otherwise, increase k by 1 and go back to Step 1. Output: Compute the final output x¯ n as: x¯ n :=
n
/ 1 xk. (n + 1)
(36)
k=0
The main tasks of Algorithm 1 include: (i) computing a projection point (34), (ii) solving a strongly convex subproblem (35), and (iii) calculating a subgradient vector −w k . If the feasible set C has a simple structure such as box, simplex, ellipsoid or polytope then problem (i) can be explicitly solved. If f (x, ·) is differentiable then problem (iii) collapses to calculate the gradient vector ∇2 f (x, ·) of f (x, ·). The number of iterations n in Algorithm 1 is chosen such that n ≤ n ε , where n ε is the maximum number of iterations for the worst case determined in Theorem 1 below.
123
J Glob Optim
The convergence of Algorithm 1 is stated in the following theorem. Theorem 1 Suppose that Assumptions A.1–A.3 are satisfied and {(u k , x k , s k , w k )}nk=0 is a sequence generated by Algorithm 1. Suppose further that ∂2 f (·, x k )(k ≥ 0) are Lipschitz continuous on C with the same Lipschitz constant L > 0. Then the final output x¯ n computed by (36) satisfies: g R (x¯ n ) ≤
β R2 . 2(n + 1)
(37)
Consequently, the sequence {g R (x¯ n )}n≥0;converges to 0 and the number of iterations attained
0 then: terminate. Otherwise, compute a new step size as βk :=
2M √ k + 1, R
(44)
increase k by 1 and go back to Step 1. Output: Compute the final output x¯ n as (36).
To compute the quantity r R (s k , x) ¯ at Step 4 of Algorithms 1 and 2, a convex program with linear objective function needs to be solved. The solution of this problem lies at the boundary of its feasible set C R (x). ¯ However, solving this problem is usually expensive if C is complex. In practice, instead of measuring r R , we can use the condition ∥x¯ n+1 − x¯ n ∥ ≤ ε to terminate Algorithms 1 and 2. This condition ensures that the approximation x¯ n is not significantly improved in the next iterations. The following theorem shows the convergence of Algorithm 2. Theorem 2 Suppose that Assumptions A.1–A.3 are satisfied and the sequence {(u k , x k , s k , w k )} is generated by Algorithm 2. Suppose further that the mappings ∂2 f (·, x k )(k ≥ 0) are uniformly bounded by the same constant M > 0. Then the final output x¯ n computed by (36) satisfies g R (x¯ n ) ≤ 2M R √
1
n+1
.
(45)
Consequently, the sequence {g R (x¯ n )}n≥0 converges < 0 and the number of iterations ; 2 2 to attained an ε-solution of (PEP) is at most n ε := 4Mε2 R .
Proof It is sufficient to prove the main estimation (45). The remaining conclusions of this theorem immediately follow from (45). Similar to Algorithm 1, the calculations at Steps 1, 2 and 3 of Algorithm 2 are indeed of the scheme (21) with λk = 1 for all k. Therefore, Sn defined by (12) satisfies Sn = n + 1. Since s −1 = 0, it follows from Step 3 of Algorithm 2 that sn =
123
n / k=0
wk .
(46)
J Glob Optim
On the other hand, from (14), (46) and Lemma 4, we have # $ # $ (n + 1)g R x¯ n = Sn g R x¯ n ≤ ' nR ≤
n / # $ βn R 2 1 . (w k )T (x¯ − x k ) + Q βn x, ¯ sn + 2βn 2
(47)
k=0
& Now, let us define bn := nk=0 (w k )(x¯ − x k ) + 2β1n Q βn (x, ¯ s n ), applying the inequality (32) with ρk = 1, and Remark 1 with the fact that βn−1 < βn , we deduce $ # $ # $ # $T # 1 1 x¯ − x n + ¯ sn − ¯ s n−1 Q βn x, Q βn−1 x, bn − bn−1 = w n 2βn 2βn−1 $ # $ # n $T # 6 1 5 n n x¯ − x + Q βn x, ¯ s − Q βn (x, ≤ w ¯ s n−1 ) 2βn MR 2M 2 = √ . ≤ βn n+1
(48)
Moreover, it follows from the definition of bn and Lemma 6 that
$ # $ # $ 2M 2 # $T # 1 1 x¯ − x 0 + ¯ s0 ≤ ¯ s −1 + b0 ≡ w 0 Q β0 x, Q β0 x, = M R. (49) 2β0 2β0 β0
From (48) and (49), by induction, we have bn ≤ M R
n / k=0
√ 1 βn R 2 ≤ MR n + 1 ≡ . √ 2 k+1
(50)
Substituting (50) into (47) we get
√ (n + 1)g R (x¯ n ) ≤ βn R 2 = 2M R n + 1,
which implies that g R (x¯ n ) ≤
2M R √ . n+1
(51) 2 ⊓
Remark 3 Theorem 1 shows that the worst-case complexity bound of Algorithm 1 is O( n1 ), where n is the number of iterations, while, in Algorithm 2, this quantity is O( √1n ) according to Theorem 2. 4 Numerical results In this section we apply Algorithms 1 and 2 to solve an equilibrium problem arising from Nash-Cournot oligopolistic equilibrium models of electricity markets. This problem has been investigated in many research papers (see, e.g. [6]). Instead of using a quadratic cost function as in [6], the cost function in our example is slightly modified. It is still convex but nonsmooth [6,22]. Thus the resulting equilibrium problem can not be transformed into a variational inequality problem. Consider a Nash-Cournot oligopolistic equilibrium model arising in electricity markets with n c (n c = 3) generating companies and each company i (i = 1, 2, 3) (com.#) may possess several generating units n ic (gen.#), as shown in Table 1. The quantities x and x c are the power generation of a unit and a company, respectively. Assuming that the electricity demand is a strictly decreasing function of the price p, the demand function in an interval of time during a day of study considered standard can
123
J Glob Optim Table 1 The lower and upper bounds of the power generation of the generating units and companies (n g = 6)
Table 2 The parameters of the generating unit cost functions c j ( j = 1, . . . , 6)
g
g
c xmin
c xmax
gen.#
xmin
xmax
1
1
0
80
0
80
2
2
0
80
0
130
2
3
0
50
0
130
3
4
0
55
0
125
3
5
0
30
0
125
3
6
0
40
0
125
com.#
gen.# αˆ j [$ MW2 h] βˆ j [$/MWh] γˆ j [$/h] α˜ j [$/MWh] β˜ j
γ˜ j
1
0.0400
2.00
0.00
2.0000
1.0000 25.0000
2
0.0350
1.75
0.00
1.7500
1.0000 28.5714
3
0.1250
1.00
0.00
1.0000
1.0000 8.0000
4
0.0116
3.25
0.00
3.2500
1.0000 86.2069
5
0.0500
3.00
0.00
3.0000
1.0000 20.0000
6
0.0500
3.00
0.00
3.0000
1.0000 20.0000
0 ( p) + σ p, where P 0 ( p) is the total power demand level be expressed as Pload ( p) = Pload load expected for a selected time interval, and σ represents the elasticity of the demand with respect &n c c to price. Let us denote by n g := i=1 n i the number of generating units of all companies and n c I = I := {1, 2, . . . , n g } Ii the index set of all generating units of the company i, where ∪i=1 i and Ii ∩ I j = ∅(i, j = 1, . . . , n c , i ̸ = j). Similar to [6], in this example, we assume that Pload ( p) = 189.2 − 0.5 p which can be expressed inversely as p = 378.4 − 2Pload ( p), where & g Pload ( p) := nj=1 x j − Ploss , Ploss represents the transmission losses throughout the system (Ploss is assumed to be zero in this example). The cost of a generating unit j is given as " % c j (x j ) := max cˆ j (x j ), c˜ j (x j ) , (52)
where
αˆ j 2 x + βˆ j x j + γˆ j , and 2 j β˜ j −1/β˜ j ˜ ˜ γ˜ j (x j )(β j +1)/β j , c˜ j (x j ) := α˜ j x j + β˜ j + 1 cˆ j (x j ) :=
(53)
and the parameters αˆ j , βˆ j , γˆ j , α˜ j , β˜ j and γ˜ j , ( j = 1, . . . , n g ), are given in Table 2. The cost function c j of each generating unit j does not depend on other units. The profit made by company i that owns n ic generating units is f i (x) := p
/ j∈Ii
xj −
/ j∈Ii
g
⎛
c j (x j ) = ⎝378.4 − 2 g
g
n / l=1
⎞
xl ⎠
/ j∈Ii
xj −
/
c j (x j ).
(54)
j∈Ii
subject to the constraints (xmin ) j ≤ x j ≤ (xmax ) j ( j = 1, . . . , n g ), where x := (x1 , . . . , xn g )T .
123
J Glob Optim
For each i = 1, . . . , n c , let us define ⎡ ⎛ ϕi (x, y) := ⎣378.4−2 ⎝ c
and f (x, y) :=
n / i=1
/ j ∈I / i
xj+
/ i∈Ii
⎞⎤ / / y j ⎠⎦ y j − c j (y j ), j∈Ii
(55)
j∈Ii
[ϕi (x, x) − ϕi (x, y)] ,
(56)
Then the oligopolistic equilibrium model of electricity markets [16] can be reformulated as an equilibrium problem of the form (PEP): Find x ∗ ∈ C g such that f (x ∗ , y) ≥ 0
for all y ∈ C g ,
where C g is the feasible set defined by 1 0 # g $ # g $ g , j = 1, . . . , n g . C g := x g ∈ Rn | xmin j ≤ x j ≤ xmax j
(57)
(58)
Let us introduce two vectors q i := (q1i , . . . , qni g )T with ! 1 if j ∈ Ii q ij = 0 otherwise,
and q¯ i := (q¯1i , . . . , q¯ni g )T with q¯ ij := 1 − q ij ( j = 1, . . . , n g ), and then define c
A := 2
n / i=1
* +T q¯ i q i ,
a := −378.4
nc / i=1
i
c
B := 2
n / i=1
* +T qi qi
q , and c(x) :=
c
n / / i=1 j∈Ii
(59)
g
c j (x j ) =
n /
c j (x j ),
j=1
then the bifunction f defined by (56) can be expressed as f (x, y) = [(A + B)x + By + a]T (y − x) + c(y) − c(x).
(60)
Note that since c is nonsmooth and convex and B is symmetric positive semidefinite, f (x, ·) is nonsmooth and convex for all x ∈ C g . Moreover, f is continuous on C g × C g . The function c is subdifferentiable and its subdifferential at x is given by ∂c(x) = (∂c1 (x1 ), . . . , ∂cn g (xn g ))T , where 1 ⎧0 ˆj , ⎪ α ˆ if cˆ j (x j ) > c˜ j (x j ), x + β j j ⎪ ⎪ ⎪ ⎪ * +1/β˜ j . ⎨ x if cˆ j (x j ) = c˜ j (x j ), j = 1, · · · , n g . αˆ j x j + βˆ j , α˜ j + γ˜ jj ∂c j (x j ) = ⎪ ! 4 ⎪ + * ˜ ⎪ ⎪ x 1/β j ⎪ ⎩ α˜ j + γ˜ j , if cˆ j (x j ) < c˜ j (x j ), j
Since f (x, y)+ f (y, x) = −(y − x)T A(y − x) and A is not positive semidefinite, the bifunction f is not monotone. Thus we can not directly apply Algorithms 1 or 2 to solve problem (57). However, the following lemma shows that (57) can be reformulated equivalently to a monotone equilibrium problem.
123
J Glob Optim
Lemma 7 Suppose that the cost functions c j ( j = 1, . . . , n g ) is defined by (52) with the parameters are given in Table 2. Then the equilibrium problem (57) can be reformulated equivalently to an equilibrium problem of a monotone bifunction f 1 given by: f 1 (x, y) := [A1 x + B1 y + a]T (y − x) + c(y) − c(x),
(61)
where A1 := A + 23 B and B1 := 21 B.
Proof From the formula (53), and the parameters given in Table 2, it is easy to check that the functions cˆ j and c˜ j (i = 1, . . . , n g ) are convex (by computing explicitly their second derivatives). Since c j is defined by (52), it is also convex [30]. On the other hand, it follows from (60) that -' ( .T 3 1 1 f (x, y) = A + B x + By + a (y − x) + (y − x)T B(y − x) + c(y) − c(x). 2 2 2 If we define f 1 as in (61) then f (x, y) = f 1 (x, y) + 21 (y − x)T B(y − x). Since B1 = 21 B which is symmetric positive semidefinite and c is convex, the bifunction f 1 is still convex with respect to the second argument for all x. Now, if we define h(x, y) := 21 (y − x)T B(y − x) then, by the symmetric positive semidefiniteness of B, h(x, y) is convex and differentiable with respect to the second argument for all x. Moreover, it is obvious that h(x, y) is nonnegative, h(x, x) = 0 and ∇2 h(x, x) = 0 for all x and y. Applying Proposition 2.1 in [18], it implies that the equilibrium problem: Find x ∗ ∈ C g such that: f 1 (x ∗ , y) ≥ 0
for all y ∈ C g
is equivalent to an auxiliary equilibrium problem: Find x ∗ ∈ C g such that: f 1 (x ∗ , y) + h(x ∗ , y) ≥ 0
for all y ∈ C g .
However, since f 1 (x, y) + h(x, y) = f (x, y), we conclude that (57) is equivalent to an equilibrium problem of the bifunction f 1 defined by (61). Finally, it is necessary to show that f 1 is monotone. Indeed, from the definition of f 1 we have f 1 (x, y) + f 1 (y, x) = −(y − x)T (A + B)(y − x) ≤ 0. Here, we use the fact that A + B is positive semidefinite by virtue of (59).
2 ⊓
Note that matrices A1 and B1 in Lemma 7 are not unique. There are many possible choices of A1 and B1 such that f 1 defined by (61) is still monotone. Now, we need to estimate the constants M and L indicated in Theorems 1 and 2 that will be used in our implementation. Since c is subdifferentiable on C g , from the definition of f 1 , we can explicitly compute ∂2 f 1 (x, y) = (A + B)x + By + a + ∂c(y). Hence, ∥∂2 f 1 (u k , x k ) − ∂2 f 1 (x k , x k )∥ = ∥(A + B)(u k − x k )∥
≤ ∥A + B∥∥x k − u k ∥, ∀u k , x k ∈ C g .
This inequality shows that ∂2 f 1 (·, x k ) is uniformly Lipschitz continuous with the same Lipschitz constant L := ∥A + B∥ on C g . Moreover, since C g is bounded and ∂c(x) is given by (61), ∂2 f 1 (·, x k ) is bounded on C g by a positive constant M defined as ¯ . M := (∥A + B∥ + ∥B∥)Mx + max {max {∥ξ ∥ | ξ ∈ ∂c(x)} | x ∈ C R (x)}
123
(62)
J Glob Optim g
g
g
g
Here, Mx := 21 (∥xmax +xmin ∥+∥xmax −xmin ∥) is the diameter of the feasible set C g . Besides, g ¯ := {x ∈ Rn x ∥x∥ ≤ R} with R := ∥xmax − it is easy to check that the box C g ⊆ B R (x) g g 1 g xmin ∥∞ and x¯ := 2 (xmin + xmax ). With these choices of R and x, ¯ we have C R (x) ¯ ≡ C g. These constants will be used in the implementation of Algorithms 1 and 2. It remains to express Algorithms 1 and 2 for this specific application. Three main steps of Algorithms 1 and 2 are specified as follows. 1 0 g g 1. Projection: u k := argmin ∥x − x¯ − β1k s k ∥2 | xmin ≤ x ≤ xmax . This problem is indeed a convex quadratic program. 2. Convex program: Problem (35) (resp., (43)) is reduced to the following convex program: ! 4 1 T g g min x Hk x + h kT x + c(x) | xmin ≤ x ≤ xmax , (63) 2 3.
where Hk := B + βk I and h k := [(A + B) − βk I ]u k + a. Subgradient: Problem of calculating vector w k ∈ −∂2 f 1 (x k , x k ) is solved explicitly by taking: w k := −(A + 2B)x k − a − ξ k ,
(64)
where ξ k ∈ ∂c(x k ) defined by (61). The dual step s k is updated by s k+1 := s k + w k .
The parameter √βk is chosen by βk = ∥A + B∥ in Algorithm 1 for all k, and is computed by βk := 2M k + 1 in Algorithm 2. To terminate Algorithms 1 and 2 we use the stopping R criterion ∥x¯ n+1 − x¯ n ∥ ≤ ε with a tolerance ε = 10−4 for both algorithms. We implement Algorithms 1 and 2 in Matlab 7.11.0 (R2010b) running on a PC Desktop Intel(R) Core(TM)2 Quad CPU Q6600 with 2.4 GHz, 3Gb RAM. We use quadprog, a built-in Matlab solver for convex quadratic programming, to solve the projection problem at Step 1 and Ipopt package (a C++ open source software at http://www.coin-or.org/Ipopt/ [35]) to solve the convex program at Step 2. Note that the convex programs (35) and (43) are nonsmooth. However, Ipopt still works well for these problems. We solve the problem which corresponds to the first model in [6], where three companies (n c = 3) are considered. The first company F1 possesses one generating unit {1}, the second company F2 has two generating units {2, 3} and the third one, F3 , has three generating units {4, 5, 6}. The computational results of this model are reported in Table 3. Here, the notation in this table includes: co#, gu.# and Prof. stand for the coalition, generating unit and profit, respectively; x j and xic are the total power of the generating unit j and the company i, respectively. Table 3 The total power and profit made by three companies Companies Algorithm 1
Algorithm 2
xic
xic
co# gu.# x j
Prof.[$/h] x j
ExtraGrad[29] Prof.[$/h] x j
xic
Prof.[$/h]
F1
1
46.6296 46.6296 4397.51
46.0871 46.0871 4334.75
F2
2
32.1091
31.6670
32.1468
3
15.0358 47.1449 4480.97
15.9337 47.6007 4511.42
15.0011 47.1479 4477.99
4
24.8218
22.6606
25.1062
5
10.8560
10.1875
10.8537
6
11.1277 46.8056 4395.13
14.1755 47.0236 4402.89
10.8545 46.8143 4392.72
F3
46.6524 46.6524 4396.42
123
J Glob Optim Table 4 The performance information of three algorithms Algorithm 1 iter. error 4416
Algorithm 2 cputime[s] iter. error
9.9972e-05 164.20
6850
ExtraGrad[29] cputime[s] iter. error
9.9992e-05 241.22
2101
cputime[s]
9.9939e-05 185.17
To compare, we also implement Algorithm 1 of the extragradient methods in [29] for this example. The computational results are also shown in Table 3 with the stopping criterion ∥x n+1 − x n ∥ ≤ 10−4 . The results reported by three algorithms are almost similar to each other. The performance of three algorithms are reported in Table 4, where iter indicates the number of iterations, error is the norm ∥x¯ n+1 − x¯ n ∥ (or ∥x n+1 − x n ∥) and cputime is the CPU time in second. From Table 4 we see that the number of iterations of Algorithms 1 and 2 is much smaller than the number of iterations n ε in the worst case. According to Theorems 1 and 2, these numbers are 96 × 106 and 4.389 × 1018 for Algorithms 1 and 2, respectively, in this example (if β is chosen by β = L in Algorithm 1). The number n ε crucially depends on the estimations of the constants L , M, Mx , the radius R and the center point x. ¯ In this example, these constants are estimated quite roughly which lead to very large values of n ε . The computational time of the ExtraGrad algorithm is greater than of Algorithm 1 even though the ExtraGrad algorithm requires fewer iterations. This happens because there are two general convex programs need to be solved at each iteration in the ExtraGrad algorithm instead of one as in Algorithm 1. Acknowledgments The authors would like to thank the anonymous referees and the editor for their comments and suggestions that helped to improve the presentation of the paper. This research was supported in part by NAFOSTED, Vietnam, Research Council KUL: CoE EF/05/006 Optimization in Engineering(OPTEC), GOA AMBioRICS, IOF-SCORES4CHEM, several PhD/postdoc & fellow grants; the Flemish Government via FWO: PhD/postdoc grants, projects G.0452.04, G.0499.04, G.0211.05, G.0226.06, G.0321.06, G.0302.07, G.0320.08 (convex MPC), G.0558.08 (Robust MHE), G.0557.08, G.0588.09, research communities (ICCoS, ANMMM, MLDM) and via IWT: PhD Grants, McKnow-E, Eureka-Flite+EU: ERNSI; FP7-HD-MPC (Collaborative Project STREP-grantnr. 223854), Contract Research: AMINAL, and Helmholtz Gemeinschaft: viCERP; Austria: ACCM, and the Belgian Federal Science Policy Office: IUAP P6/04 (DYSCO, Dynamical systems, control and optimization, 2007–2011).
References 1. Blum, E., Oettli, W.: From optimization and variational inequality to equilibrium problems. Math. Student 63, 127–149 (1994) 2. Bruck, R.E.: On the weak convergence of an Ergodic iteration for the solution of variational inequalities for monotone operators in Hilbert space. J. Math. Anal. Appl. 61, 159–164 (1977) 3. Chinchuluun, A., Pardalos, P.M., Migdalas, A., Pitsoulis, L. (eds.): Pareto Optimality, Game Theory and Equilibria. Springer, Berlin (2008) 4. Cohen, G.: Auxiliary problem principle and decomposition of optimization problems. J. Optim. Theory Appl. 32, 277–305 (1980) 5. Cohen, G.: Auxiliary principle extended to variational inequalities. J. Optim. Theory Appl. 59, 325–333 (1988) 6. Contreras, J., Klusch, M., Krawczyk, J.B.: Numerical solutions to Nash-Cournot equilibria in coupled constraint electricity markets. IEEE Trans. Power Syst. 19(1), 195–206 (2004) 7. Facchinei, F., Pang, J.-S.: Finite-Dimensional Variational Inequalities and Complementarity Problems, vol I. II. Springer, New York (2003)
123
J Glob Optim 8. Flam, S.D., Antipin, A.S.: Equilibrium programming using proximal-like algorithms. Math. Program. 78, 29–41 (1997) 9. Giannessi, F., Maugeri, A., Pardalos, P.M. (eds.): Equilibrium Problems: Nonsmooth Optimization and Variational Inequality Models. Kluwer, Dordrecht (2004) 10. Konnov, I.V.: Combined relaxation methods for variational inequalities. Springer-Verlag, Berlin (2001) 11. Konnov, I.V., Kum, S.: Descent methods for mixed variational inequalities in Hilbert spaces. Nonlinear Anal. 47, 561–572 (2001) 12. Konnov, I.V.: Generalized convexity and related topics. In: Konnov, I.V., Luc, D.T., Rubinov, (eds.) A.M. Combined Relaxation Methods for Generalized Monotone Variational Inequalities, pp. 3–331. Springer, Berlin (2007) 13. Korpelevich, G.M.: Extragradient method for finding saddle points and other problems. Matecon 12, 747– 756 (1976) 14. Lalitha, C.S.: A note on duality of generalized equilibrium problem. Optim. Lett. 4(1), 57–66 (2010) 15. Li, S.J., Zhao, P.: A method of duality for mixed vector equilibrium problem. Optim. Lett. 4(1), 85– 96 (2010) 16. Maiorano, A., Song, Y.H., Trovato, M.: Dynamics of noncollusive oligopolistic electricity markets. In: Proceedings IEEE Power Engineering Society Winter Meeting, pp. 838–844, Singapore Jan (2000) 17. Martinet, B.: Régularisation d’inéquations variationelles par approximations successives. Revue Française d’Automatique Et d’Informatique Recherche Opérationnelle 4, 154–159 (1970) 18. Mastroeni, G.: On auxiliary principle for equilibrium problems. Publicatione del Dipartimento di Mathematica DellUniversita di Pisa 3, 1244–1258 (2000) 19. Mastroeni, G.: Gap function for equilibrium problems. J. Global Optim. 27(4), 411–426 (2003) 20. Moudafi, A.: Proximal point algorithm extended to equilibrium problem. J. Nat. Geom. 15, 91–100 (1999) 21. Muu, L.D., Oettli, W.: Convergence of an adaptive penalty scheme for finding constrained equilibria. Nonlinear Anal. 18(12), 1159–1166 (1992) 22. Muu, L.D., Quoc, T.D.: Regularization algorithms for solving monotone Ky Fan inequalities with application to a Nash-Cournot equilibrium model. J. Optim. Theory Appl. 142(1), 185–204 (2009) 23. Nemirovskii, A.S.: Effective iterative methods for solving equations with monotone operators. Ekon. Matem. Met. (Matecon) 17, 344–359 (1981) 24. Nesterov, Y.: Dual extrapolation and its applications to solving variational inequalities and related problems. Math. Program. Ser. B 109(2–3), 319–344 (2007) 25. Nguyen, V.H.: Lecture Notes on Equilibrium Problems. CIUF-CUD Summer School on Optimization and Applied Mathematics. Nha Trang, Vietnam (2002) 26. Panicucci, B., Pappalardo, M., Passacantando, M.: On solving generalized Nash equilibrium problems via optimization. Optim. Lett. 3(3), 419–435 (2009) 27. Pardalos, P.M, Rassias, T.M., Khan, A.A. (eds.): Nonlinear Analysis and Variational Problems. Springer, Berlin (2010) 28. Quoc, T.D., Muu, L.D.: Implementable quadratic regularization methods for solving pseudomonotone equilibrium problems. East West J. Math. 6(2), 101–123 (2004) 29. Quoc, T.D., Muu, L.D., Nguyen, V.H.: Extragradient algorithms extended to equilibrium problems. Optimization 57(6), 749–776 (2008) 30. Rockafellar, R.T.: Convex Analysis. Princeton University Press, Princeton (1970) 31. Rockafellar, R.T.: Monotone operators and the proximal point algorithm. SIAM J. Control Optim. 14, 877– 898 (1976) 32. Taskar, B., Lacoste-Julien, S., Jordan, M.I.: Structured prediction, dual extragradient and Bregman projections. J. Mach. Learn. Res. 7, 1627–1653 (2006) 33. Van, N.T.T., Strodiot, J.J., Nguyen, V.H.: The interior proximal extragradient method for solving equilibrium problems. J. Global Optim. 44(2), 175–192 (2009) 34. Van, N.T.T., Strodiot, J.J., Nguyen, V.H.: A bundle method for solving equilibrium problems. Math. Program. 116, 529–552 (2009) 35. Wachter, A., Biegler, L.T.: On the implementation of a primal-dual interior point filter line search algorithm for large-scale nonlinear programming. Math. Program. 106(1), 25–57 (2006) 36. Zhu, D.L., Marcotte, P.: An extended descent framework for variational inequalities. J. Optim. Theory Appl. 80, 349–366 (1994)
123
