WEAK AND STRONG CONVERGENCE OF PROX

NUMERICAL ALGEBRA, CONTROL AND OPTIMIZATION Volume 3, Number 2, June 2013

doi:10.3934/naco.2013.3.353 pp. 353–366

WEAK AND STRONG CONVERGENCE OF PROX-PENALIZATION AND SPLITTING ALGORITHMS FOR BILEVEL EQUILIBRIUM PROBLEMS

Zaki Chbani and Hassan Riahi Cadi Ayyad University, Faculty of Sciences Semlalia Mathematics, 40000 Marrakech, Morroco

(Communicated by Soon-Yi Wu ) Abstract. The aim of this paper is to obtain, in a Hilbert space H, the weak and strong convergence of a penalty proximal algorithm and a splitting one for a bilevel equilibrium problem: find x ∈ SF such that G(x, y) ≥ 0 for all y ∈ SF , where SF := {y ∈ K : F (y, u) ≥ 0 ∀u ∈ K}, and F, G : K × K −→ R are two bifunctions with K a nonempty closed convex subset of H. In our framework, results of convergence generalize those recently obtained by Attouch et al. (SIAM Journal on Optimization 21, 149-173 (2011)). We show in particular that for the strong convergence of the penalty algorithm, the geometrical condition they impose is not required. We also give applications of the iterative schemes to fixed point problems and variational inequalities.

1. Introduction. Let (H, k.k) be a Hilbert space, K ⊂ H a nonempty closed convex set. For a bifunction G : K × K −→ R that verifies G(x, x) = 0 for all x ∈ K, the equilibrium problem (for short EP ) is defined as follows: Find x ∈ K such that G(x, y) ≥ 0 for all y ∈ K.

(1.1)

Let F : K × K −→ R be another bifunction, we consider the following bilevel equilibrium problem, shortly BEP , introduced by Chadli, Chbani and Riahi [4] and called viscosity principle for equilibrium problems, which general form is given by: find x ∈ SF such that G(x, y) ≥ 0 for all y ∈ SF , (1.2) where SF := {y ∈ K : F (y, u) ≥ 0 ∀u ∈ K}, the set of solutions to the equilibrium problem associated to F , is taken as the constraints set of BEP . In other words, BEP is an equilibrium problem with K = SF . S will denote the set of solutions of BEP . In [4, section 6], the authors study the sequence {xεn } generated by the approximate problems EPn , find xεn ∈ K such that F (xεn , y) + εn G(xεn , y) ≥ 0, for each y ∈ K. This perturbation scheme can be seen as a penalty method for optimization problems with parameters ε1n and penalty function F . Theorems 6.1-6.3 in [4] give some conditions under which the viscosity principle generates a family of solutions of perturbed problems EPn accumulating to points x which are not only belonging in 2010 Mathematics Subject Classification. Primary: 46N10, 65K15, 90C33; Secondary: 47H05, 65K10. Key words and phrases. Bilevel equilibrium problem, penalty proximal algorithm, splitting algorithm, fixed point, weak and strong convergence.

353

354

ZAKI CHBANI AND HASSAN RIAHI

SF , but also verifying a selection principle G(x, y) ≥ 0, ∀y ∈ SF . Moudafi [10] has adapted to BEP the proximal algorithm generalized by himself to equilibrium problems [8]. He shows that the bilevel problem BEP can be solved by a proximal method, iteratively applied to the perturbed equilibrium problem EPn . To show weak convergence of the sequence {xn } generated by the proximal iteration, to the solution of BEP , he needs to assume that kxn+1 −xn k = ◦(εn ). This assumption is not realistic at all, because one cannot a priori estimate kxn+1 − xn k. More recently, H. Attouch and al. [1], studied variational inclusion of the form Ax + ℵC (x) 3 0,

(1.3)

H

where H is a real Hilbert space, A : H → 2 is a maximal monotone operator and ℵC is the outward normal cone to a closed convex set C ⊂ H. Given ψ : H → R ∪ {+∞} which acts as a penalization function with respect to the constraint x ∈ C, and a penalization parameter βn , they consider two algorithms, a diagonal proximal one: xn = [I + λn (A + βn ∂ψ)]−1 xn−1 , and a splitting one: yn = (I + λn A)−1 xn−1 xn = (I + λn βn ∂ψ)−1 yn . Under certain hypotheses on the sequences λn and βn , and assuming a geometric condition involving the Fenchel Pn conjugate of ψ, they Pn prove that the sequence of weighted averages zn = τ1n k=1 λk xk where τn = k=1 λk weakly converges to a solution of (1.3) and that {xn } converges strongly in the case where A is strongly monotone. If A = ∂ϕ for some proper lower-semicontinuous convex function ϕ : H → R ∪ {+∞}, then the sequence {xn } itself converges weakly to a solution of (1.3). In this paper, we first consider a proximal method, iteratively applied to the parameterized family of bifunctions Fβ (x, y) = βF (x, y) + G(x, y) where β varies along the iterations. Namely, if xn , (n = 0) is the current iterate and βn the current parameter, then compute a solution xn+1 ∈ K of the regularized problem 1 βn F (xn+1 , y) + G(xn+1 , y) + hxn+1 − xn , y − xn+1 i ≥ 0, ∀y ∈ K, (1.4) λn where x0 ∈ K, {λn } and {βn } are two positive sequences. Following [1] and under the same geometric assumption but reinforcing hypothesis Σn λn = +∞ by lim inf n λn > 0, we prove (see Theorem 3.1) that not the average but the sequence itself {xn } weakly converges to a solution of (1.2). Then, under an assumption of strong monotonicity but completely getting rid of the geometric assumption considered in [1], we show (see Theorem 3.2) that the sequence converges strongly to the unique solution of BEP . Secondly, a splitting algorithm is considered for BEP : λn G(yn+1 , y) + hyn+1 − xn , y − yn+1 i ≥ 0 ∀y ∈ K, λn βn F (xn+1 , y) + hxn+1 − yn+1 , y − xn+1 i ≥ 0 ∀y ∈ K. A similar algorithm was introduced by Passty [12] to approximate a zero of the sum of maximal monotone operators, and extended by Moudafi [9], for solving F (x, y) + G(x, y) ≥ 0 ∀y ∈ K. However, our viscosity term βn ensures convergence to a solution of the bilevel equilibrium BEP. Two results of weak and strong convergence

PENALIZATION AND SPLITTING ALGORITHMS FOR BE

355

are given, see Theorems 4.1 and 4.2. The latter is obtained by weakening the geometric assumption in [1]. As applications, we consider first the case where the lower-level problem of (1.2) is in finding a fixed point for nonexpansive mapping, namely it is Find x∗ ∈ F ix(T ) such that h(I − V )x∗ , x − x∗ i ≥ 0 ∀x ∈ F ix(T ), T, V : K → K being nonexpansive mappings. Maing´e and Moudafi propose in [7], a hybrid iterative method for solving this problem. By taking G(x, y) = h(I − V )x, y − xi, this problem can be seen as a BEP . Secondly, the case where le leaderlevel problem is a set valued variational inequality is regarded and the variational inclusion (1.3) is found as a special case. 2. Preliminaries. In the sequel, we will assume that the bifunctions all satisfy the following usual conditions: (H1 ) for every x ∈ K F (x, x) = 0 (H2 ) F is monotone, i.e. for every x, y ∈ K F (x, y) + F (y, x) ≤ 0; (H3 ) limx↓0 F (tz + (1 − t)x, y) ≤ F (x, y) for any x, y, z ∈ K; (H4 ) for each x ∈ K, y 7→ F (x, y) is convex and lower semicontinuous. Lemma 2.1. (Minty’s lemma [3]) Let K be a nonempty closed convex subset of H and let F : K × K → R and consider the equilibrium problem (EP): find x ∈ K such that F (x, y) ≥ 0, ∀y ∈ K, and the associated dual problem (DEP): find x ∈ K such that F (y, x) ≤ 0, ∀y ∈ K. (i) If F satisfies [(H2 )], then each solution of (EP) is a solution of (DEP). (ii) Conversely, if F satisfies [(H1 ), (H3 )] and for each x ∈ K, y → F (x, y) is convex, then each solution of (DEP) is a solution of (EP). Lemma 2.2. [5] Let K be a nonempty closed convex subset of H and let F : K × K → R satisfying [(H1 ) − (H4 )]. Then the following are equivalent: (i) F is maximal: (x, u) ∈ K × H and F (x, y) ≤ hu, x − yi, ∀y ∈ K imply that F (y, x) + hu, x − yi ≥ 0, ∀y ∈ K; (ii) for each x ∈ H and λ > 0, there exists a unique z = JλF (x), called the resolvent of F at x such that λF (z, y) + hy − z, z − xi ≥ 0, ∀y ∈ K.

(2.1)

Notice that this notion of maximal monotonicity for bifunctions is inspired by those already known for monotone operators. In fact, inspired by [4, 3], since the maximal monotonicity of operators in a Hilbert is equivalent to the existence and uniqueness of the associated resolvent we used this notions for bi-functions. Let us mention, see [4, Lemma 2.1], that maximality of F is assured when F (x, x) ≥ 0 for each x ∈ K, F is upper hemicontinuous and convex in the second argument. Remark also that, x is a solution of (1.1) if, and only if, JλF (x) = x for all λ > 0 and xn+1 , the (n + 1)th iteration of (1.4), can be rewriting as: xn+1 = JλFnn (xn ), where Fn = βn F + G. We will also need the following technical lemma. Lemma 2.3. Let {an } be a sequence of real numbers that does not decrease at infinity, in the sense that there exists a subsequence {ank }k≥0 of (an ) which satisfies

356


ank < ank +1 for all k ≥ 0. Also consider the sequence of integers {σ(n)}n≥n0 defined by σ(n) := max{k ≤ n : ak < ak+1 }. (2.2) Then {σ(n)}n≥n0 is a nondecreasing sequence verifying limn→∞ σ(n) = ∞ and, for all n ≥ n0 aσ(n) < aσ(n)+1 and an ≤ aσ(n)+1 . (2.3) Proof. By the hypotheses on {an }, σ(n) is well defined and nondecreasing. Moreover, since there exists a subsequence {ank } such that ank < ank +1 , then σ(nk ) = nk , and thus limn→∞ σ(n) = limk→∞ σ(nk ) = +∞. From the definition of σ(n), we deduce that for all n ≥ n0 , aσ(n) < aσ(n)+1 . Let us prove the second part of (2.3). By definition of σ(n), we have σ(n) ≤ n for each n ≥ n0 . Fix n ≥ n0 and distinguish three cases: If σ(n) = n, the relation aσ(n) < aσ(n)+1 assures (2.3). If σ(n) = n − 1 or σ(n) + 1 = n, we obtain the equality in (2.3). If σ(n) < n − 1, then σ(n) + 1 ≤ n − 1. By (2.2) we easily observe that for each k between σ(n) + 1 and n − 1, ak ≥ ak+1 , i.e. aσ(n)+1 ≥ aσ(n)+2 ≥ · · · ≥ an−1 ≥ an ; which gives (2.3). We recall the following well known lemma of Opial, [11]. Lemma 2.4. Let H be a Hilbert space and {xn } a sequence such that there exists a nonempty set S ⊂ H verifying: • For every x ∈ S, limn kxn − xk exists; • If xν weakly converges to x ∈ H for a subsequence xν , then x ∈ S. Then, there exists x ∈ S such that {xn } weakly converges to x. Let Γ0 (H) denote the set of all proper lower-semicontinuous convex functions h : H → R ∪ {+∞}. Given h ∈ Γ0 (H) and x ∈ H, its Fenchel conjugate function h∗ : H → R ∪ {+∞} is defined by h∗ (x∗ ) := supy∈H {hx∗ , yi − h(y)}. The subdifferential of h at x ∈ H is defined by ∂h(x) := {x∗ ∈ H : hx∗ , y − xi + h(x) ≤ h(y), ∀ y ∈ X}. ∗ For x, x ∈ H, one has h(x) + h∗ (x∗ ) ≥ hx∗ , xi with equality if, and only if, ∗ x ∈ ∂h(x). For every u ∈ K, the function Fu∗ will denote the Fenchel conjugate of Fu (x) = F (u, x) if x ∈ K and Fu (x) = +∞ otherwise. Given a nonempty closed convex set C ⊂ H, its indicator function is defined as δC (x) = 0 if x ∈ C and +∞ otherwise. The support function of C at a point x∗ is σC (x∗ ) = supx∈C hx∗ , xi. The normal cone to C at x is {x∗ ∈ H : hx∗ , u − xi ≤ 0 for all u ∈ C} if x ∈ C ℵC (x) = ∅ otherwise. ∗ Observe that δC = σC and ∂δC = ℵC . We suppose in the sequel that ∀y ∈ K, ∂Gy (y) 6= ∅, and a condition that can ensure ∂Gu + ℵSF = ∂(Gu + δSF ), which is realized if we assume that int(K) ∩ SF 6= ∅ (int(K) is the k · k-interior of K in H) or more generally, the subspace spanned by K − SF is closed. We also need the following geometric condition:


(A)

357

i h ∀u ∈ SF , ∀p ∈ ℵSF (u), Σn λn βn Fu∗ ( β2pn ) − σSF ( β2pn ) < +∞,

where the positive sequences {λn } and {βn } satisfy conditions that will be proposed in the appropriate convergence results below. 3. Prox-penalization algorithm. Let {xn } be the sequence verifying algorithm (1.4): xn+1 = JλFnn xn . We have the following Lemma: Lemma 3.1. Take u ∈ SF , w ∈ (∂Gu (u) + ℵSF )(u) and let p ∈ ℵSF (u) be such that w − p ∈ ∂Gu (u). Then, for each n ≥ 1, the following inequality holds: kxn+1 − uk2 + λn βn F (u, xn+1 ) ≤ kxn − uk2 − kxn+1 − xn k2 + λn βn [Fu∗ ( β2pn ) − σSF ( β2pn )] + 2λn hw, u − xn+1 i. Proof. Since w − p ∈ ∂Gu (u), we have G(u, xn+1 ) + hw − p, u − xn+1 i ≥ 0

(3.1)

and by taking y = u in (1.4), we also have λn βn F (xn+1 , u) + λn G(xn+1 , u) + hxn+1 − xn , u − xn+1 i ≥ 0.

(3.2)

Thanks to monotonicity of G, (3.1) and (3.2) give λn βn F (xn+1 , u) + λn hw − p, u − xn+1 i + hxn+1 − xn , u − xn+1 i ≥ 0.

(3.3)

Since 2hxn+1 − xn , u − xn+1 i = kxn − uk2 − kxn+1 − uk2 − kxn+1 − xn k2 . Then, coming back to (3.3) and setting an = kxn − uk2 , we get an+1 ≤ an − kxn − xn+1 k2 + 2λn βn F (xn+1 , u) + 2λn hw − p, u − xn+1 i. Using monotonicity of F , we obtain an+1 + λn βn F (u, xn+1 ) ≤ an − kxn − xn+1 k2 − λn βn F (u, xn+1 ) +2λn hp, xn+1 − ui + 2λn hw, u − xn+1 i ≤ an − kxn − xn+1 k2 + λn βn [h β2pn , xn+1 i − F (u, xn+1 ) −h β2pn , ui] + 2λn hw, u − xn+1 i, and finally, using that p ∈ ℵSF (u), i.e. δSF (u) + σSF (p) = hp, ui, we obtain an+1 + λn βn F (u, xn+1 ) ≤ an − kxn − xn+1 k2 +λn βn [Fu∗ ( β2pn ) − σSF ( β2pn )] + 2λn hw, u − xn+1 i.

Corollary 3.1. Under hypothesis (A), we have for each u ∈ S: (i) limn→∞ kxn − uk exists; (ii) Σn kxn − xn+1 k2 < +∞; (iii) Σn λn βn F (u, xn+1 ) < +∞. Proof. By taking w = 0, and u ∈ S, inequality in the previous lemma yields for each n ≥ 1 an+1 + λn βn F (u, xn+1 ) ≤ an − kxn − xn+1 k2 +λn βn [Fu∗ ( β2pn ) − σSF ( β2pn )].

358


Summing from n = 0 to ∞ and using (A), we obtain: +∞ +∞ X X 2p 2p 2 λn βn F (u, xn+1 ) + kxn − xn+1 k ≤ a0 + λn βn [Fu∗ ( ) − σSF ( )] < +∞, β β n n n=0 n=0

which assures (ii) and (iii), while (i) follows from (A) and an+1 ≤ an + λn βn [Fu∗ ( β2pn ) − σSF ( β2pn )]. 3.1. Weak convergence. Theorem 3.1. Suppose that conditions (H1 )−(H4 ) are verified by F and G. Under hypothesis (A), and by assuming that lim inf n λn > 0, βn → +∞, we have that the sequence {xn } weakly converges to a solution of BEP . Proof. By Lemma 2.4 and part (i) of Corollary 3.1, it suffices to prove that every weak cluster point x ¯ of the sequence {xn } lies in S. By using (3.2) and monotonicity of F and G, we have for every u ∈ SF λn βn F (u, xn+1 ) + λn G(u, xn+1 ) ≤ hxn − xn+1 , xn+1 − ui thanks to (i) and (ii) of Corollary 3.1, hxn − xn+1 , xn+1 − ui →n→+∞ 0. By exploiting (iii) of the same Corollary, we obtain that lim supn→∞ λn G(u, xn+1 ) ≤ 0. Both of assumption that lim inf n λn > 0 and the usual condition (H4 ) give G(u, x ¯) ≤ 0. Lemma (2.1) allows to conclude that G(¯ x, u) ≥ 0 for all u ∈ SF . It remains to show that x ¯ ∈ SF . Recall that, since ∂Gy (y) 6= ∅, one can find x? (y) ∈ H such that for every u ∈ K, G(y, u) ≥ hx? (y), u − yi ≥ −kx? (y)k · ky − uk. Thus there exists γ(y) > 0 such that G(y, u) ≥ −γ(y)ky − uk for every u ∈ K.

(3.4)

Returning to (1.4), we can write F (y, xn+1 ) ≤ ≤

− β1n G(y, xn+1 ) + γ(y) βn ky − xn+1 k +

1 βn λn hxn+1 − xn , y − xn+1 i 1 βn λn kxn+1 − xn kky − xn+1 k.

Passing to the limit, using that the sequence {xn } is bounded, βn → +∞,

lim inf λn > 0 n

and yet thanks to (H4 ), we deduce that F (y, x ¯) ≤ 0 for all y ∈ K. Lemma 2.1 leads to x ¯ ∈ SF . 3.2. Strong convergence. We will show that in this case, the algorithm strongly converges without the need of the hypothesis (A), what is an improvement of [1, Theorem 2.4 ]. Theorem 3.2. Let F and G be two bifunctions satisfying (H1 ) − (H4 ) with SF 6= ∅ and G ρ−strongly monotone: G(u, v) + G(v, u) ≤ −ρku − vk2 , ∀u, v ∈ K. Suppose that ∞ X λn → 0, λn = +∞, βn → +∞ and lim inf λn βn > 0. (3.5) 0

n→∞

Then, the sequence {xn } generated by the algorithm (1.4) converges strongly to the unique solution u of BEP . Proof. Let us first remark that the solution set of BEP is nonempty, see [4].


359

Step 1. (Uniqueness) Uniqueness of solution for BEP follows from strong monotonicity of G. Indeed, if u and v are two solutions for BEP , then G(u, v) ≥ 0 and G(v, u) ≥ 0. Using strong monotonicity of G, we have −ρku − vk2 ≥ G(u, v) + G(v, u) ≥ 0; and, then u = v. Step 2. (Boundedness) Since {xn } is generated by (1.4), we have for each y ∈ K 0 ≤ λn βn F (xn+1 , y)+λn G(xn+1 , y)+

1 ky − xn k2 − ky − xn+1 k2 − kxn − xn+1 k2 . 2 (3.6)

Fix y ∈ SF , then for each n ≥ 0, 0 ≤ λn G(xn+1 , y) +

1 ky − xn k2 − ky − xn+1 k2 − kxn − xn+1 k2 . 2

(3.7)

Set an (y) = 21 ky − xn k2 . Using strong monotonicity of G and relation (3.4), we √ deduce by putting γ 0 = 2γ p −γ 0 an+1 (y) ≤ G(y, xn+1 ) ≤ −G(xn+1 , y) − ρan+1 (y). Thus G(xn+1 , y) ≤

p

an+1 (y)(γ 0 − ρ

p

an+1 (y)).

(3.8)

Let us show by induction that for each n ≥ 0 an (y) ≤ max{(

γ0 2 ) , kx0 − yk2 } := M. ρ

For n = 0, we have obviously a0 (y) ≤ M . Suppose that until n we have an (y) ≤ M . 0 2 0 2 We have to distinguish two cases : either an+1 (y) ≤ γρ or an+1 (y) > γρ . 0 2 The first case assures an+1 (y) ≤ M ; let us suppose an+1 (y) > γρ , then p γ 0 − ρ an+1 (y) < 0, and by (3.8) we obtain G(xn+1 , y) < 0. Return to (3.6), we deduce 0 ≤ an (y) − an+1 (y). Thus an+1 (y) ≤ an (y) ≤ M. We conclude an (y) ≤ M for each n ; which means the sequence {an (y)} is bounded and thus {xn } is also bounded. Step 3. (Strong convergence) We use two possible cases to confirm strong convergence of the whole sequence {xn } to u, the unique solution of BEP . (i) Suppose there is n0 ≥ 0 such that for each n ≥ n0 , an (u) ≥ an+1 (u). In this case {an (u)} converges and limn→∞ (an (u) − an+1 (u)) = 0. Set y = u in (3.6) and use u solution of the first level, λn → 0 and G(., y) lower semi-continuous, we get kxn − xn+1 k2 → 0 and − λn G(xn+1 , u) ≤ an (u) − an+1 (u). By summing infinitely the second relations, we deduce P∞ 0 (−λn G(xn+1 , u)) ≤ a0 (u).

360


Using Σn λn = ∞, we obtain lim inf n→∞ −G(xn+1 , u) ≤ 0. G is ρ-strongly monotone, then an+1 (u) =

1 1 1 kxn+1 − uk2 ≤ − G(xn+1 , u) − G(u, xn+1 ). 2 ρ ρ

Passing to the limit we have lim an+1 (u)

n→∞

=

lim inf an+1 (u) n→∞

1 1 lim inf (−G(xn+1 , u)) + lim sup(−G(u, xn+1 )) ρ n→∞ ρ n→∞ 1 − lim inf G(u, xn+1 ). ρ n→∞

≤ ≤

To deduce the convergence of the sequence {an+1 (u)} to zero it is enough to verify that lim inf n→∞ G(u, xn+1 ) ≥ 0. Let x be a weak cluster point of {xn }, which exists since {xn } is bounded, i.e. x = w − limn∈I⊂N xn . We confirm x ∈ SF . Indeed, let y be an arbitrary point of K, using (3.4) and that F, G are monotone, then for each n ≥ 0, F (y, xn+1 ) ≤

γ(y) 1 ky − xn+1 k + (an (y) − an+1 (y)). βn λ n βn

Returning to the relationship an (y) − an+1 (y) = an (u) − an+1 (u) + hxn − xn+1 , u − yi to infer that an (y) − an+1 (y) converges to zero, since an (u) − an+1 (u) → 0 and kxn − xn+1 k → 0. By using weak lower semicontinuity of F (y, ·), boundedness of {xn }, and convergence to ∞ of {βn } and lim inf n λn βn > 0, we deduce F (y, x) ≤ lim inf F (y, xn+1 ) ≤ 0 n∈I

so that F (y, x) ≤ 0. This is true for all y in K, it is claimed that x is a solution of the dual problem of the lower-level equilibrium problem. Using Minty’s lemma, we deduce x ∈ SF . Thus, we have for any subsequence {xn }n∈I , lim inf n∈I G(xn+1 , u) ≥ 0. It confirms that lim inf n→∞ G(xn+1 , u) ≥ 0; which ensures limn→∞ an+1 (u) = 0, and thus strong convergence of xn to the unique solution u of BEP . (ii) In the contrary to (i), suppose the existence of an increasing sequence {kn } such that for every n ≥ 0, akn +1 (u) > akn (u). By Lemma 2.3, for σn := max{k ≤ n : ak (u) < ak+1 (u)}, one has (σn )n≥n0 nondecreasing, limn→∞ σn = ∞ and, for all n ≥ n0 , for some n0 > 0, aσn < aσn +1 and an ≤ aσn +1 . Return to (3.6) for y = u and replace n by σn , we get for each n > n0 0 ≤ λσn G(xσn +1 , u) + aσn (u) − aσn +1 (u) < λσn G(xσn +1 , u), which yields to G(xσn +1 , u) ≥ 0, and thus lim supn→∞ −G(xσn +1 , u) ≤ 0. Return also to ρ-strong monotonicity of G and passing to the limit we have


lim sup aσn +1 (u) ≤ n→∞

≤ ≤

361

1 1 inf G(u, xσn +1 ) ρ lim sup (ρaσn +1 (u) + G(u, xσn +1 )) − ρ lim n→∞ n→∞ 1 1 inf G(u, xσn +1 ) ρ lim sup (−G(xσn +1 , u)) − ρ lim n→∞ n→∞ 1 − ρ lim inf G(u, xσn +1 ). n→∞

Under boundedness of {xn }, and using lines of the proof in (i) one can confirm lim inf n→∞ G(u, xσn +1 ) ≥ 0. Thus limn→∞ aσn +1 (u) = 0. On the other hand, since an (u) ≤ aσ(n)+1 (u) for each n ≥ n0 , we confirm limn an = 0, which guarantee strong convergence of the whole sequence {an } to u. 4. Splitting proximal algorithm. Let {βn } and {λn } be sequences of positive numbers. In this section we study the alternating algorithm given by: λn G(yn+1 , y) + hyn+1 − xn , y − yn+1 i ≥ 0 ∀y ∈ K (4.1) λn βn F (xn+1 , y) + hxn+1 − yn+1 , y − xn+1 i ≥ 0 ∀y ∈ K Just as in Lemma 3.1, we begin by establishing the following estimation result. Lemma 4.1. For u ∈ SF , take w ∈ (∂Gu + ℵSF )(u) so that w = v + p for some v ∈ ∂Gu (u) and p ∈ ℵSF (u). For each n ≥ 1 the following inequality holds: 1 kxn+1 − uk2 − kxn − uk2 + kxn − yn+1 k2 + kxn+1 − yn+1 k2 + λn βn F (u, xn+1 ) 2 ≤ 2λn hw, u − xn+1 i + λn βn [Fu∗ ( β2pn ) − σSF ( β2pn )] + 2λ2n kvk2 . Proof. Since v ∈ ∂Gu (u), we have G(u, yn+1 ) ≥ hv, yn+1 − ui. By monotonicity of G and the first inequality in (4.1), we obtain hxn − yn+1 , yn+1 − ui ≥ λn hv, yn+1 − ui, that is equivalent to kxn − uk2 − kxn − yn+1 k2 − kyn+1 − uk2 ≥ 2λn hv, yn+1 − ui.

(4.2)

On the other hand, the second inequality in (4.1) gives, thanks to the monotonicity of F , kyn+1 − uk2 − kxn+1 − yn+1 k2 − kxn+1 − uk2 ≥ 2λn βn F (u, xn+1 ). Adding inequalities (4.2) and (4.3) we deduce that kxn − uk2 − kxn+1 − uk2

≥

kxn − yn+1 k2 + kxn+1 − yn+1 k2 +2λn hv, yn+1 − ui + 2λn βn F (u, xn+1 ).

But 2λn hv, yn+1 − ui = 2λn hv, xn+1 − ui + 2λn hv, yn+1 − xn+1 i ≥ 2λn hv, xn+1 − ui − 2λ2n kvk2 − 21 kxn+1 − yn+1 k2 . Replacing in the previous inequality we obtain kxn − uk2 − kxn+1 − uk2 ≥ kxn − yn+1 k2 + 12 kxn+1 − yn+1 k2 + 2λn hv, xn+1 − ui +2λn βn F (u, xn+1 ) − 2λ2n kvk2 , and then, the same reasoning as in Lemma 3.1 gives the result. Corollary 4.1. Under hypothesis (A), if Σn λ2n < ∞ then for each u ∈ S: (i) limn→∞ kxn − uk exists ;

(4.3)

362


(ii) the series Σn kxn − yn+1 k2 and Σn kxn − yn k2 are convergent; (iii) Σn λn βn F (u, xn+1 ) < +∞. Proof. Take u ∈ S, then 0 ∈ (∂Gu + ℵSF )(u). Overwriting the inequality in Lemma 4.1, we obtain the result. 4.1. Weak convergence. Theorem 4.1. Suppose that conditions (H1 )−(H4 ) are verified by F and G. Under hypothesis (A), and by assuming that lim inf n λn βn > 0, Σn λn = ∞ and Σλ2n < +∞, then the sequence {xn } generated by Algorithm 4.1 weakly converges to a solution of BEP . Proof. Let x be a weak cluster point of {xn }. Thanks to monotonicity of F , the second inequality in(4.1) gives for all y ∈ K 1 hxn+1 − yn+1 , y − xn+1 i ≥ F (y, xn+1 ). λ n βn Using that lim inf n λn βn > 0, kxn − yn k → 0 and xn weakly converges to x for a subsequence, we obtain by passing to the limit 0 ≥ F (y, x). Minty’s lemma allows us to say that x ∈ SF . On the other hand, by kxn − yn k → 0, we have that x is also a weak cluster point of {yn }. Take v ∈ SF , the first inequality in (4.1) leads to kxn − vk2 − kyn+1 − xn k2 − kyn+1 − vk2 ≥ 2λn G(v, yn+1 ). The second one gives kyn+1 − vk2 − kyn+1 − xn+1 k2 − kxn+1 − vk2 ≥ 0. By summing the two inequalities, it follows 2λn G(v, yn+1 ) ≤ kxn − vk2 − kxn+1 − vk2 , so that Σn λn G(v, yn+1 ) < ∞. The assumption that Σn λn = ∞ induces G(v, x) ≤ lim inf G(v, yn+1 ) ≤ 0 and at the end, by means of Minty’s Lemma, x ∈ S. By using (i) of Corollary (4.1) and Opial’s Lemma (2.4), we conclude that {xn } weakly converges to x. 4.2. Strong convergence. Here, we suppose that G is ρ−strongly monotone, the sequence xn defined by 4.1 strongly converges to the unique u ∈ S. We will prove this result by reducing the assumption (A) to (A’) ∀u ∈ SF , ∀p ∈ ℵSF (u), limn→+∞ λn βn [Fu∗ ( β2pn ) − σSF ( β2pn )] = 0. Theorem 4.2. Suppose that conditions (H1 ) − (H4 ) are verified by F , G, assumption (A’) is fulfilled by F , and in addition G is ρ−strongly monotone. By assuming moreover that Σn λn = +∞, limn λn = 0 and lim inf n λn βn > 0, we have that the sequence {xn } strongly converges to the unique solution of BEP . Proof. By taking y = u the unique solution of BEP , then u ∈ Sf and the second inequality in (4.1) gives kyn+1 − uk2 − kyn+1 − xn+1 k2 − kxn+1 − uk2 ≥ 0 and the first one leads to kxn − uk2 − kyn+1 − xn k2 − kyn+1 − uk2 ≥ −2λn G(yn+1 , u).


363

By posing an = kxn − uk2 and summing the last two inequalities, we obtain: an − an+1 ≥ kyn+1 − xn k2 + kyn+1 − xn+1 k2 − 2λn G(yn+1 , u),

(4.4)

then −2λn G(yn+1 , u) ≤ an − an+1 and consequently Σn − λn G(yn+1 , u) < +∞. The condition Σn λn = +∞ gives lim inf −G(yn+1 , u) ≤ 0.

(4.5)

n

By strong monotonicity of G, we have 1 1 kyn+1 − uk2 ≤ − G(yn+1 , u) − G(u, yn+1 ). ρ ρ

(4.6)

Passing to the limit and using (4.5), we obtain 1 lim inf kyn+1 − uk2 ≤ − lim inf G(u, yn+1 ). n ρ n

(4.7)

Before discussing the cases following the behaviour of the sequence {an }, we establish that {yn } is bounded. Let us suppose by contradiction that {yn } is unbounded. We can then construct a subsequence {ynk } such that kynk +1 − uk → +∞ and limk (−G(ynk +1 , u)) ≤ 0. Using (4.6) and (3.4), we have for each n γ 1 kyn+1 − uk2 ≤ − G(yn+1 , u) + kyn+1 − uk ρ ρ and then γ 1 γ2 (kyn+1 − uk − )2 ≤ (−(G(yn+1 , u)) + 2 . (4.8) 2ρ ρ 4ρ 2

γ Passing to the limit in (4.8) expressed for {ynk }, we obtain +∞ ≤ 4ρ 2 , which is a contradiction. Case 1. Suppose that there exists n0 such that {an }n≥n0 is nonincreasing, then {an } converges because it is also nonnegative. Coming back to (4.4), we can also deduce that Σn kxn − yn k2 < +∞ and then limn kxn − yn k= 0. Whereby limn kyn − uk2 = limn kxn − uk2 . Let x∗ be a weak cluster point of {xn }, then there exists a subsequence {xnk } which weakly converges to x∗ . By replacing n by nk − 1 in the second inequality of (4.1), we obtain for all y ∈ K 1 hxnk − ynk , y − xnk i ≥ F (y, xnk ). λnk −1 βnk −1

Using that lim inf n λn βn > 0, {xn } is bounded, kxn − yn k → 0 and lower semicontinuousness of F (y, ·), we have, for all y ∈ K 0 ≥ lim inf F (y, xnk ) ≥ F (y, x∗ ). k

Lemma 2.1 gives F (x∗ , y) ≥ 0 for all y ∈ K, i.e. x∗ ∈ SF . It is easy to see that x∗ is also a weak cluster point of {yn } and we can suppose that {ynk } weakly converges to x∗ . Reconsidering (4.7) and G(u, ·) being lower semi-continuous, we obtain 1 lim inf kyn+1 − uk2 ≤ − lim G(u, ynk +1 ) ≤ G(u, x∗ ) ≤ 0. (4.9) n ρ k The last inequality is because u ∈ S and x∗ ∈ SF . Since lim kyn+1 − uk2 exists, then kyn+1 − uk2 → 0 and so that kxn+1 − uk2 → 0, hence {xn } converges strongly to u. Case 2. Suppose there exists a subsequence {ank }k≥0 of {an }n≥0 such that ank < ank +1 for all k ≥ 0. By Lemma (2.3) for σ(n) := max{k ≤ n : ak < ak+1 }, one has

364


for all n ≥ n0 , for some n0 > 0, aσ(n) < aσ(n)+1 . By replacing n by σ(n) in (4.4), we obtain 0 ≥ kyσ(n)+1 − xσ(n) k2 + kyσ(n)+1 − xσ(n)+1 k2 − 2λn G(yσ(n)+1 , u).

(4.10)

and then lim supn −G(yσ(n)+1 , u) ≤ 0. Return also to ρ-strong monotonicity of G and passing to the limit we have 1 lim sup kyσ(n)+1 − uk2 ≤ − lim inf G(u, yσ(n)+1 ). (4.11) ρ n n We have already show that {yn } is bounded. Let y be a weak cluster point of {yσ(n) }. It is also a one of {xσ(n) }. Indeed, since u ∈ S, 0 ∈ (∂Gu + ℵSF )(u) and by Lemma 4.1, we get aσ(n)+1 −aσ(n) +kxσ (n)−yσ(n)+1 k2 +kxσ(n)+1 −yσ(n)+1 k2 +λσ(n) βσ(n) F (u, xσ(n)+1 ) 2p 2p ) − σSF ( βσ(n) )] + 2λ2σ(n) kpk2 . ≤ λσ(n) βσ(n) [Fu∗ ( βσ(n)

Under A and limn λn = 0, we obtain that kxσ(n)+1 − yσ(n)+1 k goes to zero. Then y is a weak cluster point of {xσ(n) }. As in the proof of the first case, we can show that y ∈ SF . Reconsidering (4.11), we have 1 lim sup kyσ(n)+1 − uk2 ≤ − G(u, y) ≤ 0. ρ n Then limn kyσ(n)+1 − uk2 = 0, which yields limn aσ(n)+1 = 0. Recalling that an ≤ aσ(n)+1 , we get limn→∞ an = 0, so that xn → u strongly. 5. Application. 5.1. Optimization in the lower-level problem. If F (x, y) := ψ(y)−ψ(x), where ψ : K → R is convex lower semicontinuous function, then SF = MinK ψ, the minimum set of ψ on K, and BEP becomes : find x ∈ MinK ψ such that G(x, y) ≥ 0 ∀y ∈ MinK ψ. In this case F obviously satisfies all conditions of (H1 ) − −(H4 ). Without any loss of generality we assume minK ψ = 0. Set M = MinK ψ, ψ(x) = ψ(x) if x ∈ K, and ψ(x) = +∞ if x ∈ / K; then ψ(x) − δM (x) ≤ 0 for all x ∈ H. Using the reverse ∗ inequality for their Fenchel conjugates, we deduce ψ (p) − σM (p) ≥ 0 for all p ∈ H, and then condition (A) becomes : X 2p 2p ∗ ∀u ∈ M, ∀p ∈ ℵM (u), 0 ≤ λ n βn ψ − σM < +∞. (5.1) β β n n n This is simply the condition (H4 ) proposed in [1]. Applying Theorem 3.1, and suppose that M is nonempty with lim inf n λn > 0 and limn βn = ∞ then we have the whole sequence {xn } generated by (1.4) weakly converges to a point x in M . Consider the particular case ψ(x) = 12 d(x, C)2 , where C ⊂ K is a nonempty ∗ closed convex set, then ψ (p) − σM (p) = 21 kpk2 ∀p ∈ H. We deduce that Condition P λn (A) is equivalent to n βn < +∞. ∗

When ψ(x) ≥ 21 d(x, C)2 , then ψ (p) − σM (p) ≤ 12 kpk2 ∀p ∈ < (ℵM ), and λn n βn < ∞ assurs (A). P Suppose moreover 0 < lim inf λn βn ≤ lim sup λn βn < ∞, then n λβnn < ∞ is P 2 equivalent to n λn < ∞. P


365

5.2. Fixed point in the lower-level problem. Let T : K → K be a nonexpansive mapping. Denote by F ix(T ) the set of fixed points of T ; that is, F ix(T ) = {x ∈ K : T (x) = x}. It is assumed throughout that F ix(T ) 6= ∅ and we concern the following problem: find u ∈ F ix(T ) such that G(u, y) ≥ 0, ∀y ∈ F ix(T ) For every x, y ∈ K, set F (x, y) = hx − T (x), y − xi. Then, u is an equilibrium point of F , if and only if u ∈ F ix(T ). It is clear that F verifies conditions (H1 ) − (H4 ) and algorithm (4.1) becomes λn G(yn+1 , y) + hyn+1 − xn , y − yn+1 i ≥ 0 ∀y ∈ K h[(1 + λn βn )I − λn βn T ]xn+1 − yn+1 , y − xn+1 i ≥ 0 ∀y ∈ K where the second inequality can be viewed as xn+1 = πK [yn+1 − λn βn (I − T )xn+1 ], with πK : H → K is the metric projection mapping. Since, in this case, u ∈ F ix(T ) implies F (u, y) = 0 ∀y ∈ K, then Fu∗ (p) = σK (p) and condition (A) reduces to ∀u ∈ F ix(T ), ∀p ∈ ℵF ix(T ) (u) lim λn [σK (p) − σF ix(T ) (p)] = 0. n

(5.2)

Remark that σF ix(T ) (p) = hp, ui, so if ∀p ∈ ℵF ix(T ) (u), σK (p) < ∞

(5.3)

relation (5.2) is then automatically verified in Theorem (4.2) because there is assumed that limn λn = 0. Condition (5.3) means that the barrier cone b(K) := {p ∈ H; σK (p) < ∞} contains ℵF ix(T ) (u)Tfor every u ∈ F ix(T ). For each x0 ∈ K, this is equivalent to the asymptotic cone t>0 t(K − x0 ) is contained in the tangent cone S 1 h>0 h (F ix(T ) − u), for every u ∈ F ix(T ), see [2]. 5.3. Set-valued variational inequalities in the leader-level problem. Let A be a monotone operator on H with a closed convex domain and a bifunction F : K × K → R satisfying (H1 ) − (H4 ). We define the operator B := A + ℵSF and we associate to it, the following bifunction GB : D(B) × D(B) → R defined by GB (x, y) = sup hξ, y − xi. ξ∈Bx

In [6], the authors show that this bifunction is monotone and verify GB (x, x) = 0, ∀x ∈ D(B). In addition, they obtain that, if the operator B is maximal monotone, then ∀x ∈ D(B), B(x) = {x∗ ∈ H : GB (x, y) ≥ hx∗ , y − xi ∀y ∈ D(B)}.

(5.4)

Let us show that the inequality find x ∈ SF such that GB (x, y) ≥ 0 for all y ∈ SF .

(5.5)

is equivalent to find x ∈ H such that 0 ∈ A(x) + ℵSF (x). (5.6) Indeed, if (5.6) is verified, then x ∈ SF and hAx, y − xi ≥ 0 for all y ∈ SF , so x ∈ SF and, since A(x) ⊂ B(x), supξ∈B(x) hξ, y − xi ≥ 0 for all y ∈ SF , hence we obtain (5.5). Conversely, (5.5) implies that x ∈ SF and GB (x, y) ≥ 0 for all y ∈ D(B), since D(B) ⊂ SF . Hence, by (5.4), 0 ∈ B(x). Let us now consider ϕ ∈ Γ0 (H) and set F (u, y) = ϕ(y) − ϕ(u). Remark that

366


SF = argmin(ϕ) (the minimum set of ϕ), and problem (5.5) is then exactly (1.3). If we suppose that min ϕ = 0 just like in [1], then Fu∗ ≡ ϕ∗ ∀u ∈ argminϕ and assumption (A) is the same for the two problems. REFERENCES [1] H. Attouch, M. O. Czarnecki and J. Peypouquet, Prox-penalization and splitting methods for constrained variational problems, SIAM J. Control Optim., 21 (2011), 149–173. [2] J. P. Aubin, “Optima and Equilibria: An Introduction to Nonlinear Analysis,” Springer, 2nd edition, 2002. [3] E. Blum and W. Oettli, From optimization and variational inequalities to equilibrium problems, Math. Student, 63 (1994), 123–145. [4] O. Chadli, Z. Chbani and H. Riahi, Equilibrium problems with generalized monotone Bifunctions and Applications to Variational inequalities, J. Optim. Theory Appl., 105 (2000), 299–323. [5] Z. Chbani and H. Riahi, Variational principle for monotone and maximal bifunctions, Serdica Math. J., 29 (2003), 159–166. [6] N. Hadjisavvas and H. Khatibzadeh, Maximal monotonicity of bifunctions, Optimization, 59 (2010), 147–160. [7] P. E. Mainge and A. Moudafi, Strong convergence of an iterative method for hierarchical fixed-points problems, Pacific J. Optim., 3 (2007), 529–538. [8] A. Moudafi, Proximal point algorithm extended for equilibrium problems, J. Nat. Geom., 15 (1999), 91–100. [9] A. Moudafi, On the convergence of splitting proximal methods for equilibrium problems in Hilbert spaces, J. Math Anal. Appl., 359 (2009), 508–513. [10] A. Moudafi, Proximal methods for a class of bilevel monotone equilibrium problems, J. Global Optimization, 47 (2010), 287–292. [11] Z. Opial, Weak convergence of the sequence of successive approximations for nonexpansive mappings, Bull. Aust. Math. Soc., 73 (1967), 591–597. [12] G. Passty, Ergodic convergence to a zero of the sum of monotone operators in Hilbert space, J. Math. Anal. Appl., 72 (1979), 383–390.

Received April 2012; 1st revision September 2012; final revision November 2012. E-mail address: [email protected] E-mail address: [email protected]