Split feasibility problem

Home

Search

Collections

Journals

About

Contact us

My IOPscience

Iterative methods for the split feasibility problem in infinite-dimensional Hilbert spaces

This article has been downloaded from IOPscience. Please scroll down to see the full text article. 2010 Inverse Problems 26 105018 (http://iopscience.iop.org/0266-5611/26/10/105018) View the table of contents for this issue, or go to the journal homepage for more

Download details: IP Address: 119.63.65.131 The article was downloaded on 09/09/2010 at 19:11

Please note that terms and conditions apply.

IOP PUBLISHING

INVERSE PROBLEMS

Inverse Problems 26 (2010) 105018 (17pp)

doi:10.1088/0266-5611/26/10/105018

Iterative methods for the split feasibility problem in infinite-dimensional Hilbert spaces Hong-Kun Xu Department of Applied Mathematics, National Sun Yat-sen University, Kaohsiung, Taiwan E-mail: [email protected]

Received 5 August 2010, in final form 17 August 2010 Published 9 September 2010 Online at stacks.iop.org/IP/26/105018 Abstract The split feasibility problem (SFP) (Censor and Elfving 1994 Numer. Algorithms 8 221–39) is to find a point x ∗ with the property that x ∗ ∈ C and Ax ∗ ∈ Q, where C and Q are the nonempty closed convex subsets of the real Hilbert spaces H1 and H2 , respectively, and A is a bounded linear operator from H1 to H2 . The SFP models inverse problems arising from phase retrieval problems (Censor and Elfving 1994 Numer. Algorithms 8 221–39) and the intensity-modulated radiation therapy (Censor et al 2005 Inverse Problems 21 2071–84). In this paper we discuss iterative methods for solving the SFP in the setting of infinite-dimensional Hilbert spaces. The CQ algorithm of Byrne (2002 Inverse Problems 18 441–53, 2004 Inverse Problems 20 103– 20) is indeed a special case of the gradient-projection algorithm in convex minimization and has weak convergence in general in infinite-dimensional setting. We will mainly use fixed point algorithms to study the SFP. A relaxed CQ algorithm is introduced which only involves projections onto half-spaces so that the algorithm is implementable. Both regularization and iterative algorithms are also introduced to find the minimum-norm solution of the SFP.

1. Introduction The split feasibility problem (SFP) in finite-dimensional Hilbert spaces was first introduced by Censor and Elfving [4] for modeling inverse problems which arise from phase retrievals and in medical image reconstruction [2]. Recently, it has been found that the SFP can also be used to model the intensity-modulated radiation therapy [5–8]. In this paper we work in the framework of infinite-dimensional Hilbert spaces. In this setting, the SFP is formulated as finding a point x ∗ with the property x∗ ∈ C

and

Ax ∗ ∈ Q,

0266-5611/10/105018+17$30.00 © 2010 IOP Publishing Ltd Printed in the UK & the USA

(1.1) 1

Inverse Problems 26 (2010) 105018

H-K Xu

where C and Q are the nonempty closed convex subsets of the infinite-dimensional real Hilbert spaces H1 and H2 , respectively, and A ∈ B(H1 , H2 ) (i.e. A is a bounded linear operator from H1 to H2 ). Some work in the finite-dimensional setting can be found in [2, 3, 6, 18, 24, 27, 30] (see also [21, 28, 29] for relevant projection methods for solving image recovery problems). A special case of the SFP (1.1) is the convexly constrained linear inverse problem x∈C

and

Ax = b

(1.2)

which has extensively been investigated in the literature ([11]) using the projected Landweber iterative method [14]. Comparatively, the SFP (1.1) has however received much less attention so far, due partially to the complexity resulted from the set Q. Therefore, whether various versions of the projected Landweber iterative method can be extended to solve the SFP (1.1) remains an interesting topic (e.g. it is yet not clear if the dual approach to (1.2) of [17] (see also [19]) can be extended to the SFP (1.1)). The original algorithm introduced in [4] involves the computation of the inverse A−1 (assuming the existence of the inverse of A) and thus does not become popular. A more popular algorithm that solves the SFP (1.1) seems to be the CQ algorithm of Byrne [2, 3] which is found to be a gradient-projection method (GPM) in convex minimization (it is also a special case of the proximal forward–backward splitting method [10]). The CQ algorithm only involves the computations of the projections PC and PQ onto the sets C and Q, respectively, and is therefore implementable in the case where PC and PQ have closed-form expressions (e.g. C and Q are the closed balls or half-spaces). It remains however a challenge how to implement the CQ algorithm in the case where the projections PC and/or PQ fail to have closed-form expressions though theoretically we can prove (weak) convergence of the algorithm. This paper aims at a continuation of study on the CQ algorithm and its convergence. After introducing our nonlinear operator tools in section 2, we look at the CQ algorithm in section 3 from two different but equivalent ways: optimization and fixed point. Since the CQ algorithm can be viewed as a fixed point algorithm for averaged mappings, we can apply Mann’s algorithm to the SFP (1.1); in particular, we obtain an averaged CQ algorithm which is proved to be weakly convergent to a solution of the SFP (1.1). We also include an example which shows that the CQ algorithm fails to converge in norm in infinitedimensional Hilbert spaces. In section 4, we introduce a relaxed CQ algorithm in which the sets C and Q are level sets of convex functions so that the projections involved in the CQ algorithm are onto half-spaces, which makes the algorithm implementable. In the final section, section 5, we discuss regularization and the minimum-norm solution of the SFP (1.1). We consider Tikhonov’s regularization for the SFP (1.1) and provide a necessary and sufficient condition for the strong convergence of the net of regularized solutions; this condition extends that for the case where Q is a singleton and which defines the constrained pseudoinverse A†C . We also devise iterative algorithms that converge to a solution of the SFP (1.1) and in particular, the minimum-norm solution of the SFP (1.1). We have noted that the SFP (1.1) has recently been extended from the Hilbert space setting to the setting of certain Banach spaces, see [20]. The convergence analysis on the CQ algorithm in Banach spaces is much more delicate than that in Hilbert spaces, which is beyond the scope of the present paper. The reader is however referred to [20] for more details in the framework of Banach spaces which satisfy certain geometric properties. Some of the results of [20] (e.g. proposition 2(c), page 11) essentially hold in the spaces p for 1 < p < ∞ (but not in Lp for 1 < p < ∞, p = 2 since the duality map of Lp is not weak-to-weak continuous unless p = 2). We note that the main algorithm in [20] (equation (25) of method 1 on page 8) is reduced to the CQ algorithm when the underlying Banach spaces are reduced to 2

Inverse Problems 26 (2010) 105018

H-K Xu

Hilbert spaces. Due to the lack of inner products in Banach spaces, the convergence theory of iterative methods for the SFP is much less fruitful in Banach spaces than that in Hilbert spaces. It is therefore nontrivial to find Banach space counterparts of the Hilbert space results on iterative methods for the SFP. We however also note that the results obtained in this paper cannot be derived from [20] (see remark 4.2 in section 4). 2. Tools: nonlinear operators Projections are an important tool for our work in this paper. Let H be a real Hilbert space with inner product ·, · and norm  · , respectively, and let K be a nonempty closed convex subset of H. Recall that the (nearest point or metric) projection from H onto K, denoted PK, is defined in such a way that, for each x ∈ H, PKx is the unique point in K with the property x − PK x = min{x − y : y ∈ K}. The following is a useful characterization of projections. Proposition 2.1. Given x ∈ H and z ∈ K. Then z = PK x if and only if x − z, y − z  0

for all

y ∈ K.

We also need other sorts of nonlinear operators which are introduced below. Let T , A : H → H be the nonlinear operators. • T is nonexpansive if T x − T y  x − y for all x, y ∈ H. • T is firmly nonexpansive if 2T − I is nonexpansive. Equivalently, T = (I + S)/2, where S : H → H is nonexpansive. Alternatively, T is firmly nonexpansive if and only if x − y, T x − T y  T x − T y2 ,

x, y ∈ H.

• T is averaged if T = (1−α)I +αS, where α ∈ (0, 1) and S : H → H is nonexpansive. In this case, we also say that T is α-averaged. A firmly nonexpansive mapping is 1 -averaged. (Averaged mappings have received many investigations, see [1, 3, 9, 25, 2 26].) • A is monotone if Ax − Ay, x − y  0 for x, y ∈ H. • A is β-strongly monotone, with β > 0, if x − y, Ax − Ay  βx − y2 ,

x, y ∈ H.

• A is ν-inverse strongly monotone (ν-ism), with ν > 0, if x − y, Ax − Ay  νAx − Ay2 ,

x, y ∈ H.

It is well known that both PK and I − PK are firmly nonexpansive and 12 -ism. Denote by Fix(T ) the set of fixed points of T (i.e. Fix(T ) = {x ∈ H : T x = x}). Proposition 2.2. (cf [3, 25]) We have the following assertions. (i) T is nonexpansive if and only if the complement I−T is 12 -ism. (ii) If T is νism and γ > 0, then γ T is γν -ism. (iii) T is averaged if and only if the complement I−T is ν-ism for some ν > 1/2. Indeed, for 1 α ∈ (0, 1), T is α-averaged if and only if I−T is 2α -ism. (iv) If T1 is α1 -averaged and T2 is α2 -averaged, where α1 , α2 ∈ (0, 1), then the composite T1 T2 is α-averaged, where α = α1 + α2 − α1 α2 . 3

Inverse Problems 26 (2010) 105018

H-K Xu

(v) If T1 and T2 are averaged and have a common fixed point, then F ix(T1 T2 ) = F ix(T1 ) ∩ F ix(T2 ). Averaged mappings are useful, due to the following. Proposition 2.3. (cf [12]) If T : H → H is an averaged mapping with Fix(T ) = ∅, then 1. T is asymptotically regular; that is, lim T n+1 x − T n x = 0

n→∞

for all

x ∈ H.

2. For any x ∈ H, the sequence {T n x}∞ n=0 converges weakly to a fixed point of T. An application of proposition 2.3 is included below. Proposition 2.4. Let K be a closed convex subset of a Hilbert space H and T : H → H be a ν-ism. Assume the variational inequality problem (VIP). Find a point x ∗ ∈ K such that x − x ∗ , T x ∗   0 for all

x∈K

(2.1)

is consistent. Then, for 0 < γ < 2ν, the sequence {xn } generated by the algorithm xn+1 = PK (I − γ T )xn converges weakly to a solution of VIP (2.1). An immediate consequence of proposition 2.4 is the convergence of the gradientprojection algorithm. Proposition 2.5. Let f : H → R be a continuously differentiable function such that the gradient ∇f is Lipschitz continuous: ∇f (x) − ∇f (y)  Lx − y,

x, y ∈ H.

Assume that the minimization problem min{f (x) : x ∈ K}

(2.2)

is consistent, where K is a closed convex subset of H. Then, for 0 < γ < 2/L, the sequence {xn } generated by the gradient-projection algorithm xn+1 = PK (xn − γ ∇f (xn ))

(2.3)

converges weakly to a solution of (2.2). Proof. The L-Lipschitz continuity of ∇f implies that the maximal monotone gradient operator ∇f is L1 -ism. The conclusion then immediately follows from proposition 2.1.  The following result is useful when proving weak convergence of a sequence. Proposition 2.6. Let K be a nonempty closed convex subset of a Hilbert space H. Let {xn } be a bounded sequence which satisfies the following properties: • every weak limit point of {xn } lies in K; and • limn→∞ xn − x exists for every x ∈ K. Then {xn } converges weakly to a point in K. 4

Inverse Problems 26 (2010) 105018

H-K Xu

3. Iterative methods We use  to denote the solution set of the SFP (1.1), i.e.  = {x ∈ C : Ax ∈ Q} = C ∩ A−1 Q, and assume the consistency of (1.1) so that  is closed, convex and nonempty. In this section we discuss iterative methods for solving the SFP (1.1). 3.1. Gradient-projection method It is well known that the GPM is one of the powerful methods for solving constrained optimization problems (see [15]). We will reformulate the SFP (1.1) as an optimization problem so that the GPM is applicable. Indeed, x ∈  means that there is an x ∈ C such that Ax − q = 0 for some q ∈ Q. This motivates us to consider the distance function d(Ax, q) = Ax − q and the minimization problem min 12 Ax − q2 . x∈C q∈Q

Minimizing with respect to q ∈ Q first makes us consider the minimization min f (x) := 12 Ax − PQ Ax2 .

(3.1)

x∈C

The objective function f is continuously differentiable with gradient given by ∇f (x) = A∗ (I − PQ )Ax.

(3.2)

∗

(Here A is the adjoint of A.) Due to the fact that I − PQ is (firmly) nonexpansive, we find that ∇f is Lipschitz continuous with Lipschitz constant L := A2 , namely ∇f (x) − ∇f (y)  A2 x − y,

x, y ∈ H1 .

(3.3)

The GPM (2.3) thus applies to solve (3.1). This method with gradient ∇f given as in (3.2) is referred to as the CQ algorithm in [2, 3] and generates a sequence {xn } via the procedure xn+1 = PC (I − γ A∗ (I − PQ )A)xn ,

n  0,

(3.4)

where the initial guess x0 ∈ H1 and γ > 0 is a parameter. By proposition 2.5, we immediately get the following convergence result. Theorem 3.1. (cf [2, 3, 24]) Assume that the SFP (1.1) is consistent. If 0 < γ < 2/A2 , then the sequence {xn } generated by the CQ algorithm (3.4) converges weakly to a solution of the SFP (1.1). 3.2. Fixed point method We can use fixed point algorithms to solve the SFP (1.1) based upon the following analysis. Let γ > 0 and assume that x ∗ ∈ . Thus Ax ∗ ∈ Q which implies the equation (I − PQ )Ax ∗ = 0 which in turns implies the equation γ A∗ (I − PQ )Ax ∗ = 0, hence the fixed point equation (I − γ A∗ (I − PQ )A)x ∗ = x ∗ . Requiring that x ∗ ∈ C, we consider the fixed point equation: PC (I − γ A∗ (I − PQ )A)x ∗ = x ∗ .

(3.5)

We will see that solutions of the fixed point equation (3.5) are exactly solutions of the SFP (1.1). 5

Inverse Problems 26 (2010) 105018

H-K Xu

Proposition 3.2. Given x ∗ ∈ H1 . Then x ∗ solves the SFP (1.1) if and only if x ∗ solves the fixed point equation (3.5). Proof. We have already proved that if x ∗ solves the SFP (1.1), then it also solves the fixed point equation (3.5). Conversely, assume that x ∗ solves the fixed point equation (3.5), which then implies by proposition 2.1 that I − γ A∗ (I − PQ )A)x ∗ − x ∗ , z − x ∗   0,

z ∈ C.

That is, A∗ (I − PQ )A)x ∗ , z − x ∗   0,

z ∈ C.

Hence, Ax ∗ − PQ Ax ∗ , Ax ∗ − Az  0,

z ∈ C.

(3.6)

On the other hand, we have, by proposition 2.1, Ax ∗ − PQ Ax ∗ , v − Ax ∗   0,

v ∈ Q.

(3.7)

Adding up the last two inequalities yields Ax ∗ − PQ Ax ∗ , v − Az  0,

v ∈ Q,

z ∈ C.

Inserting z = x ∗ ∈ C and v = PQ Ax ∗ ∈ Q in (3.8) gives that Ax ∗ = PQ Ax ∗ ∈ Q.

(3.8) 

Therefore, we can use fixed point equations to solve the SFP (1.1); in particular, we recover the CQ algorithm (3.4) as the following fixed point algorithm: xn+1 = T xn ,

n  0,

(3.9)

where the operator T is given by T = PC (I − γ A∗ (I − PQ )A).

(3.10)

Theorem 3.3. (Cf [2, 3, 24]) Assume that the SFP (1.1) is consistent. If 0 < γ < 2/A2 , then the sequence {xn } generated by the CQ algorithm (3.4) converges weakly to a solution of the SFP (1.1). Proof. The choice of 0 < γ < 2/A2 implies that the defining operator T defined by (3.10) is averaged. The consistency of the SFP (1.1) is equivalent to that Fix(T ) = ∅. Therefore, by proposition 2.3, we immediately conclude that the sequence {xn } defined by (3.9) (or equivalently, by the CQ algorithm (3.4)) converges weakly to a solution of the SFP (1.1).  3.3. Mann’s algorithm Recall that Mann’s iterative algorithm generates a sequence {xn } through the recursion xn+1 = (1 − αn )xn + αn T xn ,

n  0,

(3.11)

where T : C → C is nonexpansive and where {αn } is (usually) assumed to be a sequence in [0, 1]. The following result is known. Theorem 3.4. Suppose T : C → C is nonexpansive with a fixed point, where C is a closed convex subset of a Hilbert space. Assume that ∞  n=1

6

αn (1 − αn ) = ∞.

(3.12)

Inverse Problems 26 (2010) 105018

H-K Xu

Then the sequence {xn } defined by Mann’s algorithm (3.11) converges weakly to a fixed point of T. We next show that if T is averaged, then the choice of the parameter sequence {αn } in theorem 3.4 can be relaxed to cross the interval [0, 1]. Theorem 3.5. Let T : H → H be an α-averaged mapping with a fixed point. Assume that {αn } is a sequence in [0, 1/α] such that   ∞  1 − αn = ∞. (3.13) αn α n=1 Then the sequence {xn } defined by Mann’s algorithm (3.11) converges weakly to a fixed point of T. Proof. Since T is α-averaged, T = (1 − α)I + αS, where α ∈ (0, 1) and S : H → H is nonexpansive. Setting αn = ααn ∈ [0, 1], we can rewrite xn+1 defined by (3.11) as xn+1 = (1 − αn )xn + αn Sxn , where {αn } satisfies the condition (by virtue of (3.13)) ∞ 

αn (1 − αn ) = ∞.

n=1

Hence, by theorem 3.4, {xn } converges weakly to a fixed point of S (and of T).



Applying theorem 3.5 to the averaged mapping T defined in (3.10), we obtain the following result. Theorem 3.6. Assume that the SFP (1.1) is consistent and 0 < γ < 2/A2 . Let {xn } be defined by the following averaged CQ algorithm: xn+1 = (1 − αn )xn + αn PC (I − γ A∗ (I − PQ )A)xn ,

(3.14)

where {αn } is a sequence in the interval [0, 4/(2 + γ A2 )] satisfying the condition   ∞  4 αn − αn = ∞. 2 + γ A2 n=1 Then {xn } converges weakly to a solution of the SFP (1.1). Proof. It suffices to prove that PC (I − γ A∗ (I − PQ )A) is α-averaged, where α=

2 + γ A2 . 4

As a matter of fact, we have seen that A∗ (I − PQ )A is

1 -ism A2 ∗

and γ A∗ (I − PQ )A

1 Hence by proposition 2.2 the complement I − γ A (I − PQ )A is γ A is γ A 2 -ism. 2 1 averaged. Therefore, noting that PC is 2 -averaged and applying proposition 2.2(v), we get that PC (I − γ A∗ (I − PQ )A) is α-averaged, with 2

1 γ A2 2 + γ A2 1 γ A2 + − · = ∈ (0, 1).  2 2 2 2 4 The CQ algorithm fails in general to converge strongly in the setting of infinitedimensional Hilbert spaces, as the following example shows. α=

7

Inverse Problems 26 (2010) 105018

H-K Xu

Indeed, based on Hundal [13], we can construct a counterexample as follows. Example 3.7. In the space H = l 2 , Hundal [13] constructed two closed convex subsets C and Q satisfying the following properties: (i) C ∩ Q = ∅; (ii) the sequence generated by alternating projections xn = (PC PQ )n x0 ,

n0

(3.15)

with x0 ∈ C, converges weakly, but not strongly. We now take A = I . Then we see that the CQ algorithm (3.4) with γ = 1 generates exactly the sequence {xn } given by (3.15) which converges weakly, but not strongly. 4. A relaxed CQ algorithm via projections on half-spaces The CQ algorithm (3.4) involves two projections PC and PQ and hence might be hard to be implemented in the case where one of them fails to have a closed-form expression. We show next that if C and Q are level sets of convex functions, then we just need projections onto half-spaces, thus making the CQ algorithm implementable in this case. Define the closed convex sets C and Q as the level sets: C = {x ∈ H1 : c(x)  0}

and

Q = {y ∈ H2 : q(y)  0},

(4.1)

where c and q are the convex functions from H1 and respectively H2 to R. We assume that c and q are subdifferentiable on C and Q, respectively. Namely, the subdifferentials ∂c(x) = {z ∈ H1 : c(u)  c(x) + u − x, z,

u ∈ H1 } = ∅

for all x ∈ C, and ∂q(y) = {w ∈ H2 : q(v)  q(y) + v − y, w,

v ∈ H2 } = ∅

for all y ∈ Q. We also assume that c and q are bounded on bounded sets. (Note that this condition is automatically satisfied if H1 and H2 are finite dimensional.) This assumption guarantees that if {xn } is a bounded sequence in H1 (resp. H2 ) and {xn∗ } is another sequence in H1 (resp. H2 ) such that xn∗ ∈ ∂c(xn ) (resp. xn∗ ∈ ∂q(xn )) for each n, then {xn∗ } is bounded. Let 0 < γ < 2/A2 and x0 ∈ H1 . Define a sequence {xn } by the following relaxed CQ iterative algorithm:     n  0, (4.2) xn+1 = PCn I − γ A∗ I − PQn A xn , where {Cn } and {Qn } are the sequences of closed convex sets constructed as follows: Cn = {x ∈ H1 : c(xn ) + ξn , x − xn   0},

(4.3)

where ξn ∈ ∂c(xn ), and Qn = {y ∈ H2 : q(Axn ) + ηn , y − Axn   0},

(4.4)

where ηn ∈ ∂q(Axn ). It is easily seen that C ⊂ Cn and Q ⊂ Qn for all n. Also note that Cn and Qn are half-spaces; thus, the projections PCn and PQn have closed-form expressions. The relaxed CQ algorithm is introduced in [27] in the finite-dimensional setting. The following theorem establishes the weak convergence of this algorithm in the infinitedimensional setting. Our proof is simpler than that of [27] where sub-optimal problems were considered; while our proof just needs the Hilbert space technique. 8

Inverse Problems 26 (2010) 105018

H-K Xu

Theorem 4.1. Suppose the SFP (1.1) is consistent. Then the sequence {xn } generated by algorithm (4.2) converges weakly to a solution of the SFP (1.1). Proof. If we write

    Tn = PCn I − γ A∗ I − PQn A ,

(4.5)

then Tn is nonexpansive and the relaxed CQ algorithm (4.2) is rewritten as xn+1 = Tn xn , ∗

n  0. ∗

Moreover, we have Tn x = x for all x ∗ ∈ . (Recall that  is the solution set of the SFP (1.1).) It follows that, for each x ∗ ∈ , xn+1 − x ∗  = Tn xn − x ∗   xn − x ∗ . Hence, {xn − x ∗ } is decreasing sequence; in particular, lim xn − x ∗ 

n→∞

exists for each x ∗ ∈ .

(4.6)

However, we need more accurate estimates on xn − x ∗  as follows. Using the nonexpansivity of projections, we get  2    xn+1 − x ∗ 2   I − γ A∗ I − PQn A xn − x ∗   2   = (xn − x ∗ ) − γ A∗ I − PQn Axn    2  = xn − x ∗ 2 + γ 2 A∗ I − PQn Axn     − 2γ xn − x ∗ , A∗ I − PQn Axn  2   xn − x ∗ 2 + γ 2 A2  I − PQn Axn     (4.7) − 2γ Axn − Ax ∗ , I − PQn Axn . Since Ax ∗ ∈ Q ⊂ Qn , we have   I − PQn Axn , Ax ∗ − PQn Axn  0. This implies that     I − PQn Axn , Axn − Ax ∗ = I − PQn Axn , Axn − PQn Axn   + I − PQn Axn , PQn xn − Ax ∗  2    I − PQn Axn  .

(4.8)

Combining (4.7) and (4.8) we get

 2  xn+1 − x ∗ 2  xn − x ∗ 2 − γ (2 − γ A2 ) I − PQn Axn  .

In particular,

   lim  I − PQn Axn  = 0.

n→∞

Set

(4.9)

(4.10)

  un = A∗ I − PQn Axn → 0.

We next demonstrate that xn+1 − xn  → 0.

(4.11)

To see this, we note the identity xn+1 − xn 2 = xn − x ∗ 2 − xn+1 − x ∗ 2 + 2xn+1 − xn , xn+1 − x ∗ .

(4.12) 9

Inverse Problems 26 (2010) 105018

H-K Xu

On the other hand, since xn+1 = PCn (xn − γ un ), we have by proposition 2.1 (xn − γ un ) − xn+1 , x − xn+1   0,

x ∈ Cn .

∗

It follows that, for x ∈ Cn , xn − xn+1 , x ∗ − xn+1   γ un , x ∗ − xn+1   γ un x ∗ − xn+1  → 0. We therefore get by (4.12) that xn+1 − xn  → 0. Since {xn } is bounded, which implies that {ξn } is bounded, we see that the set of weak limit points of {xn }, ωw (xn ), is nonempty. We now show Claim: ωw (xn ) ⊂ . Indeed, assume that xˆ ∈ ωw (xn ) and {xnj } is a subsequence of {xn } which converges ˆ Since xkj +1 ∈ Cnj , we obtain weakly to x.  c(xnj ) + ξnj , xnj +1 − xnj  0. Thus

   c(xnj )  − ξnj , xnj +1 − xnj  ξ xnj +1 − xnj ,

where ξ satisfies ξn   ξ for all n. By virtue of the lower semicontinuity of c, we get by (4.11)   ˆ  lim inf c xnj  0. c(x) j →∞

Therefore, xˆ ∈ C. Next we show that Axˆ ∈ Q. To see this, set yn = Axn − PQn Axn → 0 and let η be such that ηn   η for all n. Then, since Axnj − ynj = PQnj Axnj ∈ Qnj , we get      q Axnj + ηnj , Axnj − ynj − Axnj  0. Hence,

     q Axnj  ηnj , ynj  ηynj  → 0.

By the weak lower semicontinuity of q and the fact that Axnj → Axˆ weakly, we arrive at the conclusion   ˆ  lim inf q Axnj  0. q(Ax) n→∞

Namely, Axˆ ∈ Q. Therefore, xˆ ∈ . Now we can apply proposition 2.6 to K :=  to get that the full  sequence {xn } converges weakly to a point in . Remark 4.2. In [20], Sch¨oepfer et al consider noisy perturbations C δ and Qδ for the sets C and Q in such a way that dρ (C, C δ )  δ

and

dρ (Q, Qδ )  δ,

(4.13)

where ρ > 0, δ > 0 and dρ is the ρ-distance [16, 20] defined by dρ (D1 , D2 ) = inf{λ  0 : D1 ∩ Bρ ⊂ D2 + Bλ

and

D2 ∩ Bρ ⊂ D1 + Bλ },

(4.14)

where D1 and D2 are the subsets of H1 (or H2 ), and Bλ is the ball centered at the origin with radius λ. The regularization method ([20], method 1, page 7), when stated for the SFP (1.1) in the Hilbert space setting, is reduced to the noised CQ algorithm ([20], equation (4.5), page 8): δ = PC δ (I − tn A∗ (I − PQδ )A)xnδ , xn+1

10

(4.15)

Inverse Problems 26 (2010) 105018

H-K Xu

where the initial guess x0δ = x0 and the stepsize tn is chosen in a certain appropriate manner (see [20], equation (29)). Sch¨oepfer et al [20] prove, among others, the following (reduced to the Hilbert space setting). (i) In the non-noisy case (i.e. δ = 0), the sequence (xn) generated by (4.15) converges weakly ([20], proposition 2). l (ii) In the noisy case (i.e. δ > 0), let (δ l ) be a null sequence of noisy levels, (xnδ ) be generated l by (4.15) with the same initial guess x0δ = x0 and nl = n∗ (δ l ) be determined by the stopping rule ([20], equation (39), page 11). Then ([20], proposition 3) l l • (iia ) the sequence (xnδl ) is bounded and every weak cluster point of (xnδl ) is a solution of SFP (1.1); l • (iib ) if the sequence (xn) for exact data converges strongly to x, then (xnδl ) also converges strongly to x. Our result (theorem 4.1) is different from [20] in the following ways. (i) The choice of the stepsize is different: we use a constant stepsize γ ∈ (0, 2/A2 ), while the stepsize tn in [20] is determined by minimizing a quadratic function hn (t) ([20], equation (29), page 8). (ii) Our perturbations Cn and Qn may not ρ-converge to C and Q, i.e. we may not have lim dρ (Cn , C) = 0,

n→∞

lim dρ (Qn , Q) = 0

n→∞

for sufficiently large ρ > 0. In other words, our theorem 4.1 cannot be derived from ([20], proposition 2). We also note that since in an infinite-dimensional Hilbert space, the sequence (xn) for exact data (i.e. generated by the CQ algorithm (3.4)) fails, in general, to converge strongly, as example 3.7 shows. This shows that in order to apply ([20], proposition 3(c)) (i.e. (iib ) above), additional conditions are needed (to guarantee the strong convergence of (xn)). 5. Regularization and minimum-norm solution 5.1. Regularization The minimization problem min f (x) := 12 Ax − PQ Ax2 x∈C

is, in general, ill-posed. So regularization is needed. We consider Tikhonov’s regularization min fα (x) := 12 Ax − PQ Ax2 + 12 αx2 , x∈C

(5.1)

where α > 0 is the regularization parameter. The regularized minimization (5.1) has a unique solution which is denoted as xα . The following result is easily proved and thus its proof is omitted. Proposition 5.1. If the SFP (1.1) is consistent, then the strong limα→0 xα exists and is the minimum-norm solution of the SFP (1.1). Remark 5.2. We actually have that xα converges in norm (as α → 0) if it is bounded. Below is a necessary and sufficient condition for the existence of the strong convergence of xα . 11

Inverse Problems 26 (2010) 105018

H-K Xu

Proposition 5.3. A necessary and sufficient condition for xα to converge in norm (as α → 0) is that the minimization min dist(u, Q) = min u − PQ u

u∈A(C)

(5.2)

u∈A(C)

is attained at a point in the set A(C). Remark 5.4. When Q is a singleton {b}, condition (5.2) is reduced to the well-known necessary and sufficient condition PA(C) (b) ∈ A(C)

(5.3)

for the C-constrained pseudoinverse of A, A†C , associated with the least-squares problem min Ax − b2 . x∈C

(5.4)

Namely, A†C (b) = x † , where b satisfies condition (5.3) and x † is the unique minimum-norm solution to (5.4). Proof of proposition 5.3. It suffices to prove that condition (5.2) holds if and only if xα is bounded as α → 0. First assume condition (5.2) and let u = Ax , where x ∈ C, solve the minimization (5.2); thus, for all α, Ax − PQ Ax   Axα − PQ Axα . Now since 1 Ax 2

− PQ Ax 2 + 12 αx 2  12 Axα − PQ Axα 2 + 12 αxα 2 ,

(5.5)

we immediately get xα   x . Conversely assume that xα is bounded and thus converges in norm to a point which is denoted as x ∗ ∈ C. This is immediately clear. As a matter of fact, letting α → 0 in (5.5) which holds for all x ∈ C, we obtain Ax ∗ − PQ Ax ∗   Ax − PQ Ax . This shows that the minimization (5.2) is attained at u∗ := Ax ∗ ∈ A(C).



5.2. Minimum-norm solution Assume that the SFP (1.1) is consistent and let xmin be its minimum-norm solution; namely, xmin ∈  has the property xmin  = min{x ∗  : x ∗ ∈ }. xmin can be obtained by two steps. First, observing that the gradient ∇fα = ∇f (x) + αI = A∗ (I − PQ )A + αI is (α + A2 )-Lipschitz and α-strongly monotone, the mapping PC (I − γ ∇fα ) is a contraction with coefficient

1 1 − γ (2α − γ (A2 + α))  1 − αγ , 2 where α . (5.6) 0 0, and can be obtained through the limit as n → ∞ of the sequence of Picard iterates α xn+1 = (PC (I − γ ∇fα ))xnα .

Secondly, letting α → 0 yields xα → xmin in norm. It is interesting to know if these two steps can be combined to get xmin in a single step. The following result shows that for suitable choices of γ and α, the minimum-norm solution xmin can be obtained by a single step. Theorem 5.5. Assume that the SFP (1.1) is consistent. Define a sequence {xn } by the iterative algorithm   xn+1 = PC I − γn ∇fαn xn = PC ((1 − αn γn )xn − γn A∗ (I − PQ )Axn ), (5.7) where {αn } and γn } satisfy the following conditions: (i) (ii) (iii) (iv)

0 < γn  Aα2n+αn for all (large enough) n; α n ∞→ 0 and γn → 0; n=1 αn γn = ∞; (|γn+1 − γn | + γn |αn+1 − αn |)/(αn+1 γn+1 )2 → 0.

Then {xn } converges in norm to the minimum-norm solution of the SFP (1.1). (Note that αn = n−δ and γn = n−σ with 0 < δ < σ < 1 and σ + 2δ < 1 satisfy conditions (i)–(iv).) We need the following elementary result on real sequences (cf [22, 23]). Lemma 5.6. Assume that {an } is a sequence of nonnegative real numbers such that an+1  (1 − γn )an + γn δn ,

n  0,

where {γn } and {βn } are the sequences in (0,1) and {δn } is a sequence in R such that  (i) ∞ n=1 γn = ∞;  (ii) either lim supn→∞ δn  0 or ∞ n=1 γn |δn | < ∞. Then limn→∞ an = 0. Proof  of theorem  5.5. Note that for any γ satisfying (5.6), xα is a fixed point of the mapping PC I − γ ∇fα . For each n, let zn be the unique fixed point of the contraction   Tn := PC I − γn ∇fαn . Then zn = xαn and so zn → xmin

in norm.

So to prove the theorem, it suffices to prove that xn+1 − zn  → 0. Noting that Tn has a contraction coefficient of (1 − 12 αn γn ), we have xn+1 − zn  = Tn xn − Tn zn     1 − 12 αn γn xn − zn     1 − 12 αn γn xn − zn−1  + zn − zn−1 .

(5.8)

We now estimate zn − zn−1  = Tn zn − Tn−1 zn−1   Tn zn − Tn zn−1  + Tn zn−1 − Tn−1 zn−1     1 − 12 αn γn zn − zn−1  + Tn zn−1 − Tn−1 zn−1 . 13

Inverse Problems 26 (2010) 105018

H-K Xu

This implies that zn − zn−1  

2 Tn zn−1 − Tn−1 zn−1 . αn γn

(5.9)

However, since {zn } is bounded, we have, for an appropriate constant M > 0,       Tn zn−1 − Tn−1 zn−1  = PC I − γn ∇fαn zn−1 − PC I − γn−1 ∇fαn−1 zn−1         I − γn ∇fαn zn−1 − I − γn−1 ∇fαn−1 zn−1    = γn ∇fαn (zn−1 ) − γn−1 ∇fαn−1 (zn−1 )    = (γn − γn−1 )∇fαn (zn−1 ) + γn−1 ∇fαn (zn−1 ) − ∇fαn−1 (zn−1 )   |γn − γn−1 |∇f (zn−1 ) + αn zn−1  + γn−1 |αn − αn−1 | zn−1   (|γn − γn−1 | + γn−1 |αn − αn−1 |)M. Combining (5.8), (5.9) and (5.10), we obtain   xn+1 − zn   1 − 12 αn γn xn − zn−1  + (αn γn )βn ,

(5.10)

(5.11)

where βn =

|γn − γn−1 | + γn−1 |αn − αn−1 | → 0. (αn γn )2

Now applying lemma 5.6 to (5.11) and using conditions (ii)–(iv), we conclude that  xn+1 − zn  → 0; hence, xn → xmin in norm. In the assumptions of theorem 5.7, the sequence {γn } is forced to tend to zero. If we keep it as a constant, then we have weak convergence as shown below. Theorem 5.7. Assume that the SFP (1.1) is consistent. Define a sequence {xn } by the iterative algorithm   xn+1 = PC I − γ ∇fαn xn = PC ((1 − γ αn )xn − γ A∗ (I − PQ )Axn ). (5.12)  Assume that 0 < γ < 2/A2 and ∞ n=0 αn < ∞. Then {xn } converges weakly to a solution of the SFP (1.1). Proof. We first demonstrate the boundedness of {xn }. Indeed, pick an x ∗ ∈  to get     xn+1 − x ∗  = PC I − γ ∇fαn xn − PC (I − γ ∇f )x ∗         PC I − γ ∇fαn xn − PC I − γ ∇fαn x ∗     ∗    + PC I − γ ∇fαn x − PC I − γ ∇f x ∗      xn − x ∗  +  I − γ ∇fα x ∗ − (I − γ ∇f )x ∗  n

Since

∞ n=0

= xn − x ∗  + γ αn x ∗ . αn < ∞, we conclude that lim xn − x ∗ 

n→∞

exists for each x ∗ ∈ .

In particular, {xn } is bounded. Next, setting   yn = I − γ ∇fαn xn = (1 − γ αn )xn − γ ∇f (xn ), we have xn+1 = PC yn . 14

(5.13)

Inverse Problems 26 (2010) 105018

H-K Xu

Moreover, noting that ∇f = A∗ (I − PQ )A is A2 -Lipschitz, we get f (xn+1 ) − f (xn ) = ∇f (xn ), xn+1 − xn  1 ∇f (xn + t (xn+1 − xn )) − ∇f (xn ), xn+1 − xn  dt + 0

 ∇f (xn ), xn+1 − xn  +

A2 xn+1 − xn 2 2

1 = − (1 − γ αn )xn − γ ∇f (xn ) − xn+1 , xn+1 − xn  γ   1 A2 − − xn+1 − xn 2 − αn xn , xn+1 − xn  γ 2 1 = − yn − PC yn , PC yn − xn  γ   A2 1 − xn+1 − xn 2 − αn xn , xn+1 − xn  − γ 2   1 A2  Mαn − − xn+1 − xn 2 γ 2  Mαn ,

(5.14) (5.15)

where M is a constant such that M  xn xn+1 − xn  for all n. Since ∞ n=0 αn < ∞, we obtain from (5.15) that lim f (xn )

exists

(5.16)

lim xn+1 − xn  = 0.

(5.17)

n→∞

and furthermore, by (5.14), n→∞

Since {xn } is bounded, so is {∇f (xn )}, we have another appropriate constant M > 0 satisfying the requirements in the following argument:   2  xn+1 − x ∗ 2 = PC I − γ ∇fαn xn − PC (I − γ ∇f )x ∗      I − γ ∇fα xn − (I − γ ∇f )x ∗ 2 n

= (xn − x ∗ ) − γ (∇f (xn ) − ∇f (x ∗ )) − γ αn xn 2  (xn − x ∗ ) − γ (∇f (xn ) − ∇f (x ∗ ))2 + Mαn = xn − x ∗ 2 − 2γ xn − x ∗ , ∇f (xn ) − ∇f (x ∗ ) + γn2 ∇f (xn ) − ∇f (x ∗ )2 + Mαn 1  xn − x ∗ 2 − 2γn ∇f (xn ) − ∇f (x ∗ )2 A2 +γ 2 ∇f (xn ) − ∇f (x ∗ )2 + Mαn = xn − x ∗ 2 + Mαn   2 −γ − γ ∇f (xn ) − ∇f (x ∗ )2 . A2 Hence, ∇f (xn ) − ∇f (x ∗ ) 

A2 (xn − x ∗ 2 − xn+1 − x ∗ 2 + Mαn ). γ (2 − γ A2 )

(5.18) 15

Inverse Problems 26 (2010) 105018

H-K Xu

By virtue of (5.13) and the assumption that

∞ n=0

αn < ∞, we get from (5.18) that

∗

lim ∇f (xn ) − ∇f (x ) = 0.

n→∞

(5.19)

On the other hand, using the subdifferential inequality and (5.17), we get f (xn ) − f (x ∗ )  ∇f (xn ), xn − x ∗  = ∇f (xn ), xn − xn+1  + ∇f (xn ), xn+1 − x ∗  1 = ∇f (xn ), xn − xn+1  − (1 − γ αn )xn − γ ∇f (xn ) − xn+1 , xn+1 − x ∗  γ 1 − xn+1 − (1 − γ αn )xn , xn+1 − x ∗  γn 1 = ∇f (xn ), xn − xn+1  − yn − PC yn , PC yn − x ∗  γ 1 − xn+1 − xn , xn+1 − x ∗  + αn xn , xn+1 − x ∗  γ  M(2xn+1 − xn  + αn ) → 0. Hence, by (5.16), we get lim f (xn ) = f (x ∗ ) = 0.

 ˜ then from the lower Now if xnk is a subsequence of {xn } weakly convergent to x, semicontinuity of f , we get n→∞

˜  lim inf f (xnk ) = f (x ∗ ). f (x ∗ )  f (x) k→∞

This shows that x˜ is a minimizer of f over C; namely, x˜ ∈ . Now, noting (5.13), we can apply proposition 2.6 to K :=  to get that the full sequence  {xn } converges weakly to a point in . Acknowledgments The author was supported in part by NSC 97-2628-M-110-003-MY3 (Taiwan). He is also grateful to the referees for their criticisms, comments and suggestions which improved and strengthened the presentation of this manuscript. References [1] Bauschke H and Borwein J 1996 On projection algorithms for solving convex feasibility problems SIAM Rev. 38 367–426 [2] Byrne C 2002 Iterative oblique projection onto convex subsets and the split feasibility problem Inverse Problems 18 441–53 [3] Byrne C 2004 A unified treatment of some iterative algorithms in signal processing and image reconstruction Inverse Problems 20 103–20 [4] Censor Y and Elfving T 1994 A multiprojection algorithm using Bregman projections in a product space Numer. Algorithms 8 221–39 [5] Censor Y, Bortfeld T, Martin B and Trofimov A 2006 A unified approach for inversion problems in intensitymodulated radiation therapy Phys. Med. Biol. 51 2353–65 [6] Censor Y, Elfving T, Kopf N and Bortfeld T 2005 The multiple-sets split feasibility problem and its applications for inverse problems Inverse Problems 21 2071–84 [7] Censor Y, Motova A and Segal A 2007 Perturbed projections and subgradient projections for the multiple-sets split feasibility problem J. Math. Anal. Appl. 327 1244–56 16

Inverse Problems 26 (2010) 105018

H-K Xu

[8] Censor Y and Segal A 2008 Iterative projection methods in biomedical inverse problems Mathematical Methods in Biomedical Imaging and Intensity-Modulated Radiation Therapy (IMRT) ed Y Censor, M Jiang and A K Louis (Edizioni della Normale, Pisa, Italy) pp 65–96 [9] Combettes P L 2004 Solving monotone inclusions via compositions of nonexpansive averaged operators Optimization 53 475–504 [10] Combettes P and Wajs V 2005 Signal recovery by proximal forward-backward splitting Multiscale Model. Simul. 4 1168–200 [11] Eicke B 1992 Iteration methods for convexly constrained ill-posed problems in Hilbert spaces Numer. Funct. Anal. Optim. 13 413–29 [12] Goebel K and Kirk W A 1990 Topics in Metric Fixed Point Theory (Cambridge Studies in Advanced Mathematics vol 28) (Cambridge: Cambridge University Press) [13] Hundal H 2004 An alternating projection that does not converge in norm Nonlinear Anal. 57 35–61 [14] Landweber L 1951 An iterative formula for Fredholm integral equations of the first kind Am. J. Math. 73 615V24 [15] Levitin E S and Polyak B T 1966 Constrained minimization methods Zh. vychisl. Mat. mat. Fiz. 6 787–823 [16] Penot J P 2003 Continuity properties of projection operators J. Inequalities Appl. 5 509–21 [17] Potter L C and Arun K S 1993 A dual approach to linear inverse problems with convex constraints SIAM J. Control Optim. 31 1080–92 [18] Qu B and Xiu N 2005 A note on the CQ algorithm for the split feasibility problem Inverse Problems 21 1655–65 [19] Sabharwal A and Potter L C 1998 Convexly constrained linear inverse problems: iterative least-squares and regularization IEEE Trans. Signal Process. 46 2345–52 [20] Sch¨opfer F, Schuster T and Louis A K 2008 An iterative regularization method for the solution of the split feasibility problem in Banach spaces Inverse Problems 24 055008 (20 pp) [21] Sezan M I and Stark H 1987 Applications of convex projection theory to image recovery in tomography and related areas Image Recovery Theory and Applications ed H Stark (Orlando: Academic) pp 415–462 [22] Xu H K 2002 Iterative algorithms for nonlinear operators J. London Math. Soc. 66 240–56 [23] Xu H K 2003 An iterative approach to quadratic optimization J. Optim. Theory Appl. 116 659–78 [24] Xu H K 2006 A variable Krasnosel ski˘ı–Mann algorithm and the multiple-set split feasibility problem Inverse Problems 22 2021–34 [25] Xu H K 2010 Averaged mappings and the gradient-projection algorithm J. Optim. Theory Appl. (to appear) [26] Yamada I and Oguraz N 2004 Hybrid steepest descent method for variational inequality problem over the fixed point set of certain quasi-nonexpansive mappings Numer. Funct. Anal. Optim. 25 619–55 [27] Yang Q 2004 The relaxed CQ algorithm for solving the split feasibility problem Inverse Problems 20 1261–6 [28] Youla D 1987 Mathematical theory of image restoration by the method of convex projections Image Recovery Theory and Applications ed H Stark (Orlando: Academic) pp 29–77 [29] Youla D 1990 On deterministic convergence of iterations of relaxed projection operators J. Visual Commun. Image Represent. 1 12–20 [30] Zhao J and Yang Q 2005 Several solution methods for the split feasibility problem Inverse Problems 21 1791–9

17