Further convergence results on the general ... - Semantic Scholar

MATHEMATICS OF COMPUTATION Volume 82, Number 283, July 2013, Pages 1647–1665 S 0025-5718(2012)02665-2 Article electronically published on December 31, 2012

FURTHER CONVERGENCE RESULTS ON THE GENERAL ITERATIVELY REGULARIZED GAUSS-NEWTON METHODS UNDER THE DISCREPANCY PRINCIPLE QINIAN JIN Abstract. We consider the general iteratively regularized Gauss-Newton methods xδk+1 = x0 − gαk (F (xδk )∗ F (xδk ))F (xδk )∗ F (xδk ) − y δ − F (xδk )(xδk − x0 ) for solving nonlinear inverse problems F (x) = y using the only available noise y δ of y satisfying y δ − y ≤ δ with a given small noise level δ > 0. In order to produce reasonable approximation to the sought solution, we terminate the iteration by the discrepancy principle. Under much weaker conditions we derive some further convergence results which improve the existing ones and thus expand the applied range.

1. Introduction In this paper we are interested in the ill-posed equations (1.1)

F (x) = y

arising from nonlinear inverse problems, where F : D(F ) ⊂ X → Y is a Fréchet differentiable nonlinear operator between two Hilbert spaces X and Y whose norms and inner products are denoted as · and (·, ·); we will use F (x) and F (x)∗ to denote the Fréchet derivative of F at x ∈ D(F ) and its adjoint respectively. We assume that (1.1) has a solution x† in the domain D(F ) of F , i.e., F (x† ) = y. We call (1.1) ill-posed in the sense that its solution does not depend continuously on the right-hand side. Since the data is usually obtained by measurement, instead of y, the only available data is an approximation y δ satisfying (1.2)

y δ − y ≤ δ

with a given small noise level δ > 0. Due to the ill-posedness, the computation of a stable approximation to x† from y δ becomes an important issue, and the regularization methods should be taken into account. We consider the general iteratively regularized Gauss-Newton methods ([2, 10]) (1.3) xδk+1 = x0 − gαk F (xδk )∗ F (xδk ) F (xδk )∗ F (xδk ) − y δ − F (xδk )(xδk − x0 ) , where xδ0 := x0 is an initial guess of x† , {αk } is a given sequence of numbers such that αk 1≤ ≤q and lim αk = 0 (1.4) αk > 0, k→∞ αk+1 Received by the editor June 30, 2010 and, in revised form, August 22, 2011. 2010 Mathematics Subject Classification. Primary 65J15, 65J20; Secondary 65H17. Key words and phrases. Nonlinear inverse problems, the general iteratively regularized GaussNewton methods, the discrepancy principle, convergence, order optimality. c 2012 American Mathematical Society Reverts to public domain 28 years from publication

1647

Licensed to Australian National University. Prepared on Tue Apr 23 03:05:58 EDT 2013 for download from IP 150.203.32.185. License or copyright restrictions may apply to redistribution; see http://www.ams.org/journal-terms-of-use

1648

QINIAN JIN

for some constant q > 1, and gα : [0, ∞) → (−∞, ∞) is a family of piecewise continuous spectral filter functions satisfying suitable structure conditions. In order to produce a reasonable approximation to x† , the iteration (1.3) must be terminated properly. Due to the practical applications, a posteriori rules, which use only quantities that arise during computation, should be used to choose the stopping index of iteration. In [8] we considered the discrepancy principle (1.5)

F (xδkδ ) − y δ ≤ τ δ < F (xδk ) − y δ ,

0 ≤ k < kδ

with a given number τ > 1 and we obtained several results concerning the convergence property of xδkδ to x† as δ → 0. In particular, we showed in [8] that if F satisfies the Lipschitz condition, i.e., if there is a number L such that F (x) − F (z) ≤ Lx − z

(1.6)

for all x, z ∈ Bρ (x† ) ⊂ D(F ), then the method defined by (1.3) and (1.5) is order optimal for each 1/2 ≤ ν ≤ ν¯ − 1/2, that is, when x0 − x† satisfies the source condition x0 − x† = (F (x† )∗ F (x† ))ν ω

(1.7)

for some 1/2 ≤ ν ≤ ν¯ − 1/2 and ω ∈ N (F (x† ))⊥ ⊂ X, there holds xδkδ − x† ≤ Cν ω1/(1+2ν) δ 2ν/(1+2ν) for some constant Cν depending only on ν, where ν¯ ≥ 1 denotes the qualification of the linear regularization method defined by {gα }. In order to carry out the convergence analysis under the source condition (1.7) with 0 ≤ ν < 1/2, stronger conditions must be imposed on F . In [8] it has been shown that xδkδ converges to x† as δ → 0 and the method defined by (1.3) and (1.5) is order optimal for each 0 < ν ≤ 1/2 if F satisfies the two structure conditions, i.e., there exist constants K0 and K1 such that (1.8)

(F (x) − F (z))w ≤ K0 x − zF (z)w + K1 F (z)(x − z)w

and there are bounded linear operators R(x, z) : X → X and a constant K2 such that (1.9)

F (x) = F (z)R(x, z),

I − R(x, z) ≤ K2 x − z

†

for x, z ∈ Bρ (x ) and w ∈ X. We should point out that, the above results were established under the additional condition (1.10)

rαk (λ) ≤ crαk+1 (λ),

∀k ≥ 0 and λ ∈ [0, 1]

for some constant c > 1, where rα (λ) := 1 − λgα (λ). Although it is a direct consequence of (1.4) for some choices of {gα }, condition (1.10) in general imposes further restriction on {αk }, which could exclude the case that {αk } is a geometric decreasing sequence. Several important questions arise naturally: Can we drop the additional condition (1.10) on {αk } in the convergence analysis? Is it possible to establish the convergence results under only one of the structure conditions (1.8) and (1.9) on F ? In this paper we give further convergence analysis on the methods defined by (1.3) and the discrepancy principle (1.5). Under (1.8), we show that xδkδ converges to x† as δ → 0 and derive the order optimal convergence rate for each 0 < ν ≤ 1/2.


GENERAL ITERATIVELY REGULARIZED GAUSS-NEWTON METHODS

1649

Our argument relies on neither (1.9) nor (1.10). Thus we expand the applied range of the method defined by (1.3) and (1.5). This paper is organized as follows. In Section 2 we state several structure conditions on {gα } together with some consequences which enable us to provide a unified treatment on (1.3) and (1.5). In Section 3 we present the convergence analysis. Finally, in Section 4 we provide numerical experiments to test our theoretical results. 2. Assumptions In the formulation of the method (1.3), we can choose {gα } in various ways to produce various iterative methods. In order to give a unified convergence analysis, we need to impose certain conditions on {gα }. The function rα (λ) := 1 − λgα (λ) is called the residual function associated with gα and plays a significant role in the convergence analysis. Assumption 2.1. There exist positive constants c0 and c1 such that 0 ≤ rα (λ) ≤ 1,

λrα (λ) ≤ c0 α

and

0 ≤ gα (λ) ≤ c1 α−1

for all α > 0 and λ ∈ [0, 1]. Due to the nonlinearity of F , in the convergence analysis we need to deal with terms like rα (F (x)∗ F (x)) − rα (F (z)∗ F (z)) for x, z ∈ D(F ). The following condition on {gα } will be employed to give a unified treatment, where C denotes the complex plane. Assumption 2.2. For each α > 0, gα (λ) can be extended to a complex analytic function defined on a simply connected domain Dα ⊂ C such that [0, 1] ⊂ Dα , and there is a contour Γα ⊂ Dα enclosing [0, 1] such that (2.1)

|λ| + σ ≤ b, |λ − σ|

∀λ ∈ Γα , α > 0 and σ ∈ [0, 1],

where b is a constant independent of α > 0. Moreover, for each s > 0 there is a constant bs such that |rα (λ)| (2.2) |dλ| ≤ bs α1−s |λ|s Γα and

(2.3) Γα

|gα (λ)| |dλ| ≤ bs α−s |λ|s

for all α > 0. By using the spectral integrals for self-adjoint operators, it follows easily from (2.1) in Assumption 2.2 that for any bounded linear operator A with A ≤ 1 there holds b (2.4) (λI − A∗ A)−1 (A∗ A)ν ≤ |λ|1−ν


1650

QINIAN JIN

for λ ∈ Γα and 0 ≤ ν ≤ 1. Moreover, since Assumption 2.2 implies that rα (λ) is analytic in Dα for each α > 0, there holds the Riesz-Dunford formula (see [3]) 1 rα (λ)(λI − A∗ A)−1 dλ rα (A∗ A) = 2πi Γα for any linear operator A satisfying A ≤ 1. Similarly, we have the Riesz-Dunford formula for gα (A∗ A). Example 2.3. The following spectral filter functions {gα } satisfy Assumptions 2.1 and 2.2 (see [3, 9]): m −αm (a) gα (λ) = (α+λ) λ(α+λ)m , arising from the linear iterated Tikhonov regularization of order m, where m ≥ 1 is a fixed integer. (b) gα (λ) = (1 − e−λ/α )/λ arising from the linear asymptotic regularization. (1 − λ)i arising from the linear Landweber iteration, where (c) gα (λ) = [1/α]−1 i=0 [1/α] denotes the largest integer not greater than 1/α. −i arising from the linear implicit iteration method. (d) gα (λ) = [1/α] i=1 (1 + λ) Since our goal is to give some convergence results on the method defined by (1.3) and (1.5) under only condition (1.8) on F , it is necessary to derive some useful consequences. The condition (1.8) clearly implies that F (x) is uniformly bounded in Bρ (x† ). Thus, for simplicity of exposition, we assume that the operator F is properly scaled so that √ (2.5) F (x) ≤ min{1, c0 α0 q}, ∀x ∈ Bρ (x† ). Lemma 2.4. Let {gα } satisfy Assumption 2.2 and let F satisfy (1.8). Then there hold 1 1 (2.6) rα (Ax ) − rα (Az ) K0 x − z + √ K1 (Fx (x − z) + Fz (x − z)) , α 1 (2.7) rα (Bx ) − rα (Bz ) K0 x − z + √ K1 (Fx (x − z) + Fz (x − z)) , α (2.8)

Fx [rα (Ax ) − rα (Az )] K0 x − zα1/2

(2.9)

+ K1 (Fx (x − z) + Fz (x − z)) , 1 Fx [gα (Ax ) − gα (Az )] √ K0 x − z α 1 + K1 (Fx (x − z) + Fz (x − z)) α

for all α > 0 and x, z ∈ Bρ (x† ), where Fx := F (x), Ax := Fx∗ Fx and Bx := Fx Fx∗ . Proof. We will only give the proof of (2.8), since the proofs of the other three inequalities are similar. Since rα (λ) is analytic in Dα for each α > 0, it follows from the Riesz-Dunford formula that 1 (2.10) Fx [rα (Ax ) − rα (Az )] = rα (λ)Fx (λI − Ax )−1 − (λI − Az )−1 dλ. 2πi Γα 1 Throughout this paper we will use C to denote a generic constant independent of δ and k. We will use the convention Φ Ψ to mean Φ ≤ CΨ for some generic constant C.



1651

Observe that Fx [(λI − Ax )−1 − (λI − Az )−1 ] = Fx (λI − Ax )−1 (Fx∗ − Fz∗ )Fz (λI − Az )−1 + (λI − Bx )−1 Bx (Fx − Fz )(λI − Az )−1 . Then we have from (2.4) that Fx [(λI − Ax )−1 − (λI − Az )−1 ] |λ|−1/2 (Fx − Fz )(λI − Ax )−1 Fx∗ + (Fx − Fz )(λI − Az )−1 . By (1.8) and (2.4) we then obtain Fx [(λI − Ax )−1 − (λI − Az )−1 ] |λ|−1/2 K0 x − zBx (λI − Bx )−1 + |λ|−1/2 K1 Fx (x − z)(λI − Ax )−1 Fx + K0 x − zFz (λI − Az )−1 + K1 Fz (x − z)(λI − Az )−1 K0 x − z|λ|−1/2 + K1 (Fx (x − z) + Fz (x − z)) |λ|−1 . Combining this with (2.10) yields Fx [rα (Ax ) − rα (Az )] K0 x − z

Γα

+

K1 (Fx (x

|rα (λ)| |dλ| |λ|1/2

− z) + Fz (x − z))

Γα

|rα (λ)| |dλ|. |λ|

With the help of (2.2) in Assumption 2.2, we obtain the desired estimate.

3. Convergence analysis Several convergence results have been proved in [8] on the Newton type methods (1.3) and the discrepancy principle (1.5). In this section, we derive further convergence results under weaker conditions. In contrast to the counterpart in [8], the new convergence results require neither (1.9) nor (1.10). Our analysis involves the iterative sequence {xk } defined by (1.3) corresponding to the noise-free case, i.e., (3.1)

xk+1 = x0 − gαk (F (xk )∗ F (xk )) F (xk )∗ (F (xk ) − y − F (xk )(xk − x0 )) .

For simplicity of presentation, we use the notations ek := xk − x† ,

eδk := xδk − x† ,

and A := F (x† )∗ F (x† ), Ak := F (xk )∗ F (xk ),

Aδk := F (xδk )∗ F (xδk ),

B := F (x† )F (x† )∗ ,

Bkδ := F (xδk )F (xδk )∗ .

Bk := F (xk )F (xk )∗ ,


1652

QINIAN JIN

3.1. Justification of the method. In this subsection we show that the method given by (1.3) and (1.5) with τ > 1 is well defined, i.e., xδk ∈ Bρ (x† ) for 0 ≤ k ≤ kδ and kδ is finite. To this end, we introduce the integer k˜δ defined by 2 δ < αk , 0 ≤ k < k˜δ , (3.2) αk˜δ ≤ γ0 e0 √ where γ0 > c0 q/(τ − 1) is a fixed number. Due to (1.4), such k˜δ exists and is finite. Lemma 3.1. Let F satisfy (1.8) and (2.5), let {αk } satisfy (1.4), and let {gα } √ satisfy Assumption 2.1. Let τ > 1 be a given number. If (2 + c1 γ0 )e0 < ρ and if (K0 + K1 )e0 is suitably small, then √ √ √ 1/2 F (x† )eδk ≤ 2 q ( c0 + γ0 ) e0 αk (3.3) eδk ≤ (2 + c1 γ0 ) e0 , and

√ 1/2 F (x† )eδk − y δ + y ≤ δ + ( c0 q + C(K0 + K1 )e0 ) e0 αk for 0 ≤ k ≤ k˜δ . (3.4)

Proof. We first prove (3.3) by induction. In view of the scaling condition (2.5), it is trivial for k = 0. Now we assume that (3.3) is true for some 0 ≤ k < k˜δ . We set uδk := F (xδk ) − y − F (xδk )eδk . Then, it follows from (1.3) that (3.5)

eδk+1 = rαk (Aδk )e0 − gαk (Aδk )F (xδk )∗ y − y δ + uδk . 1/2

In view of (1.8) and the induction hypotheses we have uδk (K0 + K1 )e0 2 αk . 1/2 Consequently, by using Assumption 2.1 and δ ≤ γ0 e0 αk , we obtain

√ √ −1/2 + C(K0 + K1 )e0 2 ≤ (2 + c1 γ0 ) e0 eδk+1 ≤ e0 + c1 δαk if (K0 + K1 )e0 is suitably small. Next, we multiply (3.5) by F (x† ) to obtain

F (x† )eδk+1 − y δ + y = F (xδk )rαk (Aδk )e0 + F (x† ) − F (xδk ) rαk (Aδk )e0 − F (x† ) − F (xδk ) gαk (Aδk )F (xδk )∗ y − y δ + uδk − gαk (Bkδ )Bkδ uδk − rαk (Bkδ )(y δ − y). 1/2

By using (1.8), the induction hypotheses and δ ≤ γ0 e0 αk , we can follow the argument in [8] to obtain √ 1/2 F (x† )eδk+1 − y δ + y ≤ δ + ( c0 + C(K0 + K1 )e0 ) e0 αk . By using (1.4) we then obtain

√ 1/2 F (x† )eδk+1 − y δ + y ≤ δ + ( c0 q + C(K0 + K1 )e0 ) e0 αk+1 . √ 1/2 1/2 Thus, using again the fact that δ ≤ γ0 e0 αk ≤ γ0 qe0 αk+1 , we have for suitably small (K0 + K1 )e0 such that √ √ √ 1/2 1/2 F (x† )eδk+1 ≤ 2δ + 2 c0 qe0 αk+1 ≤ 2 q ( c0 + γ0 ) e0 αk+1 .

(3.6)

By induction we therefore obtain (3.3) for 0 ≤ k ≤ k˜δ . The inequality (3.4) follows immediately from (3.6) since it is trivial for k = 0.



1653

For the noise-free iterates {xk } defined by (3.1), we have (3.7)

ek+1 = rαk (Ak )e0 − gαk (Ak )F (xk )∗ (F (xk ) − y − F (xk )ek ) .

By using the same argument as above we can derive the following estimates. Lemma 3.2. Under the conditions in Lemma 3.1, if (K0 + K1 )e0 is suitably small, then xk ∈ Bρ (x† ),

ek e0

and

F (x† )ek αk e0 1/2

for all k ≥ 0.

√ By using Lemma 3.1, (1.8), the definition of k˜δ , and γ0 > c0 q/(τ − 1), we can follow the argument in [8] to derive that √ F (xδk˜ ) − y δ ≤ (1 + c0 q/γ0 + C(K0 + K1 )e0 ) δ ≤ τ δ δ

if (K0 + K1 )e0 is suitably small. Thus the integer kδ defined by the discrepancy principle (1.5) with τ > 1 must satisfy kδ ≤ k˜δ . This together with Lemma 3.1 shows that the method given by (1.3) and (1.5) is well defined. 3.2. Stability estimates. In this subsection we derive the estimate on xδk − xk together with other useful estimates which will be used in the convergence analysis. Lemma 3.3. Let all the conditions in Lemma 3.1 and Assumption 2.2 hold. If (K0 + K1 )e0 is suitably small, then (3.8)

δ xδk − xk √ αk

and F (xδk ) − F (xk ) − y δ + y ≤ (1 + C(K0 + K1 )e0 )δ for all 0 ≤ k ≤ k˜δ .

(3.9)

Proof. By induction we first show for 0 ≤ k ≤ k˜δ that √ δ and F (x† )(xδk − xk ) ≤ 3δ. (3.10) xδk − xk ≤ 2 c1 √ αk It is clear that (3.10) is trivial for k = 0. Now we assume that (3.10) is true for some 0 ≤ k < k˜δ . From (1.8) and Lemma 3.2 one can see that (F (xk ) − F (x† ))(xδk − xk ) K0 e0 F (x† )(xδk − xk ) + K1 e0 xδk − xk αk . 1/2

Thus, with the help of the induction hypotheses, we obtain (3.11)

F (xk )(xδk − xk ) K1 e0 δ + F (x† )(xδk − xk ) δ.

Similarly, by using Lemma 3.1 we have (3.12)

F (xδk )(xδk − xk ) K1 e0 δ + F (x† )(xδk − xk ) δ.

Now we introduce the notations uk := F (xk ) − y − F (xk )ek

and uδk := F (xδk ) − y − F (xδk )eδk .

It then follows from (3.5) and (3.7) that (3.13)

xδk+1 − xk+1 = h1 + h2 + h3 + h4 ,


1654

QINIAN JIN

where

h1 := rαk (Aδk ) − rαk (Ak ) e0 , h2 := gαk (Aδk )F (xδk )∗ y δ − y , h3 := gαk (Ak )F (xk )∗ − gαk (Aδk )F (xδk )∗ uk , h4 := gαk (Aδk )F (xδk )∗ uk − uδk .

By (2.6) and (2.8) in Lemma 2.4, (3.11), (3.12) and the induction hypotheses, we obtain δ F (xδk )h1 (K0 + K1 )e0 δ. (3.14) h1 (K0 + K1 )e0 √ , αk By Assumption 2.1 we have √ δ (3.15) h2 ≤ c1 √ , αk

F (xδk )h2 − y δ + y ≤ δ.

In order to estimate h3 and F (xδk )h3 , we note that h3 = gαk (Ak ) F (xk )∗ − F (xδk )∗ uk + gαk (Ak ) − gαk (Aδk ) F (xδk )∗ uk and

F (xδk )h3 = F (xδk ) − F (xk ) gαk (Ak )F (xk )∗ uk + rαk (Bkδ ) − rαk (Bk ) uk .

Therefore, it follows from Assumption 2.1, (1.8) and Lemma 2.4 that 1 h3 √ K0 xδk − xk uk αk 1 K1 F (xk )(xδk − xk ) + F (xδk )(xδk − xk ) uk + αk and F (xδk )h3 K0 xδk − xk uk 1 + √ K1 F (xk )(xδk − xk ) + F (xδk )(xδk − xk ) uk . αk 1/2

Observe that (1.8) and Lemma 3.2 imply uk (K0 + K1 )e0 2 αk . Therefore, it follows from the induction hypotheses, (3.11) and (3.12) that δ (3.16) h3 (K0 + K1 )e0 √ , F (xδk )h3 (K0 + K1 )e0 δ. αk For h4 and F (xδk )h4 we have from Assumption 2.1 that 1 h4 √ uk − uδk , F (xδk )h4 uk − uδk . αk By using (1.8), Lemma 3.1, Lemma 3.2, the induction hypothesis, (3.12) and the √ fact that δ/ αk e0 , we have uk − uδk ≤ F (xδk ) − F (xk ) − F (xk )(xδk − xk ) + (F (xδk ) − F (xk ))eδk (K0 + K1 )e0 δ. Therefore, (3.17)

δ h4 (K0 + K1 )e0 √ , αk

F (xδk )h4 (K0 + K1 )e0 δ.



1655

Combining the estimates (3.14)–(3.17) yields √ δ xδk+1 − xk+1 ≤ ( c1 + C(K0 + K1 )e0 ) √ αk and

F (xδk )(xδk+1 − xk+1 ) − y δ + y ≤ (1 + C(K0 + K1 )e0 ) δ. Consequently, it follows from (1.8) and Lemma 3.1 that (F (xδk ) − F (x† ))(xδk+1 − xk+1 ) (K0 + K1 )e0 δ.

Hence, if (K0 + K1 )e0 is suitably small, we have F (x† )(xδk+1 − xk+1 ) ≤ F (xδk )(xδk+1 − xk+1 ) + C(K0 + K1 )e0 δ ≤ 3δ. We therefore complete the proof of (3.10). Now we can follow the argument in [8] and derive from (1.8), Lemma 3.1, and (3.8) that F (xδ )(xδ − xk ) − y δ + y ≤ (1 + C(K0 + K1 )e0 )δ, 0 ≤ k ≤ k˜δ , k

k

which in turn implies (3.9). As immediate consequences of Lemma 3.3, we have F (xkδ ) − y δ

(3.18) and

δ F (xk ) − y,

(3.19)

0 ≤ k < kδ ,

if (K0 + K1 )e0 is suitably small, where kδ is the integer determined by the discrepancy principle (1.5) with τ > 1. 3.3. Convergence. In this subsection we show that xδkδ → x† as δ → 0. In order to achieve this, we first show that xk → x† as k → ∞. Lemma 3.4. Let all the conditions in Lemma 3.3 be fulfilled and let x0 − x† ∈ N (F (x† ))⊥ . If (K0 + K1 )e0 is suitably small, then lim ek = 0

(3.20)

k→∞

and

F (x† )ek = 0. √ k→∞ αk lim

Proof. From (3.7) it follows that (3.21)

ek+1 = rαk (A)e0 + [rαk (Ak ) − rαk (A)] e0 − gαk (Ak )F (xk )∗ uk ,

where uk := F (xk )−y −F (xk )ek . By using (1.8) and (2.6) in Lemma 2.4 we obtain 1 ek+1 − rαk (A)e0 √ (K0 + K1 )ek F (x† )ek + K0 e0 ek αk 1 + √ K1 e0 F (x† )ek + F (xk )ek . αk Note that (1.8) and Lemma 3.2 imply F (xk )ek ≤ (1 + C(K0 + K1 )e0 ) F (x† )ek .

(3.22) Consequently, (3.23)

1 † ek+1 ≤ rαk (A)e0 + C(K0 + K1 )e0 ek + √ F (x )ek . αk


1656

QINIAN JIN

Next we multiply (3.21) by F (x† ) and obtain F (x† )ek+1 = F (x† )rαk (A)e0 + F (x† ) [rαk (Ak ) − rαk (A)] e0 (3.24) + F (x† ) − F (xk ) gαk (Ak )F (xk )∗ uk + gαk (Bk )Bk uk . By applying Assumption 2.1, (1.8), and (2.8) in Lemma 2.4 we have F (x† )ek+1 − F (x† )rαk (A)e0

+ K1 e0 F (x† )ek + F (xk )ek 1 + K0 (K0 + K1 )ek 2 F (x† )ek + √ K1 (K0 + K1 )ek F (xk )ek 2 αk 1/2

K0 e0 ek αk

+ (K0 + K1 )ek F (x† )ek . Thus, we may use Lemma 3.2 and (3.22) to conclude (3.25)

1/2 F (x† )ek+1 ≤ F (x† )rαk (A)e0 + C(K0 + K1 )e0 ek αk + F (x† )ek . Now we set ηk := ek +

F (x† )ek , √ αk

εk := rαk (A)e0 +

F (x† )rαk (A)e0 . √ αk

Then it follows from (3.23), (3.25), and (1.4) that √ (3.26) ηk+1 ≤ qεk + σηk , k = 0, 1, · · · , where σ := C(K0 + K1 )e0 for a generic constant C. We may assume σ < 1 by taking (K0 + K1 )e0 to be suitably small. By iterating this inequality we get √ q εj σ k−1−j + σ k−1 η0 , k−1

ηk ≤

k = 1, 2, · · · .

j=0

Since e0 ∈ N (F (x† ))⊥ , from Assumption 2.1 we have εj → 0 as j → ∞. Thus for any ε > 0 there is k0 > 0 such that εj ≤ ε for all j > k0 . Consequently, for k > k0 , ηk ≤

k0 k √ qε σ k−1−j + C σ k−1−j ≤ j=0

j=k0 +1

σ k−1−k0 ε +C . 1−σ 1−σ

Recall that σ < 1, we therefore have ηk ≤ Cε if k is sufficiently large. Since ε > 0 is arbitrary, we obtain ηk → 0 as k → ∞. We are now ready to give the main result concerning the convergence of the method (1.3) under the discrepancy principle (1.5). Theorem 3.5. Let F satisfy (1.8), let {gα } satisfy Assumptions 2.1 and 2.2, and let {αk } satisfy (1.4). Let τ > 1 be a given number. If (K0 + K1 )e0 is suitably small, then the method given by (1.3) and (1.5) is well defined, and if e0 ∈ N (F (x† ))⊥ and N (F (x† )) ⊂ N (F (x)) for all x ∈ Bρ (x† ), then lim xδkδ = x†

δ→0

for the integer kδ defined by the discrepancy principle (1.5).



1657

Proof. We have shown that the method given by (1.3) and (1.5) is well defined. It remains only to show that xδkδ → x† as δ → 0. Assume first that there is a sequence δn 0 such that kn := kδn → k as n → ∞ for some finite integer k. Without loss of generality, we can assume that kn = k for all n. Since (3.18) implies F (xkn ) − y δn , by sending n → ∞ we obtain F (xk ) = y. This together with (1.8) gives F (x† )ek = 0, i.e., ek ∈ N (F (x† )). On the other hand, since N (F (x† )) ⊂ N (F (x)) for all x ∈ Bρ (x† ), we can see from the definition of xk and the condition e0 ∈ N (F (x† ))⊥ that ek ∈ N (F (x† ))⊥ . Therefore xk = x† , which together with Lemma 3.3 implies xδknn → x† as n → ∞. Assume next that there is a sequence δn 0 such that kn := kδn → ∞ as √ n → ∞. Then Lemma 3.4 implies that ekn → 0 and F (x† )ekn −1 / αkn → 0 as n → ∞. But from (3.19) we have δn F (xkn −1 ) − y F (x† )ekn −1 .

√ Consequently, δn / αkn → 0 as n → ∞. By Lemma 3.3 we again obtain xδknn → x† as n → ∞. 3.4. Rates of convergence. Although Theorem 3.5 gives the convergence of xδkδ to x† as δ → 0, it does not give the convergence speed. For the ill-posed problem, the convergence in general could be arbitrarily slow if there is no further source condition on e0 := x0 − x† . In this section we derive the order optimal convergence rates for the method given by (1.3) and (1.5) when e0 satisfies the source condition (1.7) for some 0 < ν ≤ 1/2 and ω ∈ X. Lemma 3.6. Let F satisfy (1.8), let {gα } satisfy Assumptions 2.1 and 2.2, and let {αk } satisfy (1.4). If (K0 + K1 )e0 is suitably small and if e0 = Aν ω for some 0 < ν ≤ 1/2 and ω ∈ X, then ek ≤ Cν ωαkν ,

(3.27)

F (x† )ek ≤ Cν ωαk

ν+1/2

for all k ≥ 0, and ν+1/2

δ ≤ Cν αkδ

(3.28)

ω

for the integer kδ determined by the discrepancy principle (1.5) with τ > 1, where Cν is a positive constant depending only on ν. √ Proof. We will prove (3.27) by using (3.26). We may assume σ ≤ 1/(2 q) by taking (K0 + K1 )e0 to be suitably small. Since e0 = Aν ω for some 0 < ν ≤ 1/2, we have from Assumption 2.1 that εk ≤ Cν αkν ω, where

Cν

:=

ν/2 c0

+

ν/2+1/4 c0 .

ηk+1 ≤

k ≥ 0,

Consequently, it follows from (3.26) that

1 √ ν qCν αk ω + √ ηk , 2 q

k = 0, 1, · · · .

By using (1.4), we can derive by induction that ηk ≤ 2qCν αkν ω,

k = 0, 1, · · · .

Recalling the definition of ηk , we therefore obtain (3.27).


1658

QINIAN JIN

For the inequality (3.28), we note that (3.19) and (3.27) imply δ F (x† )ekδ −1 ≤ Cν αkδ −1 ω. ν+1/2

Consequently, by using (1.4) we obtain the desired estimate.

In the following we assume e0 satisfies the source condition (1.7) with 0 < ν ≤ 1/2 and derive the estimate on eδkδ . It follows from (3.8) and (3.28) that (3.29)

eδkδ ekδ + √

δ ≤ ekδ + Cν ω1/(1+2ν) δ 2ν/(1+2ν) . αkδ

Thus, in order to obtain the order optimality, it suffices to derive estimate on ekδ . We first consider the method (1.3) with gα (λ) = (α + λ)−1 , which is exactly the iteratively regularized Gauss-Newton method (see [1]) (3.30) −1 δ ∗ F (xk ) (F (xδk ) − y δ ) + αk (xδk − x0 ) . xδk+1 = xδk − αk I + F (xδk )∗ F (xδk ) The order optimal rates of convergence have been derived in [4] when the iteration is terminated by the discrepancy principle (1.5) with sufficiently large τ . The following result improves the one in [4] in the sense that it requires only τ > 1, which is significant for the accuracy in numerical computation. Theorem 3.7. Let F satisfy (1.8), let {αk } satisfy (1.4), and let τ > 1 be a given number. If (K0 + K1 )e0 is suitably small, and if e0 = Aν ω for some 0 < ν ≤ 1/2 and ω ∈ N (F (x† ))⊥ ⊂ X, then for the sequence {xδk } defined by (3.30) and the integer kδ determined by (1.5) there holds xδkδ − x† ≤ Cν ω1/(1+2ν) δ 2ν/(1+2ν) , where Cν is a constant depending only on ν. Proof. We first use (3.26) to give an estimate on ek . Note that rα (λ) = α(α + λ)−1 . It follows from (1.4) that rαk+1 (λ) ≤ rαk (λ) ≤ qrαk+1 (λ) for all λ ≥ 0. Thus, rαk+1 (A)e0 ≤ rαk (A)e0 ≤ qrαk+1 (A)e0 and (3.31)

F (x† )rαk+1 (A)e0 ≤ F (x† )rαk (A)e0 ≤ qF (x† )rαk+1 (A)e0 .

Consequently, εk ≤ qεk+1 for all k ≥ 0. Hence, if (K0 + K1 )e0 is so small that σ ≤ 1/(2q), we can obtain from (3.26) that ηk ≤ 2qεk for all k ≥ 0, i.e., (3.32)

ek rαk (A)e0 +

F (x† )rαk (A)e0 , √ αk

k = 0, 1, · · · .

Next we use the special form of rα (λ) to give an estimate on F (x† )rαk (A)e0 . From (3.24), (1.8), and Lemma 3.2 it follows that F (x† )ek+1 − F (x† )rαk (A)e0 ≤ F (x† ) (rαk (Ak ) − rαk (A)) e0 + C(K0 + K1 )e0 F (x† )ek .

(3.33) We use rα (λ) = α(α + λ)−1 and write

F (x† ) (rαk (Ak ) − rαk (A)) e0 = u1 + u2 ,



where

1659

u1 := F (x† )(αk I + Ak )−1 F (xk )∗ F (x† ) − F (xk ) rαk (A)e0 , u2 := F (x† )(αk I + Ak )−1 F (x† )∗ − F (xk )∗ F (x† )rαk (A)e0 .

By using (1.8) and Lemma 3.2 we have u1 ≤ βk K0 e0 F (x† )rαk (A)e0 + K1 e0 F (x† )ek , where βk := F (x† )(αk I + Ak )−1 F (xk )∗ . In view of (1.8) and Lemma 3.2 we obtain βk ≤ (αk I + Bk )−1 Bk + F (x† ) − F (xk ) (αk I + Ak )−1 F (xk )∗ 1 (3.34) ≤ 1 + K0 ek + √ K1 F (xk )ek 1. 2 αk Therefore, u1 K0 e0 F (x† )rαk (A)e0 + K1 e0 F (x† )ek . In order to estimate u2 , we note that (1.8) and (3.34) imply (F (x† ) − F (xk ))(αk I + Ak )−1 F (x† )∗ ≤ K0 ek βk + K1 F (xk )ek (αk I + Ak )−1 F (x† )∗ K0 ek + K1 F (x† )ek F (x† )(αk I + Ak )−1 . Similar to the derivation of (3.34) we can show that √ F (x† )(αk I + Ak )−1 1/ αk . Therefore, with the help of Lemma 3.2, we obtain (F (x† ) − F (xk ))(αk I + Ak )−1 F (x† )∗ (K0 + K1 )e0 . Hence u2 (K0 + K1 )e0 F (x† )rαk (A)e0 . Combining the estimates on u1 and u2 we obtain F (x† ) (rαk (Ak ) − rαk (A)) e0 (K0 + K1 )e0 F (x† )rαk (A)e0 + K1 e0 F (x† )ek . Plugging this estimate into (3.33) yields F (x† )ek+1 − F (x† )rαk (A)e0 (K0 + K1 )e0 F (x† )rαk (A)e0 + (K0 + K1 )e0 F (x† )ek .

(3.35) This in particular implies (3.36)

F (x† )ek+1 F (x† )rαk (A)e0 + (K0 + K1 )e0 F (x† )ek .

By using (3.31), we can conclude from (3.36) that if (K0 + K1 )e0 is suitably small, then F (x† )ek F (x† )rαk (A)e0 , k = 0, 1, · · · .


1660

QINIAN JIN

Combining this estimate with (3.35) we obtain for k ≥ 1 that F (x† )rαk−1 (A)e0 (K0 + K1 )e0 F (x† )rαk−1 (A)e0 + F (x† )ek + (K0 + K1 )e0 F (x† )ek−1 (K0 + K1 )e0 F (x† )rαk−1 (A)e0 + F (x† )ek . Thus, if (K0 + K1 )e0 is suitably small, this together with (3.31) implies F (x† )rαk (A)e0 F (x† )ek ,

(3.37)

k = 0, 1, · · · ,

since it is trivial for k = 0. Now we take k = kδ in (3.37) and use (3.18) to obtain F (x† )rαkδ (A)e0 δ. With the help of the source condition e0 = Aν ω with 0 < ν ≤ 1/2 and the interpolation inequality we obtain rαkδ (A)e0 ≤ rαkδ (A)ω1/(1+2ν) F (x† )rαkδ (A)e0 2ν/(1+2ν) ≤ Cν ω1/(1+2ν) δ 2ν/(1+2ν) . Combining the above two estimates with (3.32) and using (3.28) we obtain ekδ Cν ω1/(1+2ν) δ 2ν/(1+2ν) + √

δ ≤ Cν ω1/(1+2ν) δ 2ν/(1+2ν) . αkδ

This together with (3.29) then completes the proof.

Next we return to the general method (1.3) together with the discrepancy principle (1.5). We need the following additional condition. Assumption 3.8. There exist positive constants 0 < m ≤ M < ∞ such that mF (x† )h ≤ F (x)h ≤ M F (x† )h,

h∈X

†

for all x ∈ Bρ (x ) ⊂ D(F ). The following result enables us to use Assumption 3.8 in the derivation of convergence rates. Proposition 3.9. Let T, S : X → X be two self-adjoint bounded linear operators. If Sh ≤ C0 T h for all h ∈ X, then R(S) ⊂ R(T ) and T −1 S ≤ C0 .

Proof. This is [5, Proposition 2.1].

Theorem 3.10. Let F satisfy (1.8) and Assumption 3.8, let {gα } satisfy Assumptions 2.1 and 2.2, and let {αk } satisfy (1.4). Let {xδk } be defined by (1.3) and let kδ be the integer determined by (1.5) with τ > 1. If (K0 + K1 )e0 is suitably small and if e0 = Aν ω for some 0 < ν ≤ 1/2 and ω ∈ N (F (x† ))⊥ ⊂ X, then xδkδ − x† ≤ Cν ω1/(1+2ν) δ 2ν/(1+2ν) , where Cν is a constant depending only on ν. Proof. We will use (3.7). By Assumption 3.8 we have 1/2

mA1/2 h ≤ Ak h ≤ M A1/2 h,

h ∈ X,



1661

which implies that m2 A ≤ Ak ≤ M 2 A. Since the function t → t2ν is operator 4ν 2ν monotone for 0 ≤ ν ≤ 1/2, we have m4ν A2ν ≤ A2ν k ≤ M A , which implies m2ν Aν h ≤ Aνk h ≤ M 2ν Aν h,

h ∈ X.

In view of of Proposition 3.9 we obtain R(Aν ) = R(Aνk ),

ν −ν ν A−ν Ak 1. k A 1 and A

Since e0 = Aν ω, we must have ωk ∈ X such that e0 = Aνk ωk

and

ωk ω.

Therefore, it follows from (3.7) that ek+1 = rαk (Ak )Aνk ωk − gαk (Ak )F (xk )∗ (F (xk ) − y − F (xk )ek ) . By the polar factorization we can write F (xk )∗ = Ak U for some partial isometry U : Y → X. Therefore, 1/2

ek+1 = Aνk vk ,

(3.38) where

1/2−ν

vk = rαk (Ak )ωk − gαk (Ak )Ak

U (F (xk ) − y − F (xk )ek ) .

By using Assumption 2.1, (1.8), Lemma 3.2 and Lemma 3.6, for 0 ≤ k ≤ kδ we have −ν−1/2

F (xk ) − y − F (xk )ek

−ν−1/2

(K0 + K1 )ek F (x† )ek

vk ωk + αk ωk + αk

ω + (K0 + K1 )e0 ω ω. By using (3.38), Assumption 3.8, (1.8), and (3.18) we also have F (xkδ −1 )Aνkδ −1 vkδ −1 = F (xkδ −1 )ekδ F (x† )ekδ F (xkδ ) − y δ. Therefore, by using (3.38) and the two estimates above, we have from the interpolation inequality that ekδ = Aνkδ −1 vkδ −1 ≤ vkδ −1 1/(1+2ν) F (xkδ −1 )Aνkδ −1 vkδ −1 2ν/(1+2ν) ≤ Cν ω1/(1+2ν) δ 2ν/(1+2ν) . Combining this with (3.29) gives the desired estimate.

4. Numerical experiments In this section we will present the numerical results for a parameter identification problem in differential equations. Consider the identification of the diffusion parameter a in

−(au ) = f in (0, 1), (4.1) u(0) = u0 , u(1) = u1 from the L2 measurement uδ of u on (0, 1) satisfying uδ − uL2 ≤ δ, where u0 , u1 , and f ∈ L2 (0, 1) are given. It is well known that for a ∈ L∞ (0, 1) with a ≥ a > 0 on (0, 1), (4.1) has a unique solution u = u(a) ∈ H 2 (0, 1). In order to put the problem


1662

QINIAN JIN

into the framework of Hilbert space, we assume a† ∈ H 1 (0, 1) is the sought solution with a† ≥ 2a > 0 on (0, 1). We then define F as F : H 1 (0, 1) → L2 (0, 1), with

F (a) := u(a)

D(F ) := a ∈ H 1 (0, 1) : a ≥ a > 0 on (0, 1) .

Since H 1 (0, 1) embeds into L∞ (0, 1), F is well defined. It is well known that F is Fréchet differentiable on D(F ) with F (a)h = A(a)−1 ((h(u(a))) ), F (a)∗ w = −B −1 ((u(a)) (A(a)−1 w) ), for all a ∈ D(F ), h ∈ H 1 (0, 1) and w ∈ L2 (0, 1), where, for each a ∈ D(F ), A(a) : V := H01 ∩H 2 (0, 1) → L2 (0, 1) is an isomorphism defined by A(a)η = −(aη ) for η ∈ H01 ∩ H 2 , and B : D(B) := {ϕ ∈ H 2 (0, 1) : ϕ (0) = ϕ (1) = 0} → L2 (0, 1) is defined by Bϕ := −ϕ + ϕ. It was shown in [6, Example 11.1] that there is a ball Bρ (a† ) around a† such that 1 F (a2 ) − F (a1 ), a1 , a2 ∈ Bρ (a† ). 2 In the following we will verify (1.8). It is easy to see that for a1 , a2 ∈ Bρ (a† ) and h ∈ H 1 we get (F (a2 ) − F (a1 ))h = A(a2 )−1 ((a2 − a1 )(F (a1 )h) ) + (h(u(a2 ) − u(a1 )) ) . Let V denote the anti-dual of V with respect to the bilinear form Ω ϕψdx. Recall that A(a) can be extended as an isomorphism A(a) : L2 → V so that A(a)−1 : V → L2 is uniformly bounded around a† . We have from the above equation that

(4.2) F (a2 ) − F (a1 ) − F (a1 )(a2 − a1 ) ≤

(F (a2 ) − F (a1 ))hL2 ((a2 − a1 )(F (a1 )h) ) V

+ (h(u(a2 ) − u(a1 )) ) V .

(4.3)

We claim that for any h ∈ H 1 and ϕ ∈ H01 ∩ H 2 there holds (hϕ ) V hH 1 ϕL2 .

(4.4)

To see this, by the divergence theorem we have for ψ ∈ V , that 1 1 (hϕ ) ψdx = ϕ(hψ ) dx ≤ ϕL2 (hψ ) L2 . 0

0

Since H 1 → L∞ , we have (hψ ) L2 Therefore,

≤ hL∞ ψ L2 + h L2 ψ L∞ hH 1 ψV .

1

(hϕ ) ψdx hH 1 ϕL2 ψV ,

∀ψ ∈ V,

0

which implies the claim (4.4). Observe that both F (a1 )h and u(a2 ) − u(a1 ) are in H01 ∩ H 2 , we may apply the claim (4.4) to estimate the right-hand side of (4.3). Thus, (F (a2 ) − F (a1 ))hL2 a2 − a1 H 1 F (a1 )hL2 + F (a2 ) − F (a1 )L2 hH 1 . Since (4.2) implies F (a2 ) − F (a1 )L2 F (a1 )(a2 − a1 )L2 , (1.8) is thus satisfied.



1663

In the following we report some numerical results on the method given by (1.3) and (1.5) with gα (λ) = (α+λ)−1 which, in the current context, defines the iterative solutions {aδk } by −1 δ ∗ F (ak ) (F (aδk ) − y δ ) + αk (aδk − a0 ) (4.5) aδk+1 = aδk − αk I + F (aδk )∗ F (aδk ) and determines the stopping index kδ by F (aδkδ ) − y δ ≤ τ δ < F (aδk ) − y δ ,

(4.6)

0 ≤ k < kδ .

During the computation, all differential equations are solved approximately by finite difference method by dividing the interval [0, 1] into n + 1 subintervals with equal length h = 1/(n + 1); we take n = 200 in our actual computation. δ=0.05 and kδ=28

δ=0.01 and kδ=34

4

4

3

3

2

2

1

0

0.5

1

1

0

δ=0.005 and kδ=36 4

3

3

2

2

0

0.5

1

δ=0.001 and kδ=41

4

1

0.5

1

1

0

0.5

1

Figure 1. Numerical results corresponding to the initial guess given by (4.7) We consider the estimation of a in (4.1) with f = 4 − 4 cos(4t) + 2 sin(4t) + 8t cos(4t) and u0 = u1 = 0. If u = t(1 − t), then a† = 2 + sin(4t) is the desired solution. When applying the method (4.5)–(4.6), we take αk = (1.5)−k and τ = 1.01 and use random noise data uδ satisfying uδ −uL2 [0,1] = δ with noise level δ > 0. In Figure 1 and Figure 2 we plot the numerical results corresponding to two different choices of the initial guess: (4.7) and (4.8)

a0 = 3 − t a0 = 2 + sin(4t) + 3 9 − 4t + 4t2 +

4 t 4e −t e + e 1−e 1−e


1664

QINIAN JIN

δ=0.05 and kδ=20

δ=0.01 and kδ=24

4

4

3

3

2

2

1

0

0.5

1

1

0

δ=0.005 and kδ=26 4

3

3

2

2

0

0.5

1

δ=0.001 and kδ=30

4

1

0.5

1

1

0

0.5

1

Figure 2. Numerical results corresponding to the initial guess given by (4.8)

with various values of the noise level δ > 0, where the solid, dashed, and dashed-dot curves denote the exact solution a† , the initial guess a0 , and the computed solution aδkδ , respectively. For the a0 given by (4.7), a0 −a† has no sourcewise representation a0 − a† ∈ R((F (a† )∗ F (a† ))ν ) with a good ν > 0. Thus no good convergence rate can be expected if the method starts from this a0 . Theorem 3.5, however, guarantees the convergence of aδkδ to a† as δ → 0, which is confirmed by Figure 1. On the other hand, for the a0 given by (4.8) one can check a0 − a† ∈ R(F (a† )∗ ). Thus, according to Theorem 3.7, one can expect aδkδ − a† H 1 = O(δ 1/2 ). This shows the much better accuracy which is confirmed by Figure 2.

References 1. A. B. Bakushinskii, The problems of the convergence of the iteratively regularized GaussNewton method, Comput. Math. Math. Phys., 32 (1992), 1353–1359. MR1185952 (93k:65049) 2. A. B. Bakushinskii, Iterative methods for solving nonlinear operator equations without regularity. A new approach, Russ. Acad. Sci. Dokl. Math., 47 (1993), 451–454. MR1241957 (94m:65099) 3. A. B. Bakushinsky and M. Yu. Kokurin, Iterative Methods for Approximate Solutions of Inverse Problems, Mathematics and its applications, Springer, 2004. MR2133802 (2006e:47025) 4. B. Blaschke, A. Neubauer and O. Scherzer, On convergence rates for the iteratively regularized Gauss-Newton method, IMA J. Numer. Anal., 17 (1997), 421–436. MR1459331 (98f:65066) 5. A. B¨ ottcher, B. Hofmann, U. Tautenhahn and M. Yamamoto, Convergence rates for Tikhonov regularization from different kinds of smoothness conditions, Applicable Analysis, 85 (2006), 555–578. MR2213075 (2006k:65150)



1665

6. H. W. Engl, M. Hanke and A. Neubauer, Regularization of Inverse Problems, Mathematics and its Applications, 375, Kluwer Academic Publishers Group, Dordrecht, 1996. MR1408680 (97k:65145) 7. M. Hanke, A. Neubauer and O. Scherzer, A convergence analysis of the Landweber iteration for nonlinear ill-posed problems, Numer. Math., 72 (1995), 21–37. MR1359706 (96i:65046) 8. Q. Jin and U. Tautenhahn, On the discrepancy principle for some Newton type methods for solving nonlinear inverse problems, Numer. Math., 111 (2009), 509–558. MR2471609 (2010a:65090) 9. Q. Jin and U. Tautenhahn, Inexact Newton regularization methods in Hilbert scales, Numer. Math., 117 (2011), 555–579. MR2772419 (2012b:65078) 10. B. Kaltenbacher, Some Newton-type methods for the regularization of nonlinear ill-posed problems, Inverse Problems 13 (1997), 729–753. MR1451018 (98h:65025) Department of Mathematics, Virginia Tech, Blacksburg, Virginia 24061 E-mail address: [email protected] Current address: Mathematical Sciences Institute, The Australian National University, Canberra, ACT 0200, Australia E-mail address: [email protected]
