Inexact Newton's method to nonlinear functions with ... - CiteSeerX

Inexact Newton’s method to nonlinear functions with values in a cone O. P. Ferreira∗

G. N. Silva

†

August 17, 2015

Abstract The problem of finding a solution of nonlinear inclusion problems in Banach space is considered in this paper. Using convex optimization techniques introduced by Robinson (Numer. Math., Vol. 19, 1972, pp. 341-347), a robust convergence theorem for inexact Newton’s method is proved. As an application, an affine invariant version of Kantorovich’s theorem and Smale’s α-theorem for inexact Newton’s method is obtained. Keywords: Inclusion problems, inexact Newton’s method, majorant condition, semi-local convergence.

1

Introduction

In this paper we study the inexact Newton’s method for solving the nonlinear inclusion problem F (x) ∈ C,

(1)

where F : Ω → Y is a nonlinear continuously differentiable function, X and Y are Banach spaces, X is reflexive, Ω ⊆ X an open set and C ⊂ Y a nonempty closed convex cone. The idea of solving a nonlinear inclusion problems of the form (1), plays a huge role in classical analysis and its applications. For instance, the special case in which C is the degenerate cone {0} ⊂ Y, the inclusion problem in (1) correspond to a nonlinear equation. In the case for which X = Rn , Y = Rp+q and C = Rp− × {0} is the product of the nonpositive orthant in Rp with the origin in Rq , the inclusion problem in (1) correspond to a nonlinear systems of p inequalities and q equalities, for example see [2, 7, 4, 8, 9, 16, 19, 20]. In order to solving (1), Robinson in [22] proposed the following iterative methods of Newton-type:  xk+1 = xk + dk , dk ∈ arg min kdk : F (xk ) + F 0 (xk )d ∈ C , k = 0, 1, . . . . (2) d∈X

In general, this algorithm may fail to converge and may even fail to be well defined. To ensure that the method is well defined and converges to a solution of a given nonlinear inclusion, Robinson made two important assumptions: H1. There exists x0 ∈ X such that rge Tx0 = Y, where Tx0 : X ⇒ Y is the convex process given by Tx0 d := F 0 (x0 )d − C,

d ∈ X,

and rge Tx0 = {y ∈ Y : y ∈ Tx0 (x) for some x ∈ X}, see [9] for adicional details. ∗ IME/UFG, CP-131, CEP 74001-970 - Goiˆ ania, GO, Brazil (Email: [email protected]). The author was supported in part by CAPES (Projeto 019/2011- Coopera¸ca ˜o Internacional Brasil-China), CNPq Grants 471815/2012-8, 444134/2014-0 and 305158/2014-7, PRONEX–Optimization(FAPERJ/CNPq) and FAPEG/GO. † CCET/UFOB, CEP 47808-021 - Barreiras, BA, Brazil (Email: [email protected]). The author was supported in part by CAPES .

1

H2. F 0 is Lipschitz continuous with modulo L, i.e., kF 0 (x) − F 0 (y)k ≤ L kx − yk, for all x, y, ∈ X. Under these assumptions, it was proved in [22], that the sequence {xk } generate by (2) is well defined and converges to x∗ satisfying F (x∗ ) ∈ C, provided that following convergent criterion is satisfied: kx1 − x0 k ≤

1 . 2LkTx−1 0 k

The first affine invariante version in this result was presented by Li and Ng in [20]. In [21] Li and Ng introduced the notion of the weak-Robinson condition for convex processes and presented a extension of the results of [20] under a L-average Lipschitz condition. As an applications, two special cases were provided, namely, the convergence result of the method under Lipschitz’s condition and Smale’s condition. In [12], under a affine majorant condition, a robust analysis of this method were established. As in [20], the analysis under Lipschitz’s condition and Smale’s condition are also obtained as special case, see also [1, 5]. The inexact Newton method, for solving nonlinear equation F (x) = 0, was introduced by Dembo, Eisenstat, and Steihaug in [6] for denoting any method which, given an initial point x0 , generates the sequence {xk } as follows: kF (xk ) + F 0 (xk )(xk+1 − xk )k ≤ ηk kF (xk )k,

k = 0, 1, . . . ,

(3)

and {ηk } is a sequence of forcing terms such that 0 ≤ ηk < 1. In [6] was proved, under suitable assumptions, that for any starting point x0 in a certain neighborhood of the solution x∗ , the sequence {xk } generated by (3) is convergent for this solution with super-linear rate. In [18] numerical issues about this method are discussed. In the present paper, we extend the inexact Newton’s method (3), for solving nonlinear inclusion, as any method which, given an initial point x0 , generates a sequence {xk } as follows:  xk+1 = xk + dk , dk ∈ arg min kdk : F (xk ) + F 0 (xk )d + rk ∈ C , (4) d∈X

max

w∈{−rk , rk }

−1

Tx w ≤ θ Tx−1 [−F (xk )] , 0 0

(5)

for k = 0, 1, . . ., 0 ≤ θ < 1 is a fixed suitable tolerance, and  Tx−1 (y) := d ∈ X : F 0 (x0 )d − y ∈ C , y ∈ Y. 0 The analysis of this method, under Lipschitz’s condition and Smale’s condition, are provided as special case. It is well known that the mapping Tx−1 is convex process with closed graphic; see Corollary 4A.7 0 of [9]. Hence, if θ = 0 then (4)-(5) reduces to extended Newton method (2) for solving (1), and, in the case, C = {0} it reduces to affine invariante version of (3), which was studied by Ferreira and Svaiter in [15]. Up to our knowledge, this is the first time that the inexact Newton method to solving cone inclusion problems with a relative error tolerance is analyzed. The problem in (1) is a particular instance of the following generalized equation F (x) + C(x) 3 0,

(6)

when C(x) ≡ −C, where C : X ⇒ Y is a set valued mapping. In [11] (see also [3]), Dontchev and Rockafellar proposed the following iterative methods of Newton-type for solving (6): (F (xk ) + F 0 (xk )(xk+1 − xk ) + C(xk+1 )) ∩ Rk (xk , xk+1 ) 6= ∅,

k = 0, 1, . . . ,

(7)

where Rk : X × X ⇒ Y is a sequence of set-value mappings with closed graphs. Note that, in the case, when C(x) ≡ 0, θ ≡ ηk and Rk (xk , xk+1 ) ≡ Bηk kf (xk )k (0), 2

the iteration (7) reduces to (3). In the case C(x) ≡ −C, the iteration (7) has (4)-(5) as a minimal norm affine invariante version. Therefore, in some sense, our method is a particular case of [11]. However, the analysis presented in [11] is local, i.e, it is made assumption at a solution, while in our analysis we will not assume existence of solution. In fact, our aim is to prove a robust Kantorovich’s Theorem for (4)-(5), under assumption H1 and an affine invariant majorant condition generalizing H2, which in particular, prove existence of solution for (1). Moreover, the analysis presented, show that the robust analysis of the inexact Newton’s method for solving nonlinear inclusion problems, under affine Lipschitz-like and affine Smale’s conditions, can be obtained as a special case of the general theory. Besides, for the degenerate cone, which the nonlinear inclusion becomes a nonlinear equation, our analysis retrieves the classical results on semi-local analysis of inexact Newton’s method; [15]. The organization of the paper is as follows. In Section 1.1, some notations and basic results used in the paper are presented. In Section 2, the main results are stated and in Section 2.1 some properties of the majorant function are established and the main relationships between the majorant function and the nonlinear operator used in the paper are presented. In Section 3, the main results are proved and the applications of this results are given in Section 4. Some final remarks are made in Section 5.

1.1

Notation and auxiliary results

The following notations and results are used throughout our presentation. We beginning with the following elementary convex analysis result: Proposition 1. Let I ⊂ R be an interval and ϕ : I → R be convex. For any s0 ∈ int(I), the left derivative there exist (in R) ϕ(s0 ) − ϕ(s) ϕ(s0 ) − ϕ(s) D− ϕ(s0 ) := lims→s− = sups 0. If {xk } converges to x∗ and satisfies kxk+1 − xk k ≤ Θkxk − xk−1 k,

k = 1, 2, . . . ,

(8)

then {xk } converges Q-linearally to z∗ as follows lim sup k→∞

kxk+1 − x∗ k ≤ Θ. kxk − x∗ k

Proof. The proof follows the same pattern as the proof of Proposition 2 of [13]. Let X be a Banach space. The open and closed ball at x with radius δ > 0 are denoted, respectively, by B(x, δ) := {y ∈ X : kx − yk < δ} and B[x, δ] := {y ∈ X : kx − yk 6 δ}. Let X and Y be Banach spaces. A set valued mapping T : X ⇒ Y is called sublinear or convex precess when its graph is a convex cone, i.e., 0 ∈ T (0),

T (λx) = λT (x),

λ > 0,

T (x + x0 ) ⊇ T (x) + T (x0 ), x, x0 ∈ X,

(9)

(sublinear mapping has been extensively studied in [9], [23], [24] and [25]). The following definitions and results about sublinear mappings will be need: The domain and range of a sublinear mapping T are defined, respectively, by dom T := {d ∈ X : T d 6= ∅},

rge T := {y ∈ Y : y ∈ T (x) for some x ∈ X}. 3

The inverse T −1 : Y ⇒ X of a sublinear mapping T is another sublinear mapping defined by T −1 y := {d ∈ X : y ∈ T d},

y ∈ Y.

The norm (or inner norm as is called in see [9]) of a sublinear mapping T is defined by kT k := sup {kT dk : d ∈ dom T, kdk 6 1},

(10)

where kT dk := inf{kvk : v ∈ T d} for T d 6= ∅. We use the convention kT dk = +∞ for T d = ∅, it will be also convenient to use the convention T d + ∅ = ∅ for all d ∈ X. Lemma 3. Let T : X ⇒ Y be a sublinear mapping with closed graph. Then dom T = X if and only if kT k < +∞ and rge T = Y if and only if kT −1 k < +∞. Proof. See Corollary 5C.2 of [9]. Let S, T : X ⇒ Y and U : Y ⇒ Z be sublinear mappings. The scalar multiplication, addition and composition of sublinear mappings are sublinear mappings defined, respectively, by [ (αS)(x) := αS(x), (S + T )(x) := S(x) + T (x), U T (x) := {U (y) : y ∈ T (x)} , for all x ∈ X and α > 0 and the following norm properties there hold: kαSk = |α|kSk,

kS + T k 6 kSk + kT k,

kU T k 6 kU kkT k.

Remark 1. Note that definition of the norm in (10) implies that if dom T = X and A is a linear mapping from Z to X then kT (−A)k = kT Ak. Lemma 4. Let S, T : X ⇒ Y be a sublinear mappings with closed graph such that dom S = dom T = X and kT −1 k < +∞. Suppose that kT −1 kkSk < 1 and (S +T )(x) is closed for each x ∈ X then rge (S +T )−1 = X and kT −1 k k(S + T )−1 k 6 . 1 − kT −1 kkSk Proof. These results follows from Theorem 5 of [23] by taking into account Lemma 3. Lemma 5. Let G : [0, 1] → Y and g : [0, 1] → R be continuous function and Z a reflexive Banach space. Suppose that U : Y ⇒ Z is a sublinear mapping with closed graphic such that dom U ⊇ rge G. If kU G(τ )k 6 g(τ ), then there hold Z

1

G(τ )dτ 6= ∅,

U 0

τ ∈ [0, 1],

Z

U

0

1

Z

G(τ )dτ

6

1

g(τ )dτ.

0

Proof. See Lemma 2.1 of [20]. Let Ω ⊆ X be an open set and F : Ω → Y a continuously Fr´echet differentiable function. The linear map F 0 (x) : X → Y denotes the Fr´echet derivative of F : Ω → Y at x ∈ Ω. Let C ⊂ Y be a nonempty closed convex cone, z ∈ Ω and Tz : X ⇒ Y a mapping defined as Tz d := F 0 (z)d − C.

(11)

It is well known that the mappings Tz and Tz−1 are sublinear with closed graphic, dom Tz = X, kTz k < +∞ and, moreover, rge Tz = Y if and only if kTz−1 k < +∞ (see Lemma 3 above and Corollary 4A.7, Corollary 5C.2 and Example 5C.4 of [9] ). Note that Tz−1 y := {d ∈ X : F 0 (z)d − y ∈ C}, 4

z ∈ Ω, y ∈ Y.

(12)

Lemma 6. Let X and Y be Banach spaces, Ω ⊆ X an open set and F : Ω → Y a continuously Fr´echet differentiable function. Then the following inclusion holds Tz−1 F 0 (v)Tv−1 w ⊆ Tz−1 w,

v, z ∈ Ω, w ∈ Y.

As a consequence,

−1  0   

Tz F (y) − F 0 (x) ≤ Tz−1 F 0 (v)Tv−1 F 0 (y) − F 0 (x) ,

v, x, y, z ∈ Ω.

Proof. See [12].

2

Kantorovich’s Theorem for Inexact Newton’s method

Our goal is to state and prove a robust semi-local affine invariant theorem for inexact Newton’s method to solve nonlinear inclusion of the form (1), for state this theorem we need some definitions. A nonlinear continuously Fr´echet differentiable function F : Ω → Y satisfies the Robson’s Condition at x0 ∈ Ω if rge Tx0 = Y, where Tx0 : X ⇒ Y is a sublinear mapping as defined in (11). Let X and Y be a Banach spaces, Ω ⊆ X an open set and R > 0 a scalar constant. A continuously differentiable scalar function f : [0, R) → R is a majorant function at a point x0 ∈ Ω for a continuously differentiable function F : Ω → Y if

−1  0 

Tx F (y) − F 0 (x) 6 f 0 (kx − x0 k + ky − xk) − f 0 (kx − x0 k), (13) B(x0 , R) ⊆ Ω, 0 for all x, y ∈ B(x0 , R) such that kx − x0 k + ky − xk < R and satisfies the following conditions: h1) f (0) > 0, f 0 (0) = −1; h2) f 0 is convex and strictly increasing; h3) f (t) = 0 for some t ∈ (0, R). We also need of the following condition on the majorant condition f which will be considered to hold only when explicitly stated. h4) f (t) < 0 for some t ∈ (0, R). Remark 2. Since f (0) > 0 and f is continuous then condition h4 implies condition h3. Theorem 7. Let X, Y be Banach spaces, X reflexive, Ω ⊆ X an open set, F : Ω → Y a continuously Fr´echet differentiable function, C ⊂ Y a nonempty closed convex cone, R > 0 and f : [0, R) → R a continuously differentiable function. Suppose that x0 ∈ Ω, F satisfies the Robson’s condition at x0 , f is a majorant function for F at x0 and

−1

Tx [−F (x0 )] 6 f (0) . (14) 0 Let β := sup{−f (t) : t ∈ [0, R)},

τ¯ := sup{t ∈ [0, R) : f (t) < 0}.

Take 0 ≤ ρ < β/2 and define −(f (t) + 2ρ) , 0 ρ