Optimal Order of One-Point and Multipoint Iteration - Carnegie Mellon

Optimal Order of One-Point and Multipoint Iteration H . T. R U N G Carnegie-Mellon

A N D J. F . University,

TRAUB Pittsburgh,

Pennsylvania

ABSTRACT. The problem is to calculate a simple zero of a nonlinear function / by iteration. There is exhibited a family of iterations of order 2 which use n evaluations of / and no derivative evaluations, as well as a second family of iterations of order 2 based on n — 1 evaluations of / and one of / ' . In particular, w ith four evaluations an iteration of eighth order is constructed. The best previous result for four evaluations was fifth order. It is proved that the optimal order of one general class of multipoint iterations is 2 and that an upper bound on the order of a multipoint iteration based on n evaluations of / (no derivatives) is 2\ It is conjectured that a multipoint iteration without memory based on n evaluations has optimal order 2 ~K n _ 1

n _ l

T

n _ 1

n

KEY WORDS AND PHRASES: computational complexity, iteration theory, optimal algorithms, nonlinear equations, multipoint iterations CR CATEGORIES:

1.

5.10,

5.15,

5.29

Introduction

W e deal with i t e r a t i o n s for calculating simple zeros of a scalar f u n c t i o n / . T h i s problem is a p r o t o t y p e for m a n y nonlinear numerical problems [11]. N e w t o n - R a p h s o n iteration is p r o b a b l y t h e m o s t widely used algorithm for dealing w i t h such problems. I t is of second order a n d requires t h e evaluation of / a n d / , t h a t is, it uses t w o e v a l u a t i o n s . Consider an i t e r a t i o n consisting of two successive X e w t o n - R a p h s o n iterates (composition of i t e r a t e s ) . T h i s i t e r a t i o n h a s fourth order a n d requires four evaluations, two of / a n d two of / . M o r e generally, an iteration composed of n N e w t o n iterates is of order 2 a n d requires n e v a l u a t i o n s of / a n d n e v a l u a t i o n s o f / , t h a t is, 2n e v a l u a t i o n s . W e shall show t h a t an iteration of order 2 m a y be c o n s t r u c t e d from j u s t n evaluations of / . W e exhibit a second t y p e of iteration which requires n — 1 e v a l u a t i o n s of / a n d one e v a l u a t i o n of / to achieve order 2 ~ . I n p a r t i c u l a r , w i t h four evaluations we c o n s t r u c t an i t e r a t i o n of eighth order. T h e best previous result [10, p . 196] for four e v a l u a t i o n s was fifth order. N e w t o n - R a p h s o n iteration is an example of a one-point iteration. T h e basic o p t i m a l i t y t h e o r e m for one-point iteration s t a t e s t h a t an a n a l y t i c one-point iteration which is based on n e v a l u a t i o n s is of order a t m o s t n. ( T h i s t h e o r e m was first s t a t e d b y T r a u b [9; 10, Sec. 5.4]; we give an i m p r o v e d proof here.) W e conjecture t h a t a m u l t i p o i n t iteration based on n e v a l u a t i o n s h a s o p t i m a l order 2 . W e p r o v e t h a t t h e o p t i m a l order of one i m p o r t a n t family of m u l t i p o i n t iterations is 2 a n d t h a t an u p p e r b o u n d on t h e order of n

n _ 1

n

l

n _ 1

n _ 1

Copyright (c) 1974, Association for Computing Machinery, Inc. General permission to republish, but not for profit, all or part of this material is granted provided that ACM's copyright notice is given and that reference is made to the publication, to its date of issue, and to the fact that reprinting privileges were granted by permission of the Association for Computing Machinery. This research was supported in part by the N S F under Grant GJ-32111 and the Office of Naval Research under Contract N00014-67-A-0314-0010, N R 044-422. Authors' address: Department of Computer Science, Carnegie-Mellon University, Pittsburgh, PA 15213.

Journal of the Association for Computing Machinery, Vol. 21, No. 4, October 1974, pp. 643-651.

H. T. KUNG A N D J . F . TRAUB

644 n

m u l t i p o i n t iteration based on n e v a l u a t i o n s of / is 2 . T h i s u p p e r b o u n d is close t o t h e conn

j e c t u r e d o p t i m a l order of 2 ~ \ T o c o m p a r e various algorithms, we m u s t define efficiency m e a s u r e s based on speed of convergence ( o r d e r ) , cost of e v a l u a t i n g / a n d its derivatives ( p r o b l e m c o s t ) , a n d t h e cost of forming t h e iteration ( c o m b i n a t o r y c o s t ) . W e analyze efficiency in a n o t h e r p a p e r [5]. W e confine ourselves here to iterations without memory, deferring t h e analysis of iterations w i t h m e m o r y to a future paper. W e s u m m a r i z e t h e results of this paper. T h e class of problems a n d a l g o r i t h m s s t u d i e d in this p a p e r is defined in Section 2. P a r t i c u l a r families of iterations are defined in Sections 3 - 5 . T h e o p t i m a l i t y t h e o r e m for one-point i t e r a t i o n s is proven in Section 6. A n o p t i m a l order t h e o r e m for one general class of m u l t i p o i n t iterations a n d an u p p e r b o u n d for t h e order of a second class are proven in Section 7. A general conjecture is s t a t e d in Section 8. Section 9 contains a small numerical example. T h e A p p e n d i x gives ALGOL p r o g r a m s for forming t w o families of m u l t i p o i n t i t e r a t i o n s . 2.

Definitions

W e define t h e ensemble of problems a n d a l g o r i t h m s . Let D = \f | / is a real analytic function defined on an open interval / / C R which contains a simple zero a

f

of / a n d /

does

n o t v a n i s h on / / . } . Let 12 d e n o t e t h e set of functions 4> which m a p s every / £ D t o , f)

x-*a

f

exists for a c o n s t a n t S(4>, / ) and S(, / ) ^ 0 for a t least one / £ D, t h e n 4> is said t o h a v e order of convergence ( o r d e r ) p() a n d a s y m p t o t i c error c o n s t a n t S(4>,f).

Optimal

Order of One-Point

and Multipoint

645

Iteration

T h i s is p e r h a p s t h e simplest definition of order of convergence. T h e results of t h i s p a p e r hold, w i t h s u i t a b l e modifications, for weaker definitions of order [7], b u t we feel t h a t use of t h e s e definitions would obscure t h e proofs w i t h o u t s u b s t a n t i a l l y s t r e n g t h e n i n g t h e results. L e t Vi() d e n o t e t h e n u m b e r of e v a l u a t i o n s of f used t o c o m p u t e (/) ( x ) . T h e n v{4>) = 2Zi>o Vi() is t h e t o t a l n u m b e r of e v a l u a t i o n s required b y (f)(x) p e r s t e p . l)

T o simplify n o t a t i o n , we often use a, , p, v

iy

v i n s t e a d of a , f

< £ ( / ) ( x ) , p(), i\(),

v(), if t h e r e is n o a m b i g u i t y . T h e following t w o examples i l l u s t r a t e t h e definitions. Example

2 . 1 . ( N e w t o n - R a p h s o n I t e r a t i o n ) ( / ) : / / —• R, j = 1, • • • , n, as follows: 7 i ( / ) ( x ) = x a n d for n > 1, 7/+i(/)(*)

j

= 7>(/)(*) + [ ( - 1 ) V j ! ] - [ / ( ^ ) ] ^ (j)

for j = 1, • • • , n — 1. N o t e t h a t F (f(x))

( ; )

(/(^))

(3.1)

can b e expressed in t e r m s o f /

( , )

( x ) for i = 1,

2, • • • , j . I t is easy t o s h o w t h a t , for e x a m p l e , 7 i = x,

72 = 7 i ~ fix)/f(x),

73 = 72 ~

, ,

2

l/ (x)/2/(x)][/(x)//(x)] .

T h e family J 7 | h a s b e e n t h o r o u g h l y s t u d i e d [10, Sec. 5.1]. I t s essential properties a r e n

s u m m a r i z e d in THEOREM 3 . 1 . point

iteration;

Lety

be defined by ( 3 . 1 ) . Then for n > 1, ( 1 ) y

n

( 2 ) p(y )

n

6 fi, andy

is a one-

n

= n, ( 3 ) v»(7 ) = 1, i = 0, • • • , n — 1, v -(7n)

n

n

=

t

0, t

>

n — I. Hence v(y ) = n. T h u s 7„ requires t h e e v a l u a t i o n of / a n d its first n — 1 d e r i v a t i v e s . I n Section 6 we shall s h o w t h a t , u n d e r a mild s m o o t h n e s s condition on t h e i t e r a t i o n , every one-point i t e r a t i o n of o r d e r n requires t h e e v a l u a t i o n of a t l e a s t / a n d its first n — 1 d e r i v a t i v e s . n

4.

A Family

We construct points,

of Multipoint a family

no evaluation

Iterations

of multipoint

iterations,

\^ ), n

which require the evaluation

of derivatives of f, and for which p(\[/ ) n

F o r e v e r y n, define rf/jif)

C I

f

= 2

n _ 1

of f at n

.

—> 7^.,/, j = 0, • • • , n, as follows: \f/o(f)(x)

= x

a n d for n > 0, x

$i(f)( )

—

x

+

fif(x),

/3 a n o n z e r o c o n s t a n t , j

= (MO),

)

for j = 1, • • • , n — 1, where (?.,•( i/) is t h e inverse i n t e r p o l a t o r y polynomial for / a t /(&(/) (x)), k = 0, • • • , j . T h a t is, Q>(i/) is t h e p o l y n o m i a l of degree a t m o s t j such t h a t

Q,(f(Mf)(*)))

= Mf)M,k

= o, ... ,j.

T h e ypj{f), j = 1, • • • , n, a r e well defined if

T h a t (4.2) holds for I+ j jt

sufficiently small will be p a r t of t h e proof of T h e o r e m 4 . 1 .

646

H. T. KUNG AND J . F . TRAUB

I t is easy to show t h a t , for example,

^3 = ^ - K+o)f(h)/U(h)

-

Mo)}

• { ( * ! -

*o)/[/(*l) "

/(Ml

-

(fc -

*l)/[/(tf )

~

2

A n ALGOL p r o g r a m ( P r o g r a m 1) is given in t h e Appendix for c o m p u t i n g \p for n > 4. O u r interest in t h e family of iterations {\p \ is due to t h e properties proved in THEOREM 4 . 1 . Let \p be defined by ( 4 . 1 ) . Then for ?i > 1, (1) ^ £ 12, and \p is an n-point iteration, (2) p(\f/ ) = 2 ~ , ( 3 ) v (\p ) = n, * > i ( ^ n ) = 0, i > 0. Hence v(\(/ ) = n. PROOF. W e w a n t to show t h a t , for f (z D, n

n

n

n

n

0

lim(*» -

n

l

n

2n

«)/(* -

n

l

a) ~

n

=

n = 1,2, . . .

(4.3)

for c o n s t a n t s < S ( ^ „ , / ) . T h e proof is b y induction on n. Since l i m ^ (^1 — a)/(x — a) = 1 + (4.3) holds for n = 1. Assume t h a t (4.3) holds for n = 1, • • • , m — 1. F r o m general interpolatory iteration t h e o r y [10, C h . 4], we k n o w t h a t lim[(*

m

-

a)/

II

x-*a n+1

(*» -

im)

m

where Y (f) = (-l) F (0)/(m\[F'(Q)] ) (4.4) a n d t h e induction hypothesis, - a)/(x

2

-

(4.4)

m

a n d F is t h e inverse function o f / .

m

lim

«)] = F ( / ) ,

0