Convergence Speed of an Integral Method for ...

In: J. M. Bomze et al. (eds.): Developements in Global Optimization, 153-170. Kluwer Academic Publishers 1997

Convergence Speed of an Integral Method for Computing the Essential Supremum Jens Hichert and Armin Homann

Technical University of Ilmenau, Institute of Mathematics, PF 0565, D-98684 Ilmenau, Germany

Hoang Xu^an Phu

Institute of Mathematics, P.O.Box 631 Bo Ho, 10000 { Hanoi, Vietnam

Abstract. We give an equivalence between the tasks of computing the essential

supremum of a summable function and of nding a certain zero of a one-dimensional convex function. Interpreting the integral method as Newton-type method we show that in the case of objective functions with an essential supremum that is not spread the algorithm can work very slowly. For this reason we propose a method of accelerating the algorithm which is in some respect similar to the method of Aitken/Steensen. Key words: essential supremum, convergence speed, integral global optimization, Newton algorithm

1. Introduction The problem of determining the essential supremum of a summable function f over its domain D  IR n can be regarded as a generalization of the task of global optimization. If the maximum of f over D does not exist, since D is not closed or f is not upper semicontinuous, the supremum can be determined instead of the maximum. If f is not completely de ned for each point, as in Lebesgue spaces Lp , then only the essential supremum is into consideration. Even if the existence of the maximum is guaranteed and the information on f is complete, the maximum is sometimes senseless, from a practical point of view, because it re ects only an individual case which is not representative for a general situation. Moreover, in technical or natural systems the maximum can describe a not desirable unstable state of the parameters. This is again a motivation for the computation of the essential supremum. Let D  IR n be a measurable set with (D) < 1 where  denotes the n-dimensional Lebesgue measure. Furthermore let f : D ! IR be an L1 function. Our purpose is to determine the real number ess sup f := inf f 2 IR : fx 2 D : f > g = 0g; which is called the essential supremum of f over D. In this paper we sometimes make use of the essential in mum that we simply de ne by

2 Jens Hichert et al. ess inf f := ess sup ( f ). The idea of nding essential (global) optima instead of global optima is quite old. Methods for approximating the level sets are discussed for example by Chichinadze (1967), Archetti and Betro (1975), De Biase and Frontini (1978). In a former paper, Phu and Homann (1996) developed a theoretical algorithm which generates an increasing sequence (k ) of levels of f converging to ess sup f . This algorithm is based on a Newton-type method applied to a one-dimensional function. The motivation of the present paper is to look more closely at the algorithm mentioned above under two questions. The rst one concerns the convergence speed of the level sequence (k ). This question is essential because there are only a few theoretical results in evaluating and comparing dierent global optimization algorithms. As a second question, we deal with the implementation of the algorithm considered. One possible way is to apply Monte Carlo models. Some numerical results have been obtained. The paper is organized as follows. In Section 2 we state the algorithm and some of its main properties. The results are related to Phu and Homann and therefore, they are given without proofs. Section 3 discusses the convergence speed of the level sequence generated by the algorithm depending on properties of the function f . As a consequence, in Section 4 we construct a second algorithm as a faster modi cation of the rst one. Section 5 contains a brief discussion on a numerical implementation of the two algorithms. Numerical results are given where we focus on the progress achieved in the convergence speed. To simplify the notation, we introduce the following abbreviations. Throughout the paper, [f  ] stands for the level set fx 2 D : f (x)  g. We use the same notation if we deal with other relational operators, hence it is clear what [f  ], [f = ] etc. mean.

2. An Algorithm for Computing ess sup f An algorithm that computes ess sup f was developed by Phu and Homann (1996). In their paper they introduce a function Vf : IR ! IR de ned by Z Vf () := [f (x) ] d: [f ]

We call Vf the volume function of f . Some remarkable properties of Vf have been proved (recall that f is arbitrary from L1 ):

ess_sup.tex; 29/05/1996; 16:04; no v.; p.2

3 THEOREM 2.1. The function Vf is Lipschitz-continuous, non-negative, non-increasing, convex, and has almost everywhere the derivative CONVERGENCE SPEED OF AN INTEGRAL METHOD...

Vf0 () = [f  ]: Using the function Vf we can characterize the number ess sup f in an alternative way. Consider  := supf 2 IR : Vf () > 0g, then it holds: THEOREM 2.2.

 = ess sup f:

Furthermore, we have the implications

[f  ] > 0 if Vf () > 0 Vf () = 0 if [f  ] = 0: In other words, computing ess sup f can be understood as nding the smallest zero of a one-dimensional function possessing the properties stated in Theorem 2.1. Based on this result we state a Newton type algorithm for determining  . Here and in the following, let the function m : IR ! IR be de ned by m() := [f  ]. Assume that one initial value 0 satisfying m(0) > 0 has been found. Now, a sequence (k ) is generated by k+1 = k + Vmf((k)) k = 0; 1; : : : (1) k Since m() 2 @Vf () for each  2 IR (here @Vf () denotes the subdierential of Vf in , cf. Hiriart-Urruty and Lemarechal, 1993), the point k+1 is given by the intersection point of the tangent line of Vf in k with the abscissae. This leads to the result of THEOREM 2.3. The sequence (k ) generated by (1) converges monotonously to  . In order to ensure that the algorithm stops after a nite number of steps achieving a level k satisfying jk  j < " for a tolerance " > 0 given in advance, we need an additional test. So the algorithm is organized as follows: ALGORITHM 1.

Step 0: Choose " > 0 and 0 2 IR satisfying Vf (0) > 0. Set k = 0.

ess_sup.tex; 29/05/1996; 16:04; no v.; p.3

4

Jens Hichert et al.

Step 1: Compute sk = Vmf((kk)) . Step 2: If (sk > ")

Set k+1 = k + sk , set k = k + 1 and goto Step 4. Step 3: If (Vf (k + ") = 0) STOP. k is an approximate value with 0  ess sup f k  ".

Else

Set k+1 = k + ", set k = k + 1 and goto Step 1. Step 4: If (Vf (k ) = 0) STOP. k = ess sup f .

Else

Goto Step 1.

A further interesting aspect of Algorithm 1 is that, under the assumption 0 < ess inf f , after one iteration it always leads to the same level, independent of the fact how far the initial 0 lies under the essential in mum of f . Precisely, we have PROPOSITION 2.1. Let 1 be determined as in (1). Then inf f ) 2 [ess inf f; ess sup f ] 1 = ess inf f + Vf (ess (D) holds true for all 0 < ess inf f . Numerical examples of the algorithm are given in the following sections.

3. Convergence Speed of the Level Sequence The practicability and numerical eectiveness of Algorithm 1 mainly depends on the way of evaluating the functions Vf and m. Note that in Algorithm 1 both the quotient of Vf and m and for the stopping rules explicitely Vf is used. Until now, there has been just one appropriate method based on Monte Carlo models, computing the quotient in one procedure and using higher moment integral conditions for stopping rules. This approach was thoroughly developed in the book of Chew and Zheng (1988) (see also Zheng, 1992; Kostreva and Zheng, 1994; Zheng and Zhuang, 1995). Therefore, our viewpoint can shed some

ess_sup.tex; 29/05/1996; 16:04; no v.; p.4

CONVERGENCE SPEED OF AN INTEGRAL METHOD...

5

new light on those results. Nevertheless, independent of the way the algorithm is implemented, one evaluation of the n-dimensional integral Vf will be very expensive. Therefore, the convergence speed of the level sequence (k ) is of great interest. In the rst part of this section, we state some smoothness properties of Vf . Secondly, we deal with the convergence speed of the sequence (k ). LEMMA 3.1. The function m is left-hand continuous. Proof. Let us x an  2 IR. Since m is monotone, m( 0) := limt! 0 m(t) exists. We take a sequence (n ) satisfying n % . Furthermore, consider the set sequence (An ) de ned by An := [f  n ]. This de nition implies An  An+1 and \j n Aj = [f  ] for each n. Now we conclude that   1 Aj = [f  ] = m(): m( 0) = nlim  ( A ) =  \ n j =1 !1

2

PROPOSITION 3.1. Vf is continuously dierentiable in an open set U  IR i [f = ] = 0 for all  2 U . Proof. Again, we x an  2 IR and take a sequence ( n ) with the property n & . Furthermore, consider the set sequence (Bn ) de ned by Bn := [f  n ]. Then, 1 m( + 0) = nlim (2) !1  (Bn ) =  ([i=1 Bi ) = [f > ]: Now assume that Vf is continuously dierentiable in U . Therefore Vf0 () = m() holds for all  2 U . So we can conclude that m is continuous in U . Thus we get m( 0) = m( + 0) 8 2 U . From Lemma 3.1 and (2) follows that the continuity of m in U is equivalent to the condition [f = ] = 0 for all  2 U . The proof will be completed by showing that from m continuous in U follows Vf 2 C 1(U ). But this is clear because Vf is absolutely continuous and Vf0 () = m() a.e. on D. 2 A sucient condition for the dierentiability of Vf provides PROPOSITION 3.2. Let D  IR n be an open set and f 2 C 1(D). If rf (x) 6= 0 a.e. on D, then [f = ] = 0 for all  2 IR. Proof. Choose any  2 IR and de ne the set

M := fx 2 D : f (x) = ; rf (x) 6= 0g:

ess_sup.tex; 29/05/1996; 16:04; no v.; p.5

6 Jens Hichert et al. Since rf 6= 0 and f 2 C 1 (D), M is a regular manifold of the dimension n 1. Consequently, (M) = 0 (cf. Dieudonne, 1975). From [f = ] = M [ (fx 2 D : rf (x) = 0g \ [f = ]) and  (fx 2 D : rf (x) = 0g) = 0 follows that [f = ] = 0. 2 Remark 3.1. The inverse statement of Proposition 3.2 is not true without additional assumptions. This is demonstrated by the next example.

EXAMPLE 3.1. First we construct a nowhere dense set F  [0; 1] with positive measure. Let QI \ (0; 1) = frk ; k = 1; 2; : : :g be the set of all rational numbers in the interval (0; 1). Let k =  k 1 ; k = 1; 2; : : :. Then, for 4

G :=

1 [ k=1

[(rk k ; rk + k ) \ (0; 1)]

the inequality

1  1 k 2 X 1 0 < (G)  2 = 3 0 G can be represented by the union of denumerably many disjoint intervals (see, for example, Natanson, 1975) Gk = (ak ; bk ) k = 1; 2; : : : We de ne the function ' : [0; 1] 7! IR by  '(x) = (x a )(b x0) ifif xx 22 FG k k k

ess_sup.tex; 29/05/1996; 16:04; no v.; p.6

CONVERGENCE SPEED OF AN INTEGRAL METHOD...

as well as f : [0; 1] 7! IR by

f (x) =

Zx

0

7

'(t)d(t):

It is easy to prove that ' is Lipschitz-continuous and f is strictly increasing in [0; 1]. Consequently, [f = ] = 0 for all  2 IR. But f 0 (x) = '(x) = 0 holds for all x 2 F where (F )  31 > 0. In order to investigate the speed of convergence we give the DEFINITION 3.1. We say that the essential supremum of f is spread, if [f   ] > 0 holds. If ess sup f is spread, the well-known theory of the Newton algorithm is available. THEOREM 3.1. Let ess sup f be spread and assume Vf 2 C 1(0 ; ) with an 0 <  . Then, either the algorithm stops after a nite number of steps or the convergence of the level sequence (k ) generated by (1) is Q-superlinear. Proof. According to Theorem 2.3 we can assume that the algorithm started at the initial point 0. We begin by considering the case that there is an  satisfying 0   <  with the property that Vf restricted on [;  ) is a linear function. Obviously there is such an s 2 IN that s > . This clearly forces s+1 =  . We now turn to the case that Vf is strictly convex in an interval [~; ]. We de ne V~f : IR ! IR by 

Vf () if  2 ( 1;  ] ) if  2 ( ; 1) From Lemma 3.1 it is immediately clear that V~f 2 C 1(0; 1) and, moreover,  is a regular zero of V~f . Applying Algorithm 1 to both Vf and V~f provides the same level sequence (k ). Hence, due to a well known theorem (cf. Kosmol, 1993) the level sequence (k ) converges Q-superlinearly to  . 2 V~f () =

m()(

If, under the assumptions of Theorem 3.1, Vf0 additionally satis es a Lipschitzian condition in a neighbourhood of  , the convergence of the level sequence (k ) generated by (1) is Q-quadratic. This fact immediately results from the proof of Theorem 3.1 and the corresponding theorem on the Q-quadratic convergence of Newton's algorithm.

ess_sup.tex; 29/05/1996; 16:04; no v.; p.7

8 Jens Hichert et al. EXAMPLE 3.2. For each " > 0 let D" = [0; 32 + "]  IR and f" : D" ! IR de ned by (

( 23 x 1)2 if x 2 [0; 32 ) 0 if x 2 [ 32 ; 32 + "] The functions f" have a spread essential supremum because [f"    ] = [f"  0] = " > 0. Computing the volume functions yields

f" (x) =

8 < Vf" () = :

( 32 +p") 21 if  2 ( 1; 1] ( jj + ") if  2 ( 1; 0] 0 if  2 (0; 1)

Obviously, Vf" 2 C 1( 1; 0) and Vf" is strictly convex in [ 1; 0]. Hence, we get Q-superlinear convergence of the level sequence (k ). But Vf0" is not Lipschitzian in  =  = 0 and it can be shown that the convergence is not Q-quadratic. It is our conjecture that the rst assumption of Theorem 3.1 is in general not ful lled. Indeed, in many practical optimization problems the solution set [f =  ] consists of at most denumerably many elements. In these cases Algorithm 1 can converge arbitrarily slowly, which is illustrated by the following two examples. EXAMPLE 3.3. Take any a 2 IRn and let D be the closed ball with center in a and radius r > 0. De ne f : D ! IR by

f (x) =

n X i=1

(xi ai )2:

Clearly, ess sup f = 0. Below (cf. (5),(6)) we show that 8 < (D) + b if  2 ( 1; ess inf f ] Vf () = : cnjj1+n=2 if  2 (ess inf f; 0) 0 if  2 [0; 1) where cn > 0 is a certain constant and b results from the continuity of Vf in ess inf f . Due to Proposition 2.1 we can assume that 0 > ess inf f . The level sequence is the geometric sequence 

n

k

k+1 = n + 2 0 :

n Therefore, linear convergence is ensured. The convergence rate n + 2 becomes worse with increasing dimension n of the problem.

ess_sup.tex; 29/05/1996; 16:04; no v.; p.8

CONVERGENCE SPEED OF AN INTEGRAL METHOD... EXAMPLE 3.4. Consider the function f : [0; 21 ] ! IR de ned ( 0 if x = 0 f (x) = q 1 if x 2 (0; 1 ] ln x 2

9 by

Its volume function is 8 > < Vf () = > :

1=2 + b if  2 ( 1; ess inf f ] e e dt p if  2 (ess inf f; 0) 0 if  2 [0; 1) where b results from the continuity of Vf in ess inf f . Running Algorithm 1 (cf. Section 5) shows that the level sequence converges much slower than linearly. 1 2

R1 2 0

t2

4. Speeding Up the Level Sequence As we illustrated in Section 3, the Newton method applied to functions which have not a spread essential supremum converges slowly. For this reason we develop a modi ed algorithm producing a level sequence converging faster to the essential supremum. For the rest of this paper we only consider functions f whose essential supremum is not spread, without explicitely mentioning this fact again. The basic idea of our approach is to approximate the function Vf by a polynom of the order greater than 1. Precisely, we extend Algorithm 1 by choosing in the k-th iteration a stepsize k  1 such that the iteration formula (3) k+1 = k + k Vmf((k)) k = 0; 1; : : : k leads to a new level k+1 . At the same time we have to ensure k+1 <  because if the new level passes over the optimal level  , no information on the distance j k+1 j is available and thus, no error handling is possible. For a speci c class of functions an appropriate stepsize is given naturally. Consider the function

f (x) =

n X i=1

ai jxi xi jp + 

(4)

with some p  1, x 2 IRn ,  2 IR and ai > 0 for all i. Then, under some assumptions, the order of the zero  of Vf can be given exactly. In the follwing, we call a set D  IRn robust, if cl int D = cl D holds.

ess_sup.tex; 29/05/1996; 16:04; no v.; p.9

10 Jens Hichert et al. PROPOSITION 4.1. Let f be de ned by (4) in a robust D  domain  n n  IR and assume x 2 intD. Then, by choosing k = 1 + p , the iteration formula (3) yields k+1 = ess sup f =  for all k 2 (ess inf f; ess sup f ) close enough to  . Proof. Without loss of generality we can assume that x is the origin and  = 0. This can easily be seen by considering f~ : D x ! IR de ned by f~ := f (x + x)  which implies that Vf~() = Vf ( +  ). Thus, if the statement holds for f~ then it also holds for f . There exists a  > 0 such that the ball K (0;  ; p) = fx 2 IR n : kxkp <  g is contained in D. Hence, n

m() = (K (0; jj1=p; p)) = C jj p holds for all  < 0 satisfying jj <  p , where C=

n Y

(ai ) p 2n pn1 1 i=1 1

 n 1 nY1 1 p

k=1

(k + 1)  : k + 1 + 1p



(5) (6)

Therefore, the zero  = 0 of Vf has the order 1 + np and moreover,

Vf () =  ; m() 1 + np

2

which is our claim.

In order to formulate the next proposition we introduce functions

glb (x) = g ub (x) =

n X i=1 n X i=1

ai jxi xi jp +  bi jxi xi jp + 

where ai ,1 bi > 0 forQall i, as 1well as the geometric mean values R := Qn n i=1 (ai ) n and r := i=1 (bi) n .

PROPOSITION 4.2. Let D be a robust set and x 2 intD. Furthermore, for a xed  2 (ess inf f; ess sup f ) we assume the existence of functions glb and g ub such that glb (x)  f (x)  g ub (x) holds almost everywhere on nthe level set[f  ]. Set q := (r=R) p and := 1 + np q . Then, 1.  :=  + Vmf(())  

ess_sup.tex; 29/05/1996; 16:04; no v.; p.10



CONVERGENCE SPEED OF AN INTEGRAL METHOD...

11

2.    1 q 2 Proof. Let  <  be large enough, such that K (x ; ( )1=p; p)  D. Using Proposition 4.1 we obtain straightforward n n n ( )1+ p n ( )1+ p p p cr 1 + np  Vf ()  cR 1 + np (where c > 0 follows from (6)) and n

n

n

n

cr p ( ) p  m()  cR p ( ) p

which implies

  q 2 ( )  Vmf(())  ( ) :

(7)

The rst statement follows from the right inequality of (7). The second one is implied by the left inequality of (7) because   =   Vmf(())  ( )(1 q2 ):

2

Remark 4.1. For domains D with a piecewise smooth boundary (e.g.

D is a closed box of IRn ) similar considerations seem to be possible if the point x belongs to the boundary of D. This subject is being

investigated. In the general case, we can not decide whether the conditions of Proposition 4.2 are ful lled. Nevertheless, if the stepsize is feasible (that means > 1 and    ), we use the strategy of choosing also if the conditions do not hold. However, for certain classes of functions this strategy may yield good theoretical results. EXAMPLE 4.1. Let U (x ) be a neighbourhood of x and assume that f is twice continuously dierentiable and strongly and boundedly concave in D \ U (x). Choose in U (x ) the eigensystem (ei ; i) of the Hessian of f as new coordinate system. Then we have the estimations n X a"i jxi i=1

in K (x; "; 2) where

xi j2 +   f (x)  lim a" "!+0 i

n X b"ijxi i=1

xi j2 + 

= "! lim+0 b"i = i

for all i. Therefore, q tends to 1 if  !  .

ess_sup.tex; 29/05/1996; 16:04; no v.; p.11

12

Jens Hichert et al.

In all cases, the unknown parameters p and q have to be estimated in each iteration. This will be done in the following by making some ideal assumptions. Consider a sequence (k ) generated by (1). Assume that the conditions of Proposition 4.2 hold for k 1 and k with the same p. Then, from (7) follows that k+1 k = sk   k 1 :

k k

1

sk

1

 k 1 q2

As a result of Proposition 4.1 we have for functions (4), using the usual Newton algorithm, the convergence rate

 k = n :  k 1 n + p For functions satisfying the assumptions of Proposition 4.2 we have at least 1 n sk 1 q n + p s ! 1; k

where q ! 1 for a broad class of functions f (cf. Example 4.1). Thus, an estimator for the stepsize with respect to  = k is given by

k = 1 + np = s sk 1 s : (8) k 1 k Under the assumption that the stepsize k is feasible (which has to be checked) we obtain an 0k generated from k 1 , k and k+1 by 0k = k + k . Repeating this procedure for all k under the same assumptions results in a sequence (0k ) which converges faster to  than the sequence (k ) (see the method of Aitken in Stoer, 1994).

Due to an idea of Steensen (cf. Stoer) we can generate immediately a faster sequence (0k ) without computing the whole sequence (k ) in advance. The rst test (cf. Section 5) showed that the above choice of k is practicable. We use this stepsize in the following algorithm. ALGORITHM 2.

Step 0: Choose " > 0 and 0 2 IR satisfying Vf (0) > 0.

Compute s0 = Vmf((0)) and 1 = 0 + s0 . Set k = 1. 0 V (  Step 1: Compute sk = mf(kk)) . If (sk < sk 1 )

ess_sup.tex; 29/05/1996; 16:04; no v.; p.12

CONVERGENCE SPEED OF AN INTEGRAL METHOD... Compute s^k = sksk1 1 sksk and goto Step 4.

13

Step 2: If (sk > ")

Set k+1 = k + sk , set k = k + 1 and goto Step 1. Step 3: If (Vf (k + ") = 0) STOP. k is an approximate value with 0  ess sup f k  ".

Else

Set k+1 = k + ", set k = k + 1 and goto Step 1. Step 4: If (^sk < ") Set k+1 = k + " and goto Step 6. Step 5: Set ^k+1 = k + s^k . If (Vf (^k+1) = 0) Choose k+1 satisfying k + sk  k+1 and Vf (k+1 ) > 0.

Else

Set k+1 = ^k+1 . Set k = k + 1 and goto Step 1. Step 6: If (Vf (k+1) = 0) STOP. k+1 is an approximate value with 0  ess sup f k+1  ".

Else

Set k = k + 1 and goto Step 1.

5. Implementation and Numerical Results For functions with a spread essential supremum, Algorithm 1 delivers satisfactory results (cf. Section 3). Since we are interested in a comparison of Algorithms 1 and 2, we focus on examples where the objective function has an essential supremum that is not spread. Both algorithms require the computation of a Lebesgue integral Vf () and a measure m() of a level set in each iteration. Of course, Vf and m have to be approximated. It can easily be shown that the iteration (1) in Algorithm 1 is equivalent to the iteration in the mean value level set method in the book of Chew and Zheng (1988). Algorithm 1 diers from the latter by its stopping rules. Therefore, the approach developed by Chew and Zheng, which is

ess_sup.tex; 29/05/1996; 16:04; no v.; p.13

14 Jens Hichert et al. based on Monte Carlo techniques can also be applied in Algorithms 1 and 2 for approximating Vf and m. It was not our aim to implement the Monte Carlo approach in all details, trying to maximize thoroughly the numerical eectiveness of the Monte Carlo search. Such an eort has been made for example by Zheng and Zhuang (1995) and by Caselton and Yassien (1994). The computations presented in this section have been aimed at getting numerical experience and veri cation of the results concerning the speed of convergence and its acceleration. So, a simple version of Monte Carlo search was implemented. Up to this stage of implementation, only problems with box constraints have been considered. At each level k , the level set [f  k ] was approximated by an ndimensional interval Dk containing t sample pointsPxi , i = 1; : : :; t from [f  k ]. The volume Vf (k ) was estimated by (( 1t ti=1 xi ) k )(Dk ). Chew and Zheng showed that, if the set of global maximizers consists of just one point, the set sequence (Dk ) converges to the set of global maximizers. If there are more than one maximizer, the domain of f has to be subdivided for a successful working of the algorithm. After stopping the algorithm in the k-th iteration, a set of t sample points from [f  k ] is still available. The best of them is chosen as the approximate global maximizer of f . The test Vf (k+1 ) = 0 was approximated by the test whether there are at most r < t sample points of Dk with a better function value than k+1. The algorithms were implemented on the programming environment MATLAB, Version 4.1 on a SPARC station. We give three examples. The starting levels 0 of the rst two examples were generated randomly, and so they dier in Algorithms 1 and 2. Although the results depend on the random generator used, they express the typical behaviour of the algorithms with respect to the examples considered. The symbols in Tables I-III were used in the same meaning as introduced in (8) and in the Algorithms 1 and 2. The cases where Algorithm 2 recognized the stepsize k to be too large (Step 5) were marked with the symbol ''. EXAMPLE 5.1 (Rosenbrock-Function). The function f (x1; x2) = 100(x2 x21)2 (x1 1)2 was considered in the domain D = [ 1:5; 0:5]  [1:5; 2:5]. The maximum is  = 0. With both algorithms, 10 steps were performed. Table I gives a comparison. The sequences of the boxes Dk in the Algorithms 1 and 2 are presented graphically in Figures 1 and 2, respectively. In Figure 2, the boxes D9 and D10 are already too small for drawing.

ess_sup.tex; 29/05/1996; 16:04; no v.; p.14

15

CONVERGENCE SPEED OF AN INTEGRAL METHOD... 2.5

2

1.5

1

0.5

0

-0.5 -1.5

-1

-0.5

0

0.5

1

1.5

-1

-0.5

0

0.5

1

1.5

Figure 1. Rosenbrock-Function - ALG. 1 2.5

2

1.5

1

0.5

0

-0.5 -1.5

Figure 2. Rosenbrock-Function - ALG. 2 Table I. Example 5.1

ALGORITHM 1 Iter k k

ALGORITHM 2 k + sk

^ k+1

0 -38.7215 -103.0860 67.9199 1 -12.3150 -35.1662 21.0342 2 -5.9751 -14.1320 7.6794 1.5750 -6.4526 3 -2.8619 -2.0367 1.1012 1.1674 -0.9356 4 -1.5561 -0.7512 0.4195 1.6155 -0.3317 5 -0.7152 -0.0735 0.0321 1.0827 -0.0414 6 -0.3006 -0.0387 0.0212 2.9583 -0.0175 7 -0.1330 -0.0011 5.13e-04 1.0248 -6.32e-04 8 -0.0643 -6.19e-04 3.02e-04 2.4273 -3.17e-04 9 -0.0339 -4.48e-05 2.21e-05 1.0793 -2.26e-05 10 -0.0169 -2.08e-05

-2.0367 -0.7512 -0.0735 -0.0387  0.0240 -6.19e-04  1.13e-04 -2.08e-05

k

sk

k

ess_sup.tex; 29/05/1996; 16:04; no v.; p.15

16

Jens Hichert et al.

0 -0.5 -1 -1.5 -2 -2.5 -3 -3.5 -4 -4.5 -5 -2

-1.5

-1

-0.5

0

0.5

Figure 3. Plot of f (r) = br2 c 2r2

1

1.5

2

Table II. Example 5.2 ALGORITHM 1 Iter k k

ALGORITHM 2 k

sk

0 -73.5788 -59.3142 24.3535 1 -34.0383 -34.9607 18.7766 2 -16.7212 -16.1841 9.1370 3 -9.1628 -1.3402 0.5719 4 -4.8333 -0.7302 0.3784 5 -2.7384 -0.0296 0.0127 6 -1.5692 -0.0165 0.0071 7 -0.9860 -0.0018 8.62e-04 8 -0.5002 -7.82e-04 4.42e-04 9 -0.2440 -2.43e-05 1.21e-05 10 -0.1197 -1.17e-05

k + sk

^ k+1

1.9479 -7.0471 1.0668 -0.7684 2.9552 -0.3518 1.0347 -0.0169 2.2937 -0.0094 1.1372 -9.00e-04 2.0545 -3.39e-04 1.0283 -1.21e-05

 1.6134 -0.7302  0.3880 -0.0165 -1.13e-04 -7.82e-04  1.26e-04 -1.17e-05

k

EXAMPLE 5.2 (Chew and Zheng). The discontinuous function

f (x1; x2) = bx21 + x22 c 2(x21 + x22) were bz c is the integer part of z , was considered in the domain D = [ 10; 10]  [10; 10]. The maximum is  = 0. With both algorithms, 10 steps were performed. Table II gives a comparison. The function is presented graphically in Figure 3. In the third example, the functions Vf and m could be computed directly, without Monte Carlo search.

ess_sup.tex; 29/05/1996; 16:04; no v.; p.16

17 Table III. Example 5.3 ALG. 1

ALG. 2

Iter k

k

Iter k

k

sk

k

0 1 2 3 4 5 10 20 50

-1.0000 -0.7578 -0.6271 -0.5437 -0.4851 -0.4413 -0.3207 -0.2278 -0.1821

0 1 2 3 4 5 6 7 8

-1.0000 -0.7578 -0.6271 -0.3968 -0.3594 -0.2752 -0.2572 -0.2125 -0.2018

0.2421 0.1307 0.0833 0.0258 0.0197 0.0094 0.0077 0.0045 0.0038 Under ow

2.7609 1.4481 4.2641 1.9119 5.7415 2.3776 7.1858 -

100 -0.1821

9 -0.1739

EXAMPLE 5.3 (cf. Example 3.4). The function

f (x) =

(

q

0 if x = 0 1 1 ln x if x 2 (0; 2 ]

was considered in the domain D = [0; 0:5]. As a starting level, 0 = 1 was chosen for both algorithms. Table III gives the results for both algorithms.

6. Conclusions In the present paper, the convergence speed of an integral global optimization method was investigated by relating a convex problem to the global optimization problem and by using methods of the convex analysis. For functions with an essential supremum not spread it was shown that the convergence of the levels is in general slow. A strategy of speeding up the integral method was proposed, but this strategy can fail if only stochastic methods are applied to approximate the integrals. Thus, it will be investigated in future whether deterministic approaches can guarantee the feasibility of the faster strategy with a reasonable complexity.

Acknowledgements

ess_sup.tex; 29/05/1996; 16:04; no v.; p.17

18 This research was supported in part by the Deutsche Forschungsgemeinschaft, GRK 164 / 1-96.

References Archetti, F. and Betro, B. (1975), Recursive Stochastic Evaluation of the Level Set Measure in the Global Optimization Problems, Technical Report, University of Pisa, Pisa, Italy. Chichinadze, V.K. (1967), Random Search to Determine the Extremum of the Function of Several Variables, Engeneering Cybernetics 1, 115-123. Caselton, W.F, and Yassien, H. A. (1994), LSP4, public domain software, available via ftp://ftp.ruhr-unibochum.de/mirrors/simtel.coast.net/SimTel/msdos/statistic/lsp4.zip. Chew S.H. and Zheng Q. (1988), Integral Global Optimization, Springer, Berlin, Heidelberg. De Biase, L. and Frontini, F. (1978), A Stochastic Method for Global Optimization: Its Structure and Numerical Performance, in: Dixon, L.C.W. and Szego, G.P. (eds.) (1978), Towards Global Optimization 2, North-Holland, Amsterdam, 85102. Dieudonne, J. (1975), Grundzuge der modernen Analysis, Band III, Deutscher Verlag der Wissenschaften, Berlin. Hiriart-Urruty, J.-B. and Lemarechal, C. (1993), Convex Analysis and Minimization Algorithms I, Springer, Berlin, Heidelberg. Kosmol, P. (1993), Methoden zur numerischen Behandlung nichtlinearer Gleichungen und Optimierungsaufgaben, Teubner, Stuttgart. Kostreva, M.M. and Zheng Q. (1994), Integral Global Optimization Method for Solution of Nonlinear Complementarity Problems, Journal of Global Optimization 5, 181-193. Natanson, I.P. (1975), Theorie der Funktionen einer reellen Veranderlichen, Akademie-Verlag, Berlin. Phu, H.X. and Homann, A. (1996), Essential Supremum and Supremum of Summable Functions, Numerical Functional Analysis and Optimization 17 (to appear). Stoer, J. (1994), Numerische Mathematik 1, Springer, Berlin, Heidelberg. Zheng Q. (1992), Integral Global Optimization of Robust Discontinuous Functions, Dissertation, Graduate School of Clemson University, Clemson. Zheng Q. and Zhuang D. (1995), Integral Global Minimization: Algorithms, Implementations and Numerical Tests, Journal of Global Optimization 7, 421-454. Address for correspondence: armin.ho[email protected], [email protected]

ess_sup.tex; 29/05/1996; 16:04; no v.; p.18