A New Subdivision Algorithm for the Bernstein Polynomial Approach ...

International Journal of Automation and Computing

04(4), October 2007, 342-352 DOI: 10.1007/s11633-007-0342-7

A New Subdivision Algorithm for the Bernstein Polynomial Approach to Global Optimization P. S. V. Nataraj∗

M. Arounassalame

Systems and Control Engineering, ACRE Building, Indian Institute of Technology Bombay, Powai, Mumbai 400 076, India

Abstract: In this paper, an improved algorithm is proposed for unconstrained global optimization to tackle non-convex nonlinear multivariate polynomial programming problems. The proposed algorithm is based on the Bernstein polynomial approach. Novel features of the proposed algorithm are that it uses a new rule for the selection of the subdivision point, modified rules for the selection of the subdivision direction, and a new acceleration device to avoid some unnecessary subdivisions. The performance of the proposed algorithm is numerically tested on a collection of 16 test problems. The results of the tests show the proposed algorithm to be superior to the existing Bernstein algorithm in terms of the chosen performance metrics. Keywords: tion.

1

Bernstein polynomials, global optimization, nonlinear optimization, polynomial optimization, unconstrained optimiza-

Introduction

Global optimization is the task of finding the absolutely best set of parameters from the admissible set, for which the objective function obtains its smallest value, the global minimum. A standard algorithm for unconstrained global optimization is the use of a branch and bound procedure that does an exhaustive search over the region of interest[1] . Interval methods use this procedure to find the global minimum[2−5] . However, interval analysis methods require function evaluations at each and every iteration, which leads to a computationally slow algorithm. For polynomial optimization problems, an alternate approach is based on the Bernstein form of the polynomial[6−8] . Global optimization based on the Bernstein form does not require the evaluation of the objective function[7] . Moreover, the coefficients of the Bernstein form are needed to be computed only once, i.e., only on the initial domain box. The Bernstein coefficients for the subdivided domain boxes can then be obtained from the initial box[6] . One can compute a lower bound on the global minimum of the polynomial over a domain box by using the Bernstein form of the polynomial. By repeatedly applying subdivision, one can obtain a sufficiently accurate lower bound. Nataraj and Kotecha[9, 10] proposed algorithms based on these ideas for polynomial optimization. Such algorithms for polynomial optimization that are based on the Bernstein form are called Bernstein (global optimization) algorithms. A basic Bernstein algorithm is described in the following section. The efficiency of the Bernstein algorithm greatly depends on both the direction and the point at which the box subdivision is done. While several rules have been studied for Manuscript received February 26, 2007; revised June 2, 2007 *Corresponding author. E-mail address: [email protected]

selecting the subdivision direction in this paper (see Section 2.3), the choice of the point of subdivision has been restricted to the midpoint. As there is no reason why the midpoint should always turnout to be the best point for subdivision in all problems, alternate choices that are more efficient need to be explored. This motivates us in this work to investigate alternate choices for the point of subdivision. As an alternate choice, we propose the subdivision point to be located where the minimum Bernstein coefficient occurs in the Bernstein expansion matrix. To make effective use of this choice of subdivision point, we also propose to suitably modify the existing rules for subdivision direction selection, and further propose a new acceleration scheme to avoid some unnecessary subdivisions. We then propose a novel Bernstein algorithm for polynomial optimization that incorporates all these ideas. We test and compare the performance of the proposed Bernstein algorithm with that of the basic Bernstein algorithm, on a collection of 16 polynomial test problems taken from [11−13], with dimensions varying from 2 to 8. The rest of this paper is organized as follows. In Section 2, we give a brief review of the Bernstein form, its properties, and the subdivision procedure. We explain the acceleration tests that are used to speed up the global optimization algorithm using the Bernstein expansion method. We also give the basic Bernstein algorithm in this section. In Section 3, we give our propositions such as the proposed subdivision point selection method, proposed improvement in subdivision direction selection and a new acceleration scheme along with examples. Lastly, in this section, we present the proposed Bernstein global optimization algorithm, which includes all these propositions. In Section 4, we numerically test and compare the performance of this proposed algorithm with that of the basic Bernstein alg-

P. S. V. Nataraj and M. Arounassalame/ A New Subdivision Algorithm for the Bernstein Polynomial Approach to . . .

orithm. Lastly, we give the conclusions of the work in Section 5.

2 2.1

Background

Lemma. Let p be a polynomial of degree N , and let x) denote the range of p on the given domain x . Then, p(x u) of Bernstein the following property holds for a patch B(u coefficients: x) ⊆ [min(B(u u)), max(B(u u))]. p(x

The Bernstein form

Following the notations given in [14], let l ∈ N be the number of variables and x = (x1 , x2 , · · · , xl ) ∈ Rl . A multiindex I is defined as I = (i1 , i2 , · · · , il ) ∈ N l and multii2 il power xI is defined as xI = (xi1 1 , x2 , · · · , xl ). A multiindex N is defined as N = (n1 , n2 , · · · , nl ). Inequalities I ≤ N for multi-indices are meant component-wise, where 0 ≤ ik , k = 1, 2, · · · , l. With I = (i1 , · · · , ir−1 , ir , ir+1 , · · · , il ) we associate the index Ir,k given by Ir,k = (i1 , · · · , ir−1 , ir + k, ir+1 , · · · , il ), where 0 ≤ ir +k ≤ nr . Also, we write (N/I) for (n1 /i1 , n2 /i2 , · · · , nl /il ). A real bounded and closed interval x r is defined as x r ≡ (xr , xr ) = [inf x r , sup x r ] ∈ IR, where IR denotes xr ) denote the width the set of compact intervals. Let wid(x xr ) = xr − xr . of x r , that is, wid(x We can write an l-variate polynomial p in the form X p(x) = aI xI , x ∈ Rl I≤N

with N as the degree of p. We can expand a given multivariate polynomial into Bernstein polynomial to obtain bounds for its range over an l-dimensional box x = [x1 , x2 , · · · , xl ]. Without loss of generality, we consider the unit box u = [0, 1] since any nonempty box x of Rl can be mapped affinely onto this box. The I-th Bernstein polynomial of degree N is defined as n

BIN (x) = Bin11 (x1 ) · · · Bill (xl ), x ∈ Rl where, for ij = 0, 1, · · · , nj and j = 1, 2, · · · , l. ! nj nj i Bij (xj ) = xjj (1 − xj )nj −ij . ij

343

(1)

The range enclosure of the multivariate polynomial p on the domain x can be found by transforming the polynomial from power form to Bernstein form. Then, by the Lemma, the coefficients of the expansion in the Bernstein form provide lower and upper bounds for the range. The upper u))I∈S bound or lower bound is sharp if and only if min (bI (u u))I∈S ) is attained at the indices of (respectively, max(bI (u u). This condition is known as the vertices of the array B(u the vertex property. There are several methods to compute the Bernstein coefficients of a multivariate polynomial p. A difference table scheme is proposed in [6], a parallel computation scheme in [15], and a tensor multiplication scheme in [16]. Recently, a more efficient and easily generalizable two-dimensional matrix method was proposed in [17] to compute the Bernstein coefficient matrix over general box domains. We shall use this method for the computation of Bernstein coefficients in our Bernstein Global optimization algorithm below.

2.2

Subdivision procedure

The range enclosure obtained using Bernstein coefficients can be further improved either by degree elevation of Bernstein polynomial or by subdivision. The subdivision strategy is generally more efficient than the degree elevation strategy[8,14] and is therefore preferred. A subdivision in the r-th direction (1 ≤ r ≤ l) is a bisection perpendicular to this direction. Let x = [x1 , x1 ] × · · · × [xr , xr ] × · · · × [xl , xl ]

(2)

u) of p over the unit box u are The Bernstein coefficients bI (u given by ! I X J ! · aJ , I ≤ N. u) = bI (u (3) N J≤I J Thus, the Bernstein form of a multivariate polynomial p is defined by X u) BIN (x) . bI (u (4) p (x) = I≤N

u) = The Bernstein coefficients are collected in an array B(u u))I∈S , where S = {I : I ≤ N }. This array is called as (bI (u a patch. The following well-known lemma describes the range enclosure property of the Bernstein coefficients.

x) has already been be any subbox and suppose the patch B(x computed. Further suppose that x is computed, and suppose that x is bisected along the r-th component direction. Then, two subboxes x A and x B are generated as xA

=

xr )] × · · · × [xl , xl ], [x1 , x1 ] × · · · × [xr , m(x

xB

=

xr ), xr ] × · · · × [xl , xl ] [x1 , x1 ] × · · · × [m(x

xr ) denotes the midpoint of [xr , xr ]. Starting with where m(x x) = B(x x), we set for k = 1, 2, · · · , nr , B 0 (x ( (k−1) x) bI (x , ir ≤ k (k) x) = bI (x (k−1) x) + λb(k−1) x) , k ≤ ir (x (1 − λ)bIr,−1 (x I (5) where λ is subdivision parameter, which takes the value 0.5 for subdivision at the midpoint. To obtain the new coefficients, the above formula is applied for ij = 0, · · · , nj , j = xA ) = B nr (x x). 1, · · · , r − 1, r + 1, · · · , l. Then, B(x xB ) on the neighboring subThe Bernstein coefficients B(x box x B are the intermediate values of this computation,

344

International Journal of Automation and Computing 04(4), October 2007

since for k = 0, 1, · · · , nr , the following relation holds[14] xB ) = bi1 ,··· ,nr ,··· ,il (x xA ) bi1 ,··· ,nr −k,··· ,il (x

(6)

By the above subdivision procedure, the explicit transformation of the subboxes generated by the subdivisions back to the original box is avoided. By repeatedly applying this subdivision procedure and the vertex condition in the Lemma, the bounds on the range of multivariate polynomial can be improved.

2.3

Subdivision direction selection

The subdivision of Bernstein coefficients matrix can be done in any direction. A few direction selection rules are available to go in a systematic way for the selection of subdivision direction. The direction selection rules are based on the observation that in many cases it may be advantageous to subdivide in a particular direction to increase the probability of finding a sharp Bernstein coefficient, thus providing early convergence. Many direction selection rules are available for interval analysis based on global optimization[11] . A few direction selection rules are also available for the Bernstein algorithm[8] . In this subsection, we describe three such rules that are commonly used for the selection of subdivision direction in the Bernstein-based range computation algorithms. The merit function for bisection direction selection is[11] k = min{j|j ∈ {1, 2, · · · , l}, y(j) = max(y(r)), r = 1, 2, · · · , l} where y(r) is the direction factor and is determined by the given rule. The direction r for which y(r) (direction factor) is maximum is chosen for subdivision. If the maximum is achieved in several component directions, the lowest one among these is taken for subdivision. 2.3.1 Rule A (maximum width) This is a derivative-free rule commonly used in interval analysis[11,18] . It is based on the interval width. The direction factor is given by xr ) y(r) = wid(x xr ) is the width of the box in the direction r. where wid(x The selected subdivision direction is the one for which the box edge length is maximum. This rule subdivides the original bound in a uniform way. This is the simplest rule for subdivision direction selection. 2.3.2 Rule B (derivative-based) This is a derivative-based rule. This rule is used in Bernstein subdivision direction selection[8] . The subdivision direction is decided based on the value of partial derivative of p(x) over the given interval. The direction factor is given by x) − bI (x x)|. y(r) = max |bIr,1 (x x x∈x

An upper bound of the absolute value of the partial derivative for each direction is computed, and the direction of the maximum is chosen for subdivision.

2.3.3 Rule C (derivative-based) This is also a derivative-based rule. This rule is used in [3, 11]. This rule combines the advantage of both the derivative of the function and the width of the interval. This rule is used in interval analysis, and we give the Bernstein form of this rule x) − bI (x x))− y(r) =[max(bIr,1 (x x x∈x

x) − bI (x x))]wid(x xr ). min(bIr,1 (x x x∈x

The product of width of derivative function and width of bound is computed for each direction. The selected subdivision direction is the one for which the product is maximum. The derivative of the function is computed using the Bernstein coefficients of the function.

2.4

Accelerating algorithms

In the process of finding the global minimum of a polynomial, the Bernstein subdivision process continues even for the boxes where there is no possibility for the occurrence of the global minima. These regions can be deleted so that further unnecessary subdivisions can be avoided[19] . Two speeding up algorithms are very important in the global search algorithms. They are the cut-off test and monotonicity test, and also called as accelerating devices because they accelerate the process of finding the global minimum over a given region. The monotonicity test requires the computation of the derivative of the Bernstein coefficients. The derivative of the Bernstein coefficients in a particular direction can be easily calculated using the Bernstein coefficients matrix over an interval. The derivative of the Bernstein coefficients along a particular direction is obtained by finding the difference of the coefficients along that particular direction in the Bernstein coefficient matrix[20] . We will further explain these two algorithms as follows. 2.4.1 Cut-off test If the minimum of the Bernstein coefficients of any box is more than the current minimum estimate, then that box d, B(d d)} be the current item for can be deleted[20] . Let {d processing and pˆ be the current minimum estimate. If the d, B(d d)} is minimum over the second entry of the item {d greater than pˆ, then this item cannot contain global minimum and it can be discarded. 2.4.2 Monotonicity test On a box d ⊆ x , the partial derivative with respect to xr of a polynomial p(x) in Bernstein form is X d) − bI (d d)|BNr,−1 ,I (x), d ) = nr |bIr,1 (d p0r (d I≤Nr,−1

1 ≤ r ≤ l and x ∈ d . d) p0r (d

Now, contains an enclosure of the range of the partial d) derivative of p on d . In the monotonicity test, if 0 ∈ / p0r (d then the interior of d cannot contain a global minimizer of p on x . The edge of d that can still contain a global minimizer of the part of the edge which has the smallest polynomial value is also part of x . Otherwise, no global

P. S. V. Nataraj and M. Arounassalame/ A New Subdivision Algorithm for the Bernstein Polynomial Approach to . . .

minimizer of p lies in d , and d can be discarded. From the fact that the Bernstein polynomials BNr,−1 ,I (x) are always d) − bI (d d)| non-negative, it is easy to see that if all |bIr,1 (d d) > 0 (respecare positive (respectively negative), then p0r (d d) < 0) implies that p is monotonic with respect tively p0r (d to direction r on box d . Hence, the interior of d cannot contain a global minimizer of p(x)[20] .

2.5

Basic Bernstein global optimization algorithm

The basic algorithm for polynomial optimization using the Bernstein form is outlined in this subsection. The basic algorithm is similar to the one described in [20]. In this algorithm at the outset, we compute the Bernx) of the polynomial for the given box stein coefficients B(x x. We next initialize a list L with an item {x x, B(x x)}. We also initialize a solution list Lsol to the empty list. From d, B(d d)} and remove the list L, we then pick the last item {d its entry from L. We check for vertex condition satisfaction for the chosen item. If the vertex condition is satisfied within the specified tolerance ², then put this item in the list Lsol . If the vertex condition is not satisfied, we compute d, B(d d)} has to the current minimum estimate pˆ. The item {d undergo the cut-off test and monotonicity test. These tests deletes this item from further consideration if it is sure that the global minimum does not lie in this item. Otherwise, we divide the box d into two boxes to generate two items dA , B(d dA )} and {d dB , B(d dB )} and add them to the list L. {d We continue the entire process until L becomes empty. At this point, we output the global minimum and terminate the algorithm. Inputs. Degree of each variable of the polynomial N , the polynomial coefficients aI of the power form of the polynomial, the initial box x ∈ IRl and the tolerance limit ². Outputs. Global minimum pˆ to the specified tolerance ² and the global solution dˆ. Begin algorithm Step 1. Compute the Bernstein coefficients x) of the polynomial on Compute the Bernstein coefficients B(x the initial box x . The matrix algorithm in [17] is suggested for the computation. Step 2. Initialize lists x, B(x x)}, Lsol ← {}. L ← {x Step 3. Start new iteration If L is empty, go to step 9. Otherwise, pick the last item from d, B(d d)}, and delete this item entry from L. L, denote it as {d Step 4. Check if the vertex condition is met within ² d, B(d d)}, do the following: if min(B(d d)) satisFor the item {d fies vertex condition within ², then enter the item in list Lsol and return to Step 3. Step 5. Compute current minimum estimate Compute pˆ as the minimum over the second entries of all the items present in Lsol (i.e., over all the Bernstein patches present in Lsol ). Step 6. Cut-off test (see Section 2.4.1) d) > pˆ), discard the item {d d, B(d d)} and return to If min(B(d Step 3. Step 7. Monotonicity test (see Section 2.4.2)

345

d) for any r ∈ {1, 2, · · · , l}, discard the item If 0 ∈ / p0r (d d, B(d d)} and go to Step 3. {d Step 8. Subdivide and find new patches Step 8.1. Using a rule for selection of subdivision direction, choose a component direction k for subdivision of d . Step 8.2. Subdivide d at its midpoint in component direction S k to generate two sub-boxes d A and d B such that d = d A d B . dA , B(d dA )} and Step 8.3. Enter the two new items {d dB , B(d dB )} at the end of the list L in the following manner: {d dA )) > min(B(d dB )), enter B(d dA ) first and then enter If min(B(d dB ). Else, enter B(d dB ) first and then enter B(d dA ). B(d Go to Step 3. Step 9. Compute the global minimum x) = pˆ. pˆ(x Step 10. Compute the global solution Find the location of the item in Lsol for which the minimum value in the second entry is pˆ. The first entry of this item is the global solution dˆ. Step 11. Return x). Return pˆ(x End algorithm

3 3.1

Proposed algorithm Proposed rule for subdivision point selection

The proposed rule is a derivative-free rule. First, the subdivision direction r is selected using any one of the existing direction selection rules, such as rule A, B, or C. Then, the minimum Bernstein coefficient is located in the Bernstein coefficient matrix, and the subdivision is done in direction r at this point using the Bernstein subdivision procedure. The subdivision parameter λ in the direction r for this point is defined as xr−min ) wid(x (7) λr = xr ) wid(x xr−min ) is the box width up to this point in where wid(x xr ) is the box width in direction r. direction r, and wid(x The following example shows how the value of subdivision parameter, λr is computed for Bernstein patch in some particular direction. Example 1. Consider the following Bernstein coefficient matrix for two variables x1 and x2 , where x1 ∈ x 1 = [0, 4] and x2 ∈ x 2 = [0, 6] 2 6 6 6 x) = 6 B(x 6 6 4

8 3 11 13 3

5 7 13 −9 4

3 4 −2 1 8

5 1 4 5 5

10 3 −7 4 2

3 7 7 7 7. 7 7 5

The minimum Bernstein coefficient occurring in the Bernstein coefficient matrix is −9, and occurs as element (4, 2) of the matrix. Suppose the component direction r = 1 is chosen by a subdivision direction selection rule. Then, the proposed

346

International Journal of Automation and Computing 04(4), October 2007

subdivision point selection rule would select x1 = 3 (corresponding to the 4th element in the first direction) as the subdivision point. Since the box width up to this point in x1−min ) = 3 − 0 = 3, the first component direction is wid(x x1 ) = 4, the and the total box width in this direction is wid(x subdivision parameter λ takes the value λ1 = 3/4. Suppose instead the component direction r = 2 is chosen by a subdivision direction selection rule. Then, the proposed point selection rule would select x2 = 1.5 (corresponding to the 2nd element in the second direction) as the subdivision point. Since the box width up to this x2−min ) = point in the second component direction is wid(x 1.5 − 0 = 1.5, while the total box width in this direction is x2 ) = 6, the subdivision parameter λ2 takes the value wid(x λ2 = 1.5/6. With the basic Bernstein algorithm, the midpoint rule would be used for subdivision in any component direction. Hence, λ1 or λ2 would always be 0.5. Thus, the proposed rule for selection of subdivision point selects the value of λ depending on the location of the minimum Bernstein coefficient, and this value changes for every box unlike the existing midpoint rule. After subdivision, two new boxes are generated. The box having minimum Bernstein coefficient is selected for further subdivision. To perform subdivision on this box, the direction of subdivision has to be computed using the direction selection rule. Along this direction, the location of the point where the minimum Bernstein coefficient occurs is determined and subdivision carried out, etc. This procedure is repeated until the vertex condition is satisfied for the resulting boxes. At each and every iteration, the new subdivision point is calculated. This procedure aims at having the minimum Bernstein coefficient at the corner positions within few iterations so that the vertex property will be satisfied. To speed up the algorithm, the cut-off test and monotonicity tests are applied on chosen boxes before processing them for further division.

3.2

Rule B(M)

:

x) − bI (x x)|) y(r) = γr max(|bIr,1 (x

Rule C(M)

:

x) − bI (x x)) − xr )[max(bIr,1 (x y(r) = γr wid(x

Rule A(M)

:

xr ) y(r) = γr wid(x

x x∈x

x) − bI (x x))]. min(bIr,1 (x x x∈x

With the modified direction rules, when λr is 0 or 1, then the direction factor y(r) becomes zero, and the r-th direction is skipped for subdivision. The following example shows how the proposed modifications in the direction selection rules help. Example 2. Consider the following Bernstein coefficient matrix for two variables x1 and x2 , where x1 ∈ x1 = [0, 4] and x2 ∈ x2 = [0, 6] 3 2 8 5 3 5 10 6 7 6 3 7 4 1 3 7 7 6 7 x) = 6 B(x 6 11 13 −2 4 −7 7 . 7 6 1 5 −9 5 4 13 −6 3 4 8 5 2 The minimum Bernstein coefficient occurring in the Bernstein coefficient matrix is −9, and occurs as element (4, 5) of the matrix. From this element location, we obtain the subdivision parameter values as λ1 = 0.75, λ2 = 1. Suppose we apply rule A as the direction selection rule. Then, the values of direction factors are x1 ) = 4, y(1) = wid(x x2 ) = 6. y(2) = wid(x Since y(2) > y(1), the rule gives the subdivision direction as the second component direction. However, because the minimum Bernstein element occurs at x2 = 6, (which is on the edge of the box) corresponding to the value of λ2 = 1, subdivision cannot be done in this direction at the suggested point. Suppose we instead apply the modified rule A(M). Then,

Modified rules for direction selection

When the value of the subdivision parameter λ is 0 or 1 along the selected subdivision direction, then it conveys that no subdivision is needed in that direction, contrary to what is suggested by the direction selection rule. To avoid the anomaly, the subdivision direction selection rules need to be modified so that this particular direction is skipped, and the subdivision is instead carried out in a more appropriate direction. The modified rules are described next. Let γr = λr (1 − λr ), where λr is the subdivision factor in the r-th component direction, computed as described in the preceding section (the value of λr varies from 0 to 1). Denote the modified subdivision direction selection rules corresponding to rules A, B, and C, denoted as rules A(M), B(M), and C(M), respectively, where

x x∈x

γ1 = λ1 (1 − λ1 ) = 0.1875, γ2 = λ2 (1 − λ2 ) = 0 and the values of the modified direction factors are x1 ) = 0.75, y(1) = γ1 wid(x x2 ) = 0. y(2) = γ2 wid(x Since now the subdivision direction factor y(1) > y (2), the modified rule gives the subdivision direction as the first component direction. Moreover, subdivision can be performed in this direction at the suggested point, since the value of λ1 = 0.75.

3.3

New speed up test

The search for the global minimum is done to a specified accuracy ε. All the minimum Bernstein coefficients are thus found to the specified accuracy ε. Suppose several minimum Bernstein coefficients (equal to each other within ε) exist in the Bernstein coefficient matrix. If any of them

P. S. V. Nataraj and M. Arounassalame/ A New Subdivision Algorithm for the Bernstein Polynomial Approach to . . .

occurs on the edge of the box in the selected subdivision direction r, then the box gained only little by subdividing in the r-th direction. To handle these cases, the value of subdivision parameter λr needs to fixed at the value of either 0 or 1, depending on where the minimum edge occurs. The subdivision will then be skipped in the r-th direction and instead performed in another direction. We call this test as a speed-up test, since it avoids unnecessary subdivisions in certain directions. However, unlike in other acceleration tests, this test does not delete any box. The following example illustrates the idea. Example 3. Consider the following Bernstein coefficient matrix for two variables x1 and x2 , where x1 ∈ x 1 = [0, 4] and x2 ∈ x 2 = [0, 3] 2 6 6 6 x) = 6 B(x 6 6 4

8.404 3.121 11.221 13.387 3.341

5.111 7.323 13.234 −9.454 −9.451

3.276 4.544 −2.745 1.109 8.333

5.789 1.654 4.143 5.410 5.508

10.900 3.456 −7.956 −5.498 2.341

3 7 7 7 7. 7 7 5

Suppose ε = 0.01 is specified as the tolerance for the global minimum. Then, there are two minimum Bernstein coefficients (equal to each other within ε = 0.01) occurring in the Bernstein coefficient matrix. These coefficients are −9.454 and −9.451 occurring as elements (4, 2) and (5, 2) of the matrix. Suppose we apply rule A as the direction selection rule, which chooses the first component direction for the subdivision. Since a minimum Bernstein coefficient occurs on the edge in the selected subdivision direction, subdivision at this point, gives little improvement. So the value of λ1 can be fixed as 1, and subdivision will be avoided in this direction.

3.4

Proposed Bernstein global optimization algorithm

In the proposed algorithm, the selection of the item x, B(x x)} from list L, testing for vertex condition satis{x faction, cut-off and monotonicity tests are carried out in the same way as in the basic Bernstein global optimization algorithm. The subdivision of the box d is carried out using the rules proposed in Section 3. The subdivision point is computed using the rule proposed in Section 3.1. The subdivision direction is decided using the rule proposed in Section 3.2 and the subdivision direction is modified based on the speed up test proposed in Section 3.3. With these modifications, we present the proposed Bernstein global optimization algorithm in this section. Inputs. Degree of each variable of the polynomial N , the polynomial coefficients aI of the polynomial in the power form, the initial box x ∈ IRl and the tolerance limit ². Outputs. Global minimum pˆ to the specified tolerance ² and the global solution dˆ. Begin algorithm

347

Step 1. Compute the Bernstein coefficients x) of the polynomial on Compute the Bernstein coefficients B(x the initial box x . The matrix algorithm in [17] is suggested for the computation. Step 2. Initialize lists x, B(x x)}, L ← {}. L ← {x Step 3. Start new iteration If L is empty, go to step 9. Otherwise, pick the last item from d, B(d d)}, and delete this item entry from L. L, denote it as {d Step 4. Check if the vertex condition is met within ² d, B(d d)}, do the following: if min(B(d d)) satisFor the item {d fies vertex condition within ², then enter the item in list Lsol and return to Step 3. Step 5. Compute current minimum estimate Compute pˆ as the minimum over the second entries of all the items present in Lsol (i.e., over all the Bernstein patches present in Lsol ). Step 6. Cut-off test (see Section 2.4.1) d)) > pˆ, discard the item {d d, B(d d)} and return to If min(B(d Step 3. Step 7. Monotonicity test (see Section 2.4.2) d) for any r ∈ {1, 2, · · · , l}, discard the item If 0 ∈ / p0r (d d, B(d d)} and go to Step 3. {d Step 8. Subdivide and find new patches Step 8.1. Compute the subdivision parameter λr for each direction using the rule proposed in Section 3.1. Step 8.2. Compute the (modified) direction parameter y(r) using the direction selection rule proposed in Section 3.2. Step 8.3. Further modify the direction parameter y(r) using the speed-up test proposed in Section 3.3. Step 8.4. Choose the subdivision direction for which y(r) is maximum as the subdivision direction, according to Section 3.3. dA ) and B{d dB } using Section 2.2. for Step 8.5. Compute B(d the chosen direction and computed direction parameter λr . dA , B(d dA )} and {d dB , B(d dB )} Step 8.6. Enter the new items {d at the end of the list L in the following manner: dA )) > min (B(d dB )), enter B(d dA ) first and then If min(B(d dB ). Else, enter B(d dB ) first and then enter B(d dA ). enter B(d Go to Step 3. Step 9. Compute the global minimum x) = pˆ. pˆ(x Step 10. Compute the global solution Find the location of the item in Lsol for which the minimum value in the second entry is pˆ. The first entry of this item is the global solution dˆ. Step 11. Return x) and dˆ. Return pˆ(x End algorithm

4

Numerical tests

We test and compare the performances of the proposed and basic Bernstein algorithms for polynomial optimization on 16 test problems. The test problems are taken from [1113] and described in the appendix. The numerical computations are done on a desktop PC with Pentium IV 3GHz, 512 MB RAM, while the algorithms are implemented in MATLAB[21] . An accuracy ² = 10−9 is prescribed for computing the global minimum in each test problem. Table 1 summarizes the test problems with their initial domains, and also the global minima obtained with the Bernstein algorithms.

348

International Journal of Automation and Computing 04(4), October 2007

We now compare the performances of both the algorithms, for rules A, B, and C of subdivision direction selection. The performance metrics are taken as the number of subdivisions and computational time (in seconds). In Tables 2 and 3, these metric values are reported. For each performance metric, the tables also report the “Ratio” and the “Reduction” computed as Ratio

=

Reduction

=

PMBA PMPA PMBA − PMPA · 100% PMPA

Table 1

where PMBA means performance metric with basic algorithm and PMPA means performance metric with proposed algorithm. As a summary, Table 4 gives the minimum, mean, and maximum of the Ratio and Reduction over all the 16 test problems. It shows that on an average, the proposed algorithm gives 40% - 48% reduction in the number of subdivisions and 35% - 40% reduction in the computational time. For certain test problems, the proposed algorithm gives reductions of 99.9% in both these metrics. Overall, the proposed algorithm is thus found to be considerably more efficient than the basic algorithm. The test functions in Tables 1–4 will be explained in Appendix.

Test problems with their domains, dimensions and the obtained global minimum

Example

Test function

Dimension

Domain

Global minimum

1

Himmelblau

2

[-5,5]2

-7.7657E-10

2

Camel back

2

[-5,5]2

-1.031628453

3

Booth

2

[-10,10]2

-8.148E-10

4

Rosenbrok

2

[0,5]2

-7.0451E-10

5

Trid(3)

3

[-9,9]3

-7.000000000

6

Dixon and price

3

[-10,10]3

-4.02701E-10

7

Reaction diffusion

3

[-5,5]3

-10.405604030

8

Schwefel

3

[-10,10]3

-5.8806E-10

9

Caprasse0 s

4

[-.5,.5]4

-3.180096626

10

Adaptive LV

4

[-2,2]4

-20.8

11

Trid (4)

4

[-16,16]4

-16.000000000

12

A. H. Wright

5

[-5,5]5

-30.25

13

Magnetism in physics (6)

6

[-1,1]6

-0.25

14

Butcher

6

[-1,0][-.1,.9][-.1,.5] [-1,-.1][-.1,-.05][-.1,-.03]

-0.219000000

15

Magnetism in physics (7)

7

[-1,1]7

-0.25

16

Heart dipole problem

8

[-.1,.4][.4,1][-.7,-.4][-.7,.4] [.1,.2][-.1,.2][-.3,1.1][-1.1,-.3]

-1.743448579

349

P. S. V. Nataraj and M. Arounassalame/ A New Subdivision Algorithm for the Bernstein Polynomial Approach to . . . Table 2

Comparison of the number of subdivisions required by the basic and proposed Bernstein algorithms for different subdivision direction selection rules A, B, and C Rule A

Example

Test function

Dimension

1

Himmelblau

2

2

Cammelback

3

Subdivisions

Rule B

Rule C

Existing

Proposed

Existing

Proposed

Existing

Proposed

Number Ratio Reduction (%)

241

164 1.47 31.95

234

139 1.68 40.6

223

154 1.45 30.94

2

Number Ratio Reduction (%)

129

87 1.48 32.56

131

82 1.59 37.4

127

89 1.43 29.92

Booth

2

Number Ratio Reduction (%)

184

139 1.32 24.45

176

126 1.39 28.4

152

116 1.31 23.68

4

Rosenbrok

2

Number Ratio Reduction (%)

4264

2157 1.97 49.41

2485

2229 1.11 10.3

2632

2424 1.08 7.9

5

Trid(3)

3

Number Ratio Reduction (%)

284

284 1 0

330

313 1.05 5.15

284

284 1 0

6

Dixon and Price

3

Number Ratio Reduction (%)

1081

792 1.36 26.73

682

680 1 0

1081

792 1.36 26.73

7

Reaction diffusion

3

Number Ratio Reduction (%)

54

18 3 66.67

82

18 4.56 78.04

54

18 3 66.66

8

Schwefel

3

Number Ratio Reduction (%)

1038

566 1.83 45.47

820

551 1.48 32.8

839

530 1.58 36.8

9

Capprasse0 s

4

Number Ratio Reduction (%)

215

22 9.77 89.77

273

22 12.4 91.94

209

22 9.5 89.47

10

Adaptive LV

4

Number Ratio Reduction (%)

8

1 8 87.5

29

1 29 96.55

8

1 8 87.5

11

Trid(4)

4

Number Ratio Reduction (%)

839

661 1.27 21.21

897

709 1.26 20.9

1210

899 1.35 25.7

12

A. H. Wright

5

Number Ratio Reduction (%)

116

18 6.44 84.48

146

18 8.11 87.67

18

18 1 0

13

Magnetism in physics (6)

6

Number Ratio Reduction (%)

126

95 1.33 24.6

131

95 1.38 27.48

126

95 1.33 24.6

14

Butcher

6

Number Ratio Reduction (%)

611303

30 20376 99.99

97

30 3.23 69.07

2025

30 67.5 98.5

15

Magnetism in physics (7)

7

Number Ratio Reduction (%)

191

191 1 0

254

191 1.33 24.8

254

191 1.33 24.8

16

Heart dipole

8

Number Ratio Reduction (%)

111

12 9.25 89.19

199

12 16.58 93.97

111

12 9.25 89.19

350

International Journal of Automation and Computing 04(4), October 2007

Table 3

Comparison of the computational time taken by the basic and proposed Bernstein algorithms for different subdivision direction selection rules A, B, and C Rule A

Example

Test function

Dimension

CPU time

1

Himmelblau

2

2

Cammelback

3

Rule B

Rule C

Existing

Proposed

Existing

Proposed

Existing

Proposed

Time(s) Ratio Reduction (%)

.157

.094 1.67 40.12

.156

.094 1.66 39.74

.156

.094 1.66 39.74

2

Time(s) Ratio Reduction (%)

.11

.078 1.41 29.09

.11

.078 1.41 20.09

.11

.078 1.41 29.09

Booth

2

Time(s) Ratio Reduction (%)

.125

.094 1.33 24.8

.11

.078 1.41 29.09

.094

.078 1.21 17.02

4

Rosenbrok

2

Time(s) Ratio Reduction (%)

2.57

1.2 2.14 53.3

1.45

1.4 1.04 3.45

1.59

1.53 1.04 3.77

5

Trid(3)

3

Time(s) Ratio Reduction (%)

.219

.219 1 0

.297

.29 1.02 2.36

.219

.219 1 0

6

Dixon and Price

3

Time(s) Ratio Reduction (%)

1.17

1.01 1.16 13.82

.734

.734 1 0

1.06

.937 1.13 11.6

7

Reaction diffusion

3

Time(s) Ratio Reduction (%)

.047

.016 2.94 65.96

.062

.047 3.87 74.19

.047

.016 2.93 65.96

8

Schwefel

3

Time(s) Ratio Reduction (%)

1.156

.735 1.57 36.42

.906

.641 1.41 29.25

.922

.641 1.43 30.47

9

Capprasse0 s

4

Time(s) Ratio Reduction (%)

.266

.047 5.66 82.33

.359

.047 7.64 86.91

.266

.047 5.66 82.33

10

Adaptive LV

4

Time(s) Ratio Reduction (%)

.016

.016 1 0

.031

.016 1.94 48.39

.031

.016 1.94 48.39

11

Trid(4)

4

Time(s) Ratio Reduction (%)

1.157

1.031 1.12 10.9

1.29

1.125 1.15 12.79

1.516

1.407 1.07 7.19

12

A. H. Wright

5

Time(s) Ratio Reduction (%)

.157

.047 3.34 70.06

.188

.047 4 75

.047

.047 1 0

13

Magnetism in physics (6)

6

Time(s) Ratio Reduction (%)

1.09

.875 1.25 19.72

1.14

.875 1.3 23.24

1.09

.875 1.25 19.72

14

Butcher

6

Time(s) Ratio Reduction (%)

2065

.109 18944 99.99

.375

.109 3.54 71.73

6.73

.109 61.74 98.38

15

Magnetism in physics (7)

7

Time(s) Ratio Reduction (%)

9.17

9.17 1 0

12.7

9.17 1.38 27.79

12.7

9.17 1.39 27.79

16

Heart dipole

8

Time(s) Ratio Reduction (%)

23.2

1.53 15.16 93.4

42.6

1.53 27.84 96.4

23.2

1.53 15.16 93.4

351

P. S. V. Nataraj and M. Arounassalame/ A New Subdivision Algorithm for the Bernstein Polynomial Approach to . . . Table 4

Minimum, mean and maximum values of Ratio and Reduction over all the 16 test problems with different rules for direction selection Ratio

5

Reduction (%)

Performance metric

Algorithm

Minimum

Mean

Maximum

Minimum

Mean

Maximum

Subdivisions

Rule A (proposed) Rule B (proposed) Rule C (proposed)

1 1 1

1276 5.368 6.967

20376 29.00 67.50

0 0 0

48.37 46.24 41.40

99.9 93.9 98.5

Computational time

Rule A (proposed) Rule B (Proposed) Rule C (Proposed)

1 1 1

1186 3.85 6.31

18944 27.84 61.74

0 0 0

39.99 40.02 35.92

99.9 96.4 98.3

Conclusions

6. Dixon and Price: Dixon and Price function

We presented a novel Bernstein algorithm for unconstrained global optimization of multivariate polynomial functions. The new ingredients in the proposed Bernstein algorithm comprise a new rule for selecting the subdivision point, modified rules for selecting the subdivision direction, and a new acceleration test to avoid some unnecessary subdivisions. The performance of the proposed Bernstein algorithm was tested and compared with that of the existing or basic algorithm on a collection of 16 test problems. Overall, the proposed algorithm is found to be considerably more efficient than the basic algorithm, giving average Reduction of 40% - 48% in the number of subdivisions and 35% - 40% in the computational time, depending on the direction rule considered.

Appendix

f (x1 , x2 , x3 ) = (x1 − 1)2 + 2(2x22 − x1 )2 + 3(2x23 − x2 )2 where x i = [−10, 10], i = 1, 2, 3. 7. Reaction diffusion: A 3-dimensional reaction diffusion problem f (x1 , x2 , x3 ) = −x1 + 2x2 − x3 − 0.835634534 x2 (1 + x2 ) where x i = [−5, 5], i = 1, 2, 3. 8. Schwefel: f (x1 , x2 , x3 ) =(x1 − x22 )2 + (x2 − 1)2 + (x1 − x23 )2 + (x3 − 1)2 where x i = [−10, 10], i = 1, 2, 3. 9. Caprasse0 s: Caprasse0 s system f (x1 , x2 , x3 , x4 ) = − x1 x33 + 4x2 x23 x4 + 4x1 x3 x24 + 2x2 x34 +

We denote the test function as f (xi ) and the initial bounds as x i , where i = 1, 2, · · · , l. 1. Himmelblau: Himmelblau function f (x1 , x2 ) = (x21 + x2 − 11)2 + (x1 + x22 − 7)2 where x i = [−5, 5], i = 1, 2. 2. Camel back: Six hump camel back function f (x1 , x2 ) = 4x21 − 2.1x41 +

1 6 x1 + x1 x2 − 4x22 + 4x42 3

where x i = [−5, 5], i = 1, 2. 3. Booth: Booth function f (x1 , x2 ) = 74 − 38x1 + 5x21 − 34x2 + 8x1 x2 + 5x22 where x i = [−10, 10], i = 1, 2. 4. Rosenbrok: Rosenbrok function f (x1 , x2 ) = 100(x21 − x2 )2 + (x1 − 1)2 where x i = [0, 5], i = 1, 2. 5. Trid(3): 3-variable Trid function f (x1 , x2 , x3 ) =(x1 − 1)2 + (x2 − 1)2 + (x3 − 1)2 − x1 x2 − x2 x3 where x i = [−9, 9], i = 1, 2, 3.

4x1 x3 + 4x23 − 10x2 x4 − 10x24 + 2 where x i = [−.5, .5], i = 1, 2, 3, 4. 10. Adaptive LV: A neural network modeled by an adaptive Lotka-Volterra system f (x1 , x2 , x3 , x4 ) = x1 x22 + x1 x23 + x1 x24 − 1.1x1 + 1 where x i = [−2, 2], i = 1, 2, 3, 4. 11. Trid(4): 4-variable Trid function f (x1 , x2 , x3 , x4 ) =(x1 − 1)2 + (x2 − 1)2 + (x3 − 1)2 + (x4 − 1)2 − x1 x2 − x2 x3 − x3 x4 where x i = [−16, 16], i = 1, 2, 3. 12. A. H. Wright: System of A. H. Wright f (x1 , x2 , x3 , x4 , x5 ) = x1 + x2 + x3 + x4 − x5 + x25 − 10 where x i = [−5, 5], i = 1, 2, · · · , 5. 13. Magnetism in physics (6): 6-variable magnetism in physics problem f (x1 , x2 , x3 , x4 , x5 , x6 ) =2x21 + 2x22 + 2x23 + 2x24 + 2x25 + x26 − x6 where x i = [−1, 1], i = 1, 2, · · · , 6.

352

International Journal of Automation and Computing 04(4), October 2007

14. Butcher: Butcher0 s problem f (x1 , x2 , x3 , x4 , x5 , x6 )

=x6 x22

x5 x23

+ − 1 4 x1 + x4 3 3

x1 x24

+

x34

+

x24 −

where x1 = [−1, 0], x2 = [−.1, .9], x3 = [−.1, .5], x4 = [−1, −.1], x5 = [−.1, −.05], x6 = [−.1, −.03]. 15. Magnetism in physics (7): 7-variable magnetism in physics problem f (x1 , x2 , x3 , x4 , x5 , x6 , x7 ) =x21 + 2x22 + 2x23 + 2x24 + 2x25 + 2x26 + 2x27 − x1 where x i = [−1, 1], i = 1, 2, · · · , 7. 16. Heart dipole: Heart dipole problem f (x1 , x2 , x3 , x4 , x5 , x6 , x7 , x8 ) = −x1 x36 + 3x1 x6 x27 − x3 x37 + 3x3 x7 x26 − x2 x35 + 3x2 x5 x28 − x4 x38 + 3x4 x8 x25 − 0.9563453 where x 1 = [−.1, .4], x 2 = [.4, 1], x 3 = [−.7, −.4], x 4 = [−.7, .4], x 5 = [.1, .2], x 6 = [−.1, .2], x 7 = [−.3, 1.1], x 8 = [−1.1, −.3].

References

[12] A. R. H. A. Ahmed. Studies on Metaheuristics for Continuous Global Optimization Problems, Ph.D. dissertation, Kyoto University, Japan, 2004. [13] J. Verschelde. The PHC pack, the Database of Polynomial Systems, Technical Report, Mathematics Department, University of Illinois, USA, 2001. [14] J. Garloff, The Bernstein Algorithm. Interval Computations, vol. 6, no. 2, pp. 154–168, 1993. [15] Z. A. Garczarczyk. Parallel Schemes of Computation for Bernstein Coefficients and their Application. In Proceedings of International Conference on Parallel Computation in Electrical Engineering, Poland, pp. 334–337, 2002. [16] J. Berchtold, I. Voiculescu, A. Bowyer. Multivariate Bernstein Form Polynomials, Technical Report 31/98, School of Mechanical Engineering, University of Bath, UK, 1998. [17] S. Ray. A New Approach to Range Computation of Polynomials Using the BernsteinForm, Ph.D. dissertation, Indian Institute of Technology Bombay, India, 2006. [18] R. E. Moore. Methods and Applications of Interval Analysis, SIAM, Philadelphia, 1979. [19] R. Hammer, M. Hocks, U. Kulisch, D. Ratz. Numerical Toolbox for Verified Computing I, Springer-verlag, Berlin, 1991.

[1] R. Horst, P. M. Pardalos. Handbook of Global Optimization, Kluwer Academic, Netherlands, 1995.

[20] P. S. V. Nataraj, K. Kotecha. An Improved Interval Global Optimization Algorithm Using Higher Order Inclusion Function Forms. Journal of Global Optimization, vol. 32, no. 1, pp. 35–63, 2005.

[2] R. B. Kearfott. Rigorous Global Search: Continuous Problems, Kluwer Academic, Netherlands, 1996.

[21] M. R. Guide. MATLAB version 6.1, The Mathworks Inc., MA, 2001.

[3] E. Hansen, G. W. Walster. Global Optimization Using Interval Analysis, Marcel Dekker, New York, 2004. [4] L. Jaulin, M. Kieffer, O. Didrit, E. Walter. Applied Interval Analysis, Springer, London, 2001. [5] R. Vaidyanathan, M. Ed–halwagi. Global Optimization of Nonconvex Nonlinear Programs via Interval Analysis. Computers Chemical Engineering, vol. 18, no. 10, pp. 889–897, 1994. [6] J. Garloff. Convergent Bounds for Range of Multivariate Polynomials. Interval Mathematics, Lecture Notes in Computer Science, K. Nickel (ed.), Springer, Berlin, vol. 212, pp. 37–56, 1985. [7] H. Ratschek, J. Rokne. Computer Methods for the Range of Functions, Ellis Horwood, England, 1984. [8] M. Zettler, J. Garloff. Robustness Analysis of Polynomials with Polynomial Parameter Dependency Using Bernstein Expansion. IEEE Transactions on Automatic Control, vol. 43, no. 3, pp. 425–431, 1998. [9] P. S. V. Nataraj, K. Kotecha. An Algorithm for Global Optimization Using Taylor–Bernstein Form as Inclusion Function. Journal of Global Optimization, vol. 24, no. 4, pp. 417–436, 2002. [10] P. S. V. Nataraj, K. Kotecha. Global Optimization with Higher Order Inclusion Function Forms Part 1: A Combined Taylor–Bernstein Form. Reliable Computing, vol. 10, no. 1, pp. 27–44, 2004. [11] D. Ratz, T. Csendes. On the Selection of Subdivision Directions in Interval Branch-and-bound Methods for Global Optimization. Journal of Global Optimization, vol. 7, no. 2, pp. 183–207, 1995.

P. S. V. Nataraj received his Ph.D. degree from the Indian Institute of Technology (IIT) Madras, India in process dynamics and control in 1987. He worked at the CAD center at IIT Bombay, India before joining the faculty of the Systems and Control Engineering Group of IIT Bombay in 1988. Presently, he is a professor and convenor for the Systems and Control Engineering Group of IIT Bombay, India. He has published about 25 refereed journal papers and more than 50 conference papers. His research interests include quantitative feedback theory, Bernstein polynomials and its applications to control, interval global optimization, constraint propagation, and FPGA-based system design. M. Arounassalame received his B.Tech. degree from Pondicherry University, India, in 1993 and M.Tech. degree from the University of Calicut, India, in 1998. Presently, he is working as a lecturer at the Department of Electrical and Electronics Engineering, Pondicherry Engineering College, India. He is a Ph.D. candidate in Systems and Control Engineering Group of IIT Bombay, India. His interests include interval global optimization, applications of global optimization to robust stability and control, Bernstein polynomials, and FPGA-based system design.