Cubic algorithm for global optimization with box and equality ...

m

MATHEMATICAL

Available online at www.sciencedirectcom s¢,~.om

ELS~ImR

AND COMPUTER MODELLING

~D,REOV.

Mathematical and Computer Modelling 40 (2004) 63-76 www.elsevier.com/]ocate/mcm

Cubic Algorithm for Global Optimization with Box and Equality Constraints and Application to Optimal Allocation of Resources E. A. GALPERIN t D6partement de Math~matiques Universit~ du Quebec k Montreal C.P. 8888, Succ. Centre-Ville Mont%al, Quebec, Canada H3C 3P8 galperin, ef im©uqam, ca

P. YA. EKELt AND J. G. PEREIRA, JR. t Programa de Pds-Craduaq~o em Engenharia E16trica Pontificia Universidade Catdlica de Minas GerMs Av. Dom Jos~ Gaspar, 500 30.535-610, Belo Horizonte, MG, Brasil ekelOpacminas, br j oelgpj ~hotmail. com

(Received and accepted December 2003) A b s t r a c t - - C + + code of the cubic algorithm [1] is proposed for nonconvex global optimization of continuous functions with box and equality constraints, and for the solution of nonlinear systems of equations, including underdetermined and overdetermined systems. The algorithm is upgraded for application to problems with linear and nonlinear equality constraints usually encountered in optimal resource allocation, investment planning, construction projects, and many engineering applications. It is further extended for visualization analysis of high dimensional sets of global optimizers by means of projections on linear subspaces and on arbitrary planes. C + + windows are presented for input, output, algorithmic control, and special research possibilities including graphics and screen display. The code does not create ill-conditioned situations. © 2004 Elsevier Ltd. All rights reserved. K e y w o r d s - - N o n c o n v e x global optimization, Cubic Mgorithm, Numerical methods.

1. I N T R O D U C T I O N In this paper, the cubic algorithm [1] is upgraded and extended to include effective separate treatment of equality constraints, and other noteworthy features for analysis of the set of global optimal solutions. There exist codes for the cubic algorithm in MATLAB for dim = 1, see [2], and in MAPLE for arbitrary dimension n, see [3]. A code in (object oriented) C + + for this version of the cubic algorithm allows one to use vast libraries developed by Microsoft and provides convenient Window tResearch of this author was partially supported by the NSERC Grant No. RGPIN-3492-00. *Research of these authors was supported by the CNPq Grant No. 550408-2002/9 and the PUC-MG Grant No. FIP 2003/46-TLE. 0895-7177/04/$ - see front matter ~) 2004 Elsevier Ltd. All rights reserved. doi: 10.1016/j.mcm.2003.12.003

Typeset by A_A/~S-TEX

64

E.A.

GALPERIN et al.

presentation at any iteration, see Section 6. New features of this code include linear subspace projection and intersection blocks and plane intersection block which provide the possibility for analysis and visualization according to the needs of the user in a practical application. The code can be further upgraded for nonrectangular inf or sup-compact sets in R ~ of any nature (nonrobust, disconnected) given by a system of inequalities using the beta-algorithm [1, pp. 79-134] that sifts out all sets of infeasible points without loss of global optimizers. The code solves a problem of finding the global minimum value s ° = inf f(x),

x E 2 C R ~,

(1.1)

over the n-rectangle (box)

(1.2)

Jf = {x E R ~ : a~ _< x~ _< bi, i = 1 , . . . , n } , and the set of all global minimizers Z ° : {x e : ? : f ( x ) = s°}. Here ai < bi, Vi, the bar means closure, and the function continuous over .~, that is

If(x)-f(x')l < A l l x - x'll,

(1.3)

f(x)

is supposed to be Lipschitz

Vx, x' e .,,t.

(1.4)

If the minimal Lipschitz constant over J( is known (the symbol V means gradient) L = max IIVf(x)ll,

x e )?,

(1.5)

then one can take in (1.4) the value A = L in which case (1.4) is nonimprovable over Jr. A function f(x) is not supposed to be given as a formula, it is only assumed to be computable. In such cases, the constant L of (1.5) is unknown and difficult to determine. Moreover, a subset of X can have a smaller L than defined by (1.5) over the entire Jr. For these reasons, we do not associate A in (1.4) with the value in (1.5) and treat it as a bound on the slopes of f(x) within Jr. This has transparent meaning if we consider the angle 7r

a = argtanA,

0 < a < 2"

(1.6)

The reader can check that, if (1.4) is met, then in any plane section of )( passing through x, x ~ the slope of f(x) is not greater than a. Table 1 provides the visualization for the notion of a function of bounded slope. T a b l e 1. 0 A

0

30 ° 1

,/5

45 °

60 °

63 °

71 °

76 °

78 °

1

Jg

2

a

4

5

80 °

For A > 3, slopes are very steep, and scaling should be applied to assure reasonable computer time for the procedure. For universality of the code, it is expedient to transform a problem (1.1)-(1.3) into the unit cube [7, axes oriented, with the edge c -- 1 and a vertex at the origin. It is formed by the unit vectors of the Cartesian coordinate system which is supposed to be the reference system in (1.2). The linear transformation

xi=ai+(bi-ai)zi,

i=l,...,n,

(1.7)

Cubic Algorithm

65

converts problem (1.1),(1.2) into the problem

infg(z)=inff[a+(b-a)z],

zeU=(zeR~':O 0 means the deviation of x ° from one, some or all constraints (4.5). It is worth noting t h a t the intersection S* -- nS* may be empty both for linear (4.1) and/or nonlinear (4.5) manifolds. However, if c -- 0 in (4.6), then S* ¢ ~, nonempty, and S* N X ° ~ ~, and the cubic algorithm at the second stage (4.6) yields, in the limit, the entire subset

X* = ( x ~ X °nS*} c_X°

(4.7)

with respect to the intersection of S~ for which f~i > 0 in (4.6). The set X* or its approximation presents the solution of (1.1)-(1.3) with additional equality constraints (4.1) and/or (4.5) if X ° n S* ¢ 0, nonempty. If c > 0 in (4.6), then X ° N S* = 0, empty, and the cubic algorithm should be applied first to the function in (4.6) under constraints (1.2) and then to the function in (1.1) over remaining quasi-cubic subsets, or intermittently, see [1, pp. 141-145], and the global min-value s* obtained will obviously be greater than in case of (1.2): s* > s o if X ° A S* -- ~.

5. V I S U A L I Z A T I O N V I A P L A N E PROJECTIONS AND SECTIONS For multidimensional optimization problems with n > 3, especially with a large number of minimizers x~0 E X 0 or in cases when X ° is a continuum, it is convenient to visualize the distribution of minimizers through projections on planes chosen by the user which can be seen on the screen of computer. In [3], such visualization on coordinate planes is incorporated into the M A P L E code and can be printed as output instead of large tables of numbers. Here we propose visualization on any plane passing through the origin, not necessarily on a coordinate plane. Any vector (n-tuple) x = ( x l , . . . ,x~) E R ~ is represented in the memory of a computer by the numbers xi in some assumed (imaginary) orthonormal coordinate system {e~}, Ileitl =- 1, eiej = O, i,j = 1,... ,n, so that we have x = ~ = l xiei. Take two vectors n

p = Zp,

ILptE = 1,

(5.1)

Ilqll = 1,

(5.2)

i=1 n

q = i=1

where pi, qi are any numbers (specified by the user in the input information) t h a t are automatically normalized in the computer. Vectors p~, qi define a plane Q passing through the origin,

Cubic Algorithm

71

and vice versa, any plane passing through the origin can be defined by some numbers Pi, q~ from (5.1),(5.2). In the reference system {ei}, we have the projections

i=1

xq = x q = ~

(5.4)

xiqi,

i=l

that are calculated as two numbers in a computer. In general, pq = ~in=lpiqi ~ O, and hence, unit (normalized) vectors p, q from (5.t),(5.2) do not form an orthogonal basis (Cartesian reference system) in the plane Q. For this reason, the value

cos0 = cos(p, q) =

(55) i=1

should be computed and displayed on the screen together with the axes p, q at the angle 0 of (5.5) that form an affine coordinate system p, q in the plane Q. Now, the projection of x 6 R n on the plane Q can be plotted on the screen as follows:

=

+ xqq =

+ q 4=i

(5.6) i=I

where the sums are coordinates along axes p and q in the plane Q. Hence, the graphics subroutine in a computer should plot a straight line through the point xp on the axis p parallel to the axis q, and another straight line through the point xq on the axis q parallel to the axis p. The intersection of these two lines defines the projection point x e of (5.6) on the screen of computer. Of course, those auxiliary lines should not be visible on the screen, but axes with unit vectors p, q and points xQ should be clearly displayed. Doing this for all global minimizers x ° E X ° presents a picture on the screen that can be printed and used as output information for further analysis in various viewing planes Q = (p, q) chosen by the researcher. Obviously, if X ° is a continuum, the procedure would cause the overflow in a computer. To avoid this nuisance, the purification procedure (a '%acuum cleaner") should be incorporated in the code. Suppose that human eye cannot distinguish points which are at a distance 5 < 0.2 mm from each other on the screen or on the printout sheet. Then, converting the numbers (5.3),(5.4) into mm (millimeters) scale and using the cosine theorem in the plane Q, we have for two points x E X °, y E X ° 52 = (Xp - yp)2 + (Xq - yq)2 _ 2(Xp - yp)(Xq - yq) cos 0,

(5.7)

so if 5 _ 50 = 0.2, one of points x or y should be discarded from the memory before plotting on the screen. The lower limit 60 should be specified in the input information if the visualization block (screen picture) is required, otherwise, the value 6o = 0.2 mm will be used by default. Note that 50 is independent from U and ~, cf. (2.7)-(2.9), because it acts in a different space Q of dim = 2. Also, the coordinates xi in (5.6) should be converted from the unit cube U to the original rectangle (1.2) before display, otherwise, a picture in the unit cube will be shown. Instead of projection of X ° upon Q, the intersection X ° n Q may be desirable. If x ° E X°N Q, then x ° 6 Q and two unique numbers u, v must exist such that

(5.8) i=I

i=i

i=i

72

E . A . GALPEmN et al.

In the parentheses of (5.8) are coordinates x ° computed by the cubic algorithm (or their approximations). It means that the system upi + vqi = x oi ,

i = 1,.. .,

(5.9)

n,

with knownp~, q~ (from (5.1),(5.2)) and x°, must have unique solution for u, v, at least, approximately. Denote the matrix ff q

--/Pl,'-',P°/.

B

(5.1o)

Lql," • ,qn J Then, in vector notations, the least squares minimum norm solution of (5.9) is [:] = (BB')-I Bx °

(5.11)

with discrepancy, see [5,6], A = B'(BB')-IBx

° - x ° = [B'(BB')-IB

(5.12)

- I] x °.

We have t h a t x ° E Q if and only if A = 0. If 0 < IA1 < z,~ = 1 I N "~, cf. (2.7), then x ° ¢t Q but belongs to the era-layer centered at the plane Q. If ]A] > e,~ for all x ° E X °, then X ° n Q = 0, empty, up to e m distance from Q. EXAMPLE 5.1. Let us consider the example from [3, p. 84] for arbitrary n rain/(x)

=

.5 + 9.5

x

,

x e [0, 21~ C R~_.

(5.13)

Introducing radius-vector 3, it becomes clear that the global min-value for (5.13) is zero if 0.5 + 9.5r = Irk,

k = 0, 4-1, . . . .

(5.14)

Since we have 0 1. It is clear that the set X ° of all global minimizers is given by the intersection of eight spheres corresponding to the first eight natural values of k with the cube [0, 2] ~. The picture of X ° and of f ( x ) in (5.13) for n = 2 is given in [3, pp. 84,85]. By symmetry, it is obvious that intersection of X ° by any plane passing through zero is given by eight arcs identical to those displayed in [3, p. 84]. It is also clear t h a t projection of X ° on any coordinate plane is dense. Let us illustrate the computation of plane sections by (5.8)-(5.12) for n = 3 in which case the drawings can be made by the reader. Consider a plane Q defined, say, by vectors p = (1, 1, 0), q = (0, 1, 1). The normalization multiplier is l/v/2, and by (5.5) cos0 = 1/2, 0 = 60 °. The matrix B (5.10) is = ~

Lql,q2,q3

0,1,1J "

We have BB'=

[i !] [ 11] 1 '

B'(BB')-IB

2 (BB')-I = 3

1 = g

I

= [ B ' ( B B ' ) - I B - Z] x = ~

2 1

1 2

-1

1 -I

-1 2

1

,

-I

1

(5.18)

Xl

i

I

(5.17)

'

i

x2

~3

•

(5.19)

73

Cubic Algorithm

We see t h a t A = 0 iff x~ - x2 + xs = 0. The vector (1, - 1 , 1) is orthogonal to this plane, and the reader can check that this is exactly the plane Q spanned by vectors p, q above, in a different representation. Now, the plane Q contains only those minimizers x ~ X ° for which x 1 --X 2 Jr-X3 ~- O,

x~+x~+x]=r2(k),

see (5.15) for k = 1 , . . . ,8,

and this defines the same arcs as mentioned above. It is clear t h a t computations by (5.8)-(5.12) are much simpler and faster than running the cubic algorithm to determine a plane section.

6. C + +

WINDOWS SEQUENCE

C + + has a convenient window-like presentation on the screen of computer for input information, control parameters, special features, intermediate results, and final results. In this section, we list several typical windows that appear in the process of global optimization via cubic algorithm. The code itself can be requested from Joel G. Pereira Jr. at the address: Programa de Pds-Gradua§go em Engenharia El~trica, Pontif/cia Universidade Catdlica de Minas gerais, Av. Dom Jos4 Gasper, 500, 30.535-610, Belo Horizonte, MG, Brazil, E-mail: j o e l g p j @ h o t m a i l , com. Initial Window Initial Window (Table 0) displays some data t h a t will be loaded by default. To start the algorithm, do: (1) Write in the windows: F(x) in mathematical symbols, Dimension n, Number of iterations m, Coordinates a~, b~, i = 1 , . . . , n for the Box Constraints. (2) If necessary, change the data and markings in Table 0. (3) Click Load: Table 0 will reappear as modified. Click Print if printout is required.

Input: I ril:ormation :. Table O Fun~.iotito optirriize F(x):. ¢ : lvlinimize:: c Maximize :[

C Centrel CA

...........

n

Functidri #on sfo rmed to unit Cube: :g(:Z)~[~:~(bL;a)z ]

~t~'(~=oiZ:

Dimension S u b divisio r,

:N

Grid Roint: BoxConstraihts:

Slope bound

A

[0.1

Precision

Ib

a

[

IXIo. o[.iteratio r,S

rr!

Min edge 0f subcubes F Sm Control PurifiCation Ticrie .Limit.: sec

Lo~d

1

Reset - \ ............J

OK

J

Figure 1.

[ P [1.5 [ 0.01

[ 0.02

[6o P !0t

!

E.A. GALPERINe$ al.

74

(4) If all is correct, click OK to start computations, or Reset (for another modification), or Cancel for abandoning the work. N.B.: If Central CA is marked, the grid point z0 will be assigned automatically and empty boxes under z0 will appear. Initial Table 0 is kept in the memory and may be called at any time by clicking Start 0 in subsequent Windows. This will erase all accumulated data. (See Figure 1.) Intermediate Windows Current tables are numbered by iterations. They show current iteration number m, time (sec), Sm value, cube edge size, and the number of subcubes remaining at the end of previous iteration. Clicking Show will call the data at the end of current iteration with a stop. Click Restart to continue iterations, Output to call output Table with current data, Start 0 to return to Table 0 of Input Information erasing all accumulated data, Print to print the information at the end of current iteration with a stop. No clicks will continue the iteration process. (See Figure 2.)

Figure 2.

Output W i n d o w Output Window shows parameters of the last iteration stopped at a limit prescribed by the Input Information (Table 0). Clicking Show will call the final data, Print will print the window with a sheet of all final data. Click Start 0 to return to Initial Window (Table 0), erasing all final data. Click Graphics to plot the function F(x) for dimension n = 2 or n = 3. Click Projection to call the next window of Special Features. (See Figure 3.) Research Windows To account for the equality constraint, write its function in the window M i n f , then click in blank Restart. For other features, mark Linear Subspace and write column vectors al, a2,. white columns, or mark Plane and write vectors p, v in Xl, x2 columns; then click Section or -

.

Cubic Algorithm

75

Figure 3.

Figure 4. Projection. Special Features output window will appear with a tick in the output square and all necessary information in the white rectangle. Click Print if printout is required. Click Restart to call Input window of Special Features again for analysis of another situation. Click Start 0 to return to Initial Window (Table 0), erasing all data. (See Figure 4.)

76

E.A. GALPE!%INet al.

7. C O N C L U S I O N S An extension of the cubic algorithm is developed for special applications with linear and nonlinear equality constraints, without converting them to pairs of inequality constraints. This direct and separate treatment allows one to avoid complications usually caused by equality constraints. Projection and visualization methods are made available to help the user in analysis of the set of all global optimal solutions. C + + code is proposed for computational realization of monotonic setwise descent for nonconvex nonsmooth global optimization problems. The C + + window design is presented for better algorithmic control, special research possibilities, and for further improvements and extensions of the algorithm.

REFERENCES 1. E.A. Galperin, The Cubic Algorithm for Optimization and Control, NP Research Publication, Montreal, (1990). 2. A. Ferrari and E.A. Galperin, Numerical experiments with one-dimensional adaptive cubic algorithm, Computers Math. Applic. 25 (10/11), 47-56, (1999). 3. M. Delgado Pineda and E.A. Galperin, Global optimization in R '~ with box constraints and applications: A MAPLE Code, Mathl. Compu~. Modelling 38 (1/2), 77-97, (2003). 4. E.A. Galperin, Global optimization in problems with uncertainties, Nonlinear Analysis 4Y~ 941-952, (2001). 5. R. Penrose, A generalized inverse for matrices, Proc. Cambridge Phil. Soc. 51,406-413, (1955). 6. R. Penrose, On the best approximate solution of linear matrix equations, Proc. Cambridge Phil. Soc. 52, 17-19, (1956).