Incomplete series expansion for function ... - Semantic Scholar

6th World Congresses of Structural and Multidisciplinary Optimization Rio de Janeiro, 30 May - 03 June 2005, Brazil

Incomplete series expansion for function approximation Albert A. Groenwold∗ , L.F.P. Etman† , J.A. Snyman∗ , and J.E. Rooda† ∗

Department of Mechanical and Aeronautical Engineering, University of Pretoria, Pretoria, 0002, South Africa. Email: {Albert.Groenwold,Jan.Snyman}@up.ac.za † Systems Engineering Group, Department of Mechanical Engineering, Eindhoven University of Technology, Eindhoven, the Netherlands. Email: {L.F.P.Etman,J.E.Rooda}@tue.nl

1. Abstract We present an incomplete series expansion (ISE) as a basis for function approximation. The ISE is expressed in terms of an approximate Hessian matrix which may contain second, third and even higher order ‘main’ or diagonal terms, but which excludes ‘interaction’ or off-diagonal terms. From the ISE, a family of approximate interpolating functions may be derived. The interpolating functions may be based on an arbitrary number of previously sampled points, and any of the function and gradient values at previously sampled points may be enforced when deriving the approximate interpolating functions. When function values only are enforced, the approximations are spherical, and the storage requirements are minimal. Irrespective of the conditions enforced, the approximate Hessian matrix is a sparse diagonal matrix. Hence the proposed interpolating functions are very well suited for use in sequential approximate optimization (SAO), based on computationally expensive simulations. In turn, computationally expensive simulations are often required in, for example, optimal structural design problems. We derive a selection of approximations from the family of ISE approximating functions herein; these include approximations based on the substitution of (reciprocal) intervening variables. A comparison with popular approximating functions previously proposed, illustrates the accuracy and flexibility of the new family of interpolating functions. 2. Keywords: Nonlinear function approximation; sequential approximate optimization (SAO); incomplete series expansion (ISE). 3. Introduction Consider the nonlinear programming problem PNLP : minimize f (x) subject to hj (x) = 0;

j = 1, 2, · · · , ne

gj (x) ≤ 0; xiL ≤ xi ≤ xiU ;

j = 1, 2, · · · , ni i = 1, 2, · · · , n

(1)

where f is a real valued scalar objective function, and hj and gj respectively are ne equality and ni inequality constraint functions. f , hj and gj depend on n real design variables x = {x1 , x2 , · · · , xn }T ∈ Rn . xiL and xiU respectively indicate lower and upper bounds of continuous real variable x i . Problem PNLP may be solved using any number of techniques, many of which rely on sequential approximate optimization (SAO). In structural optimization in particular, SAO is often used as an optimization strategy when the objective and/or constraint functions require computationally expensive simulations. In turn, the success of SAO largely depends on the formulation of analytical approximations of high quality for the objective and constraint functions f (x), h(x) and g(x). Rather than sampling the real objective and constraint functions f (x), ˜ h(x) and g(x) during the optimization process, the (inexpensive) analytical approximation functions f˜(x), h(x) ˜ (x) are sampled in successive approximate subproblems. and g The number of points in the approximation may influence the applicability of the approximation: 1-point local approximations are based on function value and gradient values in a single point of the design space. They are valid in the local vicinity of the point. Multipoint local approximations aim to improve the approximation by inclusion of information at previous points visited during the optimization run. At the other end of the spectrum, one finds global approximations. Global approximations aim to replace the original objective and constraint functions by surrogate ones based on simulation experiments spread over the complete design space. Typically, response surface methodologies (RSM) or related techniques are employed to generate global surrogates. Global model 1

building can be used in an SAO setting by repeatedly shrinking (zooming in) the search domain and rebuilding the surrogates, or by building the surrogates in a smaller (local) search domain and iteratively repositioning the search domain. The latter two variants are often applied when ’noisy’ functional behavior is present and gradient information cannot be used. Our study considers local gradient based approximation building. The simplest approximation is the linear Taylor series approximation: f˜L (x) = f (xk ) + ∇T f (xk )(x − xk ), (2) where f (xk ) is the (simulated) function value at xk , and ∇f the corresponding column vector of gradients. The linear approximation may be quite inaccurate even in the vicinity of xk , should significant curvature in the objective or constraint functions be present. (For the sake of brevity, we only present the approximation to f ; hj , j = 1, 2, · · · , ne and gj , j = 1, 2, · · · , ni are of course approximated similarly.) From Taylor’s theorem, it follows that increased accuracy may be obtained by retaining additional terms in the Taylor series expansion about a point xk . Rather than the first order linear approximation f˜L , the second order quadratic approximation may be constructed, viz. 1 f˜Q (x) = f (xk ) + ∇T f (xk )(x − xk ) + (x − xk )T H k (x − xk ), 2

(3)

where H represents the Hessian matrix, with entries Hij = ∂ 2 f (xk )/∂xi ∂xj , i = 1, 2, · · · , n, and j = 1, 2, · · · , n. This is very expensive, and requires evaluation (and storage) of the in general fully populated Hessian matrix H of dimension n × n. As a consequence, many approximating functions use the linear first order expansion as a basis, enhanced by so-called intervening, intermediate or substitute variables. In structural optimization, for example, the reciprocal intervening variables y, such that yi = 1/xi , i = 1, 2, · · · , n, are quite popular, e.g. see [1]. In terms of the intervening variables y, the intervening approximation is constructed as f˜I (x) = f (xk ) +

n X

yi (xi ) − yi (xki )

i=1




∂f ∂xi



∂yi ∂xi

−1 !k

.

(4)

In terms of the original variables x, we obtain, after some algebra, the reciprocal approximation f˜R (x) = f (xk ) +

n X

xi −

i=1

xki


xki xi



∂f ∂xi

k

.

(5)

Similarly, the second order or quadratic Taylor series expansion results in the quadratic reciprocal approximation n  k X x

  
∂f k xki k 2− xi − x i xi xi ∂xi i=1 !   2 k  n n

xkj 1 X X xki ∂ f + xi − xki xj − xkj . 2 i=1 j=1 xi xj ∂xi ∂xj

f˜QR (x) = f (xk ) +

i

(6)

Further to the use of intervening variables, many workers have used ad hoc heuristics, e.g. see the well known CONLIN approximation of Fleury and Braibant [2], which was also independently developed by Starnes and Haftka [3] as the conservative approximation. Another example is the very popular method of moving asymptotes (MMA) approximation by Svanberg [4], and the two-point variants by Chickermane and Gea [5] and Bruyneel et al. [6]. A number of other 1-,2-,3- multi-point approximations have been proposed. We would like to mention the approximations of Canfield [7, 8] and Fadel et al. [9]. The approximation of Fadel et al. is an extension of the so-called projection method proposed by Haftka et al. [10], and the use of exponential variables as proposed by Prasad [11]. Some approximations require iterative procedures, such as the three-point approximation of Salajegheh [12] and the TANA-1 and TANA-2 approximations by Grandhi and coworkers [13, 14]. The iterative procedure has to be applied for each (constraint) function to be approximated, which may be computationally less advantageous. 2

Xu and Grandhi [15] avoid this undesired iterative solution procedure through their TANA-3 approximation, which is an incomplete second-order Taylor series expansion in terms of specific intervening variables; the Hessian has only diagonal elements. The foregoing references are by no means exhaustive. Various other accurate and important approximations may be cited. A review of early work (up to 1993) was presented by Barthelemy and Haftka [16]. One approximation that we will discuss here, is the approximation proposed by Xu et al. [17]. This is a linear combination of the linear and reciprocal approximations, and is given as   n  X (xki )2 k k k ˜ αi (xi − xi ) + βi xi − fLR0 (x) = f (x ) + + , (7) xi i=1 with  representing the ‘residue of the first order approximation’. There are 2n + 1 unknowns, being α i , βi and , which Xu at al. propose to solve using the conditions ∇f˜(xk ) =∇f (xk ), ∇f˜(xk−1 ) =∇f (xk−1 ), f˜(xk−1 ) =f (xk−1 ),

(8)

where xk−1 represents any suitable (previous) point. However, note that while the ‘residue of the first order approximation’  results in f˜(xk−1 ) = f (xk−1 ), it also (undesirably) results in f˜(xk ) 6= f (xk ). Hence we here rather propose the linear-reciprocal approximation   n  X (xk )2 , (9) αi (xi − xki ) + βi xki − i f˜LR (x) = f (xk ) + xi i=1 viz. setting  = 0, obtained by enforcing f˜(xk ) = f (xk ), rather than f˜(xk−1 ) = f (xk−1 ) in (8). Solution of the 2n unknowns αi and βi , i = 1, 2, · · · , n, is straightforward. In this study, rather than using the linear or second order quadratic truncated Taylor series as a basis for function approximation, we propose to use an alternative, incomplete series expansion (ISE) as a basis for function approximation; the fundamental idea being to construct an approximate Hessian matrix which contains second, third and even higher order ‘main’ or diagonal terms, but which excludes ‘interaction’ or off-diagonal terms. The main reason for exclusion of the interaction terms of course being the prohibitively high computational (and storage) requirements associated with these terms. The ISE may still be used as a basis for intermediate or substitute design variables. 4. Incomplete series expansion Our proposed ISE is given as f˜ISE = f (xk ) +

p X 1 kT cj hx − xk ij , j! j=1

(10)

where we have defined the vector haij = {|a1 |j , |a2 |j , · · · , |an |j }T , hence cTj haij = cj1 |a1 |j + cj2 |a2 |j + · · · + cjn |an |j . It is illustrative to rewrite (10) in the following form: f˜ISE = f (xk ) + ∇T f (xk )(x − xk ) +

p X 1 kT cj hx − xk ij , j! j=2

(11)

viz. we have enforced ck1 = {ck11 , ck12 , · · · , ck1n }T = ∇f (xk ). For the sake of clarity, we will use form (11) in the remainder of this study. Obviously, (11) reveals that f˜(xk ) =f (xk ), and ∇f˜(xk ) =∇f (xk ),

(12)

which is a prerequisite for local function approximations in trust-region frameworks, e.g. see Alexandrov et al. [18] and Conn et al. [19]. The order of the ISE in (11) is arbitrary. The ckj , j = 2, 3, · · · , p may be obtained by 3

prescribing any of the function and gradients values at selected previous points. Again: if function values only are selected, the storage requirements are minimal, while the resulting approximations are spherical. Irrespective of the conditions enforced, the approximate Hessian matrix is sparse; it is in fact a diagonal matrix. Hence the ISE is very well suited for use in optimal design. In summary: the approximate Hessian matrix may contain second, third and even higher order ‘main’ or diagonal terms, but ‘interaction’ or off-diagonal terms are not included. The latter are simply too expensive to evaluate; this simplification constitutes the main difference between our ISE and a Taylor series expansion. The incomplete series expansion of order 0 (ISE0) about xk is the constant approximation function f˜ISE0 (x) = f (xk ). Having obtained ck1 = {ck11 , ck12 , · · · , ck1n }T = ∇f (xk ), (as we will do throughout this study), the ISE of order 1 equals the linear approximation (2). The interesting ISE approximation functions are of order 2 and higher. 4.1. Incomplete series expansion of order 2: two-point spherical quadratic ISE The ISE approximation function of order 2 is given as 1 hx − xk i2 , f˜ISE2 = f (xk ) + ∇T f (xk )(x − xk ) + ckT 2 2 which may also be written as 1 f˜ISE2 (x) = f (xk ) + ∇T f (xk )(x − xk ) + (x − xk )T C k2 (x − xk ), 2

(13)

(14)

with C k2 an n × n (diagonal) matrix, viz. C k2 = diag(ck21 , ck22 , · · · , ck2n ). This form illustrates the similarity between the ISE approximation function of order 2 and the second order Taylor series expansion; C k2 may be seen as an ‘approximate diagonal Hessian matrix’. If we select ck2i ≡ ck2 ∀ i, then C k2 = ck2 I, which results in a spherical quadratic approximation, and requires the determination of the single unknown c k2 . This may be obtained by (for example) enforcing the condition f˜(xk−1 ) = f (xk−1 ), where x

k−1

(15)

represents any suitable (previous) point. Clearly, this implies ck2 =

2[f (xk−1 ) − f (xk ) − ∇T f (xk )(xk−1 − xk )] , kxk−1 − xk2 k22

(16)

where the norm k · k2 represents the Euclidean or 2-norm. Conceptually, the spherical quadratic approximation function is in all probability the simplest 2-point approximation one can envision. Further to the function and gradient information in the current point xk , the only additional information used is the function value in the ‘previous’ point xk−1 . Notwithstanding, the spherical quadratic function is able to capture the essential curvature of many a problem, as we will demonstrate in subsequent sections. (Also note that the spherical quadratic approximation function reduces to the linear approximation when no curvature is present, viz. c k2 = 0.) The spherical quadratic ISE was developed by Snyman and Stander [20], and popularized by Snyman et al. [21, 22]. We will denote the spherical quadratic ISE by f˜SQ . 4.2. ISE function of order 3 (special case): two-point spherical cubic ISE When the (cubic) ISE function of order 3 is expressed as a spherical approximation, we obtain 1 1 f˜ISE3 = f (xk ) + ∇T f (xk )(x − xk ) + ck2 hhx − xk ii22 + ck3 hhx − xk ii33 , 2 6

(17)

where hh·iipp denotes the p-criterion, given by hhyiipp =

n X

|yi |p .

i=1

k j k k j Note that if ckji ≡ ckj ∀ i, then ckT j hx − x i = cj hhx − x iij . This time, we require the determination of the two unknowns ck2 and ck3 ; again, (15) is enforced; the simplest additional condition at xk−1 probably is

hh∇f˜(xk−1 ) − ∇f˜(xk )ii11 = hh∇f (xk−1 ) − ∇f (xk )ii11 . 4

(18)

Finding ck2 and ck3 is straightforward. We will denote this spherical cubic ISE by f˜SC ; some alternatives to this spherical cubic ISE are presented in Section 7. 4.3. n-point spherical ISE function of order p The ISE functions of order p require the determination of d = p − 1 unknowns. An obvious possibility for the required conditions is f˜(xk−g ) = f (xk−g ), g = 1, 2, · · · , d. (19) This results in the backward conditions 1 f˜(xk−1 ) = f (xk ) + ∇T f (xk )(xk−1 − xk ) + ck2 hhxk−1 − xk ii22 + · · · + 2 1 f˜(xk−2 ) = f (xk ) + ∇T f (xk )(xk−2 − xk ) + ck2 hhxk−2 − xk ii22 + · · · + 2

1 k c hhxk−1 − xk iipp , p! p 1 k c hhxk−2 − xk iipp , p! p

etc., up to 1 1 f˜(xk−d ) = f (xk ) + ∇T f (xk )(xk−d − xk ) + ck2 hhxk−d − xk ii22 + · · · + ckp hhxk−d − xk iipp . 2 p! (20) After some algebra, the ckj , j = 2, · · · p, may be obtained. (Care should of course be taken to ensure that high values of p do not result in polynomial-like interpolation.) We will denote the above ISE functions by f˜F i , with i the order of the approximation. Hence f˜F 4 indicates the ISE function of order 4, obtained from enforcing conditions (19) with 3 unknowns. f˜F 2 is of course identical to the spherical quadratic approximation f˜SQ presented in Section 4.1, f˜F 3 is a 3-point spherical cubic alternative to the 2-point spherical cubic approximation presented in Section 4.2, etc. 5. Intervening variables for the ISE functions We now present an example of intervening variables for the ISE functions. We will do so for the second order spherical quadratic ISE function f˜SQ (or the identical approximation f˜F 2 ). Extension to the higher order ISE functions, or enforcement of gradient information, is possible, although the algebra is not quite as trivial. Consider the intervening variables y = y(x), such that yr = yr (x), r = 1, 2, · · · , n1 ,

(21)

and y ∈ Rn1 ; the n1 intervening variables yr are functions of the design variables xi , i = 1, 2, · · · , n. The spherical quadratic ISE function of order 2, f˜SQ , expressed in terms of intervening variables y, may be written as   n1 n1 X
∂f k 1 k X yr − yrk f˜I = f (y k ) + (yr − yrk )2 , (22) + c2 ∂y 2 r r=1 r=1 where we have again resorted to index notation for the sake of clarity. For illustrative purposes, we now again consider the simple reciprocal intervening variables yi =

1 , i = 1, 2, · · · , n. xi

Since the intervening variables yi , i = 1, 2, · · · , n are functions of a single design variable, we obtain the intervening interpolation in terms of the original design variables as   n n X
∂f ∂xi k 1 k X (yi (xi ) − yi (xki ))2 . (23) yi (xi ) − yi (xki ) + cR2 f˜I = f (xk ) + · ∂x ∂y 2 i i i=1 i=1

In terms of the original variables x, this results in the reciprocal second order ISE   k 2 n  n X
xki ∂f 1 k X 1 1 k k ˜ fRF2 = f (x ) + + cR2 − k xi − x i . xi ∂xi 2 xi xi i=1 i=1 5

(24)

Acronym

Table 1: Overview of some of the approximations used in this study Approximation

Reference

f˜L f˜Q f˜R f˜QR f˜LR0 f˜LR f˜SQ f˜SC f˜F p f˜R2 f˜∇2 f˜F ∇2

Linear 1-point approximation Quadratic 1-point approximation (with full Hessian) Reciprocal 1-point approximation Quadratic reciprocal 1-point approximation (with full Hessian) Original linear-reciprocal 2-point approximation by Xu et al. [17] Modified linear-reciprocal 2-point approximation Spherical quadratic 2-point ISE approximation Spherical cubic 2-point ISE approximation General spherical p-point ISE approximation f˜SQ (or f˜F 2 ) with reciprocal intervening variables Non-spherical 2-point ISE (gradients enforced at xk−1 ) Non-spherical 2-point ISE (function value and gradients enforced at xk−1 )

Eq. (2) Eq. (3) Eq. (5) Eq. (6) Eq. (7) Eq. (9) Section 4.1 Section 4.2 Section 4.3 Section 5 Section 6 Section 7

Again, we choose to enforce

f˜RF2 (xk−1 ) = f (xk−1 ),

(25)

which now results in "

2 f (xk−1 ) − f (xk ) − ckR2 =

n X

(xk−1 − xk )

i=1

2 n  X 1 1 − k xi xi i=1



xki xi



∂f ∂xi

k #

.

(26)

The second order ISE reciprocal approximation f˜RF2 (x) satisfies f (x) in two distinct points, Yet, in sequential approximate optimization, we require the additional storage of a single scalar only, namely f (x k−1 ). General approximations based on reciprocal variables are problematic when their denominators equal zero. While this is often unimportant in structural optimization if the design variables represent positive quantities like area, this may be overcome by an approach similar to that of Haftka and Shore [23]. For multi-point reciprocal approximations like f˜RF2 (x), it is also required that kxk−1 − xk k2 > ,  > 0 and prescribed. 6. Non-spherical ISE functions in terms of point-wise satisfied gradients Rather than prescribing the function values at the previous points xk−1 , · · · , it is also possible to prescribe the gradients at these points. For obvious reasons, this becomes storage intensive; the storage of p − 1 n-vectors is required. This is possibly unattractive, when p and n are large. Hence we only derive the ISE of order 2 with point-wise gradient conditions enforced; we will denote this approximation by f˜∇2 . (Note however that the expense associated with the (p − 1) n-vectors is storage related only; they were previously calculated anyway.) For the sake of clarity, again consider the quadratic form 1 f˜2 (x) = f (xk ) + ∇T f (xk )(x − xk ) + (x − xk )T C k2 (x − xk ). 2

(27)

This time the approximate diagonal Hessian matrix C k2 is constructed as C k2 = diag(ck21 , ck22 , ck23 · · · , ck2n ). We require n unknowns to be solved, for which the conditions ∇f˜(xk−1 ) = ∇f (xk−1 )

6

(28)

suffice. After some algebra, this results in ( ck2i =

∂f ∂xi

k−1

−



∂f ∂xi

k )


/ xk−1 − xki . i

(29)


We require xk−1 − xki 6= 0, i = 1, 2, · · · , n. In practice, we require |xk−1 − xki | > , for i = 1, 2, · · · , n, and i i k−1 k  > 0 prescribed, else we set c2i := c2i . 7. Alternative forms and conditions for the ISE One may propose many a different condition in an ISE of given form, which results in ‘tailoring’ of the higher order approximate diagonal ‘Hessian’ matrices. As a single example only, again consider the cubic ISE: 1 1 hx − xk i2 + ckT hx − xk i3 . f˜ISE3 = f (xk ) + ∇T f (xk )(x − xk ) + ckT 2 2 6 3

(30)

We now wish to enforce both f˜(xk−1 ) =f (xk−1 ), and ∇f˜(xk−1 ) =∇f (xk−1 ).

(31)

The (n + 1) conditions required may be satisfying by setting ck2 = {ck2 , ck2 , ck2 , · · · , ck2 }T = ck2 i, and ck3 = {ck31 , ck32 , ck33 , · · · , ck3n }T , where i represents the n−vector (1, 1, 1, · · · )T . The corresponding form of the underlying ISE is 1 1 f˜ISE3a = f (xk ) + ∇T f (xk )(x − xk ) + ck2 hhx − xk ii22 + ckT hx − xk i3 . 2 6 3

(32)

We will denote this non-spherical cubic 2-point approximation by f˜F ∇2a . An alternative to f˜F ∇2a is to rather enforce ck2 = {ck21 , ck22 , ck23 , · · · , ck2n }T , and ck3 = {ck3 , ck3 , ck3 , · · · , ck3 }T = ck3 i. The corresponding form of the underlying ISE now is 1 1 kT k 2 k 3 f˜ISE3b = f (xk ) + ∇T f (xk )(x − xk ) + ckT 2 hx − x i + c3 hhx − x ii3 . 2 6

(33)

We will denote this non-spherical cubic 2-point approximation by f˜F ∇2b . This approximation is somewhat similar to the multipoint cubic approximation proposed by Canfield [8]. Finally, note that the very form of the ISE presented in (11) may also be changed; the ISE may for example be formulated using only the Euclidean 2-norm.

8. Numerical experiments In evaluating some of the approximations discussed in the foregoing, we generate test points x about x k using x = xk + αd, where d represents an n-dimensional ‘search’ or sampling direction vector. The relative percentage error e r we report is defined as 100(f˜ − f )/f , where f˜ represents the approximation f˜(·) under consideration, and f represents the original objective function. For the sake of clarity, the approximations used are summarized in Table 1.

7

10.0 5.0

10.0 ×

+ × + + × × ♦ + ×  × + ♦ + + + ♦ +   + + ♦ × ×  ♦ -5.0 ♦ × ×   ♦ -10.0 × ♦ ♦  -15.0 ♦  -20.0 f˜L ♦ ♦ f˜Q +   -25.0 f˜R  ˜ fQR × -30.0 

Relative error er [%]

0.0

Relative error er [%]

4

5.0 + ? 4 ♦ + ? 4 ♦ ? 4 ? 4 + ? 4 + + × × × × × + + + ? × ♦ + ♦ ♦     + ♦ 0.0 ×     ×  ? ×  ×  ♦ ? × ♦ ? + ♦ 4 ? ♦ ♦ ? 4 -5.0 4 4 f˜SQ ♦ -10.0 4 f˜SC + -15.0 ˜ fF 3  f˜F 4 × -20.0 f˜R2 4 -25.0 ? f˜∇2 -30.0

-35.0 -0.6 -0.4 -0.2 0.0 0.2 0.4 0.6 0.8 1.0 1.2 1.4 α

-35.0 -0.6 -0.4 -0.2 0.0 0.2 0.4 0.6 0.8 1.0 1.2 1.4 α (b) Some of the new ISE approximations

(a) ‘Classical’ 1-point approximations

Figure 1: Example 1: Relative percentage error er versus α, for d = (1, 1)T . Subfigure (a) depicts the classical 1-point approximation functions f˜L and f˜Q , as well as the ‘classical’ reciprocal approximation functions f˜R and f˜QR . Subfigure (b) depicts results for selected ISE approximation functions. The subfigures are on the same scale; Subfigure (b) is zoomed in Figure 2(a) 8.1. Example 1 We study the simple bivariate objective function [17] f (x1 , x2 ) = (x1 x2 )2 . The data used are xk = (5.0, 5.0)T , xk−1 = (5.1, 5.1)T , xk−2 = (5.2, 5.2)T , and xk−3 = (5.3, 5.3)T . Numerical results are presented in Figures 1 and 2. Figure 1(a) depicts the classical 1-point approximation functions f˜L and f˜Q , as well as the ‘classical’ reciprocal approximation functions f˜R and f˜QR . Figure 1(b) depicts results for selected ISE approximation functions. The subfigures are on the same scale; Figure 1(b) is zoomed in Figure 2(a). Clearly, the new ISE functions are quite accurate; they attain the same order of accuracy as the quadratic 1-point approximation f˜Q , given in (3). The ‘polynomial’ general spherical p-point ISE approximations f˜F 3 and f˜F 4 are (almost) exact for this case. It is however easy to generate a sequence of points which illustrates the dangers of ‘polynomial-like interpolation’ (not shown in graphical form). The reciprocal ISE approximation function f˜R2 is intended for problems where the objective or constraint functions vary by some inverse proportional relationship to the design variables. Nevertheless, the reciprocal ISE f˜R2 is far more accurate than the ‘standard’ 1-point reciprocal approximation f˜R . Figure 2(b) illustrates that, for the linear-reciprocal approximation f˜LR0 proposed by Xu et al., f˜LR0 (xk ) 6= f (xk ) (this of course corresponds to α = 0 in the figures). However, for the new modified linear-reciprocal approximation f˜LR , we do obtain f˜LR (xk ) = f (xk ). (For the sake of resolution, we have used xk−1 = (5.5, 5.5)T for the linear-reciprocal approximations f˜LR0 and f˜LR . However, with xk−1 = (5.1, 5.1)T , these approximations are not as accurate as the ISE approximations depicted in Figure 1(b).) Regarding the quadratic approximations only: For this problem, little advantage is derived from the costly (storage intensive) enforcement of gradients in approximation f˜∇2 at xk−1 , rather than the enforcement of function values only at xk−1 , as in approximating function f˜SQ . 8.2. Example 2 Consider the highly nonlinear objective function [14, 17] f (x1 , x2 , x3 , x4 ) = 180x1 + 20x2 − 3.1x3 + 0.24x4 − 5x1 x2 + 37x1 x3 + 8.7x2 x4 − 3x3 x4 − 0.1x21 + 0.001x22 x3 + 95x1 x24 − 81x4 x23 + x31 − 6.2x32 + 0.48x33 + 22x34 − 1.

8

3.0 2.0

4

? ♦

+ ? 4 ♦ ? 4 + ♦ ? 4 ? × × × + + + + × ♦ ♦ + × 0.0 ×      ×  + ×  ? ×  ×  + × ♦  +  ? 4 ♦ ? ♦ -1.0 ? ♦ f˜SQ ♦ 4 -2.0 ˜ ? fSC + ♦ f˜F 3  -3.0 ? ♦ 4 f˜F 4 × ˜ -4.0 fR2 4 ˜∇2 ? f -5.0

Relative error er [%]

Relative error er [%]

1.0

1.0 +

0.5 + + 0.0

♦

+ ♦

+ ♦

♦

♦ -0.5 f˜LR0 f˜LR

4

-6.0 -0.6 -0.4 -0.2 0.0 0.2 0.4 0.6 0.8 1.0 1.2 1.4 α

+

♦ +

+ ♦

-1.0 -0.6 -0.4 -0.2 0.0 0.2 0.4 0.6 0.8 1.0 1.2 1.4 α (b) The linear-reciprocal approximations

(a) Some of the new ISE approximations

Figure 2: Example 1: Relative percentage error er versus α, for d = (1, 1)T . Subfigure (a) is a zoom of Figure 1(b). Subfigure (b) depicts results for the linear-reciprocal approximation f˜LR0 due to Xu et al., and the modification thereof, f˜LR .

Relative error er [%]

10.0

♦

f˜L × ♦

f˜LR0 f˜LR f˜SQ f˜∇2

×

♦ +  × 4

× ×

♦

♦ ♦

× × ♦ ♦ × × ♦  ♦ 4 × ♦  ♦ × 4 4 × ♦ + ♦   ×  ♦ 4 × + × ♦ 4 ♦ 4     × ×  ♦ 4 ♦ 4 0.0   4 +    4 4 4 4 4 4  + + 4 4 4 + + + + + + + + + +  -5.0 + 5.0

-10.0 -0.5 -0.4 -0.3 -0.2 -0.1 0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0 1.1 1.2 α

Figure 3: Example 2: Relative percentage error er versus α, for d = (1, 0, 1, 0)T (case 1). The data are as follows: xk = (0.8, 0.8, 0.8, 0.8)T , and xk−1 = (1.0, 1.0, 1.0, 1.0)T . Two sampling directions are considered, respectively denoted as ‘case 1’ and ‘case 2’. For case 1, d = (1, 0, 1, 0) T ; results for case 1 are depicted in Figure 3. For case 2, d = (0, 1, 0, 1)T . This time, results are depicted in Figure 4. The results illustrate the effectiveness of the non-spherical quadratic approximation f˜∇2 . For this problem, there is a huge advantage associated with the (costly) storage of gradients in the point xk−1 . f˜∇2 is by far the most accurate over the range −0.5 ≤ α ≤ 0.5. 9. Conclusions We have presented an incomplete series expansion (ISE) as a basis for function approximation. From the ISE, a family of interpolating functions may be derived. The interpolating functions may be based on an arbitrary number of previously sampled points, and any of the function or gradients values at previously sampled points may be enforced. The proposed interpolating functions are very well suited for use in optimal structural design, since they result in diagonal approximate Hessian matrices. When function values only are enforced, the approximations

9

10.0

×

Relative error er [%]

× + ×

5.0 × 0.0

-5.0

+

× + + +

+ ×

+ +

+ × ♦  × × + ♦   ♦ + × × ×  ♦ ♦  f˜L ♦  ♦ f˜LR +  ♦ ˜ fSQ   ♦ f˜∇2 ×

+ + + +

 -10.0 -0.5 -0.4 -0.3 -0.2 -0.1 0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0 1.1 1.2 α

Figure 4: Example 2: Relative percentage error er versus α, for d = (0, 1, 0, 1)T (case 2). are spherical, and the storage requirements are minimal. Irrespective of the conditions enforced, the approximate Hessian matrix is a sparse diagonal matrix. The Hessian matrix may contain second, third and even higher order ‘main’ or diagonal terms, but ‘interaction’ or off-diagonal terms are not included. A comparison with popular approximating functions previously proposed, confirms the practicality and flexibility of the new family of interpolating functions.

10. Acknowledgments This work was in part done during the first author’s sabbatical visit to the Systems Engineering Group in the Department of Mechanical Engineering at the Eindhoven University of Technology in Eindhoven, the Netherlands. He wishes to express his sincerest thanks to his hosts for making the visit possible. Support of the first author by the Dutch Technology Foundation STW, Applied Science Division of NWO and the Technology Program of the Ministry of Economic Affairs of The Netherlands, through project ADOPT DWO.5238, as well as the National Research Foundation (NRF) of South Africa, is gratefully acknowledged. References [1] R.T. Haftka and Z. G¨urdal. Elements of Structural Optimization. Kluwer Academic Publishers, Dordrecht, 1992. [2] C. Fleury and V. Braibant. Structural optimization: a new dual method using mixed variables. Int. J. Numer. Meth. Eng., 23:409–428, 1986. [3] J.H. Starnes Jr. and R.T. Haftka. Preliminary design of composite wings for buckling, stress and displacement constraints. J. Aircraft, 16:564–570, 1979. [4] K. Svanberg. The method of moving asymptotes - a new method for structural optimization. Int. J. Numer. Meth. Eng., 24:359–373, 1987. [5] H. Chickermane and H.C. Gea. Structural optimization using a new local approximation method. Int. J. Num. Meth. Eng., 39:829–846, 1996. [6] M. Bruyneel, P. Duysinx, and C. Fleury. A family of MMA approximations for structural optimization. Struct. Multidisc. Optim., 24:263–276, 2002. [7] R.A. Canfield. High-quality approximation of eigenvalue in structural optimization. AIAA J., 1990. [8] R.A. Canfield. Multipoint cubic surrogate function for sequential approximate optimization. Struct. Multidisc. Optim., 27:326–336, 2004.

10

[9] G.M. Fadel, M.F. Riley, and J.M. Barthelemy. Two point exponential approximation method for structural optimization. Struct. Optim., 2:117–124, 1990. [10] R. T. Haftka, J. A. Nachlas, L. A. Watson, T. Rizzo, and R. Desai. Two point constraint approximation in structural optimization. Comp. Meth. Appl. Mech. Eng., 60:289–301, 1987. [11] B. Prasad. Explicit constraint approximation forms in structural optimization. part 1: Analyses and projections. Comp. Meth. Appl. Mech. Eng., 40:1–26, 1983. [12] E. Salajegheh. Optimum design of plate structures using three-point approximation. Struct. Optim., 13:142– 147, 1997. [13] L. Wang and R.V. Grandhi. Efficient safety index calculations for structural reliability analysis. Comp. Struct., 52:103–111, 1994. [14] L. Wang and R.V. Grandhi. Improved two-point function approximation for design optimization. AIAA J., 33:1720–1727, 1995. [15] S. Xu and R.V. Grandhi. Effective two-point function approximation for design optimization. AIAA J., 36:2269–2275, 1998. [16] J.-F.M. Barthelemy and R.T. Haftka. Approximation concepts for optimum structural design - a review. Struct. Opt., 5:129–144, 1993. [17] G. Xu, K. Yamazaki, and G.D. Cheng. A new two-point approximation approach for structural optimization. Struct. Mult. Optim., 20:22–28, 2000. [18] N.M. Alexandrov, J.E. Dennis, R.M. Lewis, and V. Torczon. A trust region framework for managing the use of approximation models in optimization. Struct. Optim., 15:16–23, 1998. [19] A.R. Conn, N.I.M. Gould, and P.L. Toint. Trust-region methods. MPS/SIAM Series on Optimization. SIAM, Philadelphia, 2000. [20] J.A. Snyman and N. Stander. New successive approximation method for optimum structural design. AIAA J., 32:1310–1315, 1994. [21] J.A. Snyman and A.M. Hay. The spherical quadratic steepest descent (SQSD) method for unconstrained minimization with no explicit line searches. Comput. Math. Appl., 42:169–178, 2001. [22] J.A. Snyman and A.M. Hay. The Dynamic-Q optimization method: an alternative to SQP? Comput. Math. Appl., 44:1589–1598, 2002. [23] R.T. Haftka and C.P. Shore. Approximation method for combined thermal/structural design. NASA TP-1428, 1979.

11