Numerical instabilities and convergence control for convex ...

Nonlinear Dyn (2010) 61: 605–622 DOI 10.1007/s11071-010-9674-x

O R I G I N A L PA P E R

Numerical instabilities and convergence control for convex approximation methods Dixiong Yang · Pixin Yang

Received: 16 September 2009 / Accepted: 9 February 2010 / Published online: 7 March 2010 © Springer Science+Business Media B.V. 2010

Abstract Convex approximation methods could produce iterative oscillation of solutions for solving some problems in structural optimization. This paper firstly analyzes the reason for numerical instabilities of iterative oscillation of the popular convex approximation methods, such as CONLIN (Convex Linearization), MMA (Method of Moving Asymptotes), GCMMA (Global Convergence of MMA) and SQP (Sequential Quadratic Programming), from the perspective of chaotic dynamics of a discrete dynamical system. Then, the usual four methods to improve the convergence of optimization algorithms are reviewed, namely, the relaxation method, move limits, moving asymptotes and trust region management. Furthermore, the stability transformation method (STM) based on the chaos control principle is suggested, which is a general, simple and effective method for convergence control of iterative algorithms. Moreover, the relationships among the former four methods and STM are exposed. The connection between convergence control of iterative algorithms and chaotic dynamics is established. Finally, the STM is applied to the convergence control of convex approximation methods for optimizing several highly nonlinear examples. Numerical tests of convergence comparison D. Yang () · P. Yang Department of Engineering Mechanics, Dalian University of Technology, State Key Laboratory of Structural Analysis for Industrial Equipment, Dalian 116023, China e-mail: [email protected]

and control of convex approximation methods illustrate that STM can stabilize the oscillating solutions for CONLIN and accelerate the slow convergence for MMA and SQP. Keywords Convex approximation methods · Numerical instabilities · Chaotic dynamics · Convergence control · Stability transformation method

1 Introduction Nowadays, to develop more general, effective, efficient and robust optimization methods is an important issue in the community of engineering optimization. In structural optimization, there exists such a problem that always involves nonlinear constraints, for example, it is formulated in terms of the direct variables and the reciprocal variables. This problem can no longer be directly and efficiently solved by using general mathematical programming methods. To solve the above problem, Fleury and Braibant [1] recalled the key role played by reciprocal variables in the history of structural optimization and then developed the convex linearization method (CONLIN). However, it was proved that the iterations of CONLIN converge to at least a local optimum only if the addition control on the step size of the design variables is imposed, e.g. the penalized norm term on the objective function or the move limits to the variables [2, 3].

606

Svanberg [4] observed that CONLIN could produce the oscillating solutions for implementing the structural optimization, and presented the method of moving asymptotes (MMA) based on the similar type of convex approximation, in which the moving asymptotes are taken in a heuristic rule and act as move limits. He also pointed out that CONLIN and SLP (Sequential Linear Programming) are practically a special case of MMA by some transformation, respectively. In these methods, the key idea is to replace the primary problem with a sequence of convex and separable explicit subproblems having a simple algebraic structure. Each explicit subproblem represents a convex approximation to the primary problem, obtained through the first order Taylor series expansion of the objective and constraint functions in terms of intermediate variables (e.g. direct or reciprocal variables) [1, 4, 5]. These subproblems can be efficiently solved by using the dual method or interior point method etc., and then the optimal solutions of primal problem are attained. Along with the application of these methods, some non-convergence problems were found and the improvements for them were developed to overcome the shortcomings. Fleury [6] described the second order convex approximation strategies which are the groundwork of the second order MMA. Bletzinger [7] presented several examples with the non-convergence of solutions for MMA, and also suggested a similar method to the second order MMA. To conquer the iterative oscillation of MMA and achieve the global convergence, Zillober [8] proposed the SCP (Sequential Convex Programming) combining MMA with a line search in 1993. Later, Svanberg [9–11] further modified the moving asymptotes and generated the global convergent version of MMA, namely GCMMA. Moreover, some developments on the MMA were advised, such as GMMA [12], GBMMA [13], GCSCP [14], SCPIP [15] and so on. At present, MMA and GCMMA are applied widely in many fields of structural optimization, including shape optimization, topology optimization and multidisciplinary optimization as well. Additionally, Groenwold et al. [16, 17] developed the sequential approximate optimization based on convex approximate concept and incomplete series expansions. SQP (Sequential Quadratic Programming) is another convex approximation approach used popularly in structural optimization, where a convex quadratic programming is solved in each iteration [10]. Hence,

D. Yang, P. Yang

SQP is also selected to compare the convergence properties with MMA family (i.e. CONLIN, MMA, GCMMA) hereafter. According to the nature of considered problems, the convex approximation methods are divided into the following two categories, as pointed out in [13]. One class is monotonous approximation methods such as CONLIN, MMA and GMMA; another class is non-monotonous approximation methods such as GCMMA, GBMMA and SQP. Convex approximation methods constitute the essential approaches in structural optimization. However, CONLIN, MMA and SQP could not perform well in some cases, and even fail when the convexity and conservativeness of optimization problem are not represented appropriately [12]. Moreover, GCMMA could exhibit slow convergence in topology optimization [18]. This paper focuses on the convergence difficulties including mainly the iterative oscillation and slow convergence of convex approximation methods, and will explore and reveal the causes of iterative oscillation of these methods from a novel perspective of chaotic dynamics for discrete or iterative dynamical systems [19–21], quite different from the conventional viewpoints based on intuition and experience. Furthermore, a general, effective and simple convergence control approach, i.e. the stability transformation method (STM), originally developed for the chaos control of dynamical systems by Schmelcher and Diakonos [22], is suggested for controlling the convergent oscillation and speeding up the slow convergence in structural optimization. With the great progress of nonlinear science, the chaotic dynamics theory provides us a powerful approach for controlling the convergent failure of the iterative methods, besides stabilizing the unstable periodic and fixed-point orbits of chaotic dynamical systems [23]. STM is a kind of global chaos feedback control method based on solid mathematical theory [22, 24], and has some connections with the usual approaches for enhancing the iterative convergence in optimization algorithms such as the relaxation method [25, 26], move limits [27–29], moving asymptotes [4] and trust region management [30, 31]. Additionally, it should be mentioned that STM has been applied to capture the stable convergence solutions of iterative algorithms in the first order reliability method (FORM) and the performance measure approach (PMA) of reliability based structural optimization [32, 33].

Numerical instabilities and convergence control for convex approximation methods

In this paper, the essential causes of iterative oscillation of convex approximation methods are exposed using chaos theory. Furthermore, STM for the convergence control of iterative algorithms is suggested, and the relationships among STM and relaxation method, move limits, moving asymptotes and trust region are established. Moreover, numerical examples on the convergence comparison and control of convex approximation methods are illustrated. Finally, some concluding remarks and prospects are given.

Step 3, Construct the subproblem Pk Intermediate variables: (if f0j > 0),

xj 1/xj

Subject to fi (x) ≤ ci x j ≤ xj ≤ x¯j

(1) for i = 1, . . . , m

(2)

for j = 1, . . . , n

(3)

where f0 (x) is a linear or nonlinear objective function which represents any structural characteristic to be minimized with design variables xj . The inequalities (2) are the behavior constraints that impose limitations on structural response quantities (e.g. upper bounds on stresses and displacements under loads). These constraints are usually nonlinear functions, and they might also include linear functions in some situations. 2.1 Convex linearization method (CONLIN) In CONLIN, the linearization process is performed with respect to mixed variables, either direct or reciprocal, independently for each function involved in the above optimization problem. Direct variables are used for positive first derivatives, while reciprocal variables are employed for negative derivatives. Then, a convex, separable subproblem is generated, which can be efficiently solved by using a dual method. An iterative scheme for attacking such problems accords to the following [1, 4]. Step 1, Choose a starting point x0 and let the iteration index k = 0. Step 2, Given an iteration point xk , calculate the first derivatives of objective and constraint functions: ∂fi | k , for i = 0, 1, . . . , m. fik = fi (xk ) and fijk = ∂x j x

(if fij < 0)

xj

Minimize



f0j xj −

 f0j

+

−

 fij

−



xj  fij − +

To describe the convex approximation methods, i.e. CONLIN, MMA, GCMMA, and SQP, the primary problem is mathematically expressed in the following general form [1, 4].

(if f0j < 0)

Explicit subproblem:

2 Review on convex approximation methods

f0 (x)(x ∈ Rn )

1/xj

(if fij > 0),

Subject to

Minimize

607

(4)

xj

fij xj ≤ fij0 +

−



fij

(5)

+

x j ≤ xj ≤ x¯j

(6)

−

Step 4, Solving the subproblem Pk by using a dual method formulation and let the optimal solution of this subproblem be the next iteration point xk+1 and go to step 2. Usually, the CONLIN optimizer converges in less than ten iterations for sizing and shape optimization. Nevertheless, in some cases the convex approximation scheme used in CONLIN might lead to inaccurate approximations, which are either too conservative (in which case slow convergence occurs) or not sufficiently conservative (in which case oscillation can happen) [6]. 2.2 Method of moving asymptotes (MMA) In the MMA, the intermediate linearization variables are modified. Instead of just using direct and reciprocal variables, this method employs the intermediate variables: 1/(Ui − xi ) or 1/(xi − Li ). MMA can be interpreted as a further generalization of the CONLIN. The iterative scheme of MMA is the same as that described in Sect. 2.1 for CONLIN, except that Step 3 is changed as follows [4, 6]. Step 3, Construct subproblem Pk Minimize

n   j =1

Subject to

n   j =1

(k) p0j (k) Uj

− xj

+

(k)

pij (k) Uj

− xj

(k) q0j

(k)

+ r0

(k) x j − Lj (k)

+



qij

(k) x j − Lj

 (k)

+ri ≤ ci

for i = 1, . . . , m (k) max{x j , αj } ≤ xj

for j = 1, . . . , n

(7)

(8) (k) ≤ min{x¯j , βj }

(9)

608

D. Yang, P. Yang

where, (k) pij

=

(k)

qij =

(k) ri

⎧ ⎨ (U (k) − x (k) )2 ∂fi ,

if

⎩ 0,

if

j

j

∂xj

⎧ ⎨ 0,

∂fi ∂xj ∂fi ∂xj

if

⎩ −(x (k) j

(k) ∂fi − Lj )2 ∂x j

= fi (x ) − (k)

n   j =1

if

,

>0 ≤0

∂fi ∂xj ∂fi ∂xj

≥0 μ1 = 3.0, where the previous fixed point called as the repelling fixed point loses its stability (here, |f  (μ, xp )| > 1) and is attracted to the period-2 orbits (here, |(f 2 ) (μ, xp1 )| = |(f 2 ) (μ, xp2 )| < 1); period4 solutions when μ > μ2 = 3.449489, where the previous period-2 points called as the repelling period2 points lose their stability and are attracted to the period-4 orbits; period-8 solutions when μ > μ3 = 3.544090 . . . When μ equals the value of μ∞ =

610

3.569945, the dynamical system gets period-2∞ solution, and the system attains the chaotic state. The parameters μ1 , μ2 , μ3 , . . . , are called the bifurcation points, where the corresponding fixed point can be attracting, repelling, or other [20]. To compare with convex approximation methods, the graphical iteration for period-2 oscillation of Logistic map xi+1 = f (3.3, xi ) = 3.3xi (1 − xi ) with x0 = 0.25 is demonstrated in Fig. 1. The iterative history corresponding to Fig. 1 is shown in Fig. 2.

D. Yang, P. Yang

3.2 Iterative oscillation of convex approximation methods As mentioned before, the pure CONLIN could exhibit periodic solution for some optimization examples. In fact, the partial iterative histories of Example 1 and Example 2 presented in detail in Sect. 5 are shown in Fig. 3 and Fig. 4, respectively. It is found that the corresponding iterative solutions are period-3 and period2 points similar to periodic points in Fig. 2. Essentially, the pure CONLIN constructs a complicated implicit nonlinear map or discrete dynamical system xk+1 = g(xk )

Fig. 1 Graphical iteration for period-2 oscillation of xi+1 = 3.3xi (1 − xi ) with x0 = 0.25

Fig. 2 Iterative history of xi+1 = 3.3xi (1 − xi ) with x0 = 0.25

Fig. 3 Period-3 solutions of pure CONLIN for two-bar truss in Example 1

(16)

When the spectral radius (i.e. the maximum of the absolute eigenvalues) of the Jacobian matrix J (J = ∂gi ∂xj |xk ) of dynamical system (16) at the point xk is less than 1, namely, ρ(J) < 1, the fixed point xp is attracting and the stable convergence solution is obtained. Otherwise, when ρ(J) > 1, the repelling and unstable fixed point xq is attracted to the periodic solutions, as shown in Fig. 3 and Fig. 4. From the comparative analysis of numerical instability in the logistic map and CONLIN, it is seen that

Numerical instabilities and convergence control for convex approximation methods

611

Fig. 4 Period-2 solutions of pure CONLIN for cantilever beam in Example 2

the non-convergence problem of iterative algorithms is basically attributed to the Jacobian matrix and evolutional dynamics of discrete dynamical system from the iterative procedures, rather than only their high nonlinearity. Practically, Logistic map is a simple and low nonlinear system with quadratic function, but it can generate complex non-convergence phenomena. Meanwhile, it is necessary to note that in the past the convergence proof of iterative optimization algorithms was based on such a common assumption that the attracting fixed points exist in the dynamical system constructed from the iterative algorithms. Unfortunately, the dynamical system of optimization algorithms could produce repelling fixed points instead of attracting fixed points, which leads to the nonconvergence of periodic oscillation, as illustrated in CONLIN. As a result, when one attempts to prove the convergence of optimization algorithms, it is necessary to take both the existence and stability of fixed points into account. Moreover, the MMA might generate the complex oscillating solutions instead of the periodic solutions similar to those in the pure CONLIN, because the MMA adopts the moving asymptotes and its iterative scheme is interrupted, different from the iterative procedures in the FORM, PMA, Newton method and pure CONLIN. Similarly, the full stress method, SLP and SQP might yield the non-periodic oscillation of iterative solutions due to the action of the move limits and other approximation schemes. Nevertheless, a unified method (i.e. stability transformation method) based on the chaos control principle can be established to overcome the numerical instability of these iterative algorithms in the next section.

4 Convergence control: from the relaxation method and move limits to stability transformation method Before introducing the stability transformation method, the relaxation method, move limits, moving asymptotes and trust region management for improving the convergence of optimization algorithms are described. Moreover, the relationship among these methods and STM is explored. 4.1 Relaxation method Relaxation method was early applied in the convergence control for solving the linear algebraic equations. Based on the Gauss-Seidel iterative formulation, in order to realize the convergence control the relaxation method introduces a relaxation factorω, and employs the following iterative formula [37]  (k) (k+1) = g xi xi   i−1 n   (k+1) (k) aij xj − aij xj /aii = bi − j =1

j =i+1

(i = 1, 2, . . . , n) (k+1)

xi

(17)

(k)

= (1 − ω)xi

  i−1 n   (k+1) (k) + ω bi − aij xj − aij xj /aii j =1

(i = 1, 2, . . . , n)

j =i+1

(18)

where, (17) is the iterative formulation of GaussSeidel method for solving the linear algebraic equa-

612

D. Yang, P. Yang

tions Ax = b, and x = (x1 , x2 , . . . , xn )T , aij is the element of matrix A, b = (b1 , b2 , . . . , bn )T . Equation (18) represents the relaxation method for Gauss-Seidel iterative method, which is called the successive overrelaxation method (SOR method). Generally, the relaxation factor is taken as 0 < ω < 2. If ω = 1, (18) becomes the Gauss-Seidel iterative method; if 1 < ω < 2, (18) is called the over-relaxation method, where the convergence rate is sped up; if 0 < ω < 1, (18) is called the under-relaxation method [37]. Moreover, to improve the convergence of optimization algorithms, the relaxation method is also applied. For instance, in references [25, 26], the relaxation method is used to enhance the iterative convergence of optimality criterion method, and formulated as follows xk+1 = x∗k = g(xk ) xk+1 = αxk + (1 − α)g(xk ),

(19) 0 1 in (18) can speed up the slow convergence during iterative process. Moreover, when (23) has attracting fixed point, increasing the factor λ in proper range still remains the stability of the fixed point. Hence, the factor λ > 1 in (23) of STM can also accelerate slow convergence of convex approximation methods. The procedure of convergence control for optimization methods using STM formulation with a little heuristic rule is outlined as follows. (a) If the iterative oscillation of convex approxima(k) (k−1) tion methods emerges (i.e. if the signs of xj − xj (k−1)

(k−2)

and xj − xj are opposite, similar to the condition in (12)), then 0 < λ < 1 and C is one of involutory matrices in (23). Generally, 0.1 ≤ λ ≤ 0.5, and C is taken as unit matrix. Actually, λ can be changed rather

than be fixed during the iteration, and C can be set as another matrix of involutory matrices, if necessary. (b) If the objective function decreases monotonously and the iterative convergence is too slow, then 1 < λ < 2 and C is one of involutory matrices in (23). Generally speaking, 1.1 ≤ λ ≤ 1.5, and C is taken as unit matrix. (c) Otherwise, λ = 1 and C = I in (23), the original iterative algorithms are recovered. Therefore, the new convergence control scheme of STM for iterative algorithms not only utilizes the concept of the stability transform of dynamical system, but also absorbs the experience of the step-size control of the relaxation method. However, it should be also pointed out that this method has some limitations: (1) only when the original convex approximation methods present the oscillating, chaotic solution and slow convergence, can the STM formulation be applied. Otherwise, the STM scheme may not work well; (2) the choice of involutory matrix C and factor λ of STM formulation is heuristic, and the convergence proof of developed algorithm becomes very complicated owing to the incorporation of matrix C and factor λ. This means that it can probably not be used for the development of robust, globally convergent algorithms at this stage.

5 Numerical examples for convergence comparison and control of convex approximation methods This section illustrates the numerical results of several highly nonlinear examples, and stresses on the convergence comparison and control of convex approximation methods, i.e. CONLIN, MMA, GCMMA and SQP. The program codes provided by Svanberg are used herein, either MMA or GCMMA. For the CON(k) 10 LIN, L(k) j = 0 and Uj = 10 are set as the asymptotes in the MMA codes. What is more, the SQP codes in DOT program [34] are utilized to optimize the following examples. When the STM is applied for the convergence control of convex approximation methods, the iterative formula (28) is adopted, instead of xk+1 = x∗k (the solution of the kth iteration) in the program codes.  xk+1 = xk + λC x∗k − xk

(0 < λ < 2)

(28)

Numerical instabilities and convergence control for convex approximation methods

Sometimes, the other forms of involutary matrixes C can be taken to achieve the iterative convergence [33]. Here, the unit matrix of C is set, which is used most frequently. The convergent precision in the whole calculation is taken as 10−5 . Namely, convergence is (k+1) (k) − xj | ≤ ε = deemed to have occurred if maxj |xj 10−5 . The following four benchmark examples are extracted from the references [4, 9, 11], in which the first two examples of structural optimization seem to be simple, but inherent complexity such as the nonconvexity and high nonlinearity are essential to examine the effectiveness, efficiency and stability of convex approximation methods. The two examples have been frequently revisited in the literature. The later two mathematical examples are used further to test the convergent properties of different approximation methods, which are also highly nonlinear and quite difficult to solve. Example 1 Configuration optimization of two-bar truss. Consider the configuration optimization of a 2-bar truss structure shown in Fig. 5 [4, 12]. It contains two design variables of different type: x1 is defined as the cross-sectional areas of the bars; x2 , the geometrical design variable, is defined as the half of the distance between the two bottom nodes. The objective function is to minimize the weight of the truss structure. There is one force acting at node 3: |F | = 200 kN (8Fx = Fy ). Stress constraints are imposed in each bar. This problem can be analytically stated as  Minimize f (x) = x1 1 + x22

615

 Subject to 0.124 1 + x22 (8/x1 + 1/(x1 x2 )) ≤ 1.0  0.124 1 + x22 (8/x1 − 1/(x1 x2 )) ≤ 1.0 0.2 ≤ x1 ≤ 4.0,

0.1 ≤ x2 ≤ 1.6

In the MMA, s = 0 in (11), s = 0.5 in (12), s = 0.75 in (13) according to [4]. In the GCMMA, s = 0.5 in (11), s = 0.7 in (12), s = 0.83 (1/1.2) in (13) according to [10]. As a starting point, let x1 = 1.5 cm2 , x2 = 0.5 m, the results are shown in Table 1. For the pure CONLIN method, the iterative process is period-3 oscillating and cycling presented in Fig. 3 in Sect. 3.2. Then the STM for CONLIN is implemented. As expected, the correct and stable convergence solutions are captured with difference factors, listed in column 6–8 in Table 1. The optimal solution is f (x∗ ) = 1.5087 and x∗ = (1.4114, 0.3772). When the factor λ increases, the iterative number N becomes small. For instance, λ = 0.1, N = 69; λ = 0.5, N = 11. MMA, GCMMA and SQP need 9, 5 and 6 iterations to converge, respectively. Three algorithms for this case converge fast for less than 10 iterations. Whereas, the STM for CONLIN overcome the trouble of iterative cycling of pure CONLIN, and also converges quickly when λ = 0.5. Example 2 Weight optimization of a cantilever beam. Consider the weight optimization of a cantilever beam with square cross-section as shown in Fig. 6 [4, 12]. The beam is rigidly supported at node 1, and there is a given vertical force acting at node 6. The design variables are the heights xj of the different beam elements, and the thickness is held fixed. The bound constraints are set as 0.01 ≤ xj ≤ 100. The objective function is to minimize the weight of the considered structure. This problem can be expressed analytically as Minimize

f (x) = 0.0624(x1 + x2 + x3 + x4 + x5 )

Subject to 61/x13 + 37/x23 + 19/x33 + 7/x43 + 1/x53 ≤ 1.0

Fig. 5 Two-bar truss

In the MMA, Lkj = txjk and Ujk = xjk /t (here t = 2/3), as determined in [4]. In the GCMMA, s = 0.5 in (11), s = 0.7 in (12), s = 0.83 in (13) according to [10]. As a starting point, let xj0 = 5.0 for all j ,

616

D. Yang, P. Yang

Table 1 Results of configuration optimization for two-bar truss Iterative

CONLIN

MMA

GCMMA

SQP

number N

STM for CONLIN λ = 0.1

λ = 0.3

λ = 0.5

0

1.6771

1.6771

1.6771

1.6771

1.6771

1.6771

1.6771

1

1.3933

1.3954

1.5227

1.5221

1.6386

1.5682

1.5067

2

1.5356

1.1566

1.5088

1.5030

1.6193

1.5502

1.5019

3

1.5009

1.3190

1.5087

1.5094

1.6054

1.5377

1.5053

4

1.3882

1.5172

1.5087

1.5088

1.5943

1.5290

1.5070

5

1.5344

1.5084

1.5087

1.5085

1.5850

1.5229

1.5078

6

1.5009

1.5087

1.5085

1.5769

1.5186

1.5082

7

1.3875

1.5087

1.5698

1.5156

1.5084

8

1.5342

1.5087

1.5636

1.5136

1.5085

9

1.5009

1.5087

10

1.3873

1.5580

1.5121

1.5086

1.5530

1.5111

1.5086 1.5086

11

1.5342

1.5486

1.5103

12

1.5009

1.5446

1.5098

...

...

...

...

23

...

1.5199

1.5087

...

...

...

68

1.5342

...

69

1.5009

1.5087

...

...

Fig. 6 Cantilever beam

the results are listed in Table 2. For the pure CONLIN method, the iterative process is period-2 oscillating shown in Fig. 4 in Sect. 3.2. Then the STM for CONLIN is implemented. Again, the correct and stable convergence solutions are obtained, listed in column 6–8. The optimal solution is f (x∗ ) = 1.3400 and x∗ = (6.0007, 5.3067, 4.5026, 3.5086, 2.1535). When the factor λ increases, the iterative number N becomes small. For this case, MMA needs 23 iterations to converge. GCMMA and SQP converge fast for not greater than 10 iterations. The STM for CONLIN stabilizes the iterative cycling of pure CONLIN, and converges

quickly (N = 13) if λ = 0.5. On the other hand, for the slow convergence of MMA, STM can speed up the convergence with 1.1 ≤ λ ≤ 1.5, as displayed in Table 3. Moreover, the bigger the factor λ is taken, the less iterative number is needed. Example 3 Maximizing the area of a polytope with fixed circumference. Consider the optimization problem of maximizing the area of a closed polytope with 12 corners shown in Fig. 7, subjected to the constraint that the circumfer-

Numerical instabilities and convergence control for convex approximation methods

617

Table 2 Convergence comparison and control of weight optimization for cantilever beam Iterative

CONLIN

MMA

GCMMA

SQP

STM for CONLIN

t = 2/3

number N

λ = 0.1

λ = 0.3

λ = 0.5

0

1.5600

1.5600

1.5600

1.5600

1.5600

1.5600

1.5600

1

1.1979

1.4485

1.3698

1.3498

1.5238

1.4514

1.3789

2

1.0234

1.3860

1.3452

1.3100

1.4940

1.3994

1.3487

3

1.0617

1.3582

1.3403

1.2993

1.4695

1.3744

1.3440

4

1.0361

1.3469

1.3400

1.3241

1.4494

1.3618

1.3420

5

1.0931

1.3425

1.3400

1.3401

1.4329

1.3547

1.3410

6

1.0556

1.3409

1.3400

1.3398

1.4192

1.3501

1.3405

7

1.1060

1.3403

1.3400

1.3398

1.4080

1.3470

1.3402

8

1.0649

1.3401

1.3400

1.3986

1.3449

1.3401

9

1.1128

1.3400

1.3400

1.3909

1.3434

1.3400

10

1.0708

1.3400

1.3400

1.3844

1.3424

1.3400

...

...

...

...

...

... 1.3400

13

1.1217

...

1.3703

1.3408

...

...

...

...

...

23

...

1.3400

...

...

...

...

...

... 1.3400

27

1.1466

1.3465

...

1.1083

...

...

...

1.3400

81

1.2100

...

...

Table 3 Speeding up the convergence of weight optimization for cantilever beam MMA

STM for MMA t = 2/3

t = 2/3

λ = 1.1

λ = 1.2

λ = 1.3

λ = 1.4

λ = 1.5

Iterative number N

23

20

18

16

15

14

Optimal result

1.3400

1.3400

1.3400

1.3400

1.3400

1.3400

ence of the polytope must be not greater than a given constant 60 [9]. 1 rj rj +1 sin θj 2 10

Minimize

−

j =1

Subject to r1 + r11 +

10   2 1/2 rj + rj2+1 − 2rj rj +1 cos θj ≤ 60

Fig. 7 Polytope with 12 corners

j =1

1 ≤ rj ≤ 30,

j = 1, . . . , 11

1 ≤ θj ≤ 45,

j = 1, . . . , 10

According to [9], in the MMA, s = 0 in (11), s = 0.5 in (12), s = 0.75 in (13). In the GCMMA, s = 0.5 in (11), s = 0.7 in (12), s = 0.83 in (13). The starting

618

D. Yang, P. Yang (0)

(0)

point is chosen as rj = 11 and θj = 18 for all j . The analytical optimum solution of the√problem is the regular polytope whose area is 75(2 + 3) ≈ 279.904, and rj = 5 sin(15j )/ sin(15). The numerical results of convergence comparison and control for MMA family and SQP are shown in Table 4. CONLIN presents the remarkable oscillation of solutions. MMA converges to an incorrect solution (227.131) after 1327 iterations with significant oscillation, which demonstrates that MMA is failed in this example. GCMMA converges monotonously and slowly, and needs 281 iterations to obtain the optimal solution. It is seen that when the polytope is regular, its area is maximal. SQP obtains the approximately optimal solution after 823 iterations. It should be noted that the SQP program in DOT does not employ the move limits [34]. Then, we attempt to control the iterative oscillation using STM for CONLIN and MMA. The STM for CONLIN is failed in this problem, which still presents the iterative oscillation. The STM for MMA has also failed due to the premature convergence. Additionally, it is observed that SQP shows the iterative oscillation within the first 10 iterations and converges slowly. Thus, the STM for SQP with different factors (0.1 ≤ λ ≤ 0.5) is performed to control the oscillation, and the solutions with faster convergence are shown in Table 4 (λ = 0.1, N = 46; λ = 0.3, N = 34; λ = 0.5, N = 27). It can be seen that optimal solutions are acquired by the STM for SQP with λ = 0.2 and 0.3. For this example with non-monotonous and complex objective and constraint functions, MMA and CONLIN as the monotonous approximation methods are not appropriate because they cannot approximate the objective and constraints functions with enough exactness, even combining with the STM formulation. But, GCMMA and SQP as the non-monotonous approximation methods can obtain the optimal solution. Example 4 Optimization problem with 30 variables, 41 nonlinear constraints. Consider an optimization problem as follows with 41 highly nonlinear and complicated constraints [11]. Minimize

l  (xi cos αi + xl+i sin αi − 0.1x2l+i ) i=1

Subject to

l   2 2 xi + xl+i ≤l i=1

− 2 ≤ gi (x) + gi (x)7 ≤ 2,

i = 1, . . . , l

− 2 ≤ hi (x) + hi (x) ≤ 2,

i = 1, . . . , l

− 2 ≤ xj ≤ 2,

j = 1, . . . , 3l

7

where 2 −1 x 2 + xl+i (3i − 2l)π , gi (x) = i 6l δ x2l+i − 2xi xl+i hi (x) = δ

αi =

l = 10,

i = 1, . . . , l,

x = (x1 , . . . , x3l )T ,

δ = 0.1

In the MMA, s = 0.1 in (11), s = 0.95 in (12) and s = 1.0 in (13) according to [11]. In the GCMMA, s = 0.5 in (11), s = 0.7 in (12) and s = 0.83 in (13) according to [11]. The starting point is chosen (0) (0) (0) π π ), xl+i = sin(αi + 12 ), x2l+i = as xi = cos(αi + 12 sin(2αi + π6 ), for i = 1, . . . , l. The results are demonstrated in Table 5. It should be noted that the parameter s in the MMA is set based on the recommendation in [11]. If we take the same parameter s as that in previous several examples, i.e. s = 0 in (11), s = 0.5 in (12), s = 0.75 in (13), then the MMA does not work. It means that the parameter s in the MMA is quite heuristic and need to be set by trial and error depending on the user’s experience. For the CONLIN the incorrect solution is obtained after 2418 iterations. MMA and GCMMA attain the optimal solution (−10.0231) after 142 and 57 iterations, separately. SQP obtains the infeasible solution after 7 iterations. Then, for CONLIN and SQP, we try to obtain the correct solution using STM formulation, but fail. It is because the convex linear and quadratic approximation cannot build enough the curvature for the special mathematical example with complex constraint functions. To accelerate the convergence of MMA with monotonously decreasing objective function, the STM for MMA is conducted with 1.1 ≤ λ ≤ 1.5, and the results are shown in Table 5 (λ = 1.1, N = 137; λ = 1.3, N = 124; λ = 1.5, N = 120). It is also observed that the larger the factor λ is taken, the less iterative number is needed. GCMMA is more suitable and efficient for this case. To sum up, pure CONLIN, MMA and SQP could generate cycling and oscillating solutions for some nonlinear optimization problems, which is essentially

Numerical instabilities and convergence control for convex approximation methods

619

Table 4 Convergence comparison and control for Example 3 Iterative

CONLIN

MMA

GCMMA

SQP

STM for SQP λ = 0.1

number N

λ = 0.2

λ = 0.3 −186.953

0

−186.953

−186.953

−186.953

−186.953

−186.953

−186.953

1

−277.62

−228.90

−235.212

−321.984

−198.331

−221.065

241.154

2

−372.74

−239.42

−248.565

−420.227

−207.280

−241.278

−268.386

3

−647.75

−295.54

−256.607

−454.060

−214.223

−256.070

−286.695

4

−353.70

−290.68

−261.691

−458.754

−219.740

−265.266

−294.600

5

−418.12

−227.63

−264.903

−459.773

−226.335

−271.556

−297.054

6

−332.09

−237.87

−267.003

−464.161

−230.410

−274.662

−298.885

7

−423.78

−353.57

−270.386

−464.177

−235.779

−275.774

−299.752

−137.86

−272.307

−462.896

−238.590

−278.188

−299.779

−274.112

−462.287

−243.335

−278.602

−293.812

−275.742

−461.082

−246.552

−279.611

−289.603

...

...

...

...

... −279.873

8

−316.49

9

−348.64

10

−276.87

...

...

−9.6971 −29.490 ...

27

...

...

...

...

...

...

...

...

...

...

...

...

... −279.907

34

...

...

...

...

...

...

...

...

...

...

... −272.652

46

...

...

...

...

...

...

...

...

...

281

...

...

−279.904

...

...

...

...

... −279.897

823

...

...

...

...

...

1327

...

−227.131

...

...

related to the evolutional dynamics of iterative dynamical system. The STM formulation combing with CONLIN, MMA and SQP cannot only stabilize the iterative oscillation when the factor and involutory matrix are generally taken as 0.1 ≤ λ ≤ 0.5 and C = I in STM scheme, but also speed up the slow convergence of optimization methods with the monotonously decreasing objective function when 1.1 ≤ λ ≤ 1.5 and C = I. But more complicated examples in structural optimization need to be investigated to verify this statement. From the four numerical examples in this section, it is observed that GCMMA is suitable and efficient, if the proper heuristic rules for choosing the moving asymptotes are taken. Although the STM formulation involves the selection of the factor and involutory matrix, this selection is rational and easily understandable, because the STM is based on the deep understanding for the iterative behavior of optimiza-

tion methods from the viewpoint of chaotic dynamics. Moreover, it is suggested to apply the STM formulation to replace the move limits and trust region management in optimization algorithms under some conditions.

6 Conclusions To understand comprehensively the convergent difficulties of iterative procedures in structural optimization, besides the perspective of mathematical programming, this paper provides an additional perspective of nonlinear dynamics. This paper firstly analyzes the essential causes of iterative oscillation of convex approximation methods from the viewpoint of chaotic dynamics. From the comparative analysis of numerical instability in

620

D. Yang, P. Yang

Table 5 Convergence comparison and control for Example 4 Iteration

CONLIN

MMA

GCMMA

SQP

number N

STM for MMA λ = 1.1

λ = 1.3

λ = 1.5

0

9.5593

9.5593

9.5593

9.5593

9.5593

9.5593

9.5593

1

6.6971

8.4523

9.2261

9.2183

8.3417

8.1203

7.8989

2

6.3004

8.2756

9.0655

8.6075

8.2085

8.1251

8.0404

3

6.5282

7.9605

9.0061

8.0934

7.9598

8.1005

7.9770

4

6.5147

7.3826

8.9597

7.5766

7.4425

7.8657

7.8768

5

6.4692

6.5949

8.8043

6.6629

6.6082

7.2988

7.6167

6

6.4055

5.7395

8.3168

6.4320

5.6676

6.4857

6.8555

7

6.2601

4.9305

6.9495

6.4320

4.8107

5.6352

5.8137

8

6.1424

4.1397

6.6707

(Infeasible)

4.0341

4.7047

4.9437

9

5.9223

3.4348

6.0930

3.3215

3.9136

4.2601

10

5.7852

2.7335

5.6168

2.7020

3.0312

3.6081

...

...

...

...

...

...

...

57

...

−10.0133

−10.0231

...

...

...

...

...

...

...

...

...

120

...

...

...

...

−10.0231

...

...

...

...

...

124

...

...

...

−10.0231

...

...

...

...

137

...

...

−10.0231

...

...

...

142

...

−10.0231

... 2418

... −1.9209

logistic map and CONLIN, it is seen that the nonconvergence problem of iterative algorithms is basically attributed to the Jacobian matrix and evolutional dynamics of discrete dynamical system from the iterative procedures, rather than only their high nonlinearity. Then, in order to control the iterative oscillation arising from the convex approximation methods, the STM formulation based on the principle of chaos control is proposed. The relationships among STM and relaxation method, move limits, moving asymptotes and trust region management for improving the convergence of optimization algorithms are explored. The connection between convergence control of optimization algorithms and chaotic dynamics is established. Furthermore, it is expected that the STM scheme combing with CONLIN, MMA and SQP cannot only stabilize the iterative oscillation when the factor λ and involutory matrix C are generally taken as

0.1 ≤ λ ≤ 0.5 and C = I in STM formulation, but also accelerate the slow convergence of optimization methods with the monotonously decreasing objective function when 1.1 ≤ λ ≤ 1.5 and C = I. In fact, numerical tests of four highly nonlinear optimization examples illustrate that the STM stabilizes the oscillating solutions for CONLIN and accelerates slow convergence for MMA and SQP. Owing to the complexity of some optimization problems and the intrinsic drawback of CONLIN, MMA and SQP, the STM formulation sometimes cannot obtain the correct optimal solutions. GCMMA is effective and efficient for solving some problems in structural optimization. However, determining the heuristic rules for choosing its appropriate moving asymptotes might be cumbersome. Although the STM formulation involves the selection of the factor and involutory matrix, this selection is rational and based on the deep understanding of iterative

Numerical instabilities and convergence control for convex approximation methods

behavior of optimization algorithms under the framework of chaotic dynamics of discrete dynamical system. Finally, it is pointed out that the idea of convergence control of STM for iterative algorithms derived from the chaotic dynamics theory of discrete dynamical systems throws some light in both the comprehension of numerical instabilities and the development of optimization algorithms. Convex approximation methods utilize the characteristics of structural optimization problems. Consequently, it is more efficient and effective than the general mathematical programming methods which do not consider the structural behavior. It is worthy to develop new optimization methods combining the convex approximation concept with convergence control strategy of STM with certain heuristic rules. If necessary, the other method of chaos control can be introduced for convergence control of iterative algorithms. Acknowledgements The supports of the National Natural Science Foundation of China (Grant nos. 90815023, 50978047), and the National Basic Research Program of China (Grant nos. 2006CB601205, 2010CB832703) are much appreciated. The MMA and GCMMA program codes were provided by Professor Krister Svanberg of KTH in Stockholm. We thank Professor Svanberg for allowing us to use his program. Also, we greatly appreciate the two anonymous reviewers for their insightful suggestions and comments on the early version of this paper.

References 1. Fleury, C., Braibant, V.: Structural optimization: a new dual method using mixed variables. Int. J. Numer. Methods Eng. 23, 409–428 (1986) 2. Nguyen, V.H., Strodiot, J.J., Fleury, C.: A mathematical convergence analysis for the convex linearization method for engineering design optimization. Eng. Optim. 11, 195– 216 (1987) 3. Jonsson, O., Larsson, T.: A note on step-size restrictions in approximation procedure for structural optimization. Comput. Struct. 37(3), 259–263 (1990) 4. Svanberg, K.: The method of moving asymptotes—a new method for structural optimization. Int. J. Numer. Methods Eng. 24, 359–373 (1987) 5. Fleury, C.: CONLIN: an efficient dual optimizer based on convex approximation concepts. Struct. Optim. 1, 81–89 (1989) 6. Fleury, C.: First and second order convex approximation strategies in structural optimization. Struct. Optim. 1(1), 3– 10 (1989) 7. Bletzinger, K.U.: Extended method of moving asymptotes based on second-order information. Struct. Optim. 5, 175– 183 (1993)

621

8. Zillober, C.: A globally convergent version of the method of moving asymptotes. Struct. Optim. 6, 166–174 (1993) 9. Svanberg, K.: A globally convergent version of MMA without line-search. In: Proceedings of the First World Congress on Structural and Multidisciplinary Optimization, Goslar, Germany, May 28–June 2 (1995) 10. Svanberg, K.: A class of globally convergent optimization methods based on conservative convex separable approximations. SIAM J. Optim. 12(2), 555–573 (2002) 11. Svanberg, K.: On a globally convergent version of MMA. In: Proceedings of the 7th World Congress on Structural and Multidisciplinary Optimization, Seoul, Korea, May 21–25 (2007) 12. Zhang, W.H., Fleury, C.: A modification of convex approximation methods for structural optimization. Comput. Struct. 64(1–4), 89–95 (1997) 13. Bruyneel, M., Duysinx, P., Fleury, C.: A family of MMA approximations for structural optimization. Struct. Multidiscip. Optim. 24, 263–276 (2002) 14. Zillober, C.: Global convergence of a nonlinear programming method using convex approximations. Numer. Algorithms 27, 265–289 (2001) 15. Zillober, C.: SCPIP- an efficient software tool for the solution of structural optimization problems. Struct. Multidiscip. Optim. 24, 362–371 (2002) 16. Groenwold, A.A., Etman, L.E.P., Snyman, J.A., Rooda, J.E.: Incomplete series expansion for function approximation. Struct. Multidiscip. Optim. 34, 21–40 (2007) 17. Groenwold, A.A., Etman, L.E.P.: Sequential approximate optimization using dual subproblems based on incomplete series expansions. Struct. Multidiscip. Optim. 36, 547–570 (2008) 18. Zou, K.T., Chen, L.P., Zhang, Y.Q., Yang, J.Z.: Study of key algorithms in topology optimization. Int. J. Adv. Manuf. Technol. 32, 787–796 (2007) 19. MaCauley, J.L.: Chaos, Dynamics and Fractals. Cambridge University Press, Cambridge (1993) 20. Robinson, R.C.: An Introduction to Dynamical System: Continuous and Discrete. Pearson Education, New York (2004) 21. Yang, D.X., Li, G., Cheng, G.D.: Convergence analysis of first order reliability method using chaos theory. Comput. Struct. 84(8–9), 563–571 (2006) 22. Schmelcher, P., Diakonos, F.K.: Detecting unstable periodic orbits of chaotic dynamical systems. Phys. Rev. Lett. 78(25), 4733–4736 (1997) 23. Kapitaniak, T.: Controlling Chaos: Theoretical and Practical Methods in Nonlinear Dynamics. Academic Press, London (1996) 24. Pingel, D., Schmelcher, P., Diakonos, F.K.: Stability transformation: a tool to solve nonlinear problems. Phys. Rep. 400, 67–148 (2004) 25. Lin, J.H., Che, W.Y., Yu, Y.S.: Structural optimization on geometrical configuration and element sizing with statical and dynamical constraints. Comput. Struct. 15(5), 507–515 (1982) 26. Qian, L.X.: Optimum Design of Engineering Structures. China Water Power Press, Beijing (1983) 27. Wujek, B.A., Renaud, J.E.: New adaptive move-limit management strategy for approximate optimization, Part 1. AIAA J. 36(10), 1911–1921 (1998)

622 28. Bendsoe, M.P., Sigmund, O.: Topology Optimization: Theory, Methods and Applications. Springer, Berlin (2003) 29. Lamberti, L., Pattalettere, C.: Move limits definition in structural optimization with sequential linear programming. Part I: Optimization algorithms. Comput. Struct. 81, 197–213 (2003) 30. Alexandrov, N.M., Dennis, J.E., Lewis, R.M., Torczon, V.: A trust-region framework for managing the use of approximation models in optimization. Struct. Optim. 15, 16–23 (1998) 31. Rodriguez, J., Renaud, J.E., Wujek, B.A., Tappeta, R.V.: Trust region model management in multidisciplinary design optimization. J. Comput. Appl. Math. 124, 139–154 (2000) 32. Yang, D.X.: Chaos control of FORM iterative algorithm for structural reliability analysis. Chin. J. Theor. Appl. Mech. 39(5), 647–654 (2007)

D. Yang, P. Yang 33. Yang, D.X., Yi, P.: Chaos control of performance measure approach for evaluation of probabilistic constraints. Struct. Multidiscipl. Optim. 38(1), 83–92 (2009) 34. VR&D.: DOT (Design Optimization Tools) Users Manual Version 4.20, Vanderplaats Research & Development Inc., Colorado, USA (1995) 35. Bazaraa, M.S., Schetty, C.M.: Nonlinear Programming: Theory and Application. Wiley, New York (1979) 36. Wang, L.P., Grandhi, R.V.: Safety index calculation using intervening variables for structural reliability analysis. Comput. Struct. 59(6), 1139–1148 (1996) 37. Young, D.M.: Iterative Solution of Large Linear System. Academic Press, New York (1971) 38. Nocedal, J., Wright, S.J.: Numerical Optimization. Springer, New York (1999)