WCCM V Fifth World Congress on Computational Mechanics July 7–12, 2002, Vienna, Austria Eds.: H.A. Mang, F.G. Rammerstorfer, J. Eberhardsteiner
Evolution Strategies for Optimisation in Engineering Applications Thomas B¨ack NuTech Solutions GmbH Martin Schmeisser Weg 15, D-44227 Dortmund, Germany e-mail:
[email protected]
Michael Emmerich Center of Applied Systems Analysis Informatik Centrum Dortmund, Joseph Fraunhofer Strasse 20, D-44227 Dortmund, Germany e–mail:
[email protected]
Key words: Engineering Optimisation, Evolution Strategies, Computer Experiments, Computational Fluid Dynamics, Inverse Design Abstract Evolution Strategies are global, direct optimisation algorithms, which have proven their efficiency in engineering applications for a large number of industrial applications. In this paper, the basic algorithm and some of its recent variants for design optimisation are presented. In addition, we give an overview of applications and a comparison to other optimisation algorithms. As an example we present results of an optimisation study on a test case from engineering - the design of an airfoil shape for a given target pressure distribution with low-cost evolution strategies using a CFD evaluator based on Navier Stokes Equations.
Th. B¨ack, M. Emmerich
1 Introduction In automotive engineering and aeronautics, as well as in many other disciplines, there is a growing interest in the use of numerical computer models in the design process. Such models can be used to predict the quality of solutions without cost expensive physical experimentation. Typically the involved simulation codes are very time consuming, especially in the field of structural mechanics and computational fluid dynamics where FEM/FDM methods are employed to evaluate particular solutions. Based on these analysis codes, design optimisation tools can search for solutions with desired characteristics within the design space. This process, termed inverse design, is often based on numerical optimisation procedures, which explore the design space by automatic and sequential computer experimentation. There are many difficulties one has to face when optimising with realistic computer models. First of all there is typically no explicit mathematical model that can be exploited and only little information about the often nonlinear and multimodal (occurence of multiple local optima) characteristics of the response surface. Furthermore, one has to deal with time consuming evaluations, which may take from several minutes up to many hours. The response surface is often non-smooth and rugged, because of switches in the simulator code or coarse grained discretisations that lead to numerical noise. In practical optimisation, one also has to deal with failed simulations, caused by ill parameterisations and numerical instabilities. The occurence of these difficulties makes the use of classical numerical optimisation algorithms, like gradient-based methods and pattern search often difficult, because they have been designed for differentiable objective functions and/or local optimisation, where they perform very well. On multimodal and discontinuous objective functions their results depends heavily on the starting point. Evolution Strategies (ES) are an alternative approach for the solution of simulator-based global optimisation problems. Modern ES work with a population (multi-set) of search points at a time. Search operators like mutation, recombination and selection are used to generate a succession of experimental designs with the aim to approach the global optimum of the response function (e.g. a simulator-based evaluation).
2 Evolution Strategies ES are very robust global optimisation algorithms, which have proven their efficiency on a large number of industrial design problems, e.g.:
Mixed-Integer Optimisation of Optical Multilayer Filters [6, 3]
Design of stator blades in transonic compressor cascades [10]
Multipoint airfoil optimisation with CFD-Simulators based on Navier Stokes Equations [9]
Optimisation of control variables of a casting process using FEM/FDM simulators [7] Optimisation of temperature control parameters in 3D-Thermal design problems [5]
The main features of Evolution Strategies are:
their robustness to discontinuities (e.g. caused by (penalised) simulator failures),
2
WCCM V, July 7–12, 2002, Vienna, Austria
initialize population
evaluate select mating partners (terminate)
loop
recombinate
select mutate
evaluate Figure 1: Schematic view of an iteration of a simple Evolution Strategy.
their strength in global exploration of multimodal and rugged response surfaces,
their capability of self adaptation of the search operators, which makes them capable to learn features of the search space and to switch gradually from global exploration to local optimisation,
their general-search principles, which makes them easy to adapt to the application problem by adding domain-specific knowledge, the population-based search concept, which allows for flexible parallel computation in multiprocessor computing environments.
All these features make ES very attractive for tackling difficult problems in direct simulator-based optimisation. On the other hand, ES are known to require a relatively high number of computer experiments in order to approximate the optimum precisely. This shortcoming can be compensated, by exploiting the inherent parallelism of the evolutionary search process and use in multi-processor environments. Another very promising method to achieve a speed-up is to use less accurate models, which are evaluated fast, during the search process and replace them by precise evaluations if neccesary. Quite recently, it has been shown that a significant speed-up of ES can be achieved by using metamodels as fast pre-evaluators during the search, which are interpolation models generated from all points in the search space that have already been evaluated. In this paper, we begin with an introduction of the working principles of Evolution Strategies (Section 2). Then, in Section 3, we introduce techniques for making them applicable for design optimisation with time-expensive evaluations and in Section 4, we solve a challenging problem from aeronautics with ES the re-design of an multipoint airfoil shape.
3
Th. B¨ack, M. Emmerich
equal probability to place an offspring
x*
Figure 2: Illustration of an ES population. Each individual consists not only of variables that determine its position in the search space, but also of parameters determining the shape of the random distribution used in the mutation, which is indicated by the covariance ellipsoids, which centre represent the search points. These parameters are adapted during the search in order to fit to the local topology of the objective function, which is indicated by contour lines here.
The history of Evolution Strategies (ES) dates back to the early sixties, where they have been used in automatic optimisation with experimentation on physical prototypes. In these early experiments a simple (1+1)-ES has been employed, which implemented a simple sequential mutation (random variation) and selection scheme [12]. Since these years ES have been steadily improved and a sound standing convergence theory has been elaborated [2], including convergence proofs for different global and local optimisation problems. Furthermore, the convergence speed and reliability has been subject to large statistical benchmark tests that compare them with traditional optimisation strategies on a variety of target functions [14, 1]. ES matured to established numerical optimisation algorithms, which are now frequently used in simulator-based optimisation (cf. [6, 3, 10, 9, 5, 7, 1]). Modern ES variants work with a multi-set (population) of individuals (candidate solutions + strategy parameters), which are modified by the subsequent application of variation (mutation and recombination) and selection operators with the aim to find good solutions in the search space. ES have been elaborated especially for continuous parameter optimisation problems - although there exists various specialised variants for more complex and constrained search spaces, like e.g. sequences, discrete structures or parameterised networks. Nevertheless, in this paper, we will focus on continuous optimisation problems (equation 1) like they often occur in design optimisation.
" ! $ #&% ')(+* , -./ $ ! $ #&% '0(+* , In an ES for continuous optimisation an individual &12&1435
(1)
is a tuple of continuous variables representing the candidate solution and related strategy variables (e.g. variances and covariances for the mutation distribution). A population of such individuals is illustrated in Figure 2. The
4
WCCM V, July 7–12, 2002, Vienna, Austria
6 879;:&- *@?
6
:
:
6
6
:
:
: = >9ACB2 7 = >)ACB2
>$DFEG>ACB
6
parents
end while Figure 3: Main loop of a
6 879;:
