Multi-Objective Optimization: Hybridization of an Evolutionary Algorithm with Artificial Neural Networks for fast Convergence. A. Gaspar-Cunha1*, Armando S. Vieira2, Carlos M. Fonseca3 1
IPC – Institute for Polymers and Composits, Dept. of Polymer Engineering, University of Minho, Campus de Azurém, 4800-058 Guimarães, Portugal 2
[email protected]
Dept. of Physics, Instituto Superior de Engenharia do Porto, R. S. Tome, 4200 Porto, Portugal 3
[email protected]
CSI- Centre for Intelligent Systems, Faculty of Science and Technology, University of Algarve, Faro, Portugal
[email protected]
1 Introduction Multi-Objective Evolutionary Algorithms (MOEAs) are efficient methods to evaluate the Pareto-optimal set in difficult multi-objective optimization problems, such as linear programming and combinatorial optimization [1-8]. However, to reach an acceptable solution evolutionary algorithms require a large number of evaluations of the objective functions [9, 10]. In this work we developed an algorithm using Artificial Neural Networks (ANN) [11] to reduce the number of exact function evaluations for multi-objective optimization problems. The combination of ANN with Evolutionary Algorithms is a powerful approach to address the exploitation/exploration dilemma. Neural Networks can be trained to build a smooth map of the fitness landscape and are therefore adequate to perform a local search exploiting specific regions for candidate solutions. On the other hand, Evolutionary Algorithms are efficient in exploring huge search spaces of multivariate functions with many local minima and are adequate for global search. Several approaches have been used to approximate objective functions in single objective genetic algorithms, such as statistical methods or Artificial Neural Networks [9, 12]. Nain and Deb [9] proposed an iterative method where a neural network is trained with a set of exact evaluations of the fitness function. After adequate training, the genetic algorithm uses the neural network to estimate the objective function for a fixed number of generations. The process is repeated upon reaching an adequate solution. Recently, Gaspar-Cunha and Vieira extended this hybrid approach to Multi-Objective Evolutionary Algorithms [13]. In this work we introduce a further development to accelerate Multi-Objective Evolutionary Algorithms by developing an inverse neural network (IANN). The idea is to train the neural network in an inverse way: the inputs feed to the network are the criteria and the outputs the independent variables. In this approach some of the next generation individuals are obtained directly by the IANN. The advantage is to achieve a much faster population convergence to the Pareto-front.
2 The multi-objective optimisation algorithm The capabilities of Evolutionary Algorithms to explore and discover Pareto-optimal fronts on multi-objective optimization problems are well recognized [5]. For most problems MOEA outperforms traditional deterministic methods due to its capacity to explore and combine various solutions to find the Pareto front in a single run. In MOEA it is desirable to have an homogeneous distribution of population along the Pareto frontier together with an improvement of the solutions along successive generations [3 - 5]. In this work the Reduced Pareto Set Genetic Algorithm (RPSGA) is adopted [14], and a clustering technique applied to reduce the number of solutions on the efficient frontier. Initially, RPSGAe sorts the population individuals in a number of pre-defined ranks using a clustering technique, in order to reduce the number of solutions on the efficient frontier while maintaining it characteristics intact. Then, the individuals fitness is calculated through a ranking function. To incorporate this technique, the traditional GA was modified [14, 15] by introducing an external (elitist) population and a specific fitness evaluation. Initially, an internal population of size N is randomly defined and an empty external population formed. At each generation a fixed number of best individuals, obtained by reducing the internal population with the clustering algorithm [15], are copied to an external population. The process is repeated until the number of individuals of the external population becomes full. RPSGA is then applied to sort the individuals of the external population, and a pre-defined number of the *
Corresponding author
1
best individuals are incorporated in the internal population by replacing lowest fitness individuals. Detailed information about this algorithm can be found elsewhere [14, 15].
3 The ANN-MOEA hybrid algorithm 3.1 Statement of the problem The goal of our hybrid algorithm is to reduce the number of evaluations of the exact objective functions without loosing performance on establishing the Pareto-front. Feedforward Artificial Neural Networks (ANN), implemented by a Multilayer Perceptron or Radial Basis Functions, are flexible learning machines capable to approximate an arbitrary function provide enough training data is available [11, 16]. ANN have been applied with success to accelerate single objective optimization problems [8]. However, it is generally difficult to get a model with sufficient approximation accuracy, mainly due to the lack of data and the high dimensionality of many real-world problems. Therefore, ANNs often have large approximation errors and can even introduce false optima. In this case, the evolutionary algorithm with approximate models for fitness evaluation may converge incorrectly and measures must be taken to guarantee that the solution discovered is optimal or near-optimal. One main feature in managing approximate models is the interplay between the optimization and the fidelity of the approximate model based on the trust-region method, which ensures that the search process converges to a reasonable solution of the original problem. A heuristic convergence criterion is used to determine when the approximate model must be updated. An assumption is that the first sets of data points are at least weakly correlated with the global optimum of the original problem, which is not necessarily true for high-dimensional systems. The approximate model is used alternately with the original fitness function over a predefined number of generations. Data from a new generation is used to update the approximate model. The main idea is to maintain the diversity of the individuals and to select those data points that are not redundant for model updating - on-line learning. The decision of when to carry out the online learning of the approximate model is simply based on a prescribed generation delay. Caution must be taken since the evolution process may become very unstable if there is a large discrepancy between the approximate model and the original fitness function. The common weakness in the above methods is that neither the convergence of the evolutionary algorithm with approximate fitness functions (correct convergence is assumed) nor the issue of model management is addressed. In previous works [13, 17] we proposed an approach following theses lines where the evolutionary algorithm runs over p initial generations to obtain a first set of evaluations necessary to train the neural network (being the best solutions of those populations selected to train the ANN). From this point, the ANN is used to evaluate all solutions during a number of consecutive, and fixed, q generations. This process is repeated until an acceptable solution is found being the last p generations evaluated exactly. A large p is desirable for a better ANN approximation. Conversely, by increasing q we reduce the number of exact evaluations at the cost of deteriorating the Pareto-front. Although these two parameters can be pre-defined, their determination is strongly problem dependent. In another version [13, 17], we considered all N individuals of the population are evaluated with the ANN approximation but a small portion, M, are simultaneously evaluated with the exact function. This fraction is used to monitor the accuracy of the approximation. The error introduced by the approximations (eNN) is:
e NN =
M
R
j =1
i =1
∑ ∑
(C
NN i, j
− Ci, j
)
(1)
2
R M NN
where R is the number of criteria, M the number of solutions exactly evaluated, C i , j is the value of criteria i for solution j evaluated by ANN and
Ci , j is the value of criteria i for solution j evaluated by exact function.
As the algorithm evolves it drift to regions outside the domain covered by the initial training points where the neural network approximation is no longer valid. The great advantage of this method is that parameter q is dynamically adjusted by setting e NN < e0 , where e0 is a value fixed or adapted over the run that measures the desired level of discrepancy.
2
3.2 The Inverse ANN In both methods described above, neural networks were used as direct, and general, approximations of the optimization criteria. However, when the complexity of the original fitness landscape is high (i.e., the surface is uneven and/or multimodal) the evolutionary algorithm can be misled by the neural network model. Furthermore, the neural network is used as an estimator for the whole search domain, which, for high dimensional functions, may be problematic due to insufficient training data, as demonstrated in previous work [17]. In this work we propose a new approach where the neural network is trained in a reverse way, i.e., the input layer is connected to the criteria to be optimized and the output layer corresponds to the problem parameters. Instead of using the network as a global estimator, we use it to perform a local search near non-dominated solutions from the previous generation. Since the network is trained in an inverse way, it can be used to discover directly new tentative solutions with higher fitness in the vicinity of non-dominated points. This strategy has the advantage of guiding the MOEA locally thus avoiding extrapolations for regions not covered by the training data. The inverse neural network (IANN) is trained online using the available points from previous exact function evaluations. For all applications we use a neural network with a single hidden layer of variable size that is adjusted to the complexity of the problem. All inputs and outputs were normalized between 0 and 1. The additional step in the hybrid algorithm is the introduction of a local search performed by the trained IANN. In this approach all the solutions are evaluated using exact function evaluation, the IANN is only used to obtain new solutions on the parameters to optimize domain. Figure 1 illustrates the local search operator for a problem with two criteria. First we select the best individuals of the present generation with RPSGA. For extreme points, designated in Figure 1 by “e1”, and corresponding to the minimum found so far for criterion 1, and “e2”, the minimum found for criterion 2, three new individuals are obtained with IANN. These new points, identified as “a”, “b” and “c”, are generated as follows:
G Point a) : C new = (C1 + ∆C1 , C 2 + ∆C 2 ) G Point b) : C new = (C1 , C 2 + ∆C 2 ) G Point c) : C new = (C1 − ∆C1 , C 2 )
(2)
where ∆Cj is the displacement applied to each criteria (which is a parameter to be empirically defined) and
G C new sets the coordinates of criteria corresponding to the new point. The indices 1 stands for point “e1”, and 2
for point “e2”. For the remaining points 1 to 4, new tentative solutions are obtained according to the equation:
G (3) C new = (C1 + ∆C1 , C 2 + ∆C 2 ) G new For each new point C in the criteria space, the correspondent individual in the input variables space is
obtained using the inverse neural network IANN. Only a percentage of these new individuals generated by the IANN are then used to form a new population and feed to the genetic algorithm. Table 2 presents the influence of the relevant parameters on the algorithm performance. The quantities ∆C is the perturbation introduced for each criterion, Ngen the number of generations whose points are used to train the network, limit are the limits of indifference in the clustering algorithm and IR the rate of individuals generated by the IANN to be used as a new pool for the GA. We found that none of these parameters has a significant impact on the performance of the algorithm. Table 1: influence of controlling parameters on the algorithm. Parameter
Tested values(*)
Best results
Influence
Selected
Limit
0.01; 0.05; 0.1; 0.2
[0.01; 0.2]
Small
0.01
∆C
0.3; 0.4; 0.5; 0.6
[0.3; 0.5]
Small
0.5
Ngen
5; 10; 15; 20
[5; 10]
Small
5
IR
0.35; 0.50; 0.65; 0.80
[0.35; 0.8]
Small
0.8
3
a
b
Criteria 2
∆C2
c e1
1
2 3 4 e2
Criteria 1
∆C1
Fig. 1. Scheme used for the Inverse ANN (IANN) local approximation.
4 Results Our method was tested in the ZDT1, ZDT2, ZDT3, ZDT4 and ZDT6 bi-objective functions. These functions cover various types of Pareto-optimal fronts, such as convex (ZDT1), non-convex (ZDT2), discrete (ZDT3), multimodal (ZDT4) and non-uniform (ZDT6) [18]. The S-metric proposed by Zitzler [19] is adopted to compare the performance, since this metric can be estimated along the various generations allowing the attainment of as global picture of the optimization process. Each run was performed five times to take into account the variation of the random initial population. The RPSGA population size was set to 100 over 300 generations and a roulette wheel selection strategy, a crossover probability of 0.8, a mutation probability of 0.05 and a number of ranks of 30 were used [14]. Using the direct hybrid method [17], we save between 9% (ZDT6) and 35% (ZDT1) of computational time (taking into naccount the time necessary to train the ANN) to achieve the same S-metric performance. With the new method proposed here (IANN) the savings were even higher, in some cases higher than 50%. Figure 2 compares the S-metric obtained with traditional RPSGA and the results obtained with our method using IANN. Note that the rate of convergence to the desired Pareto-front is considerable accelerated. After a few generations our hybrid IANN algorithm reached an S-metric that is equalled by the RPSGA alone only after 200 generations. Moreover, for a given number of exact evaluations, the S-metric is always superior when the approximate method is used. This is particularly visible in the early convergence where the improvements are larger. After about 200 generations both curves merge to a single plateau asymptotically approaching the final solution. All optimizations were obtained by averaging over 5 runs using different initial populations.
5 Conclusions We show that an adequate hybridization of Multi-Objective Genetic Algorithms Artificial Neural Networks can be very valuable in reducing the computational effort required to reach an acceptable Pareto-front. In particular, our method, the inverse neural network, in some cases, can save more than 50% of exact function evaluations. Note that most improvements of our method are accomplished during the first generations, does properly guiding the population to the desired solution in the very early stages. This algorithm will be applied to the real problem of optimization of a polymer extrusion process.
4
2C-ZDT1 1
S metric
0.8 0.6 0.4 IANN RPSGAe
0.2 0 0
50
100
150 Generations
200
250
300
Figure 2: Comparison of the S metric as a function of the number of evaluations for ZDT1 problem.
References 1. J.D. Schafer, Some Experiments in Machine Learning Using Vector Evaluated Genetic Algorithms, Ph. D. Thesis, Nashville, TN, Vanderbilt University, 1984. 2. C.M. Fonseca, P.J. Fleming, Genetic Algorithms for Multiobjective Optimization: Formulation, Discussion and Generalization, Proc. Fifth Int. Conf. on Genetic Algorithms, Morgan Kauffman (1993)416-423. 3. A. Gaspar-Cunha, P. Oliveira, J.A. Covas, Use of Genetic Algorithms in Multicriteria Optimization to Solve Industrial Problems, Seventh Int. Conf. on Genetic Algorithms, Michigan, USA, 1997. 4. K. Deb, S. Agrawal, A. Pratap, T. Meyarivan, A Fast Elitist Non-dominated Sorting Genetic Algorithm for MultiObjective Optimization: NSGAII, Proceedings of the Parallel Problem Solving from Nature VI (PPSNVI), (2000)849858. 5. K. Deb, Multi-Objective Optimization using Evolutionary Algorithms, Wiley, 2001. 6. E. Zitzler, M. Laumanns, L. Thiele, SPEA2: Improving the Strength Pareto Evolutionary Algorithm, TIK report no. 103, Swiss Federal Institute of Technology, Zürich, Switzerland, 2001 (Downloadable from website http://www.tik.ee.ethz.ch/~zitzler/). 7. J.D. Knowles, D.W. Corne, Approximating the Non-dominated Front using the Pareto Archived Evolutionary Strategy, Evolutionary Computation Journal, (8)(2000)149-172. 8. C.A. Coello Coello, D.A. Van Veldhuizen and G.B Lamont, Evolutionary Algorithms for Solving Multi-Objective Problems, Kluwer, 2002. 9. C. Bishop Neural Networks for Pattern Recognition, Oxford University Press, 1997. 10. P.K.S. Nain, K. Deb, A Computationally Effective Multi-Objective Search and Optimization Technique Using Coarse-toFine Grain Modelling, Kangal Report No. 2002005, 2002 (Downloadable from website http://www.iitk.ac.in/kangal/deb.htm). 11. Y. Jin, M. Olhofer, B. Sendhof, A Framework for Evolutionary Optimization with Approximate Fitness Functions, IEEE Trans. on Evolutionary Computations (6)(2002)481-494. 12. C. Poloni, A. Giurgevich, L. Onesti, V. Pedirola, Hybridization of a multi-objective genetic algorithm, a neural network and a classical optimizer for complex design problem in fluid dynamics, Computer Methods in Applied Mechanics and Engineering 186 (2000)403-420. 13. A. Gaspar-Cunha and A. Vieira, A Multi-Objective Evolutionary Algorithm Using Approximate Fitness Evaluations, European Conference on Applications of Genetic Algorithms EUROGEN 2003, p. 157, Barcelona 15-17 September 2003. 14. A. Gaspar-Cunha, J.A. Covas, J.A. - RPSGAe - A Multiobjective Genetic Algorithm with Elitism: Application to Polymer Extrusion in a Lecture Notes in Economics and Mathematical Systems volume, Springer (2002). 15. A. Gaspar-Cunha, Modelling and Optimization of Single Screw Extrusion, PhD Thesis, University of Minho, Guimarães, Portugal, 2000 (Downloadable from website http://www.lania.mx/~ccoello/EMOO/). 16. L. Bull, On Model-based Evolutionary Computation, Soft. Comp, (3)(1999) 76. 17. A. Gaspar-Cunha, A. Vieira, A Multi-Objective Evolutionary Algorithm Using Neural Networks To Approximate Fitness Evaluations, in press, International Journal of Computers, Systems, and Signals (2004). 18. E. Zitzler, K. Deb, L. Thiele, Comparison of Multiobjective Evolutionary Algorithms: Empirical Results, Evolutionary Computation, (8)(2000)173—195. 19. E. Zitzler, E.: Evolutionary Algorithms for Multiobjective Optimization: Methods and Applications, PhD Thesis, Swiss Federal Institute of Technology (ETH), Zurich, Switzerland, 1999.
5
