Population Learning Algorithm-Example Implementations and ...

MIC’2001 - 4th Metaheuristics International Conference

607

Population Learning Algorithm - Example Implementations and Experiments Ireneusz Czarnowski∗ ∗

1

Piotr J¸edrzejowicz∗

Ewa Ratajczak∗

Department of Information Systems, Gdynia Maritime Academy Morska 83, 81-225 Gdynia, Poland Email: {irek, pj, ewra}@wsm.gdynia.pl

Introduction

Continuing research interest is focused on algorithms that rely on analogies to natural processes and phenomena. An important subclass of the NPB (natural processes based) class form population based methods, which also are called evolutionary algorithms. Population based methods handle a population of individuals that evolves with the help of information exchange procedures. It should be noted that many different algorithms could be described within this framework. The best known population-based algorithms include genetic algorithms [8], [6] evolution strategies and evolution programming [12], scatter search [3], adaptive memory algorithms [4], and ant systems [1]. In this paper another metaheuristic which is an extension of population-based methods and adaptive memory programming techniques, introduced in [10], is investigated. It has been inspired by analogies to a social phenomenon rather than to a natural process. Whereas evolutionary computations emulate basic features of the natural evolution including natural selection, hereditary variations, the survival of the fittest and production of far more offspring than are necessary to replace current generation, population learning algorithms (PLA) take advantage of the features common to organized education systems: - A huge number of individuals enter the system. - Individuals learn through organized tuition, interaction, self-study, trials and errors. - Learning process is inherently parallel (with different schools, curricula, teachers, etc.). - Learning process is divided into stages. - More advanced stages are entered by a diminishing number of individuals from the initial population. - At higher stages more advanced learning/improvement techniques are used; - A final stage is reached by only a fraction of the initial population. Basic idea of the population learning algorithm is shown in the following pseudo-code: Begin Chose the number of stages ns; Porto, Portugal, July 16-20, 2001

608

MIC’2001 - 4th Metaheuristics International Conference For each stage i = 1 . . . ns set the selection criterion SCi ; For each stage i = 1 . . . ns design the learning-improvement procedure Li ; Generate the initial population of individuals J ; For i = 1 to ns do For each individual in J do Apply the learning-improvement procedure Li ; End Remove from J all individuals who do not pass the selection criterion SCi ; End End

The above pseudo-code covers simple, non-parallel version of the algorithm. For the parallel PLA a structure for grouping individuals at various stages, and respective rules for carrying parallel computations as well as information exchange and co-ordination need to be designed. The paper describes implementation of several instances of the PLA including example integer optimization problem, nonlinear integer transportation problem, training of the feed-forward artificial neural networks, and multiple processor tasks scheduling. Algorithms have been evaluated by mean of computational experiments. Results of these are reported.

2

Implementations of the PLA

The first of the considered problems is a constrained integer optimization, reported in [2]. It involves minimizing a linear function of 13 variables under a set of 22 linear constraints. A vector of integers represents an individual. The respective PLA-CIO assumes that the initial population of individuals is generated randomly. The algorithm includes pre-selection and two learning stages. Pre-selection assures feasibility of the initial population. First learning stage involves simple neighborhood search. Second stage involves a similar search but performed on individuals produced by applying a crossover operator. Selection criterion is to remove from the current population individuals with fitness below an average. The second of the considered is a nonlinear integer transportation problem, reported in [12]. There are 3 sources and 4 destinations. A matrix of integers represents an individual. The respective PLANITP assumes that the initial population of individuals is generated randomly. The algorithm again includes pre-selection and two learning stages. Pre-selection assures feasibility of the initial population. First learning stage involves simple neighborhood search. Second stage involves a similar search but performed on individuals produced by applying a mutation operator. Selection criterion is to remove from the current population individuals with fitness below an average. Next, PLA have been used to train feed-forward artificial neural networks. To test and evaluate application of the PLA to training ANN several benchmark problems have been chosen, including XOR problem, 4-parity and 5-parity problems and the sonar problem described in [7], [9]. In XOR problem a binary vector of 225 bits represents an individual. It can be decoded into 9 weights which are a real value numbers form [−10, 10], with precision 10−6 . Similar approach to coding weights has been used in case of the remaining networks. The respective PLA assumes that the initial population of individuals is generated randomly. The algorithm includes four learning stages. The first stage is a simple one-point crossover. The second is a mutation based on two-point exchange of values. The third is a mutation based on negation of values in the randomly selected range of points. Finally, the fourth is a mutation based on a cyclic move of Porto, Portugal, July 16-20, 2001


609

values in the randomly selected range of points by a random number of steps. Selection criterion is to remove from the current population individuals with fitness below an average. Finally, a PLA instance has been designed to schedule multiprocessor tasks under criterion of maximizing schedule reliability. The problem at hand, denoted as P|m − p, aj , dj |R, is characterized by a set of independent, multiple-processor (m-p), non-preemptable tasks which are to be scheduled on P identical processors. Each task is characterized by its ready time - rj , due date - dj , number of available variants - N Vj , variant reliabilities - rji and variant processing times pji . Decision variables include allocation of processors to tasks and number of variants of each task to be executed. Number of processors needed to process a task is equal to the number of its variants chosen for processing. Processing time of such a multiple-processor task is equal to a processing time of the longest variant taken over all variants used to construct a multiple-processor task. All processors needed to process an m-p task are released simultaneously after its processing time has elapsed. Additional constraints require that each task is constructed from at least one of its variants and none of the tasks is delayed. Schedule reliability is calculated from: R=1−

n ' '

(1 − rij ),

(1)

j=1 i∈SVj

where: SVj - is a set of variants of the task j chosen to be executed and n is a number of tasks. The PLA - P|m − p, aj , dj |R is based on the following assumptions: - An individual is an ordered list of n objects - one for each task. Each object includes a number of the respective task and a nested inner object representing a combination of variants that are to be run. - An initial population of individuals is randomly generated and supplemented with 2 individuals one constructed using earliest ready time as the first criterion and earliest due date as the second one and the other constructed the other way round. - There are 3 learning stages. - The selection criterion requires rejecting all individuals with fitness below an average for the whole population. - Fitness of an individual is a value of the goal function. It is calculated using the list scheduling algorithm (objects from the list are one by one assigned to the least loaded processor). The following learning-improvement procedures are used: L1 - try to improve unfeasible solutions first by exchanging objects on the list and then replacing a combination of variants to be run with another one with smaller reliability. L2 - apply 2-OPT algorithm. Its idea is to pick an object with the least reliability plus another one placed immediately after on the current list. The pair is optimized assuring maximum reliability and no delay, and the result inserted into the initial list replacing the initial pair. The initial pair is, in addition, placed on a tabu moves list. Operation is repeated n times. L3 - apply standard tabu search [5] for a fixed number of iterations with moves including exchanging objects on the list and changing combinations of variants to be run. Porto, Portugal, July 16-20, 2001

610


Figure 1: Mean relative error (MRE) versus the initial population size (PLA- CIO)

Figure 2: Mean relative error (MRE) versus the initial population size at different iteration levels (PLANIPT)

3

Computational experiments

To evaluate PLA-CIO a computational experiment has been carried. It involved three basic modes of using the algorithm. In the single mode one randomly generated initial population is passing through learning stages. In the 2-parallel mode two randomly generated initial populations of identical size are learning independently in parallel. Best individual from both populations is considered a solution. In the 3-parallel mode there are three independent, randomly generated populations of identical size. Learning process is carried as in case of the 2-parallel mode. Best individual from three populations is considered a solution. For each mode and each initial population size, 20 initial populations have been randomly generated. Number of crossovers in a single run has been set to 60 and number of iterations in the local search procedure to 10. Mean relative errors (MRE) from optimum solution are shown in Figure 1. The second experiment involved the PLA-NIPT. Local search has been carried at 5 different iteration levels (20, 40, 60, 80, 100 iterations). For each iteration level and each initial population size, 20 initial populations have been randomly generated. Mean relative errors (MRE) from optimum solution are shown in Figure 2. Further experiments have been carried with implementations designed for training of neural networks. For the PLA-XOR there have been 4 pairs of training patterns, for the PLA-4PARITY - 16 pairs, for the PLA-5PARITY - 32 pairs, and for the PLA-SONAR - 103 pairs, respectively. Performance of Porto, Portugal, July 16-20, 2001

MIC’2001 - 4th Metaheuristics International Conference Problem

XOR

4-PARITY

5-PARITY

SONAR

Approach QP GA PLA QP GA PLA QP GA PLA QP GA PLA

Mean time (sec.) 0.71 3.04 0.20 139.76 403.46 3.70 291.69 843.81 6.20 136.94 589.24 63.90

Mean no. of epoch 190 959 1000 560 5242 1640 4487 6251 2861 384 3126 998

611 MRE 0.074 0.089 0.085 0.086 0.076 0.082 0.162 0.097

Table 1: Comparison of algorithms used for ANN training L-K heuristics 1.32

Neural Network 6.58

IBGA 0.26

PLA 0.16

Table 2: Mean relative errors from the best known result (%) - PLA - P|m − p, rj , aj |R the PLA implementations has been compared with two other approaches - quickpropagation (QP) and genetic algorithm (GA) constructed in accordance with the specifications given in [7] and [9]. Three performance measures have been used - mean time needed for training, mean number of epochs and mean error. The results of the computational experiment are shown in Table 1. Finally, to evaluate the PLA - P|m − p, rj , aj |R another computational experiment has been carried. It involved 20 randomly generated instances of the problem. The number of available processors varied between 4 and 10 and number of tasks between 8 and 28. Four different algorithms have been used: heuristics based on Lin-Kernighan approach [11], neural network, island-based genetic algorithm (IBGA) and the PLA - P|m − p, rj , aj |R. In Table 2 mean relative errors from the best known solution are shown. Computational effort needed for the PLA algorithm has been about 2.5 times smaller in case of the PLA as compared to IBGA. Neural Network based algorithm has been on averages 2 - 3 times quicker then the PLA.

4

Conclusion

The paper proposes using a new population-based metaheuristic to solving some difficult combinatorial optimization problems. Computational experiments have shown that for several example problems even quite basic and simple population learning schemes can produce remarkably competitive results as compared to other approaches. Strength of the PLA can be, in part, attributed to its ability to easily incorporate elements of knowledge about the problem at hand into a process of searching for its solution. PLA can be also viewed as a convenient framework for hybrid computations. The concept of PLA requires further research aiming at introducing more formalized structures and tools and improving effectiveness and quality of this class of algorithms. At this stage it can be concluded that the idea of PLA extends the existing range of population-based and adaptive memory programming techniques. It is an open concept allowing a lot of freedom in designing PLA-schemes solving particular problems. Porto, Portugal, July 16-20, 2001

612


PLA, being naturally parallel, can also effectively use available parallel computation environments producing even more effective solutions.

References [1] A. Colorni, M. Dorigo and V. Maniezzo. Distributed optimization by ant colonies. In Varela F.J., Bourgine P., eds. Proc. of the 1st European Conference on Artificial Life - ECAL’91, , The MIT Press, Cambridge, pages 134-142, 1991. [2] C.A. Floudas and P.M. Pardalos. Recent Advances in Global Optimization. Princeton Series in Computer Science, Princeton University Press, Princeton N.J., 1992. [3] F. Glover. Genetic Algorithms and Scatter Search: unsuspected Potentials. In Gutenberg, editor, Statistics and Computing 4:131-140, 1994. [4] F. Glover. Tabu search and adaptive memory programming - advances, applications and challenges. In Barr R., Helgason R., Kennington J., eds., Interfaces in Computer Science and Operations Research, Kluwer, Boston, pages 1-75, 1996. [5] F. Glover and M. Laguna. Tabu Search. Kluwer, Norwell, 1997. [6] D.E. Goldberg. Genetic Algorithms in Search, Optimization and Machine Learning. AddisonWesley, Reading, 1989. [7] R.P. Gorman and T.J. Sejnowski. Analysis of Hiden Units in a Layered Network Trained to Classify Sonar Targets. Neural Networks 1:75-89, 1988. [8] J.M. Holland. Adaptation in Natural and Artificial Systems. The University of Michigan Press, Ann Arbor, 1975. [9] J. Hertz, A. Krogh and R.G. Palmer. Wst¸ep do Teorii Oblicze´ n Neuronowych. WNT, Warszawa, 1995 (in Polish). [10] P. J¸edrzejowicz. Social Learning Algorithm as a Tool for Solving Some Difficult Scheduling Problems. Foundation of Computing and Decision Sciences 24:51-66, 1999. [11] S. Lin and B.W. Kernighan. An effective heuristic algorithm for the traveling salesman problem. Operations Research 21:498-516, 1973. [12] Z. Michalewicz. Genetic Algorithms + Data Structures = Evolution Programs. Springer, Berlin, 1996. Acknowledgement: The research has been supported by the KBN grant no 8T11F02019

Porto, Portugal, July 16-20, 2001