Using Region-Schema to Solve Nonlinear Constraint Optimization Problems: a Cultural Algorithm Approach Robert G. Reynolds
Xidong Jin
Computer Science Department Wayne State University Detroit, MI 48202
[email protected]
Computer Science Department Wayne State University Detroit, MI 48202
[email protected]
Abstract- The key idea behind Cultural Algorithms is to explcitly acquire problem-solving knowledge (beliefs) from the evolving population in the form of schemata and in return apply that knowledge to guide the search [Reynolds 1993, 1996]. In solving nonlinear constraint optimization problems, the key problem is how to represent and store the knowledge about the constraints. Previously, Chung [1996, 1998] used Cultural Algorithms to solve unconstraint optimization problems. There, he used interval schemata proposed by Eshelman and Schaffer [Eshelman 1992] to represent global knowledge about the independent problem parameters. However, in constraint optimization the problem intervals must be modified dependently. In order to solve constraint optimization problems, we need to extend the interval representation to allow for the representation of constraints. In this paper, we define an n-dimensional regional-based schema in terms of beliefcells, which can provide an explicit mechanism to support the acquisition, storage and integration of schematic knowledge about the constraints. In a Cultural Algorithm framework, the belief space can "contain" a set of these schemata, each of them can be used to guide the search of the evolving population. This kind of region-based schemata can be used to guide the optimization search in a direct way by pruning the unfeasible regions and promoting the promising regions. We compared the results of 4 CA configurations that manipulate these schemata for an example problem. An application to a problem in Cultural Evolution is also demonstrated.
the problem solving process of future generations [Reynolds 1986, 1993, 1996; Chung 1996]. The knowledge, patterns identified during the search, can be used to guide the generation of candidate solutions, to promote more instances of desirable candidates, or to reduce the number of less desirable candidates in the population in order to facilitate the optimization process. However, traditional ECs have limited or implicit mechanisms for representing and storing individual's global acquired knowledge to be passed form generation to generation. Cultural Algorithms, which model the evolution of the culture component of an evolutionary computational system over time, provides an explicit mechanism for acquisition, storage and integration of individual and group's problem solving experience and behavior. In human societies, culture can be viewed as a vehicle for the storage of information that is globally accessible to all members of the society and that can be useful in guiding their problem solving activities. As a result, Cultural Algorithms, which model the evolution of the cultural component of an EC system, can provide an explicit mechanism for Knowledge-Based Evolutionary Computation. The general constraint optimization problem is to find x so as to:
1 Introduction Genetic Algorithms, and related evolutionary computational methods, have been used widely to solve optimization problems due to their unbiased nature which enables them perform well in situations with little/no domain knowledge [Holland 1975]. However, the incorporation of knowledgebased mechanisms can considerably improve an evolutionary algorithm's performance when problemsolving knowledge acquired during evolution is used to bias
minimize f (x) , x = ( x1 ,..., x m ) ∈ ℜ m , subject to
h ( x) = 0 g ( x) ≤ 0 Where h(x) and g (x) are problem constraints, which can be linear or nonlinear. h(x) represents a set of p equality constraints and g (x) represents a set of q inequality constraints. Due to the existence of problem constraints, non-linear constraint optimization problems are not easy to solve even though they are very important in practical use. For example, it is hypothesized here that many complex state organizations must deal with the solution of such problems constantly. Whether the solutions result in optimal or sub-
optimal results will depend upon the knowledge and experience of the participants. In this paper, the mechanisms that Culture Algorithms use to solve nonlinear constraint optimization problems are presented. The population component of the CA will be guided by region-based schemata that extends the interval schemata used by Chung [Chung 1996]. These region-based schemata are called Belief-Cells. The exploration of the EP population will serve to modify these belief-cell schemata which in return are used to guide the production of new individuals in the population component. In section 2 a brief introduction to Cultural Algorithms, will be presented. Then, in section 3, the mechanism used to handle constraint knowledge by Cultural Algorithms here is described. The various versions of the CA to solve constraint optimization problems are presented in section 4. In section 5 the test results are discussed and section 6 presents our conclusions.
2 Cultural Algorithms, EP and CAEP Evolutionary Computation can be generally described by the following equation:
P[nc, np ]t +1 = select ( generate( P[nc, np ]t )) t where P[ nc, np ] is the population which consists of nc candidate solutions to the problem with np parameters at generation t . generate() is an operator to generate new solutions, and select () is the operator to select those that will be the parent solutions in the next generation
P[nc, np]t +1 . For example, select () is often implemented by tournament selection mechanism. 2.1 Cultural Algorithms The key idea behind Cultural Algorithms is to acquire problem-solving knowledge (beliefs) from the evolving population and in return make use of that knowledge to guide the search. A Culture Algorithm is a dual inheritance system that models two levels of evolution: at the population level and at the belief space level. Besides a population space, CA has a belief space in which the problem-solving knowledge acquired from the evolving population can be stored and integrated. A pseudo-code description of Cultural Algorithms is described as follows: (1) t = 0 ;
Pt ; t (3) Initialize Belief Space B ; (2) Initialize population
(4) Evaluation the performance scores of (5)
update( B t , accept ( P t )) ;
generate( P t , influence( B t )) ; (7) t = t + 1 ; t t −1 (8) select P form P ; (6)
(9) Repeat to (4) until the termination condition is achieved. A framework for a CA can be depicted in Fig.1. update( )
Belief Space accept( ) generate( )
Population
select( )
obj( )
Fig.1 A framework of Cultural Algorithms As shown in Fig.1, the accept() function is used to generate beliefs by gleaning the experience of selected individuals from the population. In return, the influence() function can make use of the problem-solving knowledge to guide the evolution of the population component. The belief space also can be improved by update() function. Culture Algorithms for optimization problems can be represented in the above framework as:
P[nc, np]t +1 = select ( generate( P[nc, np]t , Beliefs t ), Beliefs t ) Here, Beliefs can be used to influence generate() operator and/or select () operator. The beliefs can be adjusted as the following:
Beliefs t +1 = update( Beliefs t , accept ( P[nc, np ]t )) 2.2 The Evolutionary Population Model The basic pseudo-code for the population component is given below: (1) When generation t = 0 , randomly select an initial t
population of nc candidate solutions, P[ nc, np ] , within the given domains. (2) Evaluate the performance score of each candidate solutions by a given objective function obj () . (3) Generate nc new offspring solutions from
P[nc, np ]t Pt ;
influence( )
by
applying
generate() :
P[2 * nc, np ] = generate( P[nc, np ]t ) . Note now the size of the population is 2 * nc . t
(4) Evaluate the performance score of each offspring solution by the given objective function obj () . (5) For each candidate, randomly select c competitors from the population of 2 * nc size. Conduct pair-wise
competitions between the individual and its competitors. (6) Select the np solutions that have the greatest number of wins to be parents for the next generation. (7) t = t + 1 . Repeat to step 3 until certain termination condition is achieved. 2.3 CAEP Chung embedded an evolutionary population model into a CA framework, CAEP [Chung 1996, 1998] in order to investigate the impact of global knowledge on the solution of real-valued optimization problems. CAEP keeps in the belief space situational and/or normative knowledge. Situational knowledge is represented by a set of exemplar cases, may control the directions of the mutation. Chung used interval schemata proposed by Eshelman and Schaffer [Eshelman 1992] to explicitly store normative knowledge about the range of acceptable values for each parameter. CAEP is successful in solving diverse unconstrained optimization problems [Chung 1996, 1998]. In order to solve constrained optimization problems, however, we need an explicit mechanism to support the acquisition and manipulation of knowledge about the constraints. In this paper, we extend CAEP to solve constrained nonlinear optimization problems by explicitly storing and manipulating in the belief space a new kind of knowledge, constraint knowledge, in terms of new region-based schemata called belief-cells.
3 Mechanism to Handle Constraint Knowledge by Cultural Algorithms
knowledge residing in the belief-cells to guide the evolving population. For example: (1) Belief-cells can show the whereabouts of the promising areas in the domain space. In constraint optimization problems, a solution cannot be in an infeasible region, so belief-cells can declare the feasible/semi-feasible cells as the promising area in this case and ignore those currently labelled as infeasible. (2) Many constraint optimization problems have their optimal value on/near the frontiers between the feasible regions and infeasible regions, which corresponds to semi-feasible belief-cells. Therefore, the knowledge about the frontier can be used to speed up the optimization search. x 0
0
0.2
0.4
0.6
0.8
1
1.2
1.4
1.6
1.8
2
0.5
1
y
1.5
2
2.5
3
(a) Constraints cutting the domain space into different types X 0
In solving constrained optimization problems, a key problem is how to represent and save the knowledge about the problem constraints in the belief space.
0
0.2
0.4
0.6
0.8
1
1.2
1.4
1.6
1.8
2
0.5
1
3.1 constraint knowledge and belief-cell By observation, we can see that problem constrains cut the domain space into smaller feasible/unfeasible areas-different areas have different characters. Some areas are feasible and satisfy all of the constraints while others are infeasible. Hence we can divide the domain space into smaller regular sub-regions which are called cells. The image is just like a biological creature that consists of many regular cells. Therefore, cells may have different characters: some cells are feasible since they are totally in feasible regions, some cells are infeasible since they are totally in unfeasible regions, others are semi-feasible because they span both feasible and unfeasible areas. Consequently, the cells here can introduce a tool to represent and store certain patterns of constraint knowledge of the domain space. The tool is called a belief-cell, with which an "image" structure of domain cells is maintained in belief space and can be a vehicle to represent, store and integrate constraint knowledge. Furthermore, we can apply the constraint
y 1.5
2
2.5
3
Infeasible areas Semi-feasible areas Feasible areas (b) Belief-cells illustrating the constraint information Fig. 2 Example of Domain space and belief-cells Fig.2 is a 2-dimensional example of a domain space divided into belief-cells. In Fig. 2(a), nonlinear constraints cut the domain space into different types. In Fig. 2.2(b), the
knowledge about feasible/unfeasible regions can be explicitly identified by belief-cells. These belief-cells can be viewed as schemata in the sense that they identify a range of values for each problem parameter, but do not specify what the actual values within those ranges need to be. By identifying certain belief-cells as "containers" for the solution, they can serve to bias a systematic problem decomposition process. Simply speaking, the belief-cells act like a navigation map can guide the optimization search in a direct way by pruning the unfeasible areas and promoting the promising areas.
within the sliding window are more attractive than other regions.
3.2 Belief space structure The formal syntax of the beliefs structure defined in this paper is , where N is the normative knowledge component consisting of information about the intervals for each n parameter; C is a set of belief-cells' information which constitutes the constraint knowledge, m is the number of the belief-cells; m, is the number of the belief-cells, and it can be varied systematically during the evolution or maintain static. j , N [ j ] is represented as For parameter
3.4 Update the Belief Space The components N and C will be updated by update() function as follows. (1) Updating component N. Assume now it is the i th individual that affects the lower bound for parameter j . The lower boundary for j and its score are given below:
< I j , L j , U j >.
I j represents the closed interval of
j : I j = [l j , u j ] = {x | l j ≤ x ≤ u j , x ∈ ℜ} where l (lower bound) and u (upper bound) are initialized as the given domain values and can be changed later. L j parameter
represents the performance score of the lower bound of parameter j . U j denotes the performance score of the
u for parameter j . i , C[i ] is denoted as For belief-cell < Class i , Cnt1i , Cnt 2 i , Wi , Posi , Csizei > . Class i
upper bound
denotes features of the ith cell: feasible, infeasible, semifeasible or unknown are used in this paper. Cnt1 and Cnt2 are two counters maintained in the each belief-cell. The former counts the feasible individual candidates in the domain cell whereas the latter counts the number of infeasible individual candidates in the cell. The two counters can also provide information about the reliability of classifications. They are both initialized to zero. Wi is the weight of the ith cell, the better (more promising) the cell, the heavier the weight. Pos i is a vector denoting the leftmost corner position of the cell i, and
Csizei is a vector
denoting the size of the cell for all optimization parameters. Intuitively, the belief space is modeled as a single "sliding window with a navigation map in it". However, in general there can be a number of such active containers available. The component N determines the position and size of the n-dimensional sliding window, into which m belief-cells are embedded to serve as a navigation map. Generally speaking, the feasible and semi-feasible cells
3.3 Acceptance function The accept() function selects individuals who can directly impact the formation of current belief space. Just like that in society, different knowledge can be contributed by different people, in Cultural Algorithms the normative knowledge and constraint knowledge can be influenced by different individuals. In this paper, the acceptance function for component N selects the top 20% of the individuals in terms of the performance function.
x t if xi,jt ≤ l tj or obj(xit ) < Ltj l tj+1 = i ,t j otherwise l j t t obj ( xi ) if xi,j ≤ l j or obj(xi ) < L j Ltj+1 = t otherwise L j where
l tj represents lower bound for parameter j at
t and Ltj denotes the performance score for it. Assume now it is the k th individual that impacts the upper bound for parameter j :
generation
x if x k , j ≥ u tj or obj(x k ) < U tj u tj+1 = k , j t otherwise u j obj ( x x ) if x k , j ≥ u tj or obj(x k ) < U tj U tj+1 = t otherwise U j where
u tj represents upper bound for parameter j at
generation
t and U tj denotes the performance score for it.
(2) Updating component C As determined by the problem constraints, each trail individual can be either valid or invalid. Therefore, each individual can provide constraint information about the cell in which they reside. For a given cell i , Cnt1i counts the number of valid individuals in the cell, while invalid individuals.
Cnt2 i counts
unknown if Cnt1i = 0 and Cnt2i = 0 fesible if Cnt1i > 0 and Cnt2i = 0 Classi = unfeasible if Cnt1i = 0 and Cnt2 i > 0 semi − feasible otherwise where Class i provides the classification information of cell i . Wi will be updated directly according based upon the value stored in
Class i . Csizei , represents the granular
size of that belief-cell, and can be adjusted in either an explicit or implicit way. In the explicit method, the granularity is directly changed by corresponding rules, whereas in the implicit approach granularity is indirectly updated by other belief space parameters. For example, if we keep the number of belief cells in a sliding window invariable, then the granularity of the belief-cells can be implicitly changed by the size of the sliding window. This is the method is used in this paper. Updating both of the components, N and C, can be executed at different frequencies. For example, one might update C in every generation while updating N every k ( k ≥ 1 ) generations. In the model used here, component C is embedded in Component N. Therefore, once the position/size of the sliding window is updated, the navigation map for the new sliding window needs "drawing" again from the beginning. Consequently, it should be more conservative to update component N than to component C. An alternative method is: after updating both components every generation, use some extra individuals embedded in the new sliding window as scouts to help acquire constraint knowledge for the new location of the sliding window.
feasible/semi-feasible/unknown cell. The process is achieved by:
∀i ∈ {1,2,..., nc} and ∀j ∈ {1,2,..., np}, x nc + i , j
moveTo(choose(Cell[m])) = if x nc , j ∈ {unfeasible cells} x + (u − l ) ∗ N (0,1) / m otherwise j j nc , j j nc , j
m j is the number of belief-cells for parameter j. moveTo() is an immigration function which move a parent to a chosen cell. choose(Cell[ m]) is used to select where
the destination cell in a more promising area. Here, we use the roulette wheel selection method [Goldberg 1989] to achieve this. We define weight of Cell[k ] as:
if Cell[k] ∈ {unknown cells} w1 , w , if Cell[k] ∈ {feasible cells} Wk = 2 w3 , if Cell[k] ∈ {semi − feasible cells} w4 , if Cell[k] ∈ {unfeasible cells} Here, w1 = w2 =2, w3 =3, w4 =1. Assume that parent i should move to Cell[k ] , moveTo(Cell[k ]) is defined as follows: x nc +i , j = Pos k , j + Uniformnc , j (0,1) * Csize k , j Where
Pos k , j is the position of Cell[k ] for optimization
j , Csize k , j is the size of Cell[k ] for parameter j , and Uniform(0,1) generates a number parameter
within [0,1] by uniform distribution. 3.5 Influence Function Even though more sophisticated strategies exist, we use the simplest form in this paper; that is, move the generation of individuals from unpromising cells into promising cells.The basic idea is: (1) If a parent is outside a current sliding window it will move to the sliding window by using normative knowledge. This is achieved by:
∀i ∈ {1,2,..., nc} and ∀j ∈ {1,2,..., np},
x nc , j + (u j − l j ) ∗ N nc , j (0,1) if x nc , j < l j x nc + i , j = x nc , j − (u j − l j ) ∗ N nc , j (0,1) if x nc , j > u j where l j and u j means the current lower bound and upper bound in the belief space for the parameter j , respectively. (2) If a parent is in a current sliding window, perturb small when it is in a feasible/semi-feasible/unknown cell; otherwise the parent has higher probability to move to a
4 Versions of the Algorithm The pseudo-code of this region-based CA is given below. The operations within { } are optional. 1) t = 0 ; t
2) Initialize the population P[ nc, np ] that consists of nc candidate solutions with np parameters.
B t =< N t , C t > ; t 4) Evaluate the performance scores of P[ nc, np ] ;
3) Initialize the Belief Space
5) Update the Belief Space. a) {Under certain conditions}, Update the position/size of the sliding window determined by t
component N . {Create randomly ne extra candidates within the updated sliding window}. b) Update the Constraint knowledge component
Ct .
6) Generate new offspring candidates: a) Evaluation the whereabouts of the parents: outside the sliding window, or inside the sliding-window but in an unfeasible cell… b) Generate offspring by influence function based on 7)
the region-based knowledge in t = t + 1;
Bt .
t
8) Select the new P[ nc, np ] ; 9) Repeat to 4) until the termination condition is achieved. We have developed 4 versions to justify the methods. Version 1, 2 and 3 are different at step 5-a. Version 4 modified Version 3 on step 6. 4.1 Version 1 t
In this version, under no condition will the component N be updated. That is, the position and size of the sliding window and the number of the belief-cells it contains are all fixed in advance. In a sense, it is a "fixed" window rather than a "sliding" window. In the Belief Space, only the component C is updated in this version. 4.2 Version 2 This version contains a sliding window, in which the t
component N is updated in each generation t . However, no extra candidates or scouts are created within the updated sliding window to help acquire the constraint knowledge for the next generation. 4.3 Version 3 Here, when the position/size of the sliding window is updated ne extra candidates, called scouts, are randomly created within the new sliding window to help glean the constraint knowledge about the new area in advance. This is designed to facilitate the creation of a new constraint map because the old one disappears when the position/size of the sliding window is changed. However, the scouts only function to update the constraint knowledge, they don't generate offspring in this version. 4.4 Version 4 In this version, the scouts can also be parents to generate offspring. In step 6, there are nc+ne instead of nc parents; in step 8, nc candidates for next generation should be generated from 2*( nc+ne) instead of 2*nc.
5. Implementation and test results We now compare the different configurations on a benchmark nonlinear constraint optimization problem [Floudas 1990], which is given below:
Problem Description Min -12x-7y+y2 Domain constraints: 0 ≤ x ≤ 2, 0 ≤ y ≤ 3 Problem constraints: y ≤ -2x4+2 Global best point: x*=0.71751, y*=1.470 Global best value: -16.73889 Optimization goal: < -16.70 Each version was run 100 times, and the statistical results are shown in table 1. (each runs 100 times) Version 1 Version 2 Version 3 Version 4
Mean >100 generations 18.25 generations 14.9 generations 5.4 generations
(sample) Standard deviation … 10.87 8.09 1.58
Table 1 Statistical Results Evaluation: Version 1 is not very effective because the constraint knowledge provided by component C can't be refined due to the static nature of the window and fixed granularity of belief-cells here. It takes more than 100 generations in all cases. Version 2 does substantially better than Version 1. So, the addition of the sliding window generates a marked improvement in the performance of the system here. But there is still room for improvement since when the position/size of the sliding window is changed, there may be only small number of individuals that happen to reside in new sliding window, and hence can contribute to the updating of constraint knowledge component C. Version 3 takes three generations less to solve the problem on average when scouts are used to provide immediate window updates. Thus, while updating N makes more of a contribution to solving the problem than updating C, both make non-trivial contribution. Finally, the use of scouts to not only update knowledge about the new window and to focus the search with that window by generating new offspring produces a major improvement in performance over the use of a sliding window and non-reproductive scouts. It took far fewer generations and is very robust as suggested by the relatively small standard deviation in the number of generations taken.
6. Conclusions Intuitively speaking, the belief space in this paper maintains a sliding window with a navigable map in it. The constraint knowledge in the belief-cells constitutes a navigation map of the domain space. The experiments indicate a great potential behind this approach. The Cultural Algorithms using the belief-cell mechanism, which can represent, store and integrate
constraint knowledge as well as other region-based knowledge, provides a general tool to solve constraint optimization problems. The advantages of this mechanism include: • No/little constraint knowledge needed before optimization search, the necessary constraint knowledge can be learned during the evolutionary computation. • The capability to solve non-linear constraint optimization problems. In addition, this mechanism can be used to solve optimization problems with disjoint feasible regions as well. We also can gain some insights from the comparison of the different versions of the algorithm. Even though a precise region-based map in the belief space is not necessary, it seems that the map-refining mechanisms, no matter whether explicit or implicit, are necessary and useful. The point is that the knowledge-based evolutionary computation system needs to systematically acquire and update the region-based schemata as the region shifts. Allowing scouts to update as well as to focus the search for the next generation relative to a new window position is particularly useful. Further studies include developing a hierarchical knowledge-based EC system using region-based schemata. We also devote our efforts to more sophisticated belief-cells mechanism; testing other kinds of acceptance, influence and updating functions; and the identification of other regionbased aspects of the domain space.
Bibliography [Chung 1996] Chan-Jin Chung and Robert G. Reynolds, "The Use of Cultural Algorithms Support Selfadaptation in EP", Proc. Of 1996 Adaptive Distributed Parallel Computing Symposium, Dayton, Ohio, 1996 [Chung 1998]Chan-Jin Chung and Robert G, Reynolds, "CAEP: An Evolution-based Tool for Real-Valued Function Optimization using Cultural Algorithms", International Journal on Artificial Intelligence Tools, Vol. 7, No.3, 1998. [Eshelman 1992] L. J. Eshelman and J. D. Schaffer, "RealCoded Genetic Algorithms and IntervalSchemata", Proceedings of the Second Workshop on Foundations of Genetic Algorithms. Vail, Colorado, 1992. [Floudas 1990] C A. Floudas and P. M Pardalos, "A Collection of Test Problems for Constrained Global Optimization Algorithms", SpringerVerlag, 1990 [Fogel 1995] D. B. Fogel, "Evolutionary Computation: Toward a New Philosophy of Machine Intelligence", IEEE press, 1995
[Goldberg 1989] David E. Goldberg, Genetic Algorithms in Search, Optimization, and Machine Learning, Addison Wesley, 1989 [Holland 1975] John H. Holland, Adaptation in Natural and Artificial Systems, Ann Arbor, Michigan, University of Michigan Press. [Michalewicz 1996] Zbigniew Michalewize, Genetic Algorithms + Data Structures = Evolution Program, Springer, 1996 [Reynolds, 1986] Robert G. Reynolds, “An Adaptive Computer Model for the Evolution of Plant Collecting and Early Agriculture in the Eastern Valley of Oaxaca Mexico”, In Guila Naquitz: Archaic Foraging in Oaxaca, Mexico, K.V. Flannery, ed., San Diego, CA, Academic Press. [Reynolds 1993] Robert G. Reynolds and J. Maletic, "The Use of Version Space Controlled Genetic Algorithms to Solve the Boole Problem", International Journal on Artificial Intelligence Tools, Vol.2, No.2. [Reynolds 1996] Robert G. Reynolds and C. Chung, "A Self-Adaptive Approach to Representation Shifts in Cultural Algorithms", Proceedings of IEEE International Conference on Evolutionary Computation, Nagoya, Japan, 1996.
