Formal Descriptions of Real Parameter Optimisation Tao Gong and Andrew Tuson Abstract— The design of effective operators is a matter of some interest in the evolutionary computing community, and Radcliffe’s forma analysis is one notable approach to formally incorporate domain knowledge in operator design by manipulating the formal descriptions of problem domain. Since formal description is the key issue that affects the effectiveness of derived operator, this paper examines the concepts of Dedekind Cut and Isodedekind Cut introduced by Surry. Though they serve as very useful descriptions of the continuous domain, they have not been fully formalised. Some new concepts are developed or updated based on the original work and some ambiguous points (e.g. the derivation of operators with these formal descriptions) are also made clearer. A case study is also presented to illustrate the formal derivation of concrete operators with some predefined operator templates.
I. I NTRODUCTION The formal and rigorous design of effective operators is a matter of some interest in the evolutionary computing community. Forma analysis by Radcliffe [1] is one such approach that is based on a formal codification of beliefs about the nature and structure of the search domain using equivalence relations. Tuson [2] has successfully applied forma analysis to neighborhood search optimiser design from a knowledgebased system viewpoint. Some other specific applications of forma analysis have also been made for neural network topology [3] and gas supply network optimisation [4]. Thus, formalising appropriate descriptions of domain knowledge is of great importance to effective operator design. A rigorous study of the formal descriptions is necessary in the sense that the assumptions made on them can be formally represented in a more accurate manner, and therefor the behavior and the effectiveness of the derived operator can be justified. To analyze real-coded representations and operators for function optimisation problems in a more formal manner, Surry [5] introduced two concepts: namely Dedekind Cut and Isodedekind Cut equivalence relations. However, there are still many ambiguous points in this work which have not yet been formally clarified. The isodedekind cut was introduced in an informal manner and a formal definition was not presented. The feasibility-constraints implied by these formal descriptions were not realized and formally given in [5]. The derivation of operators with isodedekind cut was also given informally, and as this paper will show, additional concepts are needed to properly formalise this process. Tao Gong is with the Department of Computing, City University, London, EC1V 0HB, UK (phone: +44 20 7040 4048; email:
[email protected]). Andrew Tuson is with the Department of of Computing, City University, London, EC1V 0HB, UK (email:
[email protected]).
This paper aims to formally re-examine these concepts to address the above and provide a rigorous study of real parameter operators using forma analysis; some new concepts are also introduced to facilitate the study when necessary. In the following Section and , we briefly review the previous work in formal analysis and formal descriptions respectively. In Section , we present our formal study and development on these formal descriptions. A case study is then given in Section to illustrate how to formally derive useful operators with the help of our new concepts. Finally, Section gives some conclusions and suggests some potential future work on this issue. II. BACKGROUND T HEORY: F ORMA A NALYSIS Before the case study, it is necessary to understand the background theory of the derivation of operators: Forma Analysis [1]. Forma analysis is a formal but practical method that allows the problem representations and those operators working with respect to those representations to be structured in a formal manner by using equivalence relations. Such equivalence relations can divide the search space into disjoint equivalence classes (depending which value the solutions match), gathering solutions that are equivalent under a certain equivalence relation. The initial aim of forma analysis [1] was to codify knowledge of the problem domain using a set of equivalence classes (also known as formae) which is assumed to be able to cluster solutions with related performance in order to guide the search process more effectively, e.g. edges if we are considering the travelling salesman problem. Since equivalence relations/classes have the ability to capture the properties of solutions, the operators can thus be mathematically derived with regards to these equivalence relations to manipulate these properties in a formal way. Some of the characteristics and operator templates related to Forma Analysis are given below to facilitate our further discussion. For detailed mathematical definitions of all the other concepts, the readers can refer to the original work [1] and extensions due to Surry [5] and Tuson [2]. A. Describing the Search Space The key concept is that of a basis: a set of equivalence relations that allows us to properly describe the search space. Definition 1: (Basis) A subset of a set of equivalence relations is termed as a basis for the set of equivalence relations, if spans the set and is independent. Each equivalence relation, , also induces a set of equivalence classes , with individual equivalence classes being denoted by . An encoding can be derived by taking the image of the basis equivalence classes corresponding to a
particular solution in the search space. Feasibility constraints may exist between equivalence classes to ensure that only solutions that exist in the search space are represented (e.g. legal permutations in sequencing problems). B. Domain Independent Operators Forma analysis can then be used to derive operators that explicitly manipulate these equivalence relations. This is achieved by combining the basis above with domain independent operators that can be thought of as a template for specifying operator behavior in terms of an arbitrary basis. Two of these templates are key to the work presented in this paper. One such template corresponds to crossover in evolutionary algorithms: Random Transmitting Recombination (RTR). Definition 2: (Random Transmitting Recombination (RTR)) Given a basis for a set of equivalence relations over a search space , the random transmitting recombination operator with the control set
is defined to select a child solution out of the dynastic potential of the parent solutions and , formally as: RTR (1) where the actual child solution is chosen from the set above RTR
uniformly at random. RTR ensures that the child solution is constructed entirely of parental material (transmission). Mutation can be defined in a similar manner using the Binomial Minimal Mutation (BMM) template. Definition 3: (Binomial Minimal Mutation (BMM)) Given a minimal mutation defined as:
© where represents the distance measure under a certain
basis set , the binomial minimal mutation can be defined as: (2) BMM
where ( is the effective chromosome length) and is a stochastic mapping: that returns a random member of set © .
C. Forma Analysis in Practice The use of forma analysis can be informally illustrated by reference to the binary representation commonly used in the evolutionary computation literature. A set of basis features can be defined, , one for each of the bits in the problem where each feature can have an equivalence class from the set – the encoding is then just the image of this (a binary string). The distance metric for this feature set is then the number of bits that are different – the hamming distance. The BMM operator thus instantiated flips each bit in the solution with a fixed probability : the usual binary bitwise
mutation. In this case, RTR is equivalent to binary uniform crossover as each bit in the child solution may come from either of the parent solutions with a certain probability. In summary, given any defined characteristics of a certain problem domain, operator templates can be instantiated to effective search operators which in many cases are in use in the evolutionary computation community, such as blend crossover, line recombination crossover and Gaussian mutation [5] for real-parameter optimisation. This paper will formally show how these operators are derived. III. R EVIEW: F ORMAL D ESCRIPTIONS OF R EAL PARAMETERS To represent real parameters in a way that can be discretely manipulated and used in forma analysis, Surry [5] proposed two descriptions for real-parameter optimisation. This work is summarised here. A. Dedekind Cut With the motivation to describe real parameter optimisation with a “limiting sequence of discrete representations” [6], Surry introduced the dedekind cut description [5] to derive problem-specific forms of generalized genetic operators by using the sequence of discrete equivalence relations to approximate the continuous search space. Given a continuous search space
(3)
by introducing the concept of interval for the -th dimension, we are able to get a discrete approximation for the continuous space, with
where
(4)
(5)
Based on the above definition of the approximation for continuous space, Surry characterised dedekind cut defining cuts on as consisting of basic equivalence relations: if or (6)
otherwise.
where . Each of these equivalence relations induces two equivalence classes:
and
(7)
By introducing intersections to these equivalence classes, we can get set of features defining closed intervals. For example, given feature set , the intersection of these compatible formae is the closed interval . Based on the one-dimension definition of a dedekind cut, it is straightforward to extend dedekind cut to multiple dimensions, assuming that these dimensions are normal to each
other. For example, given an approximated two-dimension space , the equivalence relations can be defined as:
if or and or otherwise.
(8)
The feasibility-constraint enforces that, constrains all the equivalence relations to take the value , while constrains all the equivalence relations to take the value . In this sense, given the range of the real numbers to represent , all possible combination of values for dedekind cut can be presented as follows.
B. Isodedekind Cut To extend the dedekind cut to multiple dimensions with a more general characterization, Surry [5] further developed another formal description based on new equivalence relations. In dedekind cut, the cut directions are always along the axes, which was felt to be too “artificial” (indeed it makes an assumption about the orientation of the search space along the coordinate axes). Considering this, a general version of dedekind cut, namely Isodedekind Cut (i.e. Isotropic Dedekind Cut) was introduced to divide the space with all arbitrary orientations (thus removing the bias). With this definition, the search space description consists of a continuous infinite number of equivalence relations/classes as all possible orientations will be considered; this was the extent of the formalisation conducted [5].
IV. A F URTHER S TUDY OF F ORMAL D ESCRIPTIONS Although the formal descriptions in the original work [5] were illustrated to be very useful in informally producing “natural” operators by considering its limiting behavior when the level of approximation increases, some of the definitions and derivations are still ambiguous or not given with proper formality. For example, the feasibility constraints implied by these formal descriptions were not formally given. Since the isodedekind cut description was only informally introduced, the formal definition and the derivation of operators with isodedekind cut were not given rigorously. Making clear these issues will be beneficial for our future utilization. A further examination of these formal descriptions will be given in this section together with some new concepts. A. Feasibility Constraints Given the formal definition of dedekind cut in [5], it is straightforward to realized that there are “feasibilityconstraints” among these equivalence classes which guarantee the feasibility of solutions. For example, and are incompatible formae for a feasible solution, since they define an infeasible interval which contradicts the truth that . To facilitate our understanding of the relationship between dedekind cut equivalence relations, we formally present the definition of feasibility-constraints as follows. Definition 4: (Feasibility-constraint) Given (where ), the feasibility-constraint of dedekind cut description can be formally defined as:
if otherwise
(9)
.. .
.. .
.. .
..
.
Besides the feasibility-constraints for dedekind cut, there also exist inter-basis constraints for isodedekind cut which will be discussed into details after clarifying some concepts of isodedekind cut. B. Formal Definition of Isodedekind Cut For the isodedekind cut description, the formal definition can be given with the help of specifications of hyper cuttingplanes. Particularly for one-dimensional problems, dedekind cut is equivalent to isodedekind cut, since no direction is considered. Definition 5: (Isodedekind Cut) The equivalence relations (characterization) for isodedekind cut in a dimensional space can be formally defined as:
if or
otherwise.
(10)
where represents an isodedekind cut over the dimensional space, which is actually a hyper cutting-plane defined by parameters (as a single point with an orientation can fix an unique cutting-plane in a dimensional space which is vertical to the orientation). Remark 1: This cutting-plane can divide the space into two parts: above and below the hyperplane. The property of any point which is above the cutting-plane is , and is the property for any point that is below the cutting-plane. C. Distance Measure for Dedekind Cuts In some cases, the distance between candidate solutions needs to be measured for the algorithm to operate, such as the difference vectors in Differential Evolution [7]. It is thus necessary to introduce the concept of “distance” to facilitate our analysis work. For the consideration of simplicity, we prefer to use the term “Forma Distance” to represent the distance measure based on Forma Analysis [1], [6]. Definition 6: (Forma Distance) Forma distance is defined as the weighted sum of the number of equivalence relations for all dimensions where two individuals are different,
formally as:
(11) where represents the -th equivalence relation defined for the -th dimension over the equivalence basis for the -th dimension , while and respectively represent the number of dimension and the number of equivalence relations defined for each dimension; represents the weight for the distance calculated based on . Generally speaking, the weight for each is set to by default. However, for dedekind cut and isodedekind cut, the weight is related to the defined interval for the description of the continuous space in a way that larger interval corresponds to larger weight and vice versa. However, the concepts of interval and weight are ambiguous in the context of traditional isodedekind cut in [5], since the orientation of isodedekind cut is arbitrary. Some new concepts will be introduced later to make the understanding of distance measure of isodedekind cut more straightforward. The calculation of distance is ready-to-adopt for dedekind cut with the definition of forma distance as the weighted sum of the number of different equivalence relations over all dimensions will be considered. The weight for each equivalence relation is simply the defined interval for each dimension . So, forma distance reduces to Manhattan Distance for dedekind cut. D. A Problem: Distance Measure for Isodedekind Cuts For isodedekind cut, the distance measure is still not yet clear, because it induces an infinite number of equivalence relations/classes with an infinite number of positions and orientations . In such an ambiguous situation with infinite bases, it is hard to decide the forma distance, as:
A
not applicable undefined where , respectively represent the number of possible positions and orientations for isodedekind cuts, and is related to with defined weight .
This infinite property of isodedekind cut becomes a real difficulty when we calculate the distance between solutions. E. Some New Concepts for Isodedekind Cuts
To address the above problem, we introduce the following concepts for the isodedekind cut description. The first concept is the “Compatibility” of two bases. However, it is worthy of being mentioned that the concept of compatibility is also used for formae in a way that compatible formae construct feasible solutions. By adapting the concept of compatibility to basis, we can define Basis Compatibility as follows. Definition 7: (Basis Compatibility) A basis is said to be compatible with another basis over the same space ,
2
1 Difference Vector Projection
C 3
Fig. 1.
B
Coordinate-induced Basis
if and only if, under and , the distances between any two individuals and in the space are the same, formally as:
(12) where basis represents that is compatible with . ¼
Then, we can define the term “Forma Coordinate”, notated as . Definition 8: (Forma Coordinate) A coordinate fixed by together with a vector an isodedekind cutting-plane which is vertical to it ( ) is defined to be a forma coordinate, formally as . With the definition of forma coordinate, the basis induced by a forma coordinate is defined as follows. Definition 9: The basis induced by a forma coordinate is the set of isodedekind cut-planes that are parallel to or vertical to the projection of the distance (more specifically speaking, the difference vector) onto with a certain decided interval . In Figure 1, given that is the difference vector (where the distance needs to be measured) and is the projection of on the isodedekind cutting plane ! , cutting planes ! and ! are possible candidates for the basis induced by forma coordinate ! , since they are both vertical to the projection of the difference vector . In this sense, the basis induced by a forma coordinate can be divided into two categories of isodedekind cutting-planes: and . Based on the definition of parallel forma coordinates, we can give a possible conclusion about the basis induced by a forma coordinate as follows. Definition 10: Two forma coordinates and are defined to be parallel, if is parallel to (or is parallel to ) where and are the isodedekind cutting-plane to induce and . Theorem 1: Provided equally distributed intervals ( and
) imposed on two parallel forma coordinates ( and ) with identical difference vectors (" and " ) considered to induce bases ( and ) from forma coordinates, if there exist two identical isodedekind cutting-planes and on each category ( ) of and , their induced bases and are compatible, formally
( ) which is the forma coordinate induced by the principal cut, and is defined as:
Principal CutCut Principle
Y
B
PrincipleAxis Axis Principal
Since we are able to obtain a basis if given any coordinate with a fixed interval (refer to the aforementioned “basis induced by a forma coordinate”), among all possible bases which can be induced by ( ), we formally define the Principal Basis as follows. 11: (Principal Basis) Given a difference vector Definition and the set of all possible bases with the same defined interval
A
X
Z
Fig. 2.
Principal Cut and Principal Axis in a ¿-dimensional Space
as:
" "
(13) Proof: The proof is quite straightforward as, given that
for all with the certain offset from
, there also exists a isodedekind cutting-plane ·´ µ identical to , with a offset from .
Then, it is easy to find that those isodedekind cuts that are not parallel or vertical to each other can be categorized into different incompatible bases. Furthermore, by rotating through a fixed point with an arbian isodedekind cut trary angle # (thus rotating the forma coordinate ), we are able to obtain infinite number of bases if the interval and the difference vector are given. F. A Distance Measure for Isodedekind Cuts: Principal Cuts As noted already, it is hard to define forma distance under the context of isodedekind cut where orientations are arbitrary and infinite, thus a “composite basis” needs to be built to carry out forma-based distance measure. To fulfill the task of measuring forma distance for isodedekind cut description, the Principal Cut together with some other related concepts will now be introduced. As shown in Figure 2, given the difference vector (distance to measure) in a -dimensional space, the Principal Cut is defined to be any cutting plane that is vertical to the difference vector . The Principal Axis is defined . It will be straightforward for us to along with vector conclude that for each point on vector , there exists only one principal cut. By defining intervals along the principal axis, we are able to discretely represent the and , formally as: “absolute” distance between
where represents the distance between
(15)
(14)
and along the (p)rincipal axis, and represent the positions of and respectively in the Principal Coordinate System
a basis of isodedekind cuts with the same interval is defined as a principal basis , if the following condition is met.
(16) It is also straightforward to build the relationship between principal basis and principal coordinate system with a defined interval , as follows. Theorem 2: Provided a difference vector to be considered, a principal coordinate system together with a interval defines an unique principal basis . Proof: Given a principal coordinate system and a set of forma coordinate systems with all possible orientations
with the same interval we are able to obtain the principal basis and a set of bases which is induced by the set of all possible forma coordinates .
(17) in the space Assume there exists a difference vector with an arbitrary forma coordinate system with axes " " and a principal coordinate system with axes " " as shown in Figure 3. In the limiting case (if considering that the weight equals to the interval ), it is easy to see that the forma distance reduces to the “absolute” length of vector in the principal coordinate system , while the forma distance in the arbitrary forma coordinate system reduces to . From (basic property of triangle), it is straightforward to get
However, the above is only the limiting case proof when we are considering the distance in a geometry sense. If returning to the general case of forma distance, by dividing the difference vector into several discrete intervals, we can still
dk2
dp2
B
dp1
dk1 A
C
Fig. 4. Space
Fig. 3.
Forma Coordinate Systems in a ¾-dimensional Space
calculate that the forma distance in the principal coordinate system is:
while the forma distance in an arbitrary forma coordinate system is:
From
, we can still prove
, which means that the basis induced by the principal coordinate system together with a certain interval is a principal basis where the forma distance is minimized. With principal basis defined, since there only exists a difference along the principal axis, the distance measure is now straightforward for isodedekind cut description as forma distance actually reduces to Euclidean Distance: the straightline distance between the two solutions. It is also straightforward to realize that the principal basis is independent from the coordinate system adopted in the algorithm, since the principal basis is self-induced by the difference vector. Thus, the behaviors of the operators derived from isodedekind cut are intrinsically invariant with any change of the coordinate system (such as rotation of the coordinate system). G. Integrity Constraints Given the above definitions, we can introduce the concept of “Integrity Constraints” for isodedekind cut. Basically, integrity constraints are defined over different forma coordinates to enforce basis compatibility. Definition 12: (Integrity Constraint) Given that one of the forma coordinates is defined to induce the principal
Illustration of Forma Coordinate Projection in a ¾-dimensional
basis, the integrity constraint ensures that this forma coordinate specifies (constrains) the values of the other forma coordinates as the projection of that solution on the other forma coordinates. For example, if considering a -dimensional space, given the angle between and is #$ , and the offset from to is , the integrity constraint can be formalised as:
Proj
% %
(18)
where Proj represents the projection of solution in forma coordinate onto another coordi nate and % ½¾ . Figure 4 illustrates briefly how to calculate the projection. From the above, we are able to see that the coordinate value in forma coordinate is fully specified by the projection from the defined forma coordinates which induces the principal basis. V. A C ASE S TUDY OF R EAL PARAMETER O PERATORS In the previous work [5], most of these derivations were given in a quite informal manner (except for deriving BMM with dedekind cut). Especially, as isodedekind cut description has an infinite number of formae, the derivations of operator templates with isodedekind cut were either ignored or done quite informally, e.g. the instantiations of BMM and RTR with isodededekind cuts (in fact, Surry [5] illustrated the derivation from RTR to line recombination operator by illustrating the limiting case with a figure). In this section, we will present a case study about the formal derivation of concrete operators for real parameter optimisation with the help of our updated version of formal descriptions in a more rigorous manner.
A. Derivation of Mutation Operators For the BMM operator template, a binomial distribution
is used to decide the number of minimal mutations
taken to perturb a solution. With dedekind cut description, Surry presented rather precise work on BMM, showing that as the quantization becomes finer, the operator template with binomial distribution reduces to Gaussian Mutation [8] which is well-known in EA community. The derivation will not be repeated here and the advanced readers are directed to the original work [5]. With the isodedekind cut description, an arbitrary principal axis will be chosen among all possible axes (as shown in Figure 5), so that the derivation of BMM with isodedekind
Fig. 5.
Possible Principal Axes
cut can be regarded as the derivation of BMM with an arbitrary rotated single-dimension dedekind cut. In other words, we can instantiate BMM with isodedekind cut to a Gaussian Mutation applied along an arbitrary principal axis in a similar manner as dedekind cut description. To observe their behaviors, we simulate their behaviors in a -dimensional space, as shown in Figure 6. On the left (I) of Figure 6, to simulate the behavior of the operator derived by dedekind cut description, Gaussian mutation & is applied separately on each dimension with a sampling size of to mutate the vector . On the right (II) of Figure 6, to simulate the behavior of the operator derived by isodedekind cut description, Gaussian mutation & is then applied along uniform random orientations ' # # with a sampling size of to mutate the vector . Although derived from two formal descriptions with different assumptions, the emergent behaviors of the derived mutation operators are quite similar. To understand this bet-
ter, we can imagine that for isodedekind cut derived mutation all the Gaussian mutations are first applied along a single direction. Then, by rotating the principal basis randomly for all possible directions, the points along the single direction will be equally spread in the space resulting a sphere-like behavior, which is rotationally invariant. However, the different assumptions made for dedekind cut and isodedekind cut can result in behaviors that are intrinsically different. Because the emergent behavior of dedekind cut derived mutation is sphere-like only if the range of parameter for all dimensions are the same. Otherwise, the emergent behavior of dedekind cut derived mutation will be ellipsoid-like in the most general case, which is not rotational invariant. Even in the special case for dedekind cut as shown in Figure 6, the observant reader may find that the spread of the dedekind cut derived mutation is slightly larger than that of the isodedekind cut derived mutation. This is simply because applying BMM along arbitrary principal axis results in a smaller step size projected on the normal coordinate axes, which depends on the angle between the principal axis and normal axis. B. Derivation of Crossover Operators RTR will be used here to carry out our study of crossover operator template. With dedekind cut description, RTR generates the offspring by taking equivalence class (value) from one of the parents uniformly at random for each equivalence relation over each dimension. Given two number and
as the parents within boundary , with dedekind cut description can be represented as while
can be represented as
If we assume , then . According to the definition of RTR, forma
are safely passed to the offspring . From this, we can see that by changing the value taken for each position in
we are able to generate numbers within the interval
By considering the feasibility constraint of dedekind cut, we can find that there are possible combinations, as follows.
Fig. 6. Space
The Behaviors of Derived Mutation Operators in a ¾-dimensional
.. .
.. .
.. .
..
.
VI. C ONCLUSION
Fig. 7.
The Unique Common Principal Axis Between Two Parents
As an uniform random option will be chosen from the above combinations, we can obtain a uniform random number within the interval for the offspring. For example, given parent described as and parent described as , a possible offspring can be which is generated between with an uniform distribution. As the quantization becomes finer, a uniform random intermediate value will be generated between the parents’ values for each dimension, which makes the derived operator a Blend/Intermediate Crossover [9] commonly adopted in EA literature for real-parameter optimisation. For isodedekind cut, although there are an infinite number of principal axes for each parent, there exists an unique common principal axis between the two parents where the forma distance between the two parents is minimized, as shown in Figure 7. From Figure 7, we can find that the common principal axis is effectively the connecting line between the two parents. As the direction of the principal axis is defined along the line connecting the two parents (similar to a difference vector), a uniform random intermediate point will be selected along this line in a similar manner as how intermediate crossover is generated with dedekind cut. Thus, we obtain the Line Recombination Operator [10] which is also a widely used operator in the EA community for continuous domain. It is well-known [11] that Intermediate Crossover is not a rotationally invariant operator, because rotation of the coordinate system relocates the hypercube’s corners generated by crossover which in turn redefines the area that Intermediate Crossover searches. However, the Line Recombination Operator is rotationally invariant because the line recombination of parents is not affected by the rotation of coordinate system. From the observation above, we can find that the assumptions we make for dedekind cut and isodedekind cut are both reflected in the behaviors of their derived operators, because dedekind cut is defined for each dimension separately which relies on the definition of coordinate system, while isodedekind cut is defined with arbitrary orientations and only the principal basis is considered for distance measure.
In this paper, we have presented a study of the formal descriptions of real-parameter optimisation. The previous work is examined and some ambiguous points and potential problems are resolved to produce a rigorous and fully formal description of real parameter optimisation operators for evolutionary algorithms. With the new concepts and definitions introduced here (such as principal basis and feasibility-constraints), dedekind cut and isodedekind cut can be used to derive useful operators more accurately for real-parameter optimisation and the behavior of the derived operators can also be formally analyzed in accord with the assumptions introduced by the formal descriptions. Particularly for isodedekind cut description, the formal derivation of “natural” operators becomes straightforward with these new concepts, and clear insights into the different assumptions that these operators embody were obtained. In future research, it would be beneficial to either discover new properties of these formal descriptions for real parameters or apply them to some other optimisation techniques (especially for those not originally designed for numerical optimisation problems) to derive useful operators to manipulate real numbers effectively. R EFERENCES [1] N. Radcliffe, “The algebra of genetic algorithms,” Annals of Maths and Artificial Intelligence, vol. 10, pp. 339–384, 1994. [2] A. Tuson, “No optimization without representation: a knowledge based systems view of evolutionary/neighbourhood search optimization,” Ph.D. dissertation, University of Edinburgh, Edinburgh, 1999. [3] N. Radcliffe, “Genetic set recombination and its application to neural network topology optimisation,” Neural Computing and Application, vol. 1, no. 1, 1993. [4] P. D. Surry, N. J. Radcliffe, and I. D. Boyd, “A Multi-Objective Approach to Constrained Optimisation of Gas Supply Networks : The COMOGA Method,” in Evolutionary Computing. AISB Workshop. Selected Papers, T. C. Fogarty, Ed. Sheffield, U.K.: Springer-Verlag, 1995, pp. 166–180. [5] P. D. Surry, “A prescriptive formalism for constructing domain-specific evolutionary algorithm,” Ph.D. dissertation, University of Edinburgh, Edinburgh, Scotland, UK, 1998. [6] N. Radcliffe, “Equivalence class analysis of genetic algorithms,” Complex Systems, vol. 5, no. 2, pp. 183–205, 1991. [7] R. Storn and K. Price, “Differential evolution – a simple and efficient adaptive scheme for global optimization over continuous spaces,” ICSI, Tech. Rep. TR–95–012, March 1995. [8] T. B¨ack and H. P. Schwefel, “An overview of evolutionary algorithms for parameter optimization,” Evolutionary Computation, vol. 1, no. 1, pp. 1–23, 1993. [9] L. J. Eshelman and J. D. Schaffer, “Real-coded genetic algorithms and interval schemata,” in Proceedings of the Second Workshop on Foundations of Genetic Algorithms (FOGA). Vail, Colorado, USA, July 26-29, L. D. Whitley, Ed. Morgan Kaufmann, 1992, pp. 187–202. [10] Z. Michalewicz, Genetic Algorithms + Data Structures = Evolution Programs. Berlin, Germany: Springer, 1996. [11] K. V. Price, R. Storn, and J. Lampinen, Differential Evolution: A Practical Approach to Global Optimization. London, UK: SpringerVerlag, 2005.
