A New Performance Metric for Multiobjective Optimization: The Integrated Sphere Counting Vin´ıcius L. S. Silva, Elizabeth F. Wanner (Member, IEEE) , S´ergio A. A. G. Cerqueira, Ricardo H. C. Takahashi (Member, IEEE) Abstract— A large number of evolutionary algorithms for solving multiobjective optimization problems has been already developed. Several merit factors for comparing the outcomes of these algorithms have also been proposed. However, evaluating Pareto-surface sample sets is still considered an open problem, since the result of a multiobjective evolutionary algorithm is a collection of vectors forming a nondominated set, that can be viewed under rather different merit criteria. In this paper, we present a new performance metric: the Integrated Sphere Counting. This metric is motivated on two reasoning principles: (i) the Pareto-surface is an object that is to be described via sample sets, in a sense that is similar to the sampled function description in signal processing; and (ii) the resolution that is to be employed in the Pareto-surface sample set depends on the decision-making procedure resolution, instead of the surface structure itself. We test this metric with two benchmark problems: the 0/1 Knapsack Problem and ZDT number 6 test suite.
I. I NTRODUCTION Evolutionary algorithms (EAs) have become a main tool for solving multiobjective optimization problems, due to their capability of dealing with rather generic functions while generating, in a single execution, an entire set of Pareto-set estimates [1], [2]. Over 500 publications [3] have proposed, years ago, various multiobjective evolutionary algorithms implementations and applications. The proliferation of multiobjective evolutionary algorithms boosts the importance of performance comparison issues. In absence of any established comparison criteria, none of the different sets of estimates for the Pareto-optimal solutions generated by the different available algorithms can be said to be better than the other ones. Assessing and comparing the performance of heuristic search methods is always difficult because performance is a multidimensional attribute: solution quality, computational effort and robustness may be all important [4]. The solution quality, by itself, has also several dimensions, as a result of the multidimensional nature of the Pareto front. Due to these difficulties, it has been common to indicate the performance Vin´ıcius L. S. Silva is with the Department of Electronics Engineering, Universidade Federal de Minas Gerais, Belo Horizonte, Brazil (email:
[email protected]). Elizabeth F. Wanner is with the Department of Mathematics, Universidade Federal de Ouro Preto, Ouro Preto, Brazil (email:
[email protected] ). Sergio A. A. G. Cerqueira is with the Department of Mechanical Engineering, Universidade Federal de S˜ao Jo˜ao Del Rei, S˜ao Jo˜ao Del Rei, Brazil (email:
[email protected]). Ricardo H. C. Takahashi is with the Department of Mathematics, Universidade Federal de Minas Gerais, Belo Horizonte, Brazil (email:
[email protected]).
of any algorithm by simply plotting the nondominated solutions. However, this cannot be accepted on its own due to the lack of meaningful quantitative interpretation for expressing the quality of the Pareto front. Usually the quality of a Pareto optimal set can be assessed from three aspects: • • •
These three aspects cannot be measured adequately with a single performance metric: various performance metrics for measuring the quality of a Pareto-optimal set have been proposed to compare the performance of different multiobjective algorithms. Some of these metrics rely on knowing the entire true Pareto front; other ones give an absolute value that summarizes the quality of a nondominated set without reference to any other reference set; while other metrics can be used only to compare two or more nondominated sets. Generational Distance, Error Ratio and Maximum Pareto Front Error are popular metrics that are easy to calculate but rely on the knowledge of the true Pareto front. One metric that has been favored by many people is the S Metric [8]. It calculates the hypervolume of the multi-dimensional region enclosed by a given set and a reference point, hence computing the size of the region that the set dominates. Despite the popularity of the S Metric, it has some problems: The S Metric requires defining an upper boundary (the reference point) of the region within which all feasible points will lie and this choice affects the ordering of nondominated sets. Moreover, it has a large computational overhead, rendering it unusable for many objectives or large sets. Reviews of those performance metrics can be found in [5], [6], [7]. In this paper, we propose a new performance measure, the Integrated Sphere Counting (ISC). This metric is a further development from a simple performance measure that has been proposed in [9] and that receives a re-interpretation here. ISC is a very simple measure for which the computational complexity of its evaluation is not significantly affected, neither by the number of objectives nor by the size of the Pareto optimal set. The proposed ISC metric is also motivated on a re-interpretation of the usual “distribution and spread of the solutions”, in terms of two concepts that are usually employed in the field of signal processing: • •
3625 c 1-4244-1340-0/07$25.00 2007 IEEE
the number of Pareto optimal solutions in the set; the closeness of the solution to the theoretical Pareto front; the distribution and spread of the solutions.
The surface sampling extension; The surface sampling resolution.
Differently from the signal processing traditional view of a sampling process, the sampling interval to be used in ISC is not related only to the Pareto-surface structure: the main guide for defining such interval is the resolution to be used in the decision-making process. II. T ERMINOLOGY A general multiobjective optimization problem can be stated as: min y = f (x) = (f1 (x), f2 (x), · · · , fm (x)) subject to: x = (x1 , x2 , · · · , xn ) ∈ Fx ⊂ X where x ∈ X is called the decision vector, X is the optimization parameter space, f ∈ Y is the objective vector, Y is the objective space, and the set Fx is the feasible set (this set satisfies the problem constraints). Let the vector f be described component-wise by f = (f1 , f2 , · · · , fm ), and let the set Fy ⊂ Y represent the image set of region Fx for the mapping f (·) : X → Y . The set of solutions of a multiobjective problem consists of all decision vectors for which the corresponding objective vector cannot be improved in any dimension without degradation in another one. This set of solutions is known as the Pareto-optimal set. More formally, given two decision vectors, a and b, f (a) is said to dominate f (b) if and only if ∀i ∈ {1, 2, · · · , m} fi (a) ≤ fi (b) ∧ ∃j ∈ {1, 2, · · · , m} fj (a) < fj (b) All decision vectors that are not dominated by any other decision vector of a given set are called nondominated regarding this set. A Pareto-optimal solution is a vector x ∈ Fx that is not dominated regarding to the feasible set Fx . The Pareto-optimal set is the set of all Pareto-optimal solutions. Another concept, the -dominance, is stated as: An ob1 ) is said to -dominate jective vector f 1 = (f11 , f21 , · · · , fm 2 2 2 ) , if and only another objective vector f = (f1 , f22 , · · · , fm if fi1 ≤ fi2 ∀i = 1, . . . , m for a given > 0. Loosely speaking, a vector f 1 is said to -dominated another vector f 2 , if we can multiply each coordinate of f 2 by a factor of and the resulting vector is still dominated by f 1 . All decision vectors that are not -dominated by any other decision vector of a given set are called -nondominated regarding this set. The plot of the objective functions whose nondominated vectors are in the Pareto optimal set is called the Pareto front. III. T HE I NTEGRATED S PHERE C OUNTING The Integrated Sphere Counting (ISC) metric proposed here is a non-cardinal performance metric since no knowledge of any reference set (or reference point) is necessary. It is an unary quality metric, i.e., this metric assigns each approximation set a number that reflects a certain quality aspect, in this case a mix of the “sampling density” and the
3626
“sampling extension” of the approximation set. Besides, the addition of a point in a nondominated set never degrades the evaluation of this set concerning with the Integrated Sphere Counting metric (this property is known as weak monotony property). The Integrated Sphere Counting is derived from the Sphere Counting metric proposed in [9]. The Sphere Counting was a simple metric that was intended to measure the “sampling extension” under a given “sampling density” of a Pareto front. However, it was dependent on an external parameter: the radius of the spheres, that was related to the sampling density. With the new proposed procedure, the influence of this external parameter on the Integrated Sphere Counting will disappear. In this section, we will explain all the steps to construct the methodology for calculating the Integrated Sphere Counting (ISC). A. The Sphere Counting metric: a signal processing interpretation The Sphere Counting metric proposed in [9] is first presented here under a new viewpoint: a signal processing interpretation. Firstly, we should normalize all the objective vectors f i , i = 1, · · · , m into the interval [0, 1]. After the normalization, the maximum distance amongst the points is given by √ R = m, where m is the number of objective functions of the problem. The following procedure was employed for calculating the Sphere Counting metric: 1) Obtain an estimate of the Pareto front; 2) Define a radius r in the objective space; 3) Place a sphere of radius r centered at any point of the estimate set; 4) Initialize a sphere count as equal to one; 5) Exclude all points which are located within this sphere; 6) Among the remaining points, center the sphere at the point which is closest to the previous center and increment the sphere count; 7) Go to step 5 until there is no more remaining points in the estimate set. Through this measure, it is possible to identify the front with greater spread, which is simply that one with more spheres. However, the number of spheres is dependent on the selected radius r. The figure 1 shows the influence of the choice of the radius on the number of spheres. It is interesting to relate the problem of describing the Pareto-set surface, that appears in the field of multiobjective optimization, with the problem of signal reconstruction, that appears in the field of signal processing. In signal processing, a finite-bandwidth signal can be exactly reconstructed from a set of samples, provided that: • •
The sampling frequency is at least twice the maximum signal frequency, and The samples cover a whole period of the signal.
2007 IEEE Congress on Evolutionary Computation (CEC 2007)
•
45
40
35
30
•
25
20
15
10
Using a radius greater than this value, would lead to a loss in surface description: the smaller details would not be captured by such sample set. This effect can be called under-sampling. Using a radius smaller than this value would lead to sample sets that would be redundant for surface description. In such case, the cardinality of the sample set would not be related to the surface estimate quality: it would rather reflect the information redundancy. This effect can be called over-sampling.
5
0
0
0.2
0.4
0.6
0.8
1
1.2
1.4
Fig. 1. This figure shows the influence of the radius on the Sphere Counting measure on a generic Pareto optimal set. The horizontal axis represents the value of the radius r and the vertical axis represents the number of spheres.
Due to several issues, it is often convenient to deal with more samples and higher sampling frequencies, for reconstructing empirical signals. The underlying structure of this procedure, that is to be extrapolated to Pareto-set evaluation, is: • The ability to represent the “details” of a signal is related to the distance between samples that is adopted. Smaller distances (which mean higher sampling frequencies) allow a better description of signal details. More details mean that variations of the signal occurring in a smaller scale become visible. After reaching the distance between samples, that is suitable for representing the signal, further reducing this distance is not useful (the signal representation will remain the same). • The ability to cover the whole signal extension is related to the product of the number of samples times the sampling period. If the distance between samples is small, more samples will be necessary for covering the signal extension. After reaching the whole signal extension, no more samples are necessary for representing the signal. In terms of this last “rough” interpretation of signal reconstruction, a merit factor of a set of samples of a Pareto-set could be established with the intent of measuring the ability of that sample set to reconstruct the “true” Pareto-set. This is the idea behind the Sphere Counting measure, in which the sphere radius has a role that is analogous to the role of the sampling period. Following this interpretation, the sphere counting measure can be thought as: • Given a Pareto-set surface, there is a maximal sphere radius that can be associated to sample sets that still describe the surface, in the sense that a radius greater than this value would be associated to sample sets that would not catch the variations in surface curvature. • Using exactly this maximal radius, the potential of the sample set for producing a high quality of surface description would become assured: all surface details would be accessible to such set. The cardinality (the number of samples) of such set would be related to the effective surface estimate quality.
Although the Sphere Counting metric seems to be interesting for measuring the Pareto-surface estimate quality, in this sense of surface description, there is a main difficulty in its application: choosing a radius is a very difficult task, that is strongly problem-dependent. Such choice involves some experiments, that lead to the following empirical trade-off: The higher the value of the radius, the smaller the number of spheres. For a high value of the radius r, few members of the Pareto front are evaluated and a small variation on the position of these few points creates a high percent difference on the sphere counting. On the other hand, for a small value of the radius r, the value of the sphere counting converges to the exact number of points on the Pareto front, favoring the Pareto front with a high number of points. There is no clear criterion for defining a final choice. B. Decision-making relevant Pareto-set description To overcome the difficulties of Sphere Counting metric, instead of choosing a single radius r, we propose here a further motivation that can be used for defining a Paretoset merit factor. The idea is to relate the signal processing reasoning, that was behind the SC metric, to the endapplication of a Pareto-set sample set: a decision-making procedure. The points to be noticed are: (a)
(b)
For the purpose of decision-making, a very small scale of surface description is useless. Even if a mathematical description of a surface reveals, for instance, variations in curvature that are of the order of one thousandth of surface range, these variations would be irrelevant in a decision-making procedure over such surface. Also, for decision-making purposes, a very large scale of surface description would be uncomfortable. For instance, even a linear surface (that could be described by end-points only) is better suited for a decision-making process when sampled with some samples that cover its range, with intermediate points being sampled.
These observations allow to define a range of sphere radius that can become problem-independent, being instead related to the dynamics of decision-making procedures. This dynamic is relatively problem-independent, being mostly determined by the decisor structure. This reasoning leads to define here an interval for the radius variation. The idea is to cope with a scale range
2007 IEEE Congress on Evolutionary Computation (CEC 2007)
3627
that seems to be relevant, under the viewpoint of decisionmaking. The maximum value for r is suggested to be chosen as 10% of the maximum distance R among the points in the objective space, in order to deal with observation (b) above, and the minimum value for r is suggested to be chosen as 1% of this range, in order to deal with observation (a). The proposed procedure is: to integrate the sphere counting within this interval. The resulting Integrated Sphere Counting metric is expect to represent the Pareto-set evaluation under the several scales that are relevant for decision-making purposes. C. Computation of Integrated Sphere Counting With the sphere counting calculated for each radius in the pre-determined interval, we can obtain a merit factor for the Pareto front. This factor represents the numerical integral of the number of spheres for each radius. The higher the value of this integral, the better the spread of the Pareto front. The algorithm which describes the Integrated Sphere Counting procedure is shown in the algorithm 1. The symbol nr is the number of radius for which the sphere counting is applied, nb is a vector containing the number of spheres and rd is the vector of radius to be evaluated.
We compare the performance of the Integrated Sphere Counting metric on two different benchmark problems: the knapsack problem and the ZDT number 6 test suite. These two problems are understandable, easy to formulate and represent a certain class of real-world problems. Besides, due to their practical relevance, these problems have been subject to several investigations in the evolutionary multiobjective computation field. The problems are described below. 1) 0/1 Knapsack Problem A 0/1 Knapsack problem consists of a set of items, weight and profit associated with each item, and an upper bound for the capacity of the knapsack. The goal is to find a subset of items which maximizes the total of profits in the subset, yet all the selected items fit in the knapsack, i.e, the total weight does not exceed the given capacity [10]. This is a single objective problem, although it can be transformed into a multiobjective problem by allowing an arbitrary number of knapsacks. Given a set of n items and m knapsacks, the multiobjective knapsack problem is then defined by: max ⎧f (x) = (f1 (x), f2 (x), ·n· · , fm (x)) ⎪ ⎨ ∀i ∈ (1, 2, · · · , m) : wi,j × xj ≤ ci st: j=1 ⎪ ⎩ n x = (x1 , x2 , · · · , xn ) ∈ {0, 1}
Algorithm 1 Pseudocode for the ISC Algorithm Normalize fi , for i = 1, · · · , m; for i = 1 to nr do nb (i) = Sphere Counting(rd (i)) end for Do Numerical integration for (nb , rd ).
where fi (x) =
Although the procedure formally involves a numerical integration, this in fact can be performed exactly via a finite and small sequence of summations, because the function that is being integrated is piecewise constant. An important property of a performance metric is the computational cost derived from the evaluation of the metric. We can construct a relation describing the worst case complexity of the ISC algorithm by considering the operation of each step. Each step starts with a list of n points in m objective points. In the worst case, in the ISC procedure, it is necessary to compare the distances between each pair of points, which gives a complexity of the order . O n∗(n−1) 2 Thus, the ISC algorithm complexity is given by: n ∗ (n − 1) O ∗ (1 + nr ) . 2
(1)
IV. C OMPARISONS Performance metrics have been introduced to compare the outcomes of multiobjective algorithms in a quantitative manner. The simple comparison methods would be to check whether one outcome is better than another. The key question when designing performance metrics is how to best summarize the estimate sets by means of a few characteristic numbers [7].
3628
n
(2)
pi,j × xj and xj = 1 iff j is selected,
j=1
and • • •
pi,j being the profit of item j according to knapsack i, wi,j the weight of item j according to knapsack i and ci the capacity of knapsack i.
In this work, we use 2 knapsacks. This configuration produces a bi-objective knapsack problem. 2) ZDT6 ZDT6 is a part of six test functions suggested in [11]. This test function presents two difficulties caused by the non-uniformity of the search space: the Pareto optimal solutions are non-uniformly distributed along the global Pareto front (the front is biased for solutions for which f1 (x1 ) is near one) and the density of the solutions is lower near the Pareto front and higher away from the front. The ZDT6 problem is defined below: min(f ⎧ 1 (x1 ), f2 (x)) ⎨ f2 (x) = g(x2 , · · · , xn )h(f1 (x1 ), g(x2 , · · · , xn )) st: ⎩ xi ∈ [0, 1] , i = 1, 2, · · · , 10 (3) where
2007 IEEE Congress on Evolutionary Computation (CEC 2007)
f1 (x1 ) = 1 − exp(−4x1 ) sin6 (6πx1 )
9600
f2 (x) = g(x2 , · · · , x10 )[1 − (f1 (x1 )/g(x2 , · · · , x10 ))2 ]
9400
10 g(x2 , · · · , x10 ) = 1 + 9[( i=2 xi )/9]0.25 . Since we want only to verify the efficiency of the new performance metric here proposed, we use a set of Pareto front (simulation results) from http://www.tik.ee.ethz.ch/pisa for the 0/1 Knapsack problem and ZDT6. With two different sets of data, for example PA and PB , for each problem, a -dominance algorithm was run. Two new -nondominated sets, newPA and newPB , were then obtained and, using these new sets the Integrated Sphere Counting procedure was applied. The Pareto set with a higher ISC is considered the best one. This method of comparison is shown in the algorithm 2. Pa and Pb are the Pareto fronts to be compared, newPA and newPB are the -nondominated Pareto front and I(A) and I(B) represent the ISC value. Algorithm 2 How to compare two different Pareto sets using the Integrated Sphere Counting procedure. [newPA ] = -dominance(PA , PB ) [newPB ] = -dominance(PB , PA ) I(A) = ISC(newPA ) I(B) = ISC(newPB )
P1 P2
9200
9000
8800
8600
8400 8400
8600
8800
9000
9200
9400
9600
Fig. 2. This figure shows the two different Pareto fronts, P1 and P2 , represented respectively by ‘o’ and ‘*’ for the Knapsack problem. The horizontal axis represents the values for objective function 1 and the vertical axis the objective function 2. 9600 P1 P2 9400
9200
9000
8800
The purpose of using an -dominance algorithm is to eliminate from both sets PA and PB the points that are “clearly dominated” by some point in the other set, but keeping that points in both sets that are “almost non-dominated” (or nondominated), before performing the ISC computation. In the first test, two Pareto fronts for the Knapsack problem were used. The figure 2 shows these two sets. P1 is the Pareto front represented by ‘o’ and P2 is the Pareto front represented by ‘*’. After the -dominance algorithm, the nondominated points of each front are represented in the figure 3. Using these two -nondominated sets, the Integrated Sphere Counting was applied. The resulting values are shown in table I. TABLE I I NTEGRATED S PHERE C OUNTING VALUE FOR THE K NAPSACK PROBLEM USING THE NONDOMINATED FRONTS P1 AND P2 . Pareto Front: P1 P2
ISC value 1.3759 0.7018
Analyzing visually the Pareto fronts P1 and P2 in figure 3, we can observe that the Pareto front P1 has more nondominated points than the Pareto set P2 and these points are better distributed in the objective space. This fact is confirmed by the high value for ISC, shown in table I. In the second test, two Pareto fronts for the ZDT6 problem were used. The figure 4 shows these two sets. P3 is the
8600
8400 8400
8600
8800
9000
9200
9400
9600
Fig. 3. This figure shows the two -nondominated Pareto fronts, P1 and P2 , represented respectively by ‘o’ and ‘*’ for the Knapsack problem after an -dominance algorithm. The horizontal axis represents the values for objective function 1 and the vertical axis the objective function 2.
Pareto front represented by ‘o’ and P4 is the Pareto front represented by ‘*’. After the -dominance algorithm, the nondominated points of each front are represented in the figure 5. Using these two -nondominated sets, the Integrated Sphere Counting was applied. The resulting values are shown in table II. TABLE II I NTEGRATE BALL VALUE FOR THE ZDT6 PROBLEM USING THE NONDOMINATED FRONTS P3 AND P4 . Pareto Front: P3 P4
ISC value 0.7736 0.9577
Analyzing visually the Pareto fronts P3 and P4 in figure 5, the difference between them is more subtle. The value
2007 IEEE Congress on Evolutionary Computation (CEC 2007)
3629
measures the extension and the resolution with which a sample set describes a Pareto-set. These measures are explicitly related to the decision-making purpose of a Pareto-surface sample set. The computational effort for evaluating this metric is relatively insensitive to the increment in the number of objective functions of the multiobjective problem. In this way, the Integrated Sphere Counting enables calculation of larger fronts in more objectives than other existing metrics.
3.45 P3 P4
3.4 3.35 3.3 3.25 3.2
ACKNOWLEDGMENT
3.15
The authors acknowledge the support of the Brazilian agencies CAPES, CNPq and FAPEMIG.
3.1 3.05 0.2
R EFERENCES 0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
Fig. 4. This figure shows the two Pareto fronts, P3 and P4 , represented respectively by ‘o’ and ‘*’ for the ZDT6 problem. The horizontal axis represents the values for objective function 1 and the vertical axis the objective function 2. 3.45 P3 P4
3.4 3.35 3.3 3.25 3.2 3.15 3.1 3.05 0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
Fig. 5. This figure shows the two -nondominated Pareto fronts, P3 and P4 , represented respectively by ‘o’ and ‘*’ for the ZDT6 problem after a dominance algorithm. The horizontal axis represents the values for objective function 1 and the vertical axis the objective function 2.
[1] Coello Coello, C.A., 1999 “A Comprehensive Survey of EvolutionaryBased Multiobjective Optimization Techniques,” Knowledge and Information Systems, 1999, 1(3), p. 269–308. [2] Fonseca, C.M., Fleming, P.J., 1995 “An Overview of Evolutionary Algorithms in Multiobjective Optimization,” Evolutionary Computation, 1995, 7(3), p. 205–230. [3] Coello Coello, C.A., 1999 “List of References on Evolutionary Multiobjective Optimization,” Online. Available at: http://www.lania.mx/EMOO. [4] Barr, R.S., Golden, B.L., Kelly, J. P.,Resende, M.G.C., Stewart, W.L., 1995 “Designing and Reporting on Computational Experiments with Heuristic Methods ,” Proceedings of the International Conference on Metaheuristics for Optimization,pp. 1-17, Kluwer Publishing. [5] Coello Coello, C.A., Van Veldhuizen, D.A., Lamont, G. B. 2001 “Evolutionary Algorithms for Solving Multi-Objective Problems,” Kluwer Academic Publishers. [6] Knowles, J., Corne, D. 2002 “On Metrics for Comparing Nondominated Sets,” Proceedings of IEEE Congress on Evolutionary Computation,2002, pp. 711–716. [7] Zitzler, E., Thiele, L., Laumanns,M., Fonseca, C.M., Fonseca, V.G., 2003 “Performance Assessment of Multiobjective Optimizers: An Analysis and Review,” IEEE Transactions on Evolutionary Computation,2003, 7(2), pp. 117–132. [8] Zitzler, E., 1999 “Evolutionary Algorithms for Multiobjective Optimization: Methods and Application,” Ph.D. dissertation,Swiss Federal Int. Technology (ETH), Zurich, Switzerland, 1999. [9] Wanner,E.F., Guimaraes, F.G., Takahashi, R.H.C., Fleming, P.J. 2006 “A Quadratic Approximation-Based Local Search Procedure for Multiobjective Genetic Algorithms,” Proceedings of IEEE Congress on Evolutionary Computation,2006. [10] Martello,S., Toth, P., Knapsack Problems: Algorithm and Computer Implementations, Wiley, Chichester,1990. [11] Zitzler, E., Deb, K., Thiele,L. 2000 “Comparisons of Multiobjective Evolutionary Algorithms: Empirical Results,” Evolutionary Computation,8(2), pp. 173–195.
obtained by the Integrated Sphere Counting for P3 is smaller than the value obtained by P4 , but the difference is not very large. The main point that is revealed by the ISC merit factor is related to the fact that P3 has several points near one to another, while P4 has the points more spaced one to the other – what means that the points of P4 describe a larger extension of the Pareto set. This means that P4 is indeed a better description of the Pareto-set, even though each set P3 and P4 , lonely, don’t present a complete description. V. C ONCLUSION We have proposed a performance metric, the Integrated Sphere Counting, for evaluating the quality of sample sets, originated from multiobjective evolutionary algorithms, that are intended to describe Pareto-sets. The Integrated Sphere Counting is a simple, non-cardinal and unary metric that
3630
2007 IEEE Congress on Evolutionary Computation (CEC 2007)
