2013 BRICS Congress on1st Computational BRICS Countries Intelligence Congress & on 11th Computational Brazilian Congress Intelligence on Computational Intelligence
Estimation of Distribution Algorithm based on a Multivariate Extension of the Archimedean Copula Harold D. de Mello Jr, André V. Abs da Cruz, Marley M. B. R. Vellasco Department of Electrical Engineering Pontifical Catholic University of Rio de Janeiro Rio de Janeiro, RJ, Brazil
[email protected],
[email protected],
[email protected]
Copula theory allows one to express any multivariate distribution as a function (the copula) of its marginal distributions, which describe the behavior of each individual variable; the copula contains all of the required information to describe the dependency among the variables, and it is invariant to any non-linear transformation [7].
Abstract— This paper presents a Copula-based Estimation of Distribution Algorithm with Parameter Updating for numeric optimization problems. This model implements an estimation of distribution algorithm using a multivariate extension of the Archimedean copula (MEC-EDA) to estimate the conditional probability for generating a population of individuals. Moreover, the model uses traditional crossover and elitism operators during the optimization. We show that this approach improves the overall performance of the optimization when compared to other copula-based EDAs.
In the case of multidimensional problems, the EDA probabilistic models are built, essentially, from bivariate copula [8] because these represent the majority of parametric copulas. Besides, multivariate copulas are not indicated for problems where at least one pair of variables have a different dependency structure from the other variables. [9]. By using bivariate copula, it becomes easier to estimate the multivariate system. In [10], the use of empirical bivariate Archimedean copulas in EDAs was investigated to optimize n-dimensional problems but using a constant value for the parameters of the copula.
Keywords— continuous numeric optimization; evolutionary computation; estimation of distribution algorithms; copulas.
I.
INTRODUCTION
Numerical optimization is an important task that arises in several different knowledge domains. Its importance has motivated the development of new algorithms that are capable of finding better solutions with a lower computational cost. Evolutionary algorithms play an important role in this improvement and present promising results in different optimization problems.
This study discusses an EDA (MEC-EDA) that is similar to the approach proposed in [10] and is applied to some benchmark functions. Our approach differs in some aspects: the copula parameters are estimated dynamically, by means of using dependency measures; we use a crossover operator and elitism; and we use a heuristic for reinitializing the population throughout evolution. This paper is organized as follows: section II discusses some general aspects regarding copula theory. In Section III, we present copula-based EDAs as well as the model we propose. Section IV presents experimental results on benchmark functions as well as a discussion about the results. Section V presents some general conclusions.
Nevertheless, conventional evolutionary algorithms have difficulty when addressing optimization problems, when the number of variables, constraints and goals grow. Some of this difficulty relates to the association between the performance and the number of parameters, which must be determined for each problem, and the inability to extract and use knowledge that is acquired throughout the search process [1]. One possible approach to overcoming these problems is to use the Estimation of Distribution Algorithms (EDAs) [2]. EDAs are a class of evolutionary algorithms that construct a probability distribution throughout evolution by analyzing the most promising solutions. This probability distribution is used to generate new individuals instead of using mutation and crossover operators as in traditional evolutionary algorithms.
II.
A. Theorems, properties and definitions A function (, ): [0, 1] → [0,1] is an n-dimensional copula if and only if it satisfies the following basic conditions: C is increasing; For any vector u ∈ [0,1] , () = 0, at least one of the coordinates of u is zero;
Different EDA models, which differ in how the probabilistic model is estimated, have been proposed, including but not limited to models that are based on copula. The concept of copula was introduced by Sklar [3], and it has been extensively used in finance. More recently, copula theory has been used in Evolutionary Computation [5,6]. This work has been financially supported by CNPq and FAPERJ.
978-1-4799-3194-1/13 $31.00 © 2013 IEEE DOI 10.1109/BRICS-CCI-CBIC.2013.23 10.1109/BRICS-CCI.&.CBIC.2013.19
COPULA THEORY
75
For ∈ [0, 1], ( ) = (1, … ,1, … , , 1, … ,1) = for every ∈ {1,2, … , }
where / [ ] /() = , ∀ ∈ 3. If / [ ] = / .
Thus, given a generator function and its pseudo inverse, a bivariate Archimedean copula can be defined as:
where ( ), = 1, … , are marginal univariate distribution functions of copula Thus, copula is a distribution function in [0,1] that has marginal uniform functions in (0, 1). According to Sklar’s Theorem, a joint Cumulative Distribution Function (CDF) F of random variables , , … , with continuous marginal distributions , , … , , respectively, can be characterized by a single n-dimensional dependency function or copula , such that : for all vectors ∈ ℝ
( , , … , ) = ( ), ( ), … , ( )
( , ) = / [ ] /( ) + /( )
(1)
(2)
where () = { ∈ ℝ: ( ) ≥ } = 1, … , is the generalized inverse function of the marginal distribution function ∈ (0,1). The joint Probability Density Function (PDF) is defined as the nth-order derivative of copula with respect to the marginal distribution functions: ( , … , ) =
! ("! ),… , (" ) ! ("! ),… , (" )
∏$ ( )
TABLE 1 Clayton Copula
C(DE , DF ) M + M − 1
(3)
It is possible to obtain the conditional distribution by using the copula function. If % , … , % have a joint distribution function , then the conditional distribution % , given the values of % , … , % , can be determined by:
'*! '*! (+! ,…,+'*! )
+! ⋯ +'*!
+! ⋯ +'*!
.
(4)
JK
JL
2 /M
0
(7)
( , … , ) = / Q/ ( ) + / ∘ / S/ ( ) + / ∘ ⋯ (8) ∘ / / ( ) + / ( )TU
Conditions for / in terms of a Laplace Transform to turn (8) into a real copula were described in [13].
Archimedean copulas can be constructed using a continuous, decreasing, convex function /: 3 → ℝ∗5 , such that /(1) = 0. / is called the generator function of the copula, and it is strict whenever /(0) = +∞. The pseudo inverse can also be defined as: / (), 0 ≤ ≤ /(0) 0, /(0) ≤ ≤ +∞
I [0, ∞]
This multivariate extension, named as exchangeable multivariate Archimedean copula (EAC), nevertheless, poses a limitation: the dependency structure between any pair of variables is described by the same parameter ?. On the other hand, extending Archimedean copulas to the multivariate case, as proposed in [12], the nested Archimedean constructions (NACs), allows greater flexibility in choosing marginal distributions because there is a generator function for each pair of random variables. The generalization of (6) for the n-dimensional case is given by:
B. Archimedean Copulas Archimedean copulas are widely used in financial risk analysis [11]. In addition to the fact that many parametric copulas are Archimedean, their most common subclasses can i) be expressed in an explicit closed form and ii) represent a wide variety of dependency structures. Nevertheless, this type of copula is not directly derived from Sklar’s Theorem, and thus, it is necessary to follow some conditions in order for these functions to be considered to be copulas.
/ [ ] () = 8
G(H) ? O M − 1
( , … , ) = / [ ] /( ) + ⋯ + /( )
( | , … , ) = = P{% ≤ | % = , … , % = } '*! ' (+! ,…,+' )
/M
An additional advantage of Archimedean copulas is that they can be easily used to generate multivariate distributions. The n-dimensional copula is given by:
where , are the distribution functions and the marginal densities.
=
(6)
An Archimedean copula C shows the following properties, which are very useful for extending bivariate copulas into multivariate copulas: C is symmetric: ( , ) = ( , ); C is associative: (( , ), > ) = , ( , > ). Several copula families are described in [4]; all of them are dependent on a single parameter (?) that controls the dependency structure. However, this work used only one of them in the construction of the EDA. Table 1 shows the generator function, the lower tail λA , and the upper tail λB , of Clayton’s bivariate copula. In this case, the perfect dependency among the random variables occurs when ? → ∞, while ? = 1 means total independence among the variables.
Similarly, for any vector u ∈ [0,1] ( , , … , ) = ( ), ( ), … , ( )
is strict, then
More flexibility may yet be gainned by considering paircopulas models [14]. Pair-copulas is an hierarquichal decomposition of a n-copula into a cascade of of potentially different bivariate copulas. The composing bivariate copulas may vary freely, with respect to choice of the parametric families and parameter values. Therefore, all types and
(5)
76
negative dependence and 1 indicates a perfect positive dependency.
strengths of dependence can be covered. We intend to pursue this investigation in future work. However, in this paper, we consider an exchangeable multivariate Archimedean copula, allowing the specification of only one generator, regardless of dimension.
III.
COPULA-BASED ESTIMATION OF DISTRIBUTION ALGORITHMS (CEDAS)
A. General Aspects of EDAs C. Parameter Estimation
EDAs are optimization algorithms that are based on the estimation of a probability distribution. A general EDA is shown in figure 1. EDAs typically work with a set of candidate solutions for a problem. They are initialized by sampling a uniform distribution over a feasible set of solutions. On each generation, the population can be fully sampled from the probability distribution, or partially replaced. The most important step and the bottleneck of this class of algorithms is in estimating the joint probability distribution that is associated with the variables from the most promising solutions found by the evaluation function. This approach is also the aspect that differs most from one EDA to the other. The complexity of this probabilistic model can be classified into the following categories: independent, pairwise dependent, multivariate dependency and mixed models [18].
The parameter estimation of copulas in risk analysis [15] is accomplished through one of the following different approaches: i) maximum likelihood; ii) inference functions for the margins method; or iii) semi-parametric maximum likelihood. In [16], it was shown that the semi-parametric maximum likelihood – a model in which the marginal distributions are replaced by equivalent empirical distributions and the parameters are estimated afterward – is better than using methods i) and ii). Apart from the methods described above, one can estimate the parameters using empirical copulas (non parametric estimation), which are constructed from sample data or from dependency measurements. The empirical copula can be seen as a rank transformed empirical distribution. From a sequence of observations ( , V ) for = 1,2, … , , from the random variables X and Y, we initially sort values. The rank is defined as the position of the value in the ordered sequence divided by the number n of observations. The rank variables ( , W ) are estimators for = ( ) where are the marginal uniform distributions on the interval [0, 1]. Thus, it is possible to estimate the copula from the cumulative rank distribution: Y
number of rank pairs (\^ ,_^ ) such that \^ ≤\(`) , _^ ≤_(') ,
X , Z =
Algorithm 1: General EDA 1. 2. 3. 4. 5. 6. 7. 8. 9. 10. 11.
(9)
Another technique that is used in this work, in addition to the empirical copulas, is the estimation of moments based on Kendall’s tau () [17]. In this type of estimation, the relationship between the rank correlation and the copula’s ? parameter is used. For two continuous random variables ( , W) and a copula , Kendall’s tau is defined as: d = 4 ∬[l, ]q (, ) d(, ) − 1
Fig. 1. EDA pseudocode
B. Copula-based EDAs Recently a new approach to developing EDAs to solve realvalued optimization problem has been developed that is based on copula theory. In [6], it is possible to find a review of this
(10)
In the case of Archimedean copulas, (10) can be simplified to:
x(y)
d = 1 + 4 ∫l
research and to find an R package for working with copulabased EDAs. In the copula-based EDAs, the step for estimating the probabilistic model is divided into two parts: i) estimating the marginal distributions, , for = 1, … , , and ii) estimating the dependency structure. Usually, a specific distribution is assumed to be the correct distribution for each marginal distribution, and its parameters are estimated by maximum likelihood. In other cases, kernel density estimators (KDE), marginal empirical distributions or estimators based on Kendall’s inversion were used. Once the marginal distributions are estimated, the selected population is transformed into uniform variables in the interval [0,1] through the evaluation of each cumulative
(11)
xz (y) ~ (O)
is its first where /(O) is the generator function, and / derivative. For Clayton’s copula, (11) can be further simplified to: dM =
M M5
Begin t 0; generate initial population P(t) randomly; while (termination criteria are not met) do evaluation: (P(t)); selection: S(P(t)); estimate a probabilistic model, , for S(P(t)); sample a new population, P(t+1), from ; t t 1; End End
(12)
Generally speaking, the dependency measure is defined as the difference between the number of concordant pairs and discordant pairs among the pairs of random variables. Its values are in the range [-1,1] such that -1 indicates a perfect
77
marginal distribution. This transformed population is then used to estimate a copula-based model that describes the dependency among the variables. The sampling step then creates a population by sampling the dependent distributions ( ) ( ) ( ) , , … , for = 1, … , samples that were created by the copula. New individuals are calculated by ( ) = ( ), where is the inverse function of . According to (2) the new individual ( , , … , ) is a sample that obeys the joint distribution combined by the copula and the marginal distributions. This whole process is repeated until a stop condition is reached and the best individual is found, as can be seen in figure 2.
The next step comprises sampling a new population using the probabilistic model generated by Clayton’s copula. This sampling is performed by using Clayton’s copula to generate the joint empirical cumulative distribution for each variable using equation (9). Using empirical marginals CDF instead of raw data directly makes experiments comparable and the rank statistics tends to cope better with real-world systematic biases and errors. The inverse function of () used in this paper is: Y Y5 (13) ( , ) where denote two consecutive samples of Y Y5 the sorted current population and , means a random number inside this interval.
= () = Y Y5 and
With this new population, the algorithm makes a crossover between these and the best evaluated individuals of the current population at a specific rate (which is a parameter for the algorithm). A new evaluation is performed and the worst individuals are eliminated. The algorithm proceeds with the joint distribution estimation and sampling once again until a stop criterion is reached. A pseudocode of this algorithm is given in the following figure.
Algorithm 2: MEC-EDA 1. 2. 3. 4. 5. 6. 7. 8. 9.
Fig. 2. Flowchart of a general copula-based EDA. Adapted from [19].
10.
C. Proposed Model
11. 12.
The main reason to propose a copula-based EDA is the flexibility that copulas have to build effective joint distributions of the search space at the expense of common and convenient assumption made in most EDAs, the variables follow a Gaussian distribution. This conjecture often leads to inconsistent results and, therefore, the poor performance of the algorithm, especially in problems of numerical optimization in multivariate spaces. Thus, the use of copulations in EDAs can simplify their operations, once the construction of the probability models is performed in less time and with greater accuracy than other estimation techniques.
13. 14. 15. 16. 17. 18.
Begin O → 0; Generate initial population P(t) randomly; while (termination criteria are not met) do Evaluate (P(t)); Compute matrix of Kendall’s tau: ( ,
); if (any of n variables get a same value in all individual) do Restart population P(t), holding only the first individual; End Compute Clayton´s copula parameter ? by Kendall’s tau average; Select the best k individuals of P(t): S(P(t)); Get the marginal empirical cumulative distributions; Sample k individuals from step 12 using = () = Y Y5 ( , ); Make crossover between new individuals and the best individuals from current population; Merge the new individuals with current population. Sort the list according to fitness. Eliminate the worst individuals; O → O + 1; End End Fig. 3 MEC-EDA pseudocode
IV.
The first step of MEC-EDA initializes a random population within the search space for the problem. This population is then evaluated, and the correlation matrix for all of the variables for the problem is calculated using the current population as the data. Subsequently, it is possible to calculate Kendall’s tau and to obtain the parameter ? for Clayton’s copula using equation (12). The algorithm works in a recursive way to construct a n-dimensional from bivariate copulas and their associative and symmetry properties.
EXPERIMENTAL RESULTS AND DISCUSSION
A. Benchmark Functions The performance of the proposed model was evaluated using the following benchmark functions: Sphere: () = ∑$ , −600 ≤ ≤ 600, min() = (0, … ,0) = 0
78
(14)
TABLE III. ACKLEY PROBLEM
Ackley: () = −20 exp −0.2 ∑$ −
exp X ∑$ cos(2π )Z
Algorithm
(15)
+ 20 + exp(1),
Pop. Best Worst Size Fitness Fitness
Average Fitness
5.59e-7 9.98e-7 8.66e-7 + 1.07e-7
No Eval.
MEC-EDA
30
UMDA
81
8.1e-7 + 1.1e-7
4,998 + 88.0
GCEDA
310
6.5e-7 + 2.0e-7
13,082 + 221.4
CVEDA
104
8.1e-7 + 1.0e-7
6,330 + 163.2
DVEDA
111
7.9e-7 + 1.4e-7
6,678 + 133.8
Copula MIMIC
188
8.0e-7 + 1.2e-7
10,784 + 143.7
2,860 + 164
−30 ≤ ≤ 30, min() = (0, … ,0) = 0 Rastrigin: () = − 10cos(2 ) + 10), −5.12 ≤ ≤ 5.12, min() = (0, … ,0) = 0 ∑$ (
Rosenbrock: () = ∑ $ (100( − 5 ) + ( − 1) ), −9 ≤ ≤ 11, min() = (1, … ,1) = 0
(16)
(16)
where n is the number of variables to be optimized. All of the tests were performed while considering n=10. The Sphere function is unimodal, while all of the other functions are multimodal functions. For each function, 30 independent experiments were performed using two stop criteria: i) reaching the global minimum with a precision of 1e-6 or ii) reaching 100,000 evaluations. In all of the experiments, we used a crossover rate of 30%, replacing only 20% of the population between each generation. The results are shown in the following tables. For each function, we present results for our model (MEC-EDA) and for the models described in [6]. Each column shows the population size, the best fitness found, the worst fitness found (at the end of the optimization), the average final fitness and the number of evaluations that were made. It is possible to verify that the proposed algorithm provides superior results in all of the functions, which results in a smaller number of evaluations to reach the optimization target. Even though, MEC-EDA failed to optimize Rosenbrock, it got best average fitness than others copula-based EDAs.
TABLE IV. RASTRIGIN PROBLEM
Algorithm
Pop. Best Worst Size Fitness Fitness
Average Fitness
7.13e-7 9.83e-7 7.13e-7 + 2.12e-7
Average Fitness
3.83e-7 9.97e-7 8.26e-7 + 1.57e-7
No Eval.
MEC-EDA
50
6,093 + 1,809
UMDA
447
6.7e-7 + 2.3e-7
33,614 + 2,452.2
GCEDA
721
6.8e-7 + 1.8e-7
46,095 + 2,158.2
CVEDA
447
6.6e-7 + 1.7e-7
32,914 + 2,011.0
DVEDA
325
7.3e-7 + 1.7e-7
24,710 + 1,754.3
Copula MIMIC
386
6.4e-7 + 2.1e-7
27,315 + 1,673.9
TABLE V. ROSENBROCK PROBLEM
Algorithm
TABLE II. SPHERE PROBLEM Algorithm
Pop. Best Worst Size Fitness Fitness
No Eval.
Pop. Best Size Fitness
Worst Fitness
Average Fitness
No Eval.
MEC-EDA
70
9.93e-2 8.09e+0 2.6e+0 + 2.37e+0 100,010 +19.24
UMDA
2000
8.0e+0 + 2.6e-2
300,000 + 0.0
GCEDA
2000
7.5e+0 + 1.9e-1
300,000 + 0.0
CVEDA
2000
7.5e+0 + 1.1e-1
193,867 +48,243
DVEDA
2000
7.5e+0 + 1.5e-1
172,200 +35,184
Copula MIMIC 2000
7.6e+0 + 1.3e-1
139,000 + 5,139
2,252 + 144
MEC-EDA
30
UMDA
81
6.9e-7 + 2.3e-7
3,823 + 128.3
GCEDA
310
6.5e-7 + 2.0e-7
13,082 + 221.4
CVEDA
104
6.7e-7 + 1.8e-7
4,777 + 118.8
DVEDA
104
6.7e-7 + 2.0e-7
4,787 + 100.2
Copula MIMIC
150
6.9e-7 + 1.7e-7
6,495 + 209.0
79
[12] H. Joe, Multivariate Models and Dependence Concepts. Chapman & Hall, London, 1997. [13] P. Embrechts, F. Lindskog, and A. McNeil, Modelling dependence with copulas and applications to risk management, Handbook of heavy tailed distributions in finance, Elsevier/North-Holland, Amsterdam, 2003. [14] K. Aas, C. Czado, A. Frigessi, and H. Bakken. Pair-copula constructions of multiple dependence. Insurance: Mathematics and Economics, 2, 1, p.1-25, 2007. [15] G. N. F. Weiss, Copula Parameter Estimation – Numerical Considerations and Implications for Risk Management. Journal of Risk, Vol. 13, No. 1, pp. 17-53, 2010. Available at SSRN: http://ssrn.com/abstract=1648983 [16] G. Kim, M. Silvapulle, and P. Silvapulle, Comparison of semiparametric and parametric methods for estimating copulas. Computational Statistics & Data Analysis, n. 51, pp. 2836–2850, 2007. [17] C. Genest C, and L-P. Rivest, Statistical inference procedures for bivariate Archimedean copulas. J Am Stat Assoc 88(423):1034– 1043, 1993. [18] P. Larrañaga, J.A. Lozano, and E. Bengoetxea. Estimation of distribution algorithms based on multivariate normal and Gaussian networks. Technical Report KZZA-IK-1-01, Department of Computer Science and Artificial Intelligence, University of the Basque Country, 2001. [19] X. Guo; L. Wang; J. Zeng; X. Zhang. VQ Codebook Design Algorithm Based on Copula Estimation of Distribution Algorithm, First International Conference on Robot, Vision and Signal Processing, p.178-181, 2011.
B. Discussion The results show that, despite the simplicity of the algorithm, small improvements can bring better performance to a copula-based EDA. Our approach uses alternative method for the parameter estimation of the Archimedean copula, as well as crossover and elitism. The preliminary results are good and suggest that with some fine tuning, it might be possible to improve even further the results, especially for the Rosenbrock function, for which the algorithm was not capable of reaching the global minimum with the desired precision. Nevertheless, an additional great number of benchmark functions needs to be used to test the performance of the proposed algorithm. V.
CONCLUSIONS
This paper presented a new model based on Estimation of Distribution Algorithms and copulas (MEC-EDA) for the optimization of numerical functions. The presented results are promising, although further tests must be performed. Specifically, there is a need to check the impact of estimating Kendall’s tau on each generation of the EDA. Additionally, more tests could be performed with different Archimedean copulas to determine whether the type of copula used is important. VI.
BIBLIOGRAPHY
[1]
M. Hauschild and M. Pelikan, “An introduction and survey of estimation of distribution algorithms,” Swarm and Evolutionary Computation, vol. 1, no. 3, pp.111 – 128, September 2011. [2] P. Larrañaga and J. A. Lozano, editors. Estimation of distribution algorithms. A new tool for evolutionary computation. Kluwer Academic Publishers, Boston/Dordrecht/London, 2002. [3] A. Sklar, "Fonctions de Répartition à n Dimensions et Leurs Marges," Publications de l’Institut de Statistique de L’Université de Paris, 8, 229–231, 1959. [4] R. B. Nelsen, An introduction to copula. Springer-Verlag, New York, 1998. [5] L. Wang, J. Zeng, and Y. Hong. “Estimation of distribution algorithm based on copula theory.” In Proceedings of the IEEE Congress on Evolutionary Computation (CEC 2009), pp. 1057–1063, May 2009. [6] Y.González-Fernández and M. Soto. copulaedas: Estimation of distribution algorithms based on copulas, 2011. R package version 1.1.0. Unpublished. Available at http://CRAN.Rproject.org/package=copulaedas. [7] B. Póczos, Z. Ghahramani, and J. Schneider, “Copula-based Kernel dependency measures,” In Proceedings of CoRR, 2012. [8] H. Joe. “Families of m-variate distributions with given margins and m(m-1)/2 bivariate dependence parameters. In L. Rüschendorf, B. Schweizer, and M. D. Taylor, editors, Distributions with fixed marginals and related topics, pp. 120–141, 1996. [9] M. Soto, Y. González-Fernández, and A. Ochoa, “Modeling with copulas and vines in estimation of distribution algorithms”, 2012. Unpublished. Avaible at http://arxiv.org/abs/1210.5500 [10] A. Cuesta-Infante, R. Santana, J.I. Hidalgo, C. Bielza, and P. Larrañaga, “Bivariate empirical and n-variate archimedean copulas in estimation of distribution algorithms.” In Proceedings of the IEEE Congress on Evolutionary Computation (CEC 2010), pp. 1355–1362. [11] P. Embrechts, A. McNeil, and D. Staumann, “Correlation and dependence in risk management: Properties and pitfalls”. In Risk Management: Value at Risk and Beyond. Ed. M.Dempster, Cambridge University Press, pp. 176–223, 2002.
80
