The Genetic Algorithm Estimates for the Parameters of Order p Normal Distributions (1) Un Algoritmo Genetico per la Stima dei Parametri di Distribuzioni Normali di Ordine p Salvatore Vitrano Dipartimento di Sociologia, Universita’ degli Studi “La Sapienza” Via Salaria 113, I-00198 Roma, e-mail:
[email protected]
Roberto Baragona Dipartimento di Sociologia, Universita’ degli Studi “La Sapienza” Via Salaria 113, I-00198 Roma, e-mail:
[email protected] Riassunto: Le distribuzioni normali di ordine p, anche note come distribuzioni esponenziali potenziate, costituiscono una naturale generalizzazione della distribuzione normale. La stima dei parametri presenta delle difficoltà non trascurabili sia per quanto riguarda la distorsione che la variabilità. Gli algoritmi genetici, sperimentati con successo in molte applicazioni dell’inferenza statistica, possono costituire in questo senso una valida alternativa nell’ambito dei metodi di stima proposti. Sono qui considerate la stima del parametro di locazione per piccoli campioni, e dell’ordine p per campioni di numerosità moderata. Un esperimento di simulazione illustra i risultati e i vantaggi che possono derivare dall’uso degli algoritmi genetici e permette un confronto con alcuni metodi precedentemente introdotti e largamente sperimentati. Keywords: Exponential power functions, Genetic algorithms, Jackknife estimators, Maximum-likelihood, Normal distributions of order p.
1. Introduction th
Subbotin (1923), by generalizing the 2 Schiaparelli axiom, introduced a family of error distributions named Exponential Power Functions (EPF). Several formulations of the EPF were extensively analyzed in the literature, but here we refer to the model known as Normal Distribution of Order p, defined on the whole real axis and characterized by the location µ, the scale σp, and the shape parameter p (p>0). A family of unimodal symmetric curves from this density function arises with different shapes for different values of p. The Laplace (p=1), the Normal (p=2), and the Uniform (p → ∞ ) distributions arise as special cases. Let x=(x1, x2, …, xn) be a random sample drawn from this probability density function. The maximum-likelihood (ML) estimator of the parameter vector θ =(µ, σp, p) is any value θˆ which maximizes the likelihood function: n
Lθ (x) = {2 p1/pσp Γ(1+1/p)}–n exp{–(pσ pp ) −1 ∑ | x i i =1
(1)
- µ | p }.
(1)
The present paper is financially supported by the University of Rome “La Sapienza” and by M.U.R.S.T. (Italy) as a part of the grant “Non linear models and new computational methods in time series analysis and forecasting,“ 2000.
Properties and parameter estimation methods for the distributions in this class were extensively studied (see Chiodi, 2000, for a comprehensive review). We propose a genetic algorithm (GA) approach for estimating θ . Holland (1975) studied the evolution of a population of individuals in a given environment by means of a class of analytical models called GA. Each individual in the population, which has not to be given a strictly statistical meaning, needs to be characterized by a chromosome, i.e. a string of characters. Each character in the string possesses a specific meaning that may be decoded from both its value, called allele, and position, called locus. The population evolves through several generations towards the best adaptation to the environment. A basic assumption is that a function may be defined that maps the genetic pool (the set of all admissible strings) into the positive real axis. This function is called the fitness function, and it has to increase as soon as the adaptation to the environment increases. The special applications of the GA for function optimization assume the objective function as the fitness function, and the string owned by the best fit individual as the solution to the optimization problem. Chatterjee and Laudatto (1997) investigated using GA for statistical inference, and we closely followed the guidelines provided therein. The plan of the paper is as follows. The next section is devoted to the illustration of the GA we implemented for assessing estimates and standard errors. Comparison with alternative approaches by means of a simulation experiment is provided in Section 3. The estimation of the location parameter µ in case of small samples, for given order p, and the estimation of all three parameters, for rather large sample sizes, is considered.
2. Genetic algorithm implementation The main features of our design of the GA are the tournament selection for reproduction, and the use of three operators, crossover, inversion, and mutation. Unlike many GA applications, inversion proved to constitute a valid support to increase the speed of convergence to the solution. Further, coding the potential solutions, that are real numbers in a given interval (a,b), say, was done by using the following device. Let us choose a positive integer as the binary string length. Then, 2 different numbers x may be coded according to x = a + c (b – a) / (2 – 1),
(2)
where c represents the real value corresponding to the given binary string. As c may vary from 0 (the null string 000…000) through 2–1 (the all-one string 111…111), then (2) encodes a finite subset of real numbers in the interval (a,b), at equispaced intervals of length (b–a)/(2–1) each. As far as the GA parameters choice is concerned, for the present problem, we assumed the selection pressure ps=0.75, the crossover, mutation, and inversion probability pc=0.7, pi=0.65, and pm=0.01 respectively, and 50 as the number of generations. We assumed the population size rather small, s=50, as no substantial improvement was observed by taking larger values. The likelihood (1) provided the fitness function.
Table 1: Order-p Normal distribution parameter estimates, all parameters unknown µ p=1 n=50 n=100 p=1.25 n=50 n=100 p=1.50 n=50 n=100 p=1.75 n=50 n=100 p=2 n=50 n=100 p=2.25 n=50 n=100 p=2.5 n=50 n=100 p=2.75 n=50 n=100 p=3 n=50 n=100 p=3.25 n=50 n=100
Maximum likelihood p σp
µ
Genetic algorithm σp
p
3.0193
2.1429
1.2462
3.0267
2.1341
(.2868)
(.3198)
(.3390)
(.2796)
(.3171)
(.3279)
2.9781
2.0606
1.1144
3.0074
2.0403
1.1099
(.2004)
(.1917)
(.1602)
(.2169)
(.1988)
(.1762)
2.9765
2.0486
1.5072
3.0235
2.0528
1.4813
(.3578)
(.3436)
(.7016)
(.3176)
(.3237)
(.6211)
3.0447
2.0468
1.3691
3.0066
2.0593
1.3774
(.2181)
(.2495)
(.2954)
(.2063)
(.2820)
(.3766)
2.9694
1.9805
1.7014
2.9812
2.0096
1.8719
(.3106)
(.3483)
(.7612)
(.3167)
(.3783)
(.9849)
2.9924
1.9949
1.5935
2.9922
2.0094
1.5980
(.1959)
(.2634)
(.3806)
(.2009)
(.2324)
(.4399)
3.0454
2.1079
2.2741
2.9633
2.0760
2.1561
(.3220)
(.4431)
(1.0201)
(.2962)
(.3908)
(.9725)
2.9570
1.9861
1.8479
2.9894
1.9606
1.8190
(.1987)
(.2150)
(.3964)
(.2111)
(.2353)
(.4464)
3.0064
2.0554
2.5580
3.0510
2.0858
2.5986
(.2908)
(.3666)
(1.1405)
(.2963)
(.3090)
(1.0214)
2.9938
2.0654
2.2843
3.0127
2.024
2.2229
(.2196)
(.2362)
(.7219)
(.1815)
(.2246)
(.6650)
3.0160
2.0807
3.0267
2.9586
2.0637
2.8037
(.2985)
(.3049)
(1.1726)
(.2848)
(.3257)
(1.1711)
3.0314
1.9904
2.4970
2.9944
2.0054
2.4778
(.2077)
(.2345)
(.6559)
(.1883)
(.2125)
(.7259)
2.9395
2.0682
3.2349
2.9985
1.9875
3.0510
(.2562)
(.2919)
(1.1717)
(.2670)
(.3054)
(1.2151)
3.0771
2.0807
2.9473
3.0098
2.0395
2.7787
(.1718)
(.2123)
(.8330)
(.1661)
(.2171)
(.7440)
2.9752
2.0459
3.2150
3.0213
1.9974
3.2069
(.2416)
(.3026)
(1.1268)
(.2671)
(.3146)
(1.1257)
2.9778
2.0149
3.0505
2.9904
1.9498
2.8491
(.1846)
(.2213)
(.8972)
(.1846)
(.1966)
(.7361)
2.9970
2.0584
3.6761
3.0009
2.0152
3.5714
(.2448)
(.2661)
(1.1710)
(.2509)
(.3022)
(1.2258)
3.0188
2.0110
3.4036
3.0051
2.0365
3.3715
(.1733)
(.2124)
(.9240)
(.1624)
(.1970)
(.8667)
3.0054
1.9841
4.0412
3.0103
1.9782
3.7364
(.2199)
(.2248)
(1.0605)
(.2260)
(.2782)
(1.2226)
2.9912
2.0242
3.6140
3.0063
2.0174
3.5910
(0.1609)
(.1932)
(.9203)
(.1639)
(.2161)
(.9309)
Note: the estimated standard errors are given in parentheses.
K
1.2250 10 5 5 6 2 1 2
1
3. Simulation and results To compare the parameter estimates obtained by the GA with the ML estimates obtained by the common search numerical algorithms, two distinct Monte Carlo simulations were designed. The numerical inspection of the sample mean and standard deviation of the parameter estimates was made by drawing 100 samples using of size n= 10,20,50, when p was supposed known, and n=50, 100, when p was unknown, from a Normal distribution of order p (µ=3; σp=2) with p = 1-3.25 (0.25), through a generator algorithm (Chiodi, 1986). Some simulation results are given in Table 1. The number K of samples, for which the ML procedure has not reached any optimum point of (1), was also reported along with the parameter estimates by the two approaches. When p>2, a less bias was observed using the GA than the ML approach. Opposite results occur if p≤2. In this case, however, some samples were discarded by the ML procedure. Similar findings were obtained, except some particular cases, for the estimates of the two other parameters. When p was assumed known, the GA and two different scale parameter estimators (Chiodi, 1988; La Tona, 1994) were considered. The estimators gave results that were found to differ less than 1/1000, with best results by using the GA, when p>2. Finally, in both types of Monte Carlo simulations (p known and unknown), the standard deviations of the estimates from the two approaches were obtained that had the same order of magnitude.
References Chatterjee, S., Laudatto, M. (1997) Genetic algorithms in statistics: procedures and applications, Communications in Statistics - Simulation and Computation, 26(4), 16171630. Chiodi, M. (1986) Procedures for generating pseudo-random numbers from a normal distribution of order p (p>1), Statistica Applicata, 1, 7-26. Chiodi, M. (1988) Sulle distribuzioni di campionamento delle stime di massima verosimiglianza dei parametri delle curve normali di ordine p, Istituto di Statistica Università di Palermo. Chiodi, M. (2000) Le curve normali di ordine p nell’ambito delle distribuzioni di errori accidentali: una rassegna dei risultati e problemi aperti per il caso univariato e per quello multivariato, in: Atti della XL Riunione Scientifica della SIS, ISTAT Centro Stampa, 5960. Extended and revised version available at http://statistica.economia.unipa.it/ Holland, J. H. (1975) Adaptation in Natural and Artificial Systems. University of Michigan Press (Second Edition: MIT Press, 1992) La Tona, L. (1994) Uno stimatore jackknife del parametro di scala di una curva normale di norma p, Metron, 52, 181-188. Subbotin, M. T. (1923) On the law of frequency of errors, Matematicheskii Sbornik, 31, 296-300.
