A Genetic algorithm to approximate convex sets of probabilities Andres CANO
Depto de Ciencias de la Computacion e I.A. Universidad de Granada 18071 - Granada - Spain
[email protected]
Abstract An Evolution Program is presented to propagate convex sets of probabilities. This algorithm is useful when the number of extreme points in the 'a posteriori' convex set for a variable is too high and a single probabilistic propagation is feasible. We have tested the algorithm in a random causal network with a random number of conditional probabilities in each one of the variables. The experimental evaluation show that the resulting intervals obtained for the cases of a variable are similar to those obtained using an exact method of propagation.
1 Introduction Graphical structures have been used to represent and manipulate independence relationships [16]. Propagation algorithms were rst developed for the probabilistic case [11, 17]. These independence relationships can be used to obtain factorizations of uncertainty representations given by means of other formalisms (see Shafer and Shenoy [19, 18]). One of the particularizations of the Shafer and Shenoy system was given by Cano, Moral and Verdegay-Lopez [3, 1] for the case of convex sets of probabilities. Some methods to reduce the cost of the calculations were given by Cano and Moral [2], among them a Montecarlo algorithm. Essentially the problem comes from the fact that when in the probabilistic case we have a probability distribution or a conditional probability for the speci cation of the problem, in the case of convex sets of probabilities, we have a convex set of probabilities or a convex set of conditional probabilities. The number of possible global probabilities is the product of the number of extreme probabilities or conditional probabilities in each one of the initial convex sets. An exact calculation is
Serafn MORAL
Depto. de Ciencias de la Computacion e I.A. Universidad de Granada 18071 - Granada - Spain
[email protected]
equivalent to repeat a single probabilistic propagation for each one of the possible global probabilities. In Cano, Cano and Moral [5, 6] several Gibb's sampling based techniques were given to select the probabilities that are important for the calculation of the convex set for a given variable. In this work we present an Evolution Program [12] to nd the convex set of probabilities of the variables of interest. This algorithm is useful when there is a combinatorial number of probabilities, that can not be propagated with an exact algorithm. It is assumed that the simple probabilistic case is feasible. That is,if we select an unique probability for each convex set then the calculations could be carried out without problems. In the algorithm it is considered that extreme points of the 'a posteriori' convex set of a variable Xi with a high normalization factor are the most important in the calculus of the intervals of the cases of Xi . Evolution Programs are useful in this task because they can obtain a population of close points to an optimal solution (that is, points with a high normalization factor). Once we have found a population of points with high normalization factor, we can give an approximate interval for each case of a variable Xi taken the maximum and minimum probability for each case. These intervals will be similar to the obtained intervals using Choquet's Integral [7, 13] applied to all the possible extreme points obtained with an exact propagation algorithm. The paper is organized in the following way: in section 2 we give the basic elements of probabilistic propagation. In section 3 we study brie y the propagation of convex sets of probabilities. In section 4 we give a description of Evolution Programs, a general tool to solve combinatorial optimization problems and we describe the algorithm used to nd the convex set of probabilities for a given variable. Finally, in section 5 we describe the experimental tests carry out with a random causal network, and in section 6 we give the conclusions of
the paper.
2 Probabilities Propagation We give only the necessary elements to describe our procedure. More details can be found in [16, 15, 11]. We follow Shenoy and Shafer [19] methodology applied to probabilities. The main di erence with the Lauritzen and Spiegelhalter [11] methodology is on the message passing system. Lauritzen and Spiegelhalter carry out a division which improves the eciency. We have not found an easy way of using division in our algorithm. Consider a set of random variables Xi i2N , with N = 1; 2; : : :; n , and assume that each variable Xi is taking values on a nite set Ui . We will denote the n-dimensional variable (X1 ; X2 ; : : :; Xn ) by X, which takes values on UN = i2N Ui . Assume that each variable Q Xi is de ned on a nite set Ui and that UI = ni=1 Ui . Following Shenoy and Shafer terminology a mapping from a set UI on [0; 1] will be called a valuation de ned on UI . Given two valuations, h1 and h2, de ned on UI and UJ , then the combination of h1 and h2 is a valuation h1 h2 de ned on UI [J by means of pointwise multiplication: h1 h2(u) = h1(u#I ):h2(u#J ) (1) f
f
g
g
where u#I is the element obtained from u by dropping the coordinates not in I. If h is a valuation de ned on UI and J I, then the marginalization of h in J, h#J is calculated by addition: X h(v) h#J (u) = (2)
v#J =u
A graph of dependences on the set of variables Xi i2N is an directed acyclic graph which expresses independence relationships among the variables by means of the d-separation criterion [16]. Initially we start with a directed acyclic graph, with a node for each variable, Xi , see Fig. 2. A conditional probability for each variable given its parents is speci ed. According to the independence relationships, these conditional probabilities determine an unique global probability distribution: p1 p2 pn . If we have a family of observations e = Oj j 2J , the global conditional N probability is proportional, to p1 p2 pn ( j 2J Oj ). The 'a posteriori' probability distribution for N variable Xk , is proportional to (p1 p2 pn ( j 2J Oj ))#fkg . This value will be denoted as PSk . The aim of propagation algorithms is to calculate the marginal valuations PSk for each variable Xk by doing f
f
g
g
a local computation. To do that we will use the Shenoy and Shafer [19] methodology. In this methodology we build a tree of cliques. Between every two cliques Ci and Cj there are two messages MCi !Cj and MCj !Ci To reach the equilibrium in the tree of cliques it is enough to visit two times every clique. This is achieved by means of a propagation algorithm [11, 19]. After reaching the equilibrium in every node, if we want to calculate the 'a posteriori' valuation for a given variable Xk , we have to determine a clique Ci containing this variable, and to calculate, PSk = (
O
m=1;:::;l
VCjm !Ci VCi )#fkg
(3)
where Cj1 ; : : :; Cjl are the neighbouring cliques connected to Ci .
3 Propagation of Convex Sets of Probabilities The propagation of convex sets of probabilities is quite analogous to the propagation of probabilities. Here, we only describe the main di erences. More details can be found in [4]. Now, valuations are convex sets of possible probabilities, with a nite number of extreme points. A conditional valuation is a convex set of conditional probability distributions. An observation of a value for a variable will be represented in the same way as in the probabilistic case. The combination of two convex sets of mappings is the convex hull of the set obtained by combining a mapping of the rst convex set with a mapping of the second convex set. If we must combine two valuations Vi and Vj with mi and mj points we should do mi mj pointwise multiplications of vectors. We have n valuations V1; : : :; Vn and each valuation is given by mi points. The marginalization of a convex set is de ned by marginalizing each of the mappings of the convex set. With these operations, we can carry out the same propagation algorithms as in the probabilistic case. The result of the propagation for the variable Xi , will be a convex sets of mappings from Ui in [0; 1]. For the sake of simplicity we assume that this variable Xi has two possible values: x1i ; x2i . The result of the propagation is a convex set on 2 of the form of gure 1 . We will call Ri to this convex set of points. Points of this convex set, Ri , can be obtained in the following way: If P is a global probability distribution, formed selecting a xed probability for each convex set, then associated to this probability, we shall obtain
