Matthias C. M. Troffaes, Gert de Cooman, and Dirk Aeyels. Imprecise probabilities — discussion and open problems. In Proceedings of the 2nd FTW PhD Symposium, Dec 2001.
Imprecise Probabilities — Discussion and Open Problems Matthias C. M. Troffaes, Gert de Cooman, and Dirk Aeyels Abstract— If only little, vague or conflicting information is available about a system, it is hard to construct and motivate a model based on the so-called precise probability theory. In this case, generalisations of this theory—called in the literature imprecise probability theories—prove to be useful. We discuss the most satisfactory of these models, the unifying behavioural theory of lower previsions, we identify a number of open problems in the theory, and we briefly point to work that is being done to solve them. Keywords— uncertainty, systems, modelling.
I. Introduction
I
N MODELLING a system, it often occurs that some of its aspects, or some of the influences acting on it, are not well known. The uncertainty this produces about the system’s behaviour is usually modelled by a probability measure, and treated using techniques from probability theory—we could call this the Bayesian approach. It can deal efficiently with uncertainty in many situations (Bayesian learning, data compression, pattern recognition, reliability, population models in biology, etc.). Such a model will often not be adequate—it might be unreasonable to identify a specific and unique probability measure. This can happen because • the information is scarce (“Tossing this coin three times resulted in tail, tail and tail. What will be the result of the next toss?”); • the information is vague, for example when only linguistic information is available (“Mary is young.”); or • the information is conflicting, for example when there is a conflict between prior information and statistical data, or when trying to combine the information of a number of experts, or a group, in order to make group decisions. In particular, the following classes of problems would benefit from a generalisation of the theory of probability: • decision making with little information [1] (e.g., processing subjective knowledge and expert estimates [2]); • optimisation using an imprecise cost criterion (e.g., selecting an optimal consumption path in a complex economical society trying to avoid a ecological catastrophe [3]); • the estimation of quantities which depend on parameters that are not well known (e.g., the time to failure of
Matthias C. M. Troffaes is a member of the SYSTeMS Research Group of the Department of Electrical Power Engineering, Ghent University (RUG), Ghent, Belgium. E-mail:
[email protected] . Gert de Cooman is a member of the SYSTeMS Research Group of the Department of Electrical Power Engineering, Ghent University (RUG), Ghent, Belgium. E-mail:
[email protected] . Dirk Aeyels is a member of the SYSTeMS Research Group of the Department of Electrical Power Engineering, Ghent University (RUG), Ghent, Belgium. E-mail:
[email protected] .
a component in a system [4]; modelling an opponent in a game-theoretical setting [5], [6]); • modelling systems based on linguistic information [4], [7]; • classification [8], [9] and pattern recognition; and • constructing, and reasoning on, hierarchical models [10] (e.g., in order to make group decisions). II. Imprecise Probabilities Generalisations of probability theory have been developed and applied in order to represent and manipulate the really available knowledge about the system. These extensions are called imprecise probabilities—most commonly found in the literature are: comparative probability orderings [11], [12], [13]; Choquet capacities [14]; belief functions [15], [16]; possibility measures [17]; fuzzy measures [18]; lower (and upper) previsions, sets of desirable gambles or convex sets of probability distributions [19]; and risk measures [20]. It has also been shown that in some situations, particularly when only little information is available, using imprecise probabilities leads to better results than using a Bayesian model. As an illustrative example [6], consider a two-player zero-sum game, in which the second player plays a constant strategy. Simulations show that, a (first) player which uses an imprecise probability model to represent the constant player, makes more profit than a player which uses a Bayesian model. The difference is most striking at the start of the game, and as the players play longer, the difference becomes negligible. This shows that, when little information is available, an imprecise probability model may perform better, and it also justifies the use of a Bayesian model when a lot of information is available. This of course motivates further research in the—far from complete—theory of imprecise probabilities. III. Lower Previsions Walley’s theory of lower previsions [19] unifies most of the above-mentioned models, and from a foundational point of view it seems to be the most satisfactory one by far. Briefly, it consists of • an assessment of the behaviour of the system in certain situations—this represents the available knowledge about the system; • rationality criteria—these are used to identify conflicts in the information and to determine whether the model is consistent or not; and • a reasoning/inference method that calculates how the system should behave in other situations—this tells us how to draw conclusions from, and make decisions based on, the available knowledge.
2
From a mathematical point of view, specifying a consistent lower prevision is equivalent to specifying a set of (finitely) additive probability distributions—thus the theory of lower previsions can in certain respects be interpreted in terms of Bayesian sensitivity analysis. However, the theory of lower previsions can be developed without such an interpretation, and without a prior need for precise Bayesian models. Precise probabilities are a special case of lower previsions, and the laws of precise probability theory follow from the above-mentioned rationality criteria, as special cases. Classical probability theory can be used as a source of inspiration for the development of new analytical methods in the theory of imprecise probabilities. Conversely, as a number of studies (see for instance [21]) have already shown, looking at classical probability theory from a broader perspective provides additional insights for that theory as well.
Acknowledgments The authors would like to thank the FWO for funding the research reported in this paper. References [1] [2]
[3] [4] [5] [6] [7]
IV. Open Problems [8]
Despite the fact that the theory of lower previsions is very appealing, it has a few shortcomings in the following respects: A technical problem is that lower previsions only deal with bounded random variables, whereas applications involving unbounded random variables abound. In particular, the following classes of problems would benefit from a generalisation of imprecise probability theory: – optimisation using an imprecise cost criterion, with an unbounded (e.g., quadratic) cost; and – the estimation of unbounded quantities which depend on parameters that are not well known. We are studying this problem in the SYSTeMS group and we have constructed an extension to unbounded random variables on which the Bayesian sensitivity interpretation also continues to hold. We are still working on the study of consistency and inference for lower previsions on unbounded random variables. • A dynamical approach to imprecise systems is still in its developing phase. It should however be mentioned that imprecise probability theory has a learning tool, called the imprecise Dirichlet model [22] which allows a type of inductive reasoning that has distinct advantages over its Bayesian counterpart. It has already been applied with success in classification problems [9], game-theoretic learning [5], [6] and we are currently studying its application to learning in hidden Markov models. There are also indications that the theory of differential inclusions might provide mathematical tools for studying so-called imprecise dynamical systems [23]. • The relation of the theory of lower previsions to the classical von Neumann utility theory [24] and its generalisations [1], [25], commonly applied in economics, needs to be clarified. There are strong indications that this could lead to interesting insights in epistemology and the study of logic [21]. •
[9] [10] [11] [12] [13] [14] [15] [16] [17] [18] [19] [20] [21] [22] [23]
[24] [25] [26]
[27]
Robert J. Aumann, “Utility theory without the completeness axiom,” Econometrica, vol. 30, no. 3, pp. 445–462, 1962. Thomas Fetz, Michael Oberguggenberger, and Simon Pittschmann, “Applications of possibility and evidence theory in civil engineering,” In de Cooman et al. [26], pp. 146–153. Morgane Chev´ e and Ronan Congar, “Consumption/pollution tradeoffs under hard uncertainty and irreversibility,” In de Cooman et al. [26], pp. 85–93. Lev V. Utkin and Sergey V. Gurov, “Imprecise reliability models for the general lifetime distribution classes,” In de Cooman et al. [26], pp. 333–342. Gert de Cooman, “Learning in two-player fictious play using the imprecise Dirichlet model,” Tech. Rep., Ghent University, 2000. Bart De Vylder, “Simulatie van speltheoretisch leren met imprecieze waarschijnlijkheden,” M.S. thesis, Ghent University, Sept. 2001. Peter Walley and Gert de Cooman, “A behavioral model for linguistic uncertainty,” Information Sciences, vol. 134, pp. 1– 37, 2001. Marco Zaffalon, “A credal approach to naive classification,” In de Cooman et al. [26], pp. 405–414. Marco Zaffalon, “Statistical inference of the naive credal classifier,” In de Cooman et al. [27], pp. 384–393. Gert de Cooman, “Lower desirability functions: A convenient imprecise hierarchical uncertainty model,” In de Cooman et al. [26], pp. 111–120. J. M. Keynes, A Treatise on Probability, Macmillan, London, 1921. Bruno de Finetti, “Sul significato soggettivo della probabilit` a,” Fundamenta Mathematicae, vol. 17, pp. 298–329, 1931. T. L. Fine, Theories of Probability, Academic Press, New York, 1973. G. Choquet, “Theory of capacities,” Annales de l’Institut Fourier, vol. 5, pp. 131–295, 1953–54. A. P. Dempster, “Upper and lower probabilities induced by a multivalued mapping,” Ann. Math. Statist., vol. 38, pp. 325– 339, 1967. G. Shafer, A Mathematical Theory of Evidence, Princeton University Press, 1976. L. A. Zadeh, “Fuzzy sets as a basis for a theory of possibility,” Fuzzy Sets Syst., vol. 1, pp. 3–28, 1978. G. J. Klir and T. A. Folger, Fuzzy Sets, Uncertainty, and Information, Prentice-Hall, Englewood Cliffs, New Jersey, 1988. Peter Walley, Statistical Reasoning with Imprecise Probabilities, Chapman and Hall, London, 1991. P. Artzner, F. Delbaen, J. M. Eber, and D. Heath, “Coherent measures of risk,” Mathematical Finance, vol. 9, no. 3, pp. 203– 228, 1999. Gert de Cooman, “Belief models: an order theoretic analysis,” In de Cooman et al. [27], pp. 93–103. Peter Walley, “Inferences from multinomial data: Learning about a bag of marbles,” Journal of the Royal Statistical Society, vol. 58, no. 1, pp. 3–34, 1996. Tadeusz Rzezuchowski and Janusz W¸ asowski, “Differential equations with fuzzy parameters via differential inclusions,” Journal of Mathematical Analysis and Applications, vol. 255, pp. 177– 194, 2001. J. von Neumann and O. Morgenstern, Theory of Games and Economic Behavior, Princeton University Press, 1944. T. Seidenfeld, M. J. Schervish, and J. B. Kadane, “A representation of partially ordered preferences,” The Annals of Statistics, vol. 23, pp. 2168–2217, 1995. G. de Cooman, F. G. Cozman, S. Moral, and P. Walley, Eds., ISIPTA ’99 – Proceedings of the First International Symposium on Imprecise Probabilities and Their Applications, Ghent, 1999. Imprecise Probabilities Project. G. de Cooman, T. L. Fine, and T. Seidenfeld, Eds., ISIPTA ’01 – Proceedings of the Second International Symposium on Imprecise Probabilities and Their Applications, Maastricht, 2001. Shaker Publishing.
