Continuous updating rules for imprecise probabilities

Size: px

Start display at page:

Download "Continuous updating rules for imprecise probabilities"

Buck Hodge
5 years ago
Views:

1 Continuous updating rules for imprecise probabilities Marco Cattaneo Department of Statistics, LMU Munich WPMSIIP 2013, Lugano, Switzerland 6 September 2013

2 example X {1, 2, 3} Marco LMU Munich Continuous updating rules for imprecise probabilities 2/9

3 example X {1, 2, 3} prior assessment: P(X ) =... Marco LMU Munich Continuous updating rules for imprecise probabilities 2/9

4 example X {1, 2, 3} prior assessment: P(X ) =... observation: A = {X 2} Marco LMU Munich Continuous updating rules for imprecise probabilities 2/9

5 example X {1, 2, 3} prior assessment: P(X ) =... observation: A = {X 2} updated prevision: P(X A) =... Marco LMU Munich Continuous updating rules for imprecise probabilities 2/9

6 example X {1, 2, 3} prior assessment: P(X ) =... observation: A = {X 2} updated prevision: P(X A) =... natural/regular extension: P(X A) P(X ) Marco LMU Munich Continuous updating rules for imprecise probabilities 2/9

7 updating likelihood of linear previsions: lik(p) P(A) Marco LMU Munich Continuous updating rules for imprecise probabilities 3/9

8 updating likelihood of linear previsions: lik(p) P(A) Bayesian updating: P P( A) when lik(p) 0 Marco LMU Munich Continuous updating rules for imprecise probabilities 3/9

9 updating likelihood of linear previsions: lik(p) P(A) Bayesian updating: P P( A) when lik(p) 0 regular updating: P inf P M(P) : lik(p)>0 P( A) when lik 0 on M(P) Marco LMU Munich Continuous updating rules for imprecise probabilities 3/9

10 updating likelihood of linear previsions: lik(p) P(A) Bayesian updating: P P( A) when lik(p) 0 regular updating: P inf P M(P) : lik(p)>0 P( A) when lik 0 on M(P) the discontinuities are caused by linear previsions P M(P) with arbitrarily small likelihood (i.e., almost contradicted by the data) Marco LMU Munich Continuous updating rules for imprecise probabilities 3/9

11 updating likelihood of linear previsions: lik(p) P(A) Bayesian updating: P P( A) when lik(p) 0 regular updating: P inf P M(P) : lik(p)>0 P( A) when lik 0 on M(P) the discontinuities are caused by linear previsions P M(P) with arbitrarily small likelihood (i.e., almost contradicted by the data) β-cut updating, with β (0, 1): P inf P M(P) : lik(p)>β P(A) P( A) when lik 0 on M(P) Marco LMU Munich Continuous updating rules for imprecise probabilities 3/9

12 updating likelihood of linear previsions: lik(p) P(A) Bayesian updating: P P( A) when lik(p) 0 regular updating: P inf P M(P) : lik(p)>0 P( A) when lik 0 on M(P) the discontinuities are caused by linear previsions P M(P) with arbitrarily small likelihood (i.e., almost contradicted by the data) β-cut updating, with β (0, 1): P inf P M(P) : lik(p)>β P(A) P( A) when lik 0 on M(P) regular updating is the limit of β-cut updating as β 0, while the limit as β 1 is a generalization of Dempster s rule of conditioning Marco LMU Munich Continuous updating rules for imprecise probabilities 3/9

13 updating likelihood of linear previsions: lik(p) P(A) Bayesian updating: P P( A) when lik(p) 0 regular updating: P inf P M(P) : lik(p)>0 P( A) when lik 0 on M(P) the discontinuities are caused by linear previsions P M(P) with arbitrarily small likelihood (i.e., almost contradicted by the data) β-cut updating, with β (0, 1): P inf P M(P) : lik(p)>β P(A) P( A) when lik 0 on M(P) regular updating is the limit of β-cut updating as β 0, while the limit as β 1 is a generalization of Dempster s rule of conditioning β-cut updating was used, e.g., in Antonucci et al. (2012); Cattaneo and Wiencierz (2012); Destercke (2013), and similar updating rules for imprecise probabilities were suggested, e.g., in Moral and de Campos (1991); Moral (1992); Cano and Moral (1996) Marco LMU Munich Continuous updating rules for imprecise probabilities 3/9

14 example β-cut updating: P(X A) β = 0.5 β = 0.1 β 0 β P(X ) Marco LMU Munich Continuous updating rules for imprecise probabilities 4/9

15 example β-cut updating: P(X A) β = 0.5 β = 0.1 β 0 β P(X ) the slope of the central line segment is 1 /β Marco LMU Munich Continuous updating rules for imprecise probabilities 4/9

16 a more interesting example Piatti et al. (2005) studied the behavior of the IDM model when the realization of each binary experiment X 1, X 2,... can be observed incorrectly with a known probability ε (the errors of observation are independent, conditional on the realizations of X 1, X 2,...) Marco LMU Munich Continuous updating rules for imprecise probabilities 5/9

17 a more interesting example Piatti et al. (2005) studied the behavior of the IDM model when the realization of each binary experiment X 1, X 2,... can be observed incorrectly with a known probability ε (the errors of observation are independent, conditional on the realizations of X 1, X 2,...) observation: X X 10 = 7 (i.e., 7 successes in 10 experiments) Marco LMU Munich Continuous updating rules for imprecise probabilities 5/9

18 a more interesting example Piatti et al. (2005) studied the behavior of the IDM model when the realization of each binary experiment X 1, X 2,... can be observed incorrectly with a known probability ε (the errors of observation are independent, conditional on the realizations of X 1, X 2,...) observation: X X 10 = 7 (i.e., 7 successes in 10 experiments) lower probability of success in the next experiment (with s = 2): P(X 11 = 1 X X 10 = 7) =... Marco LMU Munich Continuous updating rules for imprecise probabilities 5/9

19 a more interesting example Piatti et al. (2005) studied the behavior of the IDM model when the realization of each binary experiment X 1, X 2,... can be observed incorrectly with a known probability ε (the errors of observation are independent, conditional on the realizations of X 1, X 2,...) observation: X X 10 = 7 (i.e., 7 successes in 10 experiments) lower probability of success in the next experiment (with s = 2): P(X 11 = 1 X X 10 = 7) =... regular updating: P(X 11 = 1 X X 10 = 7) { if ε = 0 0 if ε > 0 Marco LMU Munich Continuous updating rules for imprecise probabilities 5/9

20 a more interesting example Piatti et al. (2005) studied the behavior of the IDM model when the realization of each binary experiment X 1, X 2,... can be observed incorrectly with a known probability ε (the errors of observation are independent, conditional on the realizations of X 1, X 2,...) observation: X X 10 = 7 (i.e., 7 successes in 10 experiments) lower probability of success in the next experiment (with s = 2): P(X 11 = 1 X X 10 = 7) =... regular updating: P(X 11 = 1 X X 10 = 7) { if ε = 0 0 if ε > 0 β-cut updating with β = 0.01: if ε = if ε = 0.01 P(X 11 = 1 X X 10 = 7) if ε = if ε = 0.1 Marco LMU Munich Continuous updating rules for imprecise probabilities 5/9

21 distances distance between two linear previsions P, P on Ω (dual/operator norm = total variation distance): d(p, P ) = sup P(X ) P (X ) X : Ω [ 1, 1] = 2 sup P(A) P (A) A Ω Marco LMU Munich Continuous updating rules for imprecise probabilities 6/9

22 distances distance between two linear previsions P, P on Ω (dual/operator norm = total variation distance): d(p, P ) = sup P(X ) P (X ) X : Ω [ 1, 1] = 2 sup P(A) P (A) A Ω distance between two (coherent) lower previsions P, P on Ω (Hausdorff distance = dual/operator norm ): { d(p, P ) = max sup inf d(p, P M(P) P M(P ) P ), = sup P(X ) P (X ) X : Ω [ 1, 1] sup P M(P ) inf d(p, P M(P) P ) } Marco LMU Munich Continuous updating rules for imprecise probabilities 6/9

23 continuity Bayesian updating is continuous on all linear previsions P with P(A) > 0 Marco LMU Munich Continuous updating rules for imprecise probabilities 7/9

24 continuity Bayesian updating is continuous on all linear previsions P with P(A) > 0 regular updating is continuous on all lower previsions P with P(A) > 0 Marco LMU Munich Continuous updating rules for imprecise probabilities 7/9

25 continuity Bayesian updating is continuous on all linear previsions P with P(A) > 0 regular updating is continuous on all lower previsions P with P(A) > 0 regular updating is not continuous on all lower previsions P with P(A) > 0, apart if A = Ω or A = 1 (and the same holds for natural extension) Marco LMU Munich Continuous updating rules for imprecise probabilities 7/9

26 continuity Bayesian updating is continuous on all linear previsions P with P(A) > 0 regular updating is continuous on all lower previsions P with P(A) > 0 regular updating is not continuous on all lower previsions P with P(A) > 0, apart if A = Ω or A = 1 (and the same holds for natural extension) β-cut updating is continuous on all lower previsions P with P(A) > 0, for each β (0, 1) Marco LMU Munich Continuous updating rules for imprecise probabilities 7/9

27 hierarchical model a likelihood function lik on the set P of all linear previsions on Ω is described by its normal hypograph {(P, β) P [0, 1] : β sup lik lik (P)} Marco LMU Munich Continuous updating rules for imprecise probabilities 8/9

28 hierarchical model a likelihood function lik on the set P of all linear previsions on Ω is described by its normal hypograph {(P, β) P [0, 1] : β sup lik lik (P)} hierarchical updating: lik lik (P ) sup P P : P( A)=P lik (P) lik(p) when lik lik 0 Marco LMU Munich Continuous updating rules for imprecise probabilities 8/9

29 hierarchical model a likelihood function lik on the set P of all linear previsions on Ω is described by its normal hypograph {(P, β) P [0, 1] : β sup lik lik (P)} hierarchical updating: lik lik (P ) sup P P : P( A)=P lik (P) lik(p) when lik lik 0 distance between two likelihood functions lik, lik on P: Hausdorff distance between their normal hypographs, with respect to a product metric on P [0, 1] Marco LMU Munich Continuous updating rules for imprecise probabilities 8/9

30 hierarchical model a likelihood function lik on the set P of all linear previsions on Ω is described by its normal hypograph {(P, β) P [0, 1] : β sup lik lik (P)} hierarchical updating: lik lik (P ) sup P P : P( A)=P lik (P) lik(p) when lik lik 0 distance between two likelihood functions lik, lik on P: Hausdorff distance between their normal hypographs, with respect to a product metric on P [0, 1] hierarchical updating is continuous on all likelihood functions lik with lik lik 0 Marco LMU Munich Continuous updating rules for imprecise probabilities 8/9

31 references Antonucci, A., Cattaneo, M., and Corani, G. (2012). Likelihood-based robust classification with Bayesian networks. In Advances in Computational Intelligence, Part 3. Springer, Cano, A., and Moral, S. (1996). A genetic algorithm to approximate convex sets of probabilities. In IPMU 96. Vol. 2. Proyecto Sur, Cattaneo, M., and Wiencierz, A. (2012). Likelihood-based Imprecise Regression. Int. J. Approx. Reasoning 53, Destercke, S. (2013). A pairwise label ranking method with imprecise scores and partial predictions. In Machine Learning and Knowledge Discovery in Databases. Springer, Moral, S. (1992). Calculating uncertainty intervals from conditional convex sets of probabilities. In UAI 92. Morgan Kaufmann, Moral, S., and de Campos, L. M. (1991). Updating uncertain information. In Uncertainty in Knowledge Bases. Springer, Piatti, A., Zaffalon, M., and Trojani, F. (2005). Limits of learning from imperfect observations under prior ignorance: the case of the imprecise Dirichlet model. In ISIPTA 05. SIPTA, Marco LMU Munich Continuous updating rules for imprecise probabilities 9/9

Credal Classification

Credal Classification A. Antonucci, G. Corani, D. Maua {alessandro,giorgio,denis}@idsia.ch Istituto Dalle Molle di Studi sull Intelligenza Artificiale Lugano (Switzerland) IJCAI-13 About the speaker PhD