Theoretical and Computational Approaches to Dempster-Shafer (DS) Inference

Transcription

1 Theoretical and Computational Approaches to Dempster-Shafer (DS) Inference Chuanhai Liu Department of Statistics Purdue University Joint work with A. P. Dempster Dec. 11, 2006

2 What is DS Analysis (DSA)? State Space Model (SSM) Reference Dempster (2006, IJAR) The problem X Poisson(L) with L 0 and X = 0, 1,... X L The SSM for the Poisson DS model: SSM = {(L,X) : L 0,X = 0, 1,...} December 11, 2006 Copyright c 2006 by Chuanhai Liu 1

3 What is DSA? DS Model (DSM) The Poisson DSM on the SSM (X,L) is defined by associated (a)- random sets based on a standard Poisson a-process that can be visualized as follows. X V 1 V 2 V 3 V 4 V 5 V 6 V 7 L A typical a-random subset is the union of intervals at levels X = 0, 1, 2,... December 11, 2006 Copyright c 2006 by Chuanhai Liu 2

4 What is DSA? DS Calculus (DSC) Combining with a fixed L yields a standard Poisson inference for X. Combining with an observed X yields an a-random inference for L. A-random sets for inference about L Given an observed count X, a-random sets for inference about L are intervals of the form [V X,V X+1 ] where V X (V X+1 V X ), V X Gamma(X, 1), and (V X+1 V X ) Gamma(1, 1) = Exp(1), that is, V X+1 Gamma(X + 1, 1). DSC in symbols (SSM + DSM) DSC = DSA DSM = DSM 1 DSM 2 DSM n December 11, 2006 Copyright c 2006 by Chuanhai Liu 3

5 What is DSA? DS Output Every assertion that you represent as meaningfully true or false has an associated (p, q, r) where p is your probability for the assertion, q is the probability against, and r is your probability of don t know. Example Suppose X = 0. Then the assertion {0 L 3} has (p,q,r) = (0.95, 0, 0.05) December 11, 2006 Copyright c 2006 by Chuanhai Liu 4

6 What is DSA? DS Output Two-cdfs Taking (p,q,r) to be functions of K in the assertion {0 L K} leads to two cdfs. One is the probability for {0 L K}. The other one is the plausibility of {0 L K}. The difference between the two cdfs at K is the posterior probability of don t know. Example Inference about L with the observed Poisson count X = 4. CDF plausibility s(k) = p(k) + r(k) probability p(k) = p({0<l<k}) r(k) = r({0<l<k}) is the prob. of "don t know" K Two cdfs with the observed count X = 4 December 11, 2006 Copyright c 2006 by Chuanhai Liu 5

7 Elements of DS calculus DS concept of DSM To specify a DSM over a given SSM you specify a mass distribution over the subsets of the SSM, later to be referred to in mathematically more conventional terms as an a-probability measure over a sample space of a-random subsets A S. Mass function m(a) for all A S. Example For the Poisson model, m([a,b]) = f VX (a)f VX+1 V X (b a), (0 a b) where f VX (.) and f VX+1 V X (.) denote the density functions of the a- random variables V X and V X+1 V X, respectively. December 11, 2006 Copyright c 2006 by Chuanhai Liu 6

8 Elements of DS calculus DS concept of DSM (continued) The mass m(a) associated with each assertion A S is an atom in the sense that it cannot be further broken down into pieces assigned to subsets of A. Logically, however, the probability p(a) representing the uncertainty that you unambiguously assign to A is different from m(a) because p(a) accumulates masses from all the assertions that imply A. In symbols, p(a) = B A m(b) where the empty set has m( ) = 0. Example For the Poisson model with X = 0, p({0 L 3}) = P(V X+1 3) = pg(3,x + 1) 0.95 q({0 L 3}) = P(V X > 3) = 0 r({0 L 3}) = 1 p({0 L 3}) q({0 L 3}) 0.05 December 11, 2006 Copyright c 2006 by Chuanhai Liu 7

9 Elements of DS calculus DS combination DS extension Input component DSM s are usually defined on a margin, and must be extended to logically equivalent DSM s before combination is defined. DS combination To combine DS-independent DSM s on a common SSM, intersect sets, multiply probabilities, cumulate, and renormalize away conflict. Commonality The collection of (p, q, r) probabilities associated with any DSM s are from a mathematical perspective set functions. Another set function c(a) = m(b) B A called commonality is technically important because it multiplies under DS combination. December 11, 2006 Copyright c 2006 by Chuanhai Liu 8

10 Poisson models and analyses Multiple Poisson counts Assume n DS-independent variables X 1,X 2,...,X n that represent counts, all occurring at the same rate L per unit time, but having differing known periods τ 1,τ 2,...,τ n. This situation requires a DSM that combines n independent Poisson counts over the state space (L,X 1, X 2,...,X n ). You create this DSM mathematically by starting from marginal pairs (L,X i ) with 2-dimensional Poisson DSM s having rates τ i L, then extending each of these to the full (n+1)-dimensional SSM, and finally performing DS-combination on the full state space. December 11, 2006 Copyright c 2006 by Chuanhai Liu 9

11 Poisson models and analyses The role of join trees Inference about L requires DS-combination of 2n input DSM s consisting of the n observations X i and the n Poisson DSM s associated with the pairs (L,X i ). Computation can be carried out in a brute force way involving extension, combination, and marginalization on the full SSM, or equivalently and much more economically by message passing through a mathematical structure called a join tree, as follows: L L,X 1 L,X 2... L,X n... X 1 X 2... X n The join tree for the multiple Poisson inference of Section 3.2. The arrows on the edges indicate the directions of inward propagation. December 11, 2006 Copyright c 2006 by Chuanhai Liu 10

12 Poisson models and analyses Predicting a new count The join tree for predicting a new count given one or more observed counts is as follows: L L,X 1 L,X 2 X 1 X 2 The join tree for prediction of X 2 with arrows indicating the direction of information flow. December 11, 2006 Copyright c 2006 by Chuanhai Liu 11

13 A model from physics Case (1) The problem Consider an interesting statistical problem in particle physics about detecting the Higgs particle. A particular non-negative quantity, denoted by B, is to be estimated with confidence bounds. Two specific models were proposed by a group of physicists and statisticians for study (Heinrich, 2006). Case (1) An experiment yields three counts X 1, X 2, and X 3 that are independent Poissons with means L 1, L 2, and L 3, respectively, where L 1 = BU + E, L 2 = au, and L 3 = be with a and b known constants coming from the experimental set-up, and U and E unknowns that need to be finessed to get at B. December 11, 2006 Copyright c 2006 by Chuanhai Liu 12

14 A model from physics Case (1) DS inference The three a-random intervals for L 1, L 2, and L 3 [V X1,V X1 +1], [V X2,V X2 +1], and [V X3,V X3 +1] where V X1, V X2, V X3, V X1 +1 V X1, V X2 +1 V X2, and V X3 +1 V X3 are independent Gamma random variables. This leads to the marginal a-random interval with the unknown E and U effectively integrated out for inference about B as follows } a max { 0,V X1 1 b V X 3 +1 V X2 +1 B a V X b V X 3 V X2 (1) subject to the non-conflict constraint V X1 +1 b 1 V X3. The two cdfs for inference about B can obtained numerically by making use of Beta distributions. December 11, 2006 Copyright c 2006 by Chuanhai Liu 13

15 A model from physics Case (1) Example An artificial data set X 1 X 2 X 3 a b a CDFS a likelihood CDF B The a-cdfs and a-likelihood function of the parameter of interest, B. December 11, 2006 Copyright c 2006 by Chuanhai Liu 14

16 DS inference a-likelihood function The degree of don t know r(b), computed as the vertical distance between the two a-cdfs, is of the type plausibility of singleton or equivalently commonality of singleton. So in particular this function r(b) is a kind of marginal likelihood function of B. We call r(b) the a-likelihood function of B. The basic reasons for thinking of r(b) as likelihood are (1) r(b) is the ordinary likelihood in simple parametric inference situations, (2) these r(b)s do in the rigorous DS theory multiply across independent replicates, and (3) if one brought in a prior on B and used r(b) as likelihood in the usual Bayesian way, we would have a rigorous DS inference for B. December 11, 2006 Copyright c 2006 by Chuanhai Liu 15

17 A model from physics Case (2) Case (2) There are 10 replicates of the experiment in case (1) with a common B but different U j, E j, a j and b j for j = 1,..., ,000 copies of the data sets of case (2) were simulated for study. We refer to these 70,000 data sets as replicates and to those 10 repeats within each of the 70,000 data sets as subreplicates. In what follows, we denote by n the number of replicates and by n l the number of subreplicates for l = 1,...,n. Simulated data for study It turns out that the prior distributions of the unreported U s, Es, Bs in the simulated data of 70,000 replicates can be easily ascertained by an exploratory data analysis with graphical techniques and the method of moments. A more realistic situation No prior information is available about the unknown U s, Es, and B. The parameter of interest is of course still B. December 11, 2006 Copyright c 2006 by Chuanhai Liu 16

18 A model from physics Case (2) The formal DSM for case (2) with subreplicates n l involves 3 n l intervals (for L (j) 1, L(j) 2, and L(j) 3 for j = 1,...,n l ). Given the fact that a common B is shared by all the n l subreplicates, the projected DS a-random interval for inference about the common B is formed by the intersection of the n l separate a-random intervals of the type (1). The a-random interval for inference about the common value of B can be obtained by MCMC methods. a-likelihood function The n l subreplicates are DS combined simply by multiplying the individual a-likelihoods r j (B), that is r(b) = n l j=1 r j (B), where r j (B) is the a-likelihood of B given the j-th subreplicate. December 11, 2006 Copyright c 2006 by Chuanhai Liu 17

19 A model from physics Case (2) Example An artificial data set Subreplicate X X X a b Inference about B The a-likelihood function of ln B, and its normal approximation. a likelihood normal approximation 1.5 density lnb December 11, 2006 Copyright c 2006 by Chuanhai Liu 18

20 Multiple significance testing The data The data The numbers of faults in 32 rolls of textile fabric (Bissell 1972; see also Cox and Snell, 1981 and Gelman et al, 1995) No. of Faults The green line was fitted by "eye" to the "good" data points The vertical green line segments are the (1/32) predictive intervals (PIs) The red dots are the points laying outside of the 1/32 PIs (371, 14) (735, 17) (895, 28) (905, 23) Roll Length (meters) December 11, 2006 Copyright c 2006 by Chuanhai Liu 19

21 Multiple significance testing A statistical model Data structure The plot indicates that it is reasonable to assume that for each i = 1,..., 32, the number of faults in roll i follows Poission with mean τ i L i, where τ i is the known roll length. One could imagine that (1) most of the rolls were manufactured with the underlying product line being under control and (2) a small unknown number of rolls were manufactured with the product line being out of control. A statistical model For i = 1,..., let X i be the number of faults in roll i and assume that X i s are independent with X i Poisson(τ i L i ). Most of L i s are the same and equal to a common unknown L 0, except for a small unknown number, denoted by K, of L i s that are different from (greater than) L 0. We refer to L i s that are different from L 0 as outliers. December 11, 2006 Copyright c 2006 by Chuanhai Liu 20

22 Multiple significance testing a-random intervals Non-conflict a-random intervals [ ] [ ] [ ] [ ] [ ] [ ] [ ] [ ] [ λ o1 ] [ λ o2 ] λ L 0 [Z l = the (K + 1)-th largest V Xi /τ i,z u = min V Xi +1/τ i ]. December 11, 2006 Copyright c 2006 by Chuanhai Liu 21

23 Multiple significance testing Inference about K The basic idea Assume K K max and compute (p,q,r)s for the sequence of assertions {K = k} for k = K max,k max 1,..., 1, i.e., (p k,q k,r k ) = (P({K = k}), 0, 1 P({K = k})) assuming K = 0,...,k for each k = 1,...,K max. A frequency coverage problem k = 1 with K max = 1. p k 1 as n, e.g., K = 0 and December 11, 2006 Copyright c 2006 by Chuanhai Liu 22

24 Multiple significance testing Inference about K Logit p k Plot of logit(p k ) for the observed data and logit(p k ) for 10 simulated data sets without outliers logit(p_k) (Simulated data) logit(p_k) (Observed data) December 11, 2006 Copyright c 2006 by Chuanhai Liu 23

25 Multiple significance testing Inference about K Ratios of minimum deviations Let Z u be the upper end point of the a- random interval for L 0, i.e., Z u = minv Xi +1. Let V (K) l be the K- th largest lower end points of the n a-random intervals. Histograms of V (K) l /Z u for k = 1,..., 8: k = 1 k = 2 k = 3 k = 4 Frequency Frequency Frequency Frequency R R R R k = 5 k = 6 k = 7 k = 8 Frequency Frequency Frequency Frequency R R R R December 11, 2006 Copyright c 2006 by Chuanhai Liu 24

26 Multiple significance testing Identification of outliers Outliers (example) Taking K = 5 leads to the following posterior probabilities of {L i > L 0 } for i = 1,...,n P(X_i is an outlier) X_i/tau_i December 11, 2006 Copyright c 2006 by Chuanhai Liu 25

27 Multiple significance testing An alternative DSM DS model (with latent variables) This model is under investigation. The SSM for each i consists of six variables I, X, Y, Z, L, and M, where Y Poisson(L), Z Poisson(M) with (M > M 0 > 0), I = 0 or 1, X = Y if I = 0 and X = Y + Z if I = 1. ( a ) I = 0 ( b ) I = 1 Y Y Z 0 Z M M The small circles in the (Z,Y ) plane represent the possible points at I = 0 and I = 1 when the observed X is fixed at 5. December 11, 2006 Copyright c 2006 by Chuanhai Liu 26