Methods of Exact Inference. Cutset Conditioning. Cutset Conditioning Example 1. Cutset Conditioning Example 2. Cutset Conditioning Example 3

Transcription

1 Methods of xact Inference Lecture 11: xact Inference means modelling all the dependencies in the data. xact Inference his is only computationally feasible for small or sparse networks. Methods: utset onditioning (Pearl) Node ombination Join rees (Lauritizen and peigelhalter) Intelligent ata nalysis and Probabilistic Inference Lecture 11 lide No 1 Intelligent ata nalysis and Probabilistic Inference Lecture 11 lide No 2 utset onditioning Pearl suggested a method for propagating probabilities in multiply connected networks. utset onditioning xample 1 Given the sia network, the minimum cutset consists of node L. If L is instantiated propagation is safe. he first step is to identify a cut set of nodes, which if instantiated makes propagation safe. L L sia ronchitis yspnea Lung ancer moking uberculosis Ray Intelligent ata nalysis and Probabilistic Inference Lecture 11 lide No 3 Intelligent ata nalysis and Probabilistic Inference Lecture 11 lide No 4 utset onditioning xample 2 If data is not available on L the network can be broken into 2 networks, one for each possible instantiated state of L. oth networks are singly connected: utset onditioning xample 3 Probabilities can be propagated in both networks safely. final value for the probability can be found by weighting the results according to the prior probability distribution for L. L1 L1 L1 L2 L2 L2 Intelligent ata nalysis and Probabilistic Inference Lecture 11 lide No 5 Intelligent ata nalysis and Probabilistic Inference Lecture 11 lide No 6

2 Problems with cutset conditioning 1 he method relies on finding a small cutset. If the cutset consists of three nodes each with 8 states then the probability propagations need to be carried out 8 3 = 512 times. hus the computation time expands quickly Problems with cutset conditioning 2 For the previous example we need to make 512 networks each with a separate instantiation of the three cutset variables. We also need to find priors for all these possible instantiations. here may not be enough data to do this reliably. Intelligent ata nalysis and Probabilistic Inference Lecture 11 lide No 7 Intelligent ata nalysis and Probabilistic Inference Lecture 11 lide No 8 Joining dependent variables Instantiating Joined Variables If and are dependent (given ) we can combine them into one new variable by joining them. & If we instantiate one of or, but not the other, (eg =d2) we must either re-calculate the link matrix: P(& ) => P( ) =d2 or instantiate several states: Given & has states {c1d1, c2d1, c3d1,c1d2,c2d2, c3d2 } If we instantiate =d2 we have P(& ) & λ(&) = {0,0,0,1,1,1} Intelligent ata nalysis and Probabilistic Inference Lecture 11 lide No 9 Intelligent ata nalysis and Probabilistic Inference Lecture 11 lide No 10 Limitation of joining variables Join rees If two variables are to be joined, having states and having states the new variable & will have * states. he increased number of states is undesirable and limits the applicability of this approach In the limit a fully connected network degenerates into one node! Join trees are a generalisation of the previous method. Given a network, with all dependencies included as arcs, the objective is to find a way of joining the variables such that propagation is possible. his idea was originally pioneered by Lauritzen and peigelhalter. Intelligent ata nalysis and Probabilistic Inference Lecture 11 lide No 11 Intelligent ata nalysis and Probabilistic Inference Lecture 11 lide No 12

3 xample We shall use the following medical example which is taken from Neapolitan s first book: a1 a2 Metastatic ancer b1 b2 otal erum alcium Increased c1 c2 rain umour d1 d2 omna e1 e2 Papilledema he join tree P() P( ) P( ) P( &) P( &) P()P( )P( ) he join tree has the same joint probability distribution as the original tree Intelligent ata nalysis and Probabilistic Inference Lecture 11 lide No 13 Intelligent ata nalysis and Probabilistic Inference Lecture 11 lide No 14 Join tree nodes Instantiation and propagation P()P( )P( ) π P()P( )P( ) π P( &) λ λ P( &) he nodes of the original tree are joined according to how the probability matrices are grouped. Propagation is done in just two passes, up followed by down. he messages are not the same as in Pearl s algorithm Intelligent ata nalysis and Probabilistic Inference Lecture 11 lide No 15 Intelligent ata nalysis and Probabilistic Inference Lecture 11 lide No 16 Potential Representations Potential Representations Grouping the link matrices of the original tree to form a join tree could be done in a number of ways, and each is called a potential representation. potential representation is a number of subsets Wi (nodes in the join tree) of our variables (nodes in the original tree), and a function ψ with the property: P(V) = Π ψ(wi) i P(V) = Π ψ(wi) i In our previous example: ψ(w1) = P()P( )P( ) ψ(w2) = P( &) ψ(w3) = P(V) = P(&&&&) P( &) P()P( )P( ) Intelligent ata nalysis and Probabilistic Inference Lecture 11 lide No 17 Intelligent ata nalysis and Probabilistic Inference Lecture 11 lide No 18

4 Intersections For a potential representation, given an ordering, we can define some intersection sets: i = Wi (W1 W2 W3.... Wi-1) (he variables in Wi which are in any lower index set) Intersection ets Wi i Ri P()P( )P( ) Ri = Wi - i (he variables in Wi that have not appeared already) P( &) N Lower index => parent Intelligent ata nalysis and Probabilistic Inference Lecture 11 lide No 19 Intelligent ata nalysis and Probabilistic Inference Lecture 11 lide No 20 he running intersection property o permit propagation, the potential representation must have the running intersection property: Given an ordered set of subsets of our variables V, for any adjacent sets Vi and Vj such that j<i: i = Wi (W1 W2 W3.... Wi-1) Wj If this is so we can write: Running Intersection Property Wi i Ri First Possibility P( &) P()P( )P( ) p P(V) = P(W1) Π P(Ri i) i=2 Intelligent ata nalysis and Probabilistic Inference Lecture 11 lide No 21 First Possible Join ree Intelligent ata nalysis and Probabilistic Inference Lecture 11 lide No 22 Running Intersection Property - another tree Wi i Ri P()P( )P( ) P( &) ummary he ordering is set up so that each node of the join tree has an associated set of variables R R is a set of variables that have not been seen above is a set of variables which must appear in the immediate parents. hus we will have a link matrix P(R ) Intelligent ata nalysis and Probabilistic Inference Lecture 11 lide No 23 Intelligent ata nalysis and Probabilistic Inference Lecture 11 lide No 24

5 Finding the ordered subset of the nodes n ordered subset of the variables can always be found from a given causal graph as follows: 1. Moralise the graph (join any unjoined parents) 2. riangulate the graph 3. Identify the cliques of the resulting graph 4. For each variable choose one clique Li such that Pa() Li (Pa() means parents of ) 5. efine the function ψ as follows: ψ(li) = Π P( Pa()) Li 1,2. Moralising and triangulation fter moralising this graph is triangulated so there is no need for a triangulation step Intelligent ata nalysis and Probabilistic Inference Lecture 11 lide No 25 Intelligent ata nalysis and Probabilistic Inference Lecture 11 lide No Finding the liques 4. llocating variables to liques lique 1 We need to allocate the variables to cliques such that their original parents are in the same clique: lique 2 lique 3 clique is a maximal set of nodes in which every node is connected to every other Variable lique lique Variables 3, 2,, 1,, 1,, 1,, Intelligent ata nalysis and Probabilistic Inference Lecture 11 lide No 27 Intelligent ata nalysis and Probabilistic Inference Lecture 11 lide No Initialising Potential functions fter allocating the variables we initialise clique potential functions from the conditional probabilities Variable lique lique Variables lique Function Ψ 3, 2,, P( &) 1,, P()P( )P( ) 1,, 1,, efining the tree of cliques We now find an ordering of the cliques with the running intersection property. hat is: here is some ordering such that if j<i i = li (l1 l2... li-1) lj lj can be taken as a parent of li Intelligent ata nalysis and Probabilistic Inference Lecture 11 lide No 29 Intelligent ata nalysis and Probabilistic Inference Lecture 11 lide No 30

6 pplying this to our example 2 = l2 l1 = {,,} {,,} = {,} learly {,} {,,} o we choose l1 to be the parent of l2 3 = l3 (l1 l2) = {,} {,,,} = {} We have that {} is a subset of both {,,} and {,,} so either l1 or l2 can be the parent of l3 he Join tree he join tree is now completed. In all future probability calculations we use just the join tree, disregarding our original network L1 L2 L2 L1 L3 L3 he two possible join trees Intelligent ata nalysis and Probabilistic Inference Lecture 11 lide No 31 Intelligent ata nalysis and Probabilistic Inference Lecture 11 lide No 32