Outline } Exact inference by enumeration } Exact inference by variable elimination } Approximate inference by stochastic simulation } Approximate infe

Transcription

1 Inference in belief networks Chapter 15.3{4 + new AIMA Slides cstuart Russell and Peter Norvig, 1998 Chapter 15.3{4 + new 1

2 Outline } Exact inference by enumeration } Exact inference by variable elimination } Approximate inference by stochastic simulation } Approximate inference by Markov chain Monte Carlo AIMA Slides cstuart Russell and Peter Norvig, 1998 Chapter 15.3{4 + new 2

3 queries: compute posterior marginal P(X i je e) Simple P (NoGasjGauge empty Lights on Starts false) e.g., Conjunctive queries: P(X i X j je e) P(X i je e)p(x j jx i E e) Inference tasks decisions: decision networks include utility information Optimal inference required for P (outcomejaction evidence) probabilistic Value of information: which evidence to seek next? Sensitivity analysis: which probability values are most critical? Explanation: why do I need a new starter motor? AIMA Slides cstuart Russell and Peter Norvig, 1998 Chapter 15.3{4 + new 3

4 P(B J true M true)p (J true M true) P(B J true M true) e a P (B true)p (e)p (ajb true e)p (J trueja)p (M trueja) P (B true) e P (e) a P (ajb true e)p (J trueja)p (M trueja) Inference by enumeration intelligent way to sum out variables from the joint without actually Slightly constructing its explicit representation query on the burglary network: Simple true M true) P(BjJ e a P(B e a J true M true) full joint entries using product of CP entries: Rewrite (B truejj true M true) P AIMA Slides cstuart Russell and Peter Norvig, 1998 Chapter 15.3{4 + new 4

5 returns a distribution over X EnumerationAsk(X,e,bn) X, the query variable inputs: X do of e with value x i extend for each value x i return Normalize(Q(X)) returns arealnumber EnumerateAll(vars,e) Empty?(vars) then return 1.0 if do else Y Y has value y in e if return P (y j Pa(Y )) EnumerateAll(Rest(vars),e) then Enumeration algorithm Exhaustive depth-rst enumeration: O(n) space, O(d n ) time e, evidence specied as an event bn, a belief network specifying joint distribution P(X 1 ::: X n ) Q(X ) a distribution over X for X Q(x i ) EnumerateAll(Vars[bn],e) irst(vars) return P y P (y j Pa(Y )) EnumerateAll(Rest(vars),e y ) else e y is e extended with Y y where AIMA Slides cstuart Russell and Peter Norvig, 1998 Chapter 15.3{4 + new 5

6 P(B) {z } B e P (e) {z } E a P(ajB e) {z } A (J trueja) {z } P J P(B) e P (e) a P(ajB e)p (J trueja)f M (a) P(B) e P (e) a P(ajB e)f J (a)f M (a) P(B)f E AJM (b) (sum out E) f B (b) f E AJM (b) (M trueja) {z } P M Inference by variable elimination is inecient: repeated computation Enumeration computes P (J trueja)p (M trueja) for each value of e e.g., elimination: carry out summations right-to-left, Variable intermediate results (factors) to avoid recomputation storing P(BjJ true M true) P(B) e P (e) a f A (a b e)f J (a)f M (a) P(B) e P (e)f AJM (b e) (sum out A) AIMA Slides cstuart Russell and Peter Norvig, 1998 Chapter 15.3{4 + new 6

7 f(x 1 ::: x j y 1 ::: y k z 1 ::: z l ) f 1 (a b) f 2 (b c) f(a b c) E.g., x f 1 f k f 1 f i x f i+1 f k f 1 f i f X Variable elimination: Basic operations product of factors f 1 and f 2 : Pointwise 1 (x 1 ::: x j y 1 ::: y k ) f 2 (y 1 ::: y k z 1 ::: z l ) f out a variable from a product of factors: move any constant Summing outside the summation: factors assuming f 1 ::: f i do not depend on X AIMA Slides cstuart Russell and Peter Norvig, 1998 Chapter 15.3{4 + new 7

8 EliminationAsk(X,e,bn) returns a distribution over X function X, the query variable inputs: X 2 e then return observed point distribution for X if [] vars Reverse(Vars[bn]) factors each var in vars do for [Makeactor(var e)jfactors] factors return Normalize(PointwiseProduct(factors)) Variable elimination algorithm e, evidence specied as an event bn, a belief network specifying joint distribution P(X 1 ::: X n ) if var is a hidden variable then factors SumOut(var,factors) AIMA Slides cstuart Russell and Peter Norvig, 1998 Chapter 15.3{4 + new 8

9 { time and space cost of variable elimination are O(d k n) Complexity of exact inference connected networks (or polytrees): Singly any two nodesare connected by at most one (undirected) path { connected networks: Multiply can reduce 3SA to exact inference ) NP-hard { { equivalent to counting 3SA models ) #P-complete 1. A v B v C A B C D 2. C v D v ~A B v C v ~D AND AIMA Slides cstuart Russell and Peter Norvig, 1998 Chapter 15.3{4 + new 9

10 Inference by stochastic simulation idea: Basic Draw N samples from a sampling distribution S 1) Compute an approximate posterior probability ^P 2) Show this converges to the true probability P 3) Outline: Sampling from an empty network { Rejection sampling: reject samples disagreeing with evidence { Likelihood weighting: use evidence to weight samples { MCMC: sample from a stochastic process whose stationary { is the true posterior distribution AIMA Slides cstuart Russell and Peter Norvig, 1998 Chapter 15.3{4 + new 10

11 PriorSample(bn) returns an event sampled from P(X 1 ::: X n ) specied by bn function an event with n elements x i 1 to n do for i a random sample from P(X i j Parents(X i )) x h0:8 0:2i P(RainjCloudy)! true sample Sampling from an empty network return x P(C).5 Cloudy 0:5i P(Cloudy)h0:5! true sample C P(S) Sprinkler Rain C P(R) :9i P(SprinklerjCloudy)h0:1! false sample Wet Grass S R P(W) etgrassj:sprinkler Rain) h0:9 0:1i P(W! true sample AIMA Slides cstuart Russell and Peter Norvig, 1998 Chapter 15.3{4 + new 11

12 P (Y y) is, estimates derived from PriorSample are consistent hat Sampling from an empty network contd. that PriorSample generates a particular event Probability PS (x 1 :::x n ) n i 1 P (x ijparents(x i )) P (x 1 :::x n ) S i.e., the true prior probability N PS (Y y) be the number of samples generated for whichy y, Let any set of variables Y. for hen ^P (Y y) N PS (Y y)n and lim N!1 ^P (Y y) h S PS (Y y H h) h P (Y y H h) AIMA Slides cstuart Russell and Peter Norvig, 1998 Chapter 15.3{4 + new 12

13 ^P(Xje) estimated from samples agreeing with e RejectionSampling(X,e,bn,N) returns an approximation to P (Xje) function avector of counts over X, initially zero N[X] j 1toN do for PriorSample(bn) x x is consistent withe then if N[x]+1 where x is the value of X in x N[x] return Normalize(N[X]) ^P(RainjSprinkler true) Normalize(h8 19i) h0:296 0:704i Rejection sampling estimate P(RainjSprinkler true) using 100 samples E.g., samples have Sprinkler true 27 Of these, 8 have Rain true and 19 have Rain false. Similar to a basic real-world empirical estimation procedure AIMA Slides cstuart Russell and Peter Norvig, 1998 Chapter 15.3{4 + new 13

14 ^P(Xje) N PS (X e) Analysis of rejection sampling defn.) (algorithm N PS (X e)n PS (e) (normalized by N PS (e)) (property ofpriorsample) P(X e)p (e) P(Xje) (defn. of conditional probability) Hence rejection sampling returns consistent posterior estimates Problem: hopelessly expensive if P (e) is small AIMA Slides cstuart Russell and Peter Norvig, 1998 Chapter 15.3{4 + new 14

15 WeightedSample(bn,e) returns an event andaweight function an event with n elements w 1 x i 1to n do for X i has a value x i in e if return x, w w w P (X i x i j Parents(X i )) then x i a random sample from P(X i j P arents(x i )) else LikelihoodWeighting(X,e,bn,N) returns an approximation to P (Xje) function ] avector of weighted counts over X, initially zero W[X j 1toN do for WeightedSample(bn) x,w return Normalize(W[X ]) Likelihood weighting x evidence variables, sample only nonevidence variables, Idea: weight each sample by the likelihood it accords the evidence and W[x ] W[x ]+w where x is the value of X in x AIMA Slides cstuart Russell and Peter Norvig, 1998 Chapter 15.3{4 + new 15

16 Likelihood weighting example P(RainjSprinkler true W etgrass true) Estimate P(C).5 Cloudy C P(S) true Sprinkler true Rain C P(R) Wet Grass S R P(W) AIMA Slides cstuart Russell and Peter Norvig, 1998 Chapter 15.3{4 + new 16

17 w P (Sprinkler truejcloudy true) 0:1 w Sample P(RainjCloudy true) h0:8 0:2i say true 4. LW example contd. generation process: Sample w 1:0 1. Sample P(Cloudy) h0:5 0:5i say true 2. Sprinkler has value true, so 3. WetGrass has value true, so 5. w P (W etgrass truejsprinkler true Rain true) 0:099 w AIMA Slides cstuart Russell and Peter Norvig, 1998 Chapter 15.3{4 + new 17

18 probability for WeightedSample is Sampling WS (y e) l i 1 P (y ijp arents(y i )) S l i 1 P (y ijparents(y i )) m i 1 P (e ijp arents(e i )) P (y e) (by standard global semantics of network) Likelihood weighting analysis pays attention to evidence in ancestors only Note: somewhere \in between" prior and posterior distribution ) for a given sample y e is Weight e) m i 1 P (e ijparents(e i )) w(y sampling probability is Weighted WS (y e)w(y e) S likelihood weighting returns consistent estimates Hence performance still degrades with many evidence variables but AIMA Slides cstuart Russell and Peter Norvig, 1998 Chapter 15.3{4 + new 18

19 MCMC-Ask(X,e,bn,N) returns an approximation to P (Xje) function variables: N[X ], a vector of counts over X, initially zero local the nonevidence variables in bn Y, the current state of the network, initially copied from e x, x with random values for the variables in Y initialize j 1toN do for Ydo in the value of Y i sample for each Y i return Normalize(N[X ]) Approximate inference using MCMC \State" of network current assignment to all variables next state by sampling one variable given Markov blanket Generate each variable in turn, keeping evidence xed Sample N[x ] N[x ] + 1 where x is the value of X in x in x from P(Y i jmb(y i )) given the values of MB(Y i )inx stationary distribution: long-run fraction of time spent in Approaches state is exactly proportional to its posterior probability each AIMA Slides cstuart Russell and Peter Norvig, 1998 Chapter 15.3{4 + new 19

20 MCMC Example Estimate P(RainjSprinkler true W etgrass true) Cloudy then Rain, repeat. Sample number of times Rain is true and false in the samples. Count blanket of Cloudy is Sprinkler and Rain Markov blanket of Rain is Cloudy, Sprinkler, andw etgrass Markov AIMA Slides cstuart Russell and Peter Norvig, 1998 Chapter 15.3{4 + new 20 C P(S) true Sprinkler P(C).5 Cloudy true Wet Grass S R P(W) Rain C P(R).80.20

21 P(CloudyjMB(Cloudy)) P(CloudyjSprinkler :Rain) 1.! false sample P(RainjMB(Rain)) P(Rainj:Cloudy Sprinkler W etgrass) 2.! true sample MCMC example contd. Random initial state: Cloudy true and Rain false 100 states Visit have Rain true, 69 have Rain false 31 true W etgrass true) ^P(RainjSprinkler Normalize(h31 69i) h0:31 0:69i AIMA Slides cstuart Russell and Peter Norvig, 1998 Chapter 15.3{4 + new 21

22 ransition probability q(y! y 0 ) condition on t denes stationary distribution (y) Equilibrium stationary distribution depends on choice of q(y! y 0 ) Note: MCMC analysis: Outline Occupancy probability t (y) at time t Pairwise detailed balance on states guarantees equilibrium sampling transition probability: Gibbs each variable given current values of all others sample ) detailed balance with the true posterior Bayesian networks, Gibbs sampling reduces to or conditioned on each variable's Markov blanket sampling AIMA Slides cstuart Russell and Peter Norvig, 1998 Chapter 15.3{4 + new 22

23 (y 0 ) y (y)q(y! y 0 ) for ally 0 Stationary distribution t (y) probability in state y at time t t+1 (y 0 ) probability in state y 0 at time t +1 t+1 in terms of t and q(y! y 0 ) t+1 (y 0 ) y t (y)q(y! y 0 ) Stationary distribution: t t+1 If exists, it is unique (specic to q(y! y 0 )) In equilibrium, expected \outow" expected \inow" AIMA Slides cstuart Russell and Peter Norvig, 1998 Chapter 15.3{4 + new 23

24 (y)q(y! y 0 )(y 0 )q(y 0! y) for ally y 0 Detailed balance \Outow" \inow" for each pair of states: balance ) stationarity: Detailed y (y)q(y! y 0 ) y (y 0 )q(y 0! y) (y 0 ) y q(y 0! y) (y 0 ) algorithms typically constructed by designing a transition MCMC q that is in detailed balance with desired probability AIMA Slides cstuart Russell and Peter Norvig, 1998 Chapter 15.3{4 + new 24

25 Sample each variable in turn, given all other variables Gibbs sampling Y i,let Y i be all other nonevidence variables Sampling values are y i and y i e is xed Current ransition probability is given by q(y! y 0 )q(y i y i! y 0 i y i )P (y 0 ij y i e) gives detailed balance with true posterior P (yje): his! y 0 ) P (yje)p (y 0 ij y i e) P (y i y i je)p (y 0 ij y i e) (y)q(y P (y i j y i e)p ( y i je)p (y 0 ij y i e) (chain rule) P (y i j y i e)p (y 0 i y ije) (chain rule backwards) q(y 0! y)(y 0 )(y 0 )q(y 0! y) AIMA Slides cstuart Russell and Peter Norvig, 1998 Chapter 15.3{4 + new 25

26 Markov blanket sampling variable is independent of all others given its Markov blanket: A (y 0 ij y i e) P (y 0 ijmb(y i )) P given the Markov blanket is calculated as follows: Probability (y 0 ijmb(y i )) P (y 0 ijp arents(y i )) Zj 2Children(Y P )P (z j jparents(z j )) i computing the sampling distribution over Y i for each ip requires Hence cd multiplications if Y i has c children and d values can cache it if just c not too large. computational problems: Main Dicult to tell if convergence has been achieved 1) bewasteful if Markov blanket is large: 2)Can (Y i jmb(y i )) won't change much (law oflarge numbers) P AIMA Slides cstuart Russell and Peter Norvig, 1998 Chapter 15.3{4 + new 26

27 Polytime approximation: poly(n ;1 log ;1 ) are NP-hard for any < 0:5 Performance of approximation algorithms Absolute approximation: jp (Xje) ; ^P (Xje)j approximation: jp (Xje); ^P (Xje)j P (Xje) Relative Relative ) absolute since 0 P 1 (may beo(2 ;n )) Randomized algorithms may fail with probability at most (Dagum and Luby, 1993):both absolute and relative heorem for either deterministic or randomized algorithms approximation (Absolute approximation polytime with no evidence Cherno bounds) AIMA Slides cstuart Russell and Peter Norvig, 1998 Chapter 15.3{4 + new 27