Lecture 2: Bayesian inference and Hidden Markov Models

Transcription

1 Lecture 2: Bayesian inference and Hidden Markov Models Scribe: Audrow Nash 1 Probability Chain rule P(X 1,..., X n ) = P(X 1 X 2,..., X n )P(X 2,..., X n ) = P(X 1 X 2,..., X n )P(X 2 X 3,..., X n )P(X 3,..., X n ). = Πi=1 n P(X i parents(x i )) A note on notation: P(A B, C) = P(A B C). 2 Bayesian Networks 1. Each node corresponds to a random variable, which is either discrete or continuous. 2. Directed links connect pairs of nodes; X Y means X is a parent of Y. 3. Each node has a conditional probability distribution: P(X parents(x)). 4. Each node is conditionally independent of its non-descendents given its parents. 1

2 2.1 Example Can-dirty P(D) Trusting P(T) Intervene-can Intervene-glass D T P(C D, T) T P(G T) f f f t t f t t t f Figure 1: The Bayesian Network for the example, which uses the example of a robot reaching for a glass or a can where a human can intervine. P(D, C, T, G) = P(C D, T)P(G T)P(D)P(G) P(G) =P(G T)P(T) + P(G T)P( T) = = 0.45 P(C) =P(C D, T)P(D, T) + P(C D, T)P( D, T) + P(C D, T)P(D, T) =P(C D, T)P(D)P(T) + P(C D, T)P( D)P(T) + P(C D, T)P( D, T) + P(C D, T)P(D, T) = =

3 P(T G) = P(G T)P(T) P(G) = 0.45 = 0.22 P(C G) =P(C D, T, G)P(D, T, G) + P(C D, T, G)P( D, T, G) + P(C D, T, G)P(D, T, G) + P(C D, T, G)P( D, T, G) = P(C D, T)P(T G)P(D) + P(C D, T)P( D)P( T G) + P(C D, T)P(D)P( T G)P(C D, T)P(T G) = Bayesian Inference Figure 2: A Hidden Markov Model, where X i and O i denotes the state and observation at time step i. We ll be focusing on Discrete Random Variables. Markov assumption: The next state depends only on the current state - not past states. P(X t X 0,..., X t 1 ) = P(X t X 0:t 1 ) = P(X t X t 1 ) Components of a Hidden Markov Model: 3

4 1. χ: set of states 2. Ω: set of observations 3. T : X Π(X): state transition probabilities 4. M : X Π(Ω): observation probabilities 3.1 Example Setup X ={Desk, Whiteboard, Door} Dynamics model: T = X t 1 = X t = Desk Whiteboard Door Desk Whiteboard Door Sensor model: M = X t = O t = Desk Whiteboard Door Desk Whiteboard Door Given observations = {desk, door, desk} P(X 0 ) = {1/3, 1/3, 1/3} 4

5 3.1.3 Exercises P(X t O 1:t ) =P(X t O 1:t 1, O t ) = P(O t O 1:t 1, X t )P(X t O 1:t 1 ) P(O t O 1:t 1 ) =ηp(o t O 1:t 1, X t )P(X t O 1:t 1 ) =ηp(o t X t )P(X t O 1:t 1 ) =ηp(o t X t ) X t 1 P(X t X t 1, O 1:t 1 )P(X t 1 O 1:t 1 ) P(X 1 O 1 ) = η µ(o 1 X 1 ) X 0 T(X 1 X 0 )P(X 0 ) P(X 1 = desk O 1 = desk) =η M(O 1 = desk X 0 = desk) (T(X 1 = desk X 0 = desk)p(x 0 = desk) + T(X 1 = desk X 0 = whiteboard)p(x 0 = whiteboard) + T(X 1 = desk X 0 = door)p(x 0 = door)) = η 0.8 (0.4 1/ / /3) = η P(X 1 = whiteboard O 1 = desk) =η 0.1 (0.4 1/ /3 + 0) =η P(X 1 = door O 1 = desk) =η 0.1 (0.2 1/ / /3) =η =P(X 1 = desk O 1 = desk) + P(X 1 = whiteboard O 1 = desk) + P(X 1 = door O 1 = desk) =η ( ) η = 1/

6 P(X 1 ) = {Desk: 0.744, Whiteboard: 0.093, Door: 0.163} Repeat the same process using the updated probabilities to solve for P(X 2 = desk O 1 = desk, O 2 = door) and beyond. t P(desk) P(whiteboard) P(door) 0 1/3 1/3 1/ Can we predict the future? Condition without observations: P(X t+k O 1:t ) = x t +k 1 P(X t+k X t+k 1, O 1:t )P(X t+k 1 O 1:t ) = P(X t+k X t+k 1 )P(X t+k 1 O 1:t ) x t +k 1 Prediction: t P(desk) P(whiteboard) P(door) Note: The door is an absorbing state, so when predicting without evidence the belief converges to the door. 3.2 Discussion of examples What is the time complexity of predictions? O(c) O(log t) O(t) (answer; aka, linear) O(t 2 ) 6

7 3.3 Misc. Notes Forward pass: P(X k O 1:t ) =ηp(x k O 1:k, O k+1:t ) =ηp(o k+1:t X k, O 1:k )P(X k O 1:k ) =ηp(o k+1:t X k, O 1:k )P(X k O 1:k ) =ηp(x k O 1:k )P(O k+1:t X k ) =η f 1:k b k+1:t Backwards pass: P(O k+1:t X k ) = P(O k+1:t X k, X k+1 )P(X k+1 X k ) = P(O k+1:t X k+1 )P(X k+1 X k ) = P(O k+1, O k+2:t X k+1 )P(X k+1 X k ) = P(O k+1 X k+1 )P(O k+2:t X k+1 )P(X k+1 X k ) = M(O k+1 X k+1 )P(O k+2:t X k+1 )T(X k+1 X k ) 7