Graphical Models. Lecture 5: Template- Based Representa:ons. Andrew McCallum

Transcription

1 Graphical Models Lecture 5: Template- Based Representa:ons Andrew McCallum Thanks to Noah Smith and Carlos Guestrin for some slide materials. 1

2 Administra:on Homework #3 won t go out un:l early March. Push back HW#2 due date? Lagrange Mul:pliers? Calendar. 2

3 BN with Repeated Structure A B 1 B 2 B 3 B N

4 Plate Model A B N

5 Unrolled Ground Network A B 1 B 2 B 3 B N Ground network

6 Students and their Grades A B N Example: A = student, B = grade

7 Student, Course, Grade, Difficulty Each student takes only one course A 1 A 2 Nes:ng B T N Example: A 1 = course difficulty, A 2 = student ap:tude for the area, B = grade

8 Student, Course, Grade, Difficulty Mul:ple courses per student A 1 A 2 Intersec:ng B N T Example: A 1 = assignment difficulty, A 2 = intelligence, B = grade

9 Plate Models: Limita:ons and Alterna:ves Limita:ons: can t have edges between two copies of the same variable, (e.g. posi%on a :me t depends on posi%on at :me :me t- 1) can t have edges between par:cular pairs selected by some other rela:on, (e.g. Genotype(U 1 ) depends on Genotype(U 2 ), where U 2 is mother of U 1. Alterna:ves Dynamic Bayesian Networks (DBNs) Specific to repe::ons over :me Probabilis:c rela:onal models More flexible; see K&F

10 Temporal Models X takes different values at each (discrete) :me step. X (t) is the random variable at :me t Markov Assump:on: X (t+1) {X (0),, X (t- 1) } X (t) Sta:onary Assump:on (aka %me invariant or homogeneous) P(X (t+1) X (t) ) is the same for all t. Can use condi%onal Bayesian network to define P(X (t+1) X (t) )

11 Hidden Markov Model S S S (0) S (1) S (2) S (3) O O (1) O (2) O (3) Time slice t Time slice t+1 2- $me- slice condi&onal BN unrolled or ground Bayesian network

12 Dynamic Bayesian Network Bayesian network over X (0), condi:onal Bayesian network for X (t+1) given X (t) (2- :me- slice) HMM is a special case. Kalman filter (linear dynamical system) is a special case.

13 Example: DBN for vehicle posi:on Weather Weather Weather Velocity Velocity Velocity Loca:on Loca:on Loca:on Failure Failure Failure Observ Observ Time slice 0 Time slice t Time slice t+1 13

14 Example: DBN for vehicle posi:on Factor template Weather 0 Weather 1 Weather 2 Weather 3 Velocity 0 Velocity 1 Velocity 2 Velocity 3 Loca:on 0 Loca:on 1 Loca:on 2 Loca:on 3 Failure 0 Failure 1 Failure 2 Failure 3 Observ 0 Observ 1 Observ 2 Observ 3 Unrolled over 3 steps 14

15 Dynamic Bayesian Networks S 1 S 1 S i S i K O 1 S 1 S 1 Factorial HMM O Coupled HMM O 1 S S Pair HMM O i 2

16 Probabilis:c Rela:onal Models Con:ngent Dependency specifies the context in which some dependency holds, with a guard a formula that must hold for the dependency to be applicable. e.g. Loca:on(V) depends on Loca:on(U) con:ngent on Precedes(U,V) e.g. Genotype(V) depends on Genotype(U) con:ngent on Mother(U,V) Rela:onal Uncertainty (one kind of structural uncertainty) The guard predicates are random variables! 16

17 Object Uncertainty The set of objects is not predetermined. Get list of authors in 100 BibTeX files. Stuart Russell Stuart Rusell S. Russell How many people are men:oned? [Milch et al BLOG ] Introduce person- objects (represents en:ty) person- reference objects (represents men:on) refers- to(m,o) rela:on Model generates (a) # of people, (b) person objects, (c) their reference objects. 17

18 Directed Factor Graph Nota:on [Laura Dietz 2010] 18

19 Variables and Constants Llatent variable / latent parameter Observed variable Constant / hyper parameter Directed factor graph var obs const Pseudocode 19

20 Factors and Densi:es Factor with one input parameter Directed factor graph in Example: σ Gaussian µ Density N out x Pseudocode 1: draw out Density(in) 1: draw x N (µ, σ) 20

21 Replica:on with Plates Directed factor graph Pseudocode global Density var Plate i {1..N} 1: for all i {1..N} do 2: draw var i global σ Example: repeated Gaussian µ N x i {1..N} 1: for all i {1..N} do 2: draw x i N (µ, σ) 21

22 Nested Plates Directed factor graph Pseudocode σ µ N x Nested plates i {1..N} k {1..K} 1: for all k {1..K} do 2: for all i {1..N} do 3: draw x k,i N (µ k, σ) 22

23 Condi:oning with Gates Minka & Winn 2008 Directed factor graph Pseudocode f Unrolled boolean gate c θ 1 c =1 c =0 g x 1: if c =1then 2: draw x f(θ 1 ) 3: else 4: draw x g(θ 2 ) θ 2 23

24 Plates & Gates (and implicit combo) Directed factor graph Pseudocode c Replicated gate d = c Multi c θ c d 1: draw x Multi(θ c ) x c Implicit notation for replicating gates Multi x θ c 1: draw x Multi(θ c ) 24

25 Latent Dirichlet Alloca:on [Blei, Ng, Jordan] Plate diagram α Directed factor graph α Dirich θ θ Multi z β z k = z w N M cf. [7], Figure 1 Multi w N M β k 25