Temporal probability models. Chapter 15, Sections 1 5 1

Transcription

1 Temporal probabiliy models Chaper 15, Secions 1 5 Chaper 15, Secions 1 5 1

2 Ouline Time and uncerainy Inerence: ilering, predicion, smoohing Hidden Markov models Kalman ilers (a brie menion) Dynamic Bayesian neworks Paricle ilering Chaper 15, Secions 1 5 2

3 Time and uncerainy The world changes; we need o rack and predic i Diabees managemen vs vehicle diagnosis Basic idea: copy sae and evidence variables or each ime sep X = se o unobservable sae variables a ime e.g., BloodSugar, SomachConens, ec. E = se o observable evidence variables a ime e.g., MeasuredBloodSugar, P ulserae, F oodeaen This assumes discree ime; sep size depends on problem Noaion: X a:b = X a, X a+1,..., X b 1, X b Chaper 15, Secions 1 5 3

4 Markov processes (Markov chains) Consruc a Bayes ne rom hese variables: parens? Markov assumpion: X depends on bounded subse o X 0: 1 Firs-order Markov process: P(X X 0: 1 ) = P(X X 1 ) Second-order Markov process: P(X X 0: 1 ) = P(X X 2, X 1 ) Firs order X 2 X 1 X X +1 X +2 Second order X 2 X 1 X X +1 X +2 Sensor Markov assumpion: P(E X 0:, E 0: 1 ) = P(E X ) Saionary process: ransiion model P(X X 1 ) and sensor model P(E X ) ixed or all Chaper 15, Secions 1 5 4

5 Example Rain 1 R 1 P(R ) Rain Rain +1 R P(U ) Umbrella 1 Umbrella Umbrella +1 Firs-order Markov assumpion no exacly rue in real world! Possible ixes: 1. Increase order o Markov process 2. Augmen sae, e.g., add T emp, P ressure Example: robo moion. Augmen posiion and velociy wih Baery Chaper 15, Secions 1 5 5

6 Inerence asks Filering: P(X e 1: ) belie sae inpu o he decision process o a raional agen Predicion: P(X +k e 1: ) or k > 0 evaluaion o possible acion sequences; like ilering wihou he evidence Smoohing: P(X k e 1: ) or 0 k < beer esimae o pas saes, essenial or learning Mos likely explanaion: arg max x1: P (x 1: e 1: ) speech recogniion, decoding wih a noisy channel Chaper 15, Secions 1 5 6

7 Filering Aim: devise a recursive sae esimaion algorihm: P(X +1 e 1:+1 ) = (e +1, P(X e 1: )) P(X +1 e 1:+1 ) = P(X +1 e 1:, e +1 ) = αp(e +1 X +1, e 1: )P(X +1 e 1: ) = αp(e +1 X +1 )P(X +1 e 1: ) I.e., predicion + esimaion. Predicion by summing ou X : P(X +1 e 1:+1 ) = αp(e +1 X +1 )Σ x P(X +1 x, e 1: )P (x e 1: ) = αp(e +1 X +1 )Σ x P(X +1 x )P (x e 1: ) 1:+1 = Forward( 1:, e +1 ) where 1: = P(X e 1: ) Time and space consan (independen o ) Chaper 15, Secions 1 5 7

8 Filering example True False Rain 0 Rain 1 Rain 2 Umbrella 1 Umbrella 2 Chaper 15, Secions 1 5 8

9 Smoohing X 0 X 1 X k X Divide evidence e 1: ino e 1:k, e k+1: : E1 E k E P(X k e 1: ) = P(X k e 1:k, e k+1: ) = αp(x k e 1:k )P(e k+1: X k, e 1:k ) = αp(x k e 1:k )P(e k+1: X k ) = α 1:k b k+1: Backward message compued by a backwards recursion: P(e k+1: X k ) = Σ xk+1 P(e k+1: X k, x k+1 )P(x k+1 X k ) = Σ xk+1 P (e k+1: x k+1 )P(x k+1 X k ) = Σ xk+1 P (e k+1 x k+1 )P (e k+2: x k+1 )P(x k+1 X k ) Chaper 15, Secions 1 5 9

10 Smoohing example True False orward smoohed backward Rain 0 Rain 1 Rain 2 Umbrella 1 Umbrella 2 Forward backward algorihm: cache orward messages along he way Time linear in (polyree inerence), space O( ) Chaper 15, Secions

11 Mos likely explanaion Mos likely sequence sequence o mos likely saes!!!! Mos likely pah o each x +1 = mos likely pah o some x plus one more sep max P(x x 1...x 1,..., x, X +1 e 1:+1 ) = P(e +1 X +1 ) max P(X+1 x x ) max P (x x 1...x 1,..., x 1, x e 1: ) 1 Idenical o ilering, excep 1: replaced by m 1: = max P(x x 1...x 1,..., x 1, X e 1: ), 1 I.e., m 1: (i) gives he probabiliy o he mos likely pah o sae i. Updae has sum replaced by max, giving he Vierbi algorihm: m 1:+1 = P(e +1 X +1 ) max x (P(X +1 x )m 1: ) Chaper 15, Secions

12 Vierbi example Rain 1 Rain 2 Rain 3 Rain 4 Rain 5 sae space pahs rue alse rue alse rue alse rue alse rue alse umbrella rue rue alse rue rue mos likely pahs m 1:2 m 1:3 m 1:4 m 1:1 m 1:5 Chaper 15, Secions

13 Hidden Markov models X is a single, discree variable (usually E is oo) Domain o X is {1,..., S} Transiion marix T ij = P (X = j X 1 = i), e.g., Sensor marix O or each ime sep, diagonal elemens P (e X = i) e.g., wih U 1 = rue, O 1 = Forward and backward messages as column vecors: 1:+1 = αo +1 T 1: b k+1: = TO k+1 b k+2: Forward-backward algorihm needs ime O(S 2 ) and space O(S) Chaper 15, Secions

14 Counry dance algorihm Can avoid soring all orward messages in smoohing by running orward algorihm backwards: 1:+1 = αo +1 T 1: O :+1 = αt 1: α (T ) 1 O :+1 = 1: Algorihm: orward pass compues, backward pass does i, b i Chaper 15, Secions

15 Counry dance algorihm Can avoid soring all orward messages in smoohing by running orward algorihm backwards: 1:+1 = αo +1 T 1: O :+1 = αt 1: α (T ) 1 O :+1 = 1: Algorihm: orward pass compues, backward pass does i, b i Chaper 15, Secions

16 Counry dance algorihm Can avoid soring all orward messages in smoohing by running orward algorihm backwards: 1:+1 = αo +1 T 1: O :+1 = αt 1: α (T ) 1 O :+1 = 1: Algorihm: orward pass compues, backward pass does i, b i Chaper 15, Secions

17 Counry dance algorihm Can avoid soring all orward messages in smoohing by running orward algorihm backwards: 1:+1 = αo +1 T 1: O :+1 = αt 1: α (T ) 1 O :+1 = 1: Algorihm: orward pass compues, backward pass does i, b i Chaper 15, Secions

18 Counry dance algorihm Can avoid soring all orward messages in smoohing by running orward algorihm backwards: 1:+1 = αo +1 T 1: O :+1 = αt 1: α (T ) 1 O :+1 = 1: Algorihm: orward pass compues, backward pass does i, b i Chaper 15, Secions

19 Counry dance algorihm Can avoid soring all orward messages in smoohing by running orward algorihm backwards: 1:+1 = αo +1 T 1: O :+1 = αt 1: α (T ) 1 O :+1 = 1: Algorihm: orward pass compues, backward pass does i, b i Chaper 15, Secions

20 Counry dance algorihm Can avoid soring all orward messages in smoohing by running orward algorihm backwards: 1:+1 = αo +1 T 1: O :+1 = αt 1: α (T ) 1 O :+1 = 1: Algorihm: orward pass compues, backward pass does i, b i Chaper 15, Secions

21 Counry dance algorihm Can avoid soring all orward messages in smoohing by running orward algorihm backwards: 1:+1 = αo +1 T 1: O :+1 = αt 1: α (T ) 1 O :+1 = 1: Algorihm: orward pass compues, backward pass does i, b i Chaper 15, Secions

22 Counry dance algorihm Can avoid soring all orward messages in smoohing by running orward algorihm backwards: 1:+1 = αo +1 T 1: O :+1 = αt 1: α (T ) 1 O :+1 = 1: Algorihm: orward pass compues, backward pass does i, b i Chaper 15, Secions

23 Counry dance algorihm Can avoid soring all orward messages in smoohing by running orward algorihm backwards: 1:+1 = αo +1 T 1: O :+1 = αt 1: α (T ) 1 O :+1 = 1: Algorihm: orward pass compues, backward pass does i, b i Chaper 15, Secions

24 Kalman ilers Modelling sysems described by a se o coninuous variables, e.g., racking a bird lying X = X, Y, Z, Ẋ, Ẏ, Ż. Airplanes, robos, ecosysems, economies, chemical plans, planes,... X X+1 X X+1 Z Z+1 Gaussian prior, linear Gaussian ransiion model and sensor model Chaper 15, Secions

25 Updaing Gaussian disribuions Predicion sep: i P(X e 1: ) is Gaussian, hen predicion P(X +1 e 1: ) = x P(X +1 x )P (x e 1: ) dx is Gaussian. I P(X +1 e 1: ) is Gaussian, hen he updaed disribuion P(X +1 e 1:+1 ) = αp(e +1 X +1 )P(X +1 e 1: ) is Gaussian Hence P(X e 1: ) is mulivariae Gaussian N(µ, Σ ) or all General (nonlinear, non-gaussian) process: descripion o poserior grows unboundedly as Chaper 15, Secions

26 Simple 1-D example Gaussian random walk on X axis, s.d. σ x, sensor s.d. σ z µ +1 = (σ2 + σ 2 x)z +1 + σ 2 zµ σ 2 + σ 2 x + σ 2 z σ 2 +1 = (σ2 + σ 2 x)σ 2 z σ 2 + σ 2 x + σ 2 z P(X) P(x1 z1=2.5) P(x0) P(x1) *z X posiion Chaper 15, Secions

27 Transiion and sensor models: General Kalman updae P (x +1 x ) = N(Fx, Σ x )(x +1 ) P (z x ) = N(Hx, Σ z )(z ) F is he marix or he ransiion; Σ x he ransiion noise covariance H is he marix or he sensors; Σ z he sensor noise covariance Filer compues he ollowing updae: µ +1 = Fµ + K +1 (z +1 HFµ ) Σ +1 = (I K +1 )(FΣ F + Σ x ) where K +1 = (FΣ F + Σ x )H (H(FΣ F + Σ x )H + Σ z ) 1 is he Kalman gain marix Σ and K are independen o observaion sequence, so compue oline Chaper 15, Secions

28 2-D racking example: ilering 12 2D ilering 11 rue observed ilered 10 Y X Chaper 15, Secions

29 2-D racking example: smoohing 12 2D smoohing 11 rue observed smoohed 10 Y X Chaper 15, Secions

30 Where i breaks Canno be applied i he ransiion model is nonlinear Exended Kalman Filer models ransiion as locally linear around x = µ Fails i sysems is locally unsmooh Chaper 15, Secions

31 Dynamic Bayesian neworks X, E conain arbirarily many variables in a replicaed Bayes ne BMeer 1 P(R ) R 0 P(R ) Baery 0 Baery 1 Rain 0 Rain 1 R 1 P(U ) X 0 X 1 Umbrella 1 XX 0 X 1 Z 1 Chaper 15, Secions

32 DBNs vs. HMMs Every HMM is a single-variable DBN; every discree DBN is an HMM X X +1 Y Y+1 Z Z +1 Sparse dependencies exponenially ewer parameers; e.g., 20 sae variables, hree parens each DBN has = 160 parameers, HMM has Chaper 15, Secions

33 DBNs vs Kalman ilers Every Kalman iler model is a DBN, bu ew DBNs are KFs; real world requires non-gaussian poseriors E.g., where are bin Laden and my keys? Wha s he baery charge? BMBroken0 BMBroken1 BMeer 1 Baery 0 Baery 1 5 E(Baery ) 4 E(Baery ) X 0 XX 0 X 1 X 1 E(Baery) P(BMBroken ) 0 P(BMBroken ) Z Time sep Chaper 15, Secions

34 Exac inerence in DBNs Naive mehod: unroll he nework and run any exac algorihm P(R 0) 0.7 Rain 0 R 0 P(R 1) Rain 1 P(R 0) 0.7 Rain 0 R 0 P(R 1) Rain 1 R 0 P(R 1) Rain 2 R 0 P(R 1) Rain 3 R 0 P(R 1) Rain 4 R 0 P(R 1) Rain 5 R 0 P(R 1) Rain 6 R 0 P(R 1) Rain 7 R 1 P(U 1) R 1 P(U 1) R 1 P(U 1) R 1 P(U 1) R 1 P(U 1) R 1 P(U 1) R 1 P(U 1) R 1 P(U 1) Umbrella 1 Umbrella 1 Umbrella 2 Umbrella 3 Umbrella 4 Umbrella 5 Umbrella 6 Umbrella 7 Problem: inerence cos or each updae grows wih Rollup ilering: add slice + 1, sum ou slice using variable eliminaion Larges acor is O(d n+1 ), updae cos O(d n+2 ) (c. HMM updae cos O(d 2n )) Chaper 15, Secions

35 Likelihood weighing or DBNs Se o weighed samples approximaes he belie sae Rain 0 Rain 1 Rain 2 Rain 3 Rain 4 Rain 5 Umbrella 1 Umbrella 2 Umbrella 3 Umbrella 4 Umbrella 5 LW samples pay no aenion o he evidence! racion agreeing alls exponenially wih number o samples required grows exponenially wih RMS error LW(10) LW(100) LW(1000) LW(10000) Time sep Chaper 15, Secions

36 Paricle ilering Basic idea: ensure ha he populaion o samples ( paricles ) racks he high-likelihood regions o he sae-space Replicae paricles proporional o likelihood or e rue Rain Rain +1 Rain +1 Rain +1 alse (a) Propagae (b) Weigh (c) Resample Widely used or racking nonlinear sysems, esp. in vision Also used or simulaneous localizaion and mapping in mobile robos dimensional sae space Chaper 15, Secions

37 Paricle ilering cond. Assume consisen a ime : N(x e 1: )/N = P (x e 1: ) Propagae orward: populaions o x +1 are N(x +1 e 1: ) = Σ x P (x +1 x )N(x e 1: ) Weigh samples by heir likelihood or e +1 : W (x +1 e 1:+1 ) = P (e +1 x +1 )N(x +1 e 1: ) Resample o obain populaions proporional o W : N(x +1 e 1:+1 )/N = αw (x +1 e 1:+1 ) = αp (e +1 x +1 )N(x +1 e 1: ) = αp (e +1 x +1 )Σ x P (x +1 x )N(x e 1: ) = α P (e +1 x +1 )Σ x P (x +1 x )P (x e 1: ) = P (x +1 e 1:+1 ) Chaper 15, Secions

38 Paricle ilering perormance Approximaion error o paricle ilering remains bounded over ime, a leas empirically heoreical analysis is diicul Avg absolue error LW(25) LW(100) LW(1000) LW(10000) ER/SOF(25) Time sep Chaper 15, Secions

39 Summary Temporal models use sae and sensor variables replicaed over ime Markov assumpions and saionariy assumpion, so we need ransiion modelp(x X 1 ) sensor model P(E X ) Tasks are ilering, predicion, smoohing, mos likely sequence; all done recursively wih consan cos per ime sep Hidden Markov models have a single discree sae variable; used or speech recogniion Kalman ilers allow n sae variables, linear Gaussian, O(n 3 ) updae Dynamic Bayes nes subsume HMMs, Kalman ilers; exac updae inracable Paricle ilering is a good approximae ilering algorihm or DBNs Chaper 15, Secions