3/6/00. Reading Assignments. Outline. Hidden Markov Models: Explanation and Model Learning

Transcription

1 3/6/ Hdden Mrkov Models: Explnton nd Model Lernng Brn C. Wllms 6.4/6.43 Sesson 2 9/3/ courtesy of JPL copyrght Brn Wllms, 2 Brn C. Wllms, copyrght 2 Redng Assgnments AIMA (Russell nd Norvg) Ch 5.-.3, 2.3 Stte Estmton nd Hdden Mrkov Models From lst Mondy: Ch 3 Ch 4.-4 Revew of Proltes Prolstc Resonng 9/3/ copyrght Brn Wllms, 2 2 Outlne Revew Explnton nd Lernng n Sttstcl Nturl Lnguge Decodng usng the Vter Algorthm Evluton v Forwrd nd Bckwrd Algorthms. Model lernng v the Bum-Welch Algorthm. 9/3/ copyrght Brn Wllms, 2 3

2 X X X N- X N T 3/6/ HMM Estmton s Pervsve Dlogue Mngement courtesy of NASA Engneerng Opertons courtesy of NASA Root Loclzton 9/3/ copyrght Brn Wllms, 2 4 Courtesy of Knn Rjn, NASA Ames. Used wth permsson. Posteror Prolty, fter Oservtons X,n = x,n PM ( ) = PM ( ) M M PM ( x ) = Px ( M) P( M x ), n n, n - P(x M) s estmted usng model, F, ccordng to: If prevous oservtons X, - = x, -, M nd F entls X = x Then P(x M) = If prevous oservtons X, - = x,-, M nd F entls X? v Then P(x c) = Assume: Apror mode ndependence. Consstent os eqully lkely Otherwse, Assume ll consstent ssgnments to X re eqully lkely oservtons: let D {x c, F s consstent wth X = x c c D X c } Then P(x M) = / D c 9/3/ copyrght Brn Wllms, 2 5 Estmtng Dynmc Systems S T Gven sequence of oservtons nd commnds: Wht s the lkelhood of prtculr stte? Belef Stte Updte: (flterng nd smoothng) Wht s the most lkely sequence of sttes tht got me here? Decodng: ( Vter Algorthm) Wht s the most lkely sequence of oservtons generted? Evluton/Predcton: Wht HMM most lkely generted these oservtons? Lernng: ( Bum-Welch Algorthm, Expectton-Mxmzton Algorthm) 9/3/ copyrght Brn Wllms, 2 6 2

3 3/6/ Wht s the lkelhood of stte? Smoothng Flterng Predcton t Flterng Proltes of current sttes Predcton Proltes of future sttes Smoothng Proltes of pst sttes 9/3/ copyrght Brn Wllms, 2 7 Notton S t+ : set of hdden vrles n the t+ tme slce s t+ : set of vlues for those hdden vrles t t+ x t+ : set of oservtons t tme t+ x :t : set of oservtons from ll tmes from to t : normlzton constnt 9/3/ copyrght Brn Wllms, 2 8 Hdden Mrkov Models Fnte Sttes S, Actons A & Oservtons W Stte trnston functon T(S,A,S + ) P(S + S, A ) Oservton functon O(S, W ) P(W S ) Intl stte dstruton Q(S): P(S ) Notton: P(S) denotes ll susets of S 9/3/ copyrght Brn Wllms, 2 9 3

4 3/6/ Mrkov Assumptons Gven dstruton over the current stte, the future sttes nd current nd future oservtons re ndependent of the pst. Frst-order Mrkov process P(S t S :t-) = P(S t S t-) Mrkov ssumpton of evdence P(X t S :t,x:t- )= P(X t S t ) 9/3/ copyrght Brn Wllms, 2 Belef Updte Exmple S ( ) = H (.7). H (.4) Ox (, S ) T( S,, S ) S ( ) T (.3) T (.6). s S.5 C.5 C 2 Oserved sequence: H T H H H H T H C.5 x.7 x [.9 x.5 +. x.5] =.35 =.64 C 2.5 x.4 x [. x x.5] =.2 =.36 s S Ps ( ) =, hence = =.82 9/3/ copyrght Brn Wllms, 2 Dgnosng Dynmc Systems: V Prolstc Constrnt Automt Devces modes Prolstc trnstons etween modes Stte constrnts for ech mode One utomt per component vlv=open => + Outflow = M z (nflow); Open Open Cost 5 Pro.9 Closed Close vlv=stuck open => + Outflow = M z (nflow); Stuck open Stuck closed Vlv = closed => vlv=stuck closed=> Outflow = ; Unknown Outflow = ; 9/3/ copyrght Brn Wllms, 2 2 4

5 3/6/ Outlne Revew Explnton nd Lernng n Sttstcl Nturl Lnguge Decodng usng the Vter Algorthm Evluton v Forwrd nd Bckwrd Algorthms Model lernng v the Bum-Welch Algorthm 9/3/ copyrght Brn Wllms, 2 3 Vrnt on Hdden Mrkov Model A HMM s defned s <s,s,w,e> Dfferences from erler HMM: S s the set of sttes, Oservtons re words s S s the sngle strt stte, Trnstons emt oservtons W s the set of oservle symols, A unque strt stte E s the set of emttng trnstons. A trnston s four tuple such s <s 2, hd, s 3,.3> denotng: " hd " P ( s f s ) =.3 P ( s," hd" s ) E comnes the trnston 2 Exmple: s = nd oservton functon n the prevous HMM def S = {, } W = {, } E = { <,,,.48>.48. <,,,.48> <,,,.4> <,,,.> } Sentence Prsng Exmple HMM = <S,s,W,E> where S={s,s 2,s 3,s 4,s 5,s 6,s 7,s 8 }; W={ Roger, }; E={trnston, } nd trnston = <s 2, hd, s 3,.3>. " hd " 2 = P ( s f s 3 ). 3 Hot.5 Roger Ordered.3.3 Bg. Dog.5 Mry Hd A Lttle Lm. s s 2 s 3 s 4.4 s 5.5 s 6.4 s John Cooked A Curry..5 Oserved Word Sequences: And.3 And.5 S : Mry hd lttle Lm nd g dog. S 2 : Roger ordered lm curry nd hot dog. S 3 : John cooked hot dog curry. s 7 P(S 3 )=.3*.3*.5*.5*.3*.5=

6 3/6/ Prolems nd Algorthms Decodng: Gven sequence of oservtons, wht s the most lkely sequence of hdden sttes? Soluton: Vter lgorthm Evluton: Wht s the prolty of gven sequence of oservtons? Soluton: forwrd/ckwrd lgorthm (These re used n the lernng lgorthm). Lernng: Gven sequence of oservtons, wht HMM trnston proltes mxmze the lkelhood of the sequence? Soluton: Bum-Welch lgorthm (form of Expectton-Mxmzton) 9/3/ copyrght Brn Wllms, 2 6 Outlne Overvew Belef Stte Updte Explnton nd Lernng n Sttstcl Nturl Lnguge Decodng usng the Vter Algorthm Evluton v Forwrd nd Bckwrd Algorthms Model lernng v the Bum -Welch Algorthm. 9/3/ copyrght Brn Wllms, 2 7 HMM Decodng Fndng the most lkely stte trjectory Prolem: Gven HMM = <S, s,w,e>, nd oservton sequence w :t-, fnd the most lkely stte sequence (denoted s (t)), endng n s t tme t: : t : t - s ( t ) = rg mx Ps ( w ).2.4 :t s s.t. s t = s Oserve w,4 = <,,, > s 5 = <,,,, > 9/3/ copyrght Brn Wllms,

7 3/6/ Prolty of the Most Lkely Stte Trjectory Prolty of the most lkely sequence s (t) endng n s gven oservtons w,t : s P( (t +) w :t ) = P( ( t+), w:t ) s Pw :t ( ) P( (t +) w : = mx P(s t+, w t s t ) P ( s : t-, s t w : t- ) j j sj S s t ) = mx t+ t sj S P(s ()) =, f s = s P(s, w t s P t w : j ) (s j (), nd, otherwse. 9/3/ copyrght Brn Wllms, 2 9 t- ) Prolem: Compute Vter Algorthm s ( t ) = rg mx : t : t - P( s w ) :t s s.t. s t = s Soluton: For n from to t, Compute the most lkely pths of length n tht end t ech s k, S. Extend to the most lkely pths of length n+ tht end t ech s, S s () = s s (t + ) = s ( t) os, mx j mx where j = rgmx S w t : P(s () t w t- ) P( k k k= s f s ) Notton: <s, s n > <s, s n, > 9/3/ copyrght Brn Wllms, 2 2 s () = s Exmple: Vter Algorthm s (t +) = s j mx ( t ) s,where j mx = rgmx P ( s k () t ) P( s k f s ) S o k= w t Sttes\ Os Strt Sequence Prolty Sequence Prolty Oserve /3/ copyrght Brn Wllms, 2 2 7

8 3/6/ Vter Pseudo Code Vter(<S,s,W,E>, w :T-, T) // prm (t) denotes P(s(t) w :t- ). egn 2. for ( ; S ; +) { 3. ntlze s() {s }, () { f s s strt stte s, else }} 4. for (t ; t < T; t t+) { // for ech os w t w,t 5. for ( ; S ; +) { // ech stte s t+ t tme t+ 6. ntlze j mx, P mx -; 7. for (k ; k S ; k k+) // compute rg mx over s t k 8. f ( k (t) * P(s t+, w t s k t ) > P mx ) { 9. P mx k (t) * P(s t+, w t s t k ), j mx k };. (t+) P mx ;. s(t+) s jmx (t) s } // postpend the next stte 2. return s(t) for < = S tht mxmzes (T) 3. end Outlne Revew Explnton nd Lernng n Sttstcl Nturl Lnguge Decodng usng the Vter Algorthm Evluton v Forwrd nd Bckwrd Algorthms Model lernng v the Bum -Welch Algorthm Appendx: Montorng nd Dgnoss v Prolstc Constrnt Automt 9/3/ copyrght Brn Wllms, 2 23 Prolty of Oservton Sequence Forwrd prolty - t : t t () = P ( w, s ) = f. : () = P ( w, s ) = otherwse f Smlr to elef stte, gven erler. c t w : t - t ( t + ) = P ( s f s ) ( t ) = P ( s fs ) P ( w, s ) j j j j = j = c t w j Oservton prolty S : t ( ) = ( t + ) = P w 8

9 3/6/ HMM Evluton Oservton nd Forwrd proltes t e (t) c.2. w t.5 (t +) = P( s j f s) j () t :t Pw j= S ( ) = (t +) = (t) P(w :t ) *.=.2 +.*.5=.5 Forwrd Algorthm Pseudo Code Forwrd(<S,s,W,E>, w :T ) // prm (t) denotes P(s t w :t- ). egn 2. for S ) 3. ntlze () { f s s strt stte s, else }; 4. for t T { 5. for j S { 6. ntlze j (t+) ; 7. for S 8. j (t+) j (t+) + P(s j, w t s )* (t); 9. }. return? (T+) for ll s S. end 9/3/ copyrght Brn Wllms, 2 26 Wht s the lkelhood of stte? Smoothng Flterng Predcton t Flterng Proltes of current sttes P(S t w :t ) Predcton Proltes of future sttes P(S k w :t ) for k > t Smoothng Proltes of pst sttes P(S k w :t ) for k < t 9/3/ copyrght Brn Wllms,

10 3/6/ Smoothng PS ( k w : t ) = P( S k w : k, w k + :t ) Dvde os : k ) ( k+: t k = P( S k w Pw, : k ) ( k +:t = P( S k w Pw PS ( k w :t ) = ( k ) ( k) :k S w ) Byes k S ) Mrkov 9/3/ copyrght Brn Wllms, 2 28 Bckwrd Proltes Bckwrd prolty T + ( T + ) = P ( e s ) = tt : t ( t ) Pw ( s ) (t) s smlr to (t) ut strts from the end. S t - w S w t - tt : t j ( t - ) = Ps ( f s j ) j ( t ) = Ps ( fsj ) P ( w s ) j j = j = : Oservton prolty Pw Pw : ( ) = () = ( T = s ) 9/3/ copyrght Brn Wllms, 2 29 Outlne Revew Explnton nd Lernng n Sttstcl Nturl Lnguge Decodng usng the Vter Algorthm Evluton v Forwrd nd Bckwrd Algorthms Model lernng v the Bum-Welch Algorthm 9/3/ copyrght Brn Wllms, 2 3

11 3/6/ HMM Trnng (Bum-Welch Algorthm) Approch: Gven trnng sequence w :T, djust the HMM stte trnston proltes to mke the oservton sequence s lkely s possle Trnng Sequence: w,8 = 2 9/3/ copyrght Brn Wllms, 2 3 Delng wth Hdden Sttes Intutvely Prolem: Sttes re not known. Soluton: Estmte sttes from Model..7 Prolem: Trnstons no longer determnstc. Soluton: Compute expected # of trnstons..3 c Prolem: Model not known.(chcken &egg) When countng trnstons Soluton: Bootstrp the model: prorte ech trnston. Guess model (trnston proltes). y ts prolty. 2. Use model to estmte sttes. 3. Count estmted trnstons to get model. 9/3/ copyrght Brn Wllms, 2 32 Expectton-Mxmzton (Bum-Welch). Guess set of trnston proltes. 2. whle (trnston proltes mprovng) {. Expectton: Use trnston P to estmte sttes P(S t w :T ).. Mxmzton: Estmte new trnston proltes y countng expected # of trnstons, gven stte estmtes.} mprovement mesured y comprng cross-entropy fter ech terton: - P (w : T )log PM ( w :T ) n w :T M - 2 Termnte when chnge n cross-entropy s less thn some q. 9/3/ copyrght Brn Wllms, 2 33

12 3/6/ Estmtng The Trnston Prolty C(s,w k,s j ): The expected count of trnstons s f s j, durng oservton sequence w :T : w k T w k Cs ( f s j ) = ( tps ) ( f s j ) j (t + ) t = P e : Estmted trnston prolty for s f s j estmted from oservton sequence w :T : w k w k Cs ( f s ) j P ( s f s ) = e j S W w m C(s ): Expected count Cs ( f s ) l of trnstons out of s l= m= w k w k 9/3/ copyrght Brn Wllms, 2 34 Bum-Welch Pseudo Code Bum-Welch (P new, w :T, q) // P new estmted P(s j,w k s ). do ( // w :T s trnng sequence nd q s convergence crter. 2. for =, j = S, = k = W 3. P old (s j,w k s ) = P new (s j,w k s ); // rememer old prolty estmte 4. compute (t), (t), for ll vlues of = = S nd = t = T; 5. for =, j = S, = k = W, nd = t = T { 6. ntlze C(s, w k, s j ) ; 7. for = t = T 8. C(s, w k, s j ) (t) P old (s j,w k s ) j (t); 9. }. for = = S {. ntlze C(s ) ; 2. for = j = S, = k = W 3. C(s ) C(s ) + C(s, w k, s j ); 4. for = j = S, = k = W 5. P new (s j, w k s )=C(s,w k, s j )/C(s ); 6. } 7. } whle (mxchnged(p new, P old ) > q) Bum-Welch exmple.48.4 Trnston proltes re ntlly guessed. Trnng sequence s (t) (t) e * (t) (t) e T(,,).52 T(,,) T(,,).3 T(,,) Totl New P C (,, ) C (,, ) + C (,, ) + C (,, ) 2

13 X X X N- X N T 3/6/ Notton Summry Prolstc trnstons: wrtten s P(s 3, hd s 2 ) =.3 or s " hd " 2 3 P ( s f s ) =.3 Oservton sequences: w,t denotes the entre sequence of oservtons. e denotes the empty sequence (no oservtons). Sttes S Sttes re suscrpted, s S, where = = S. k th Superscrpts ndcte tme, for exmple, s s the k stte n stte sequence.. Stte sequences: s (t) denotes the most lkely sequence of t sttes tht ends n st te s s t (t-) o s conctentes stte to the end of the sequence. (t) t w :t - t P(s ) denotes the forwrd prolty t tme step t of s =s (t) t:t s t ) t P(w denotes the ckwrd prolty t tme step t of s=s 9/3/ copyrght Brn Wllms, 2 37 Estmtng Dynmc Systems S T Gven sequence of oservtons nd commnds: Wht s the lkelhood of prtculr stte? Belef Stte Updte: (flterng, smoothng, predcton) Wht s the most lkely sequence of sttes tht got me here? Decodng: ( Vter Algorthm) Wht s the most lkely sequence of oservtons generted? Evluton: Wht HMM most lkely generted these oservtons? Lernng: (Bum-Welch Algorthm, Expectton-Mxmzton Algorthm) 9/3/ copyrght Brn Wllms,