CS344: Introduction to Artificial Intelligence

Transcription

1 C344: Iroduco o Arfcal Iellgece Puhpa Bhaacharyya CE Dep. IIT Bombay Lecure : Forward ad bacward; Baum elch 9 h ad 2 March ad 2 d Aprl 203 Lecure were o EM; dae 2 h March o 8 h March

2 HMM Defo e of ae: where =N ar ae 0 /*P 0 =*/ Oupu Alphabe: O where O =M Trao Probable: A= {a } /*ae o ae */ Emo Probable : B= {b o } /*prob. of emg or aborbg o from ae */ Ial ae Probable: Π={p p 2 p 3 p N } Each p =Po 0 =ε 0

3 Marov Procee Propere Lmed Horzo: Gve prevou ae a ae depede of precedg 0 o - + ae. PX = X - X -2 X 0 = PX = X - X -2 X - Order Marov proce Tme varace: how for = PX = X - = = PX = X 0 = = PX = X - =

4 Three bac problem cod. Problem : Lelhood of a equece Forward Procedure Bacward Procedure Problem 2: Be ae equece Verb Algorhm Problem 3: Re-emao Baum-elch Forward-Bacward Algorhm

5 Probablc Iferece O: Obervao equece : ae equece Gve O fd * * where arg max p / O called Probablc Iferece Ifer Hdde from Oberved How h ferece dffere from logcal ferece baed o propooal or predcae calculu?

6 Eeal of Hdde Marov Model. Marov + Nave Baye 2. Ue boh rao ad obervao probably O p p O / p / 3. Effecvely mae Hdde Marov Model a Fe ae Mache FM wh probably

7 Probably of Obervao equece p O p O = p p O / hou ay rerco earch pace ze= O

8 Coug wh he Ur example Colored Ball choog Ur # of Red = 30 # of Gree = 50 # of Blue = 20 Ur 2 # of Red = 0 # of Gree = 40 # of Blue = 50 Ur 3 # of Red =60 # of Gree =0 # of Blue = 30

9 Example cod. Trao Probably Obervao/oupu Probably Gve : U U 2 U 3 U U U ad R G B U U U Obervao : RRGGBRGR ha he correpodg ae equece?

10 Dagrammac repreeao /2 R 0.3 G 0.5 B U U R G B 0.3 R 0. G 0.4 B 0.5 U 2 0.2

11 Dagrammac repreeao 2/2 R0.03 G0.05 B0.02 R0.8 G0.03 B0.09 U R 0.08 G 0.20 B 0.2 R0.06 G0.24 B0.30 R0.5 G0.25 B0.0 U 2 R0.02 G0.08 B0.0 R0.24 G0.04 B0.2 U 3 R0.8 G0.03 B0.09 R0.02 G0.08 B0.0

12 Probablc FM a :0.3 a :0. a 2 :0.4 a :0.3 a 2 :0.2 2 a :0.2 a 2 :0.2 a 2 :0.3 The queo here : wha he mo lely ae equece gve he oupu equece ee

13 Developg he ree ar *0.= a a 2 0.*0.2= *0.4= *0.3= *0.2=0.06 Chooe he wg equece per ae per erao

14 Tree rucure cod *0.= a a The problem beg addreed by h ree * arg max P a a2 a a2 a-a2-a-a2 he oupu equece ad µ he model or he mache

15 Forward ad Bacward Probably Calculao

16 Forward probably F Defe F= Probably of beg ae havg ee o 0 o o 2 o Fp=Po 0 o o 2 o p wh a he legh of he oberved equece ee o far Poberved equece=po 0 o o 2..o m =Σ p=n Po 0 o o 2..o p =Σ p=n Fm p

17 Forward probably cod. F q = Po 0 o o 2..o q = Po 0 o o 2..o q = Po 0 o o 2..o - o q = Σ p=n Po 0 o o 2..o - p o q = Σ p=n Po 0 o o 2..o - p. Po q o 0 o o 2..o - p = Σ p=n F-p. Po q p o = Σ p=n F-p. P p q O 0 O O 2 O 3 O O + O m- O m p q m fal

18 Bacward probably B Defe B= Probably of eeg o o + o +2 o m gve ha he ae wa before O B=Po o + o +2 o m \ h m a he legh of he oberved equece Poberved equece=po 0 o o 2..o m = Po 0 o o 2..o m 0 =B0

19 B p Bacward probably cod. = Po o + o +2 o m \ p = Po + o +2 o m o p = Σ q=n Po + o +2 o m o q p = Σ q=n Po q p Po + o +2 o m o q p = Σ q=n Po + o +2 o m q. Po q p o = Σ q=n B+q. P p q O 0 O O 2 O 3 O O + O m- O m p q m fal

20 HMM Trag Baum elch or Forward Bacward Algorhm

21 Key Iuo a a b a q r b a b b Gve: Ialzao: Compue: Approach: Trag equece Probably value Pr ae eq rag eq ge expeced cou of rao compue rule probable Ialze he probable ad recompue hem EM le approach

22 Baum-elch algorhm: cou ab q a b r ab ab rg = abb aaa bbb aaa equece of ae wh repec o pu ymbol o/p eq ae eq a q r b b a q q r a a b b b a a a q r q q q r q r

23 Calculag probable from able a P q r b P q b P T=#ae w A=#alphabe ymbol 5/ 8 3/ 8 T c A l m c w w m Table of cou rc De O/P Cou q r a 5 q q b 3 r q a 3 r q b 2 Now f we have a o-deermc rao he mulple ae eq poble for he gve o/p eq ref. o prevou lde feaure. Our am o fd expeced cou hrough h. l

24 Ierplay Bewee Two Equao T l A m l m c c P 0 0 w P C w No. of me he rao occur he rg

25 Illurao b:0.7 a:0.6 q a:0.67 b:.0 r Acual Dered HMM b:0.48 a:0.48 q a:0.04 b:.0 r Ial gue

26 a Oe ru of Baum-elch algorhm: rg ababb a b b a a b b b b Ppah a q r b r q a q q q r q r q q q r q q q q q q q r q q q q q q q q b q q Rouded Toal New Probable P 0.06 =0.0/0. ae equece * ε codered a arg ad edg ymbol of he pu equece rg. Through mulple erao he probably value wll coverge.

27 Compuaoal par /2 w P P w P P P P P C ] [ ] [ ] [ ] [ w 0 w w 2 w w - w 0 - +

28 Compuaoal par 2/ B w P F B w P F B w P F P w P P w P w P w 0 w w 2 w w - w 0 - +

29 Dcuo. ymmery breag: Example: ymmery breag lead o o chage al value b:.0 b:0.5 a:0.5 a:.0 Dered a:0.5 b:0.25 a:0.25 b:0.5 a:0.5 a:0.25 b:0.5 b:0.5 Ialzed 2 ruc Local maxma 3. Label ba problem Probable have o um o. Value ca re a he co of fall of value for oher.