COMP42 Lecure 8 Hidden Markov Model he Univeriy of Mancheer
he Univeriy of Mancheer Hidden Markov Model a b2 0. 0. SAR 0.9 0.9 SOP b 0. a2 0. Imagine he and 2 are hidden o he daa roduced i a equence of a and b Daa generaion i eay a-b-a-b-a-b-a2-a2-b2 abababaab Daa decoding i ambiguou abababaab? Sae emi feaurea or b bu heir origini no known 2
he Univeriy of Mancheer Markov Chain: reminder A Markov chain i a generaivemodel of equence = 2 3... -... where i an objec from he ae ace S I ha he roery ha ( - -2... ) = ( - ) he robabiliy of a equence i ( ) = ( ) ( ) + 3 =
he Univeriy of Mancheer Hidden Markov Model (HMM) An HMM i a model of a equence of feaure or feaure vecor 2 3... - generaed according o emiion robabiliie ( ) he underlying ae equence i from a Markov chain model = 2 3 L L ( ) = ( ) ( ) + = bu he ae equence i hiddenfrom u 4
he Univeriy of Mancheer HMM Eamle a b2 0. 0. SAR 0.9 0.9 SOP b 0. a2 0. Feaure are {a b} and ae are {a a2 b b2} Emiion robabiliie ( ) = a = a = ( = b = a ) = 0 ( = a = b ) = 0 ( = b = b ) = raniion robabiliie ( ) = a2 = b = 0. ( 2 2) = a = a = 0. 45 ec ec 5
he Univeriy of Mancheer hi give imilar equence HMM Eamle 0.9 SAR a 0.9 0.9 b ab 0. 0. 0. SOP Emiion robabiliie ( ) = a = a = ( ) = a = b = 0 ( = a = ab) = 0. 5 ( ) = b = a = 0 ( ) = b = b = ( = b = ab) = 0. 5 Sae abcan emi feaure aor bwih equal robabiliy 6
he Univeriy of Mancheer So he difference a b2 0. 0. SAR 0.9 0.9 SOP b 0. a2 0. 0.9 a ab 0. 0. SAR 0.9 0.9 SOP b 0. 7
he Univeriy of Mancheer HMM for eech 0.96 0.02 ye 0.0 0.96 SAR.0 il 0.99 0.99 il 0.04 SOP 0.02 no 0.0 Emiion robabiliie: ( = SIL) ( = ye ) ( = no ) i he MFCC feaure vecor for egmen of he eech ignal We can fi normal deniie o he feaure diribuion for each ae (lighly more fleible diribuion are ued in racice) hi model wa ued o cro he eech ha you ue in Lab 2 8
he Univeriy of Mancheer Join robabiliy of aeand feaure 0.9 SAR a 0.9 0.9 b ab 0. 0. 0. SOP Emiion robabiliie: ( = a = a) = ( = a = b) = 0 ( = b = a) = 0 ( = b = b) = ( = a = ab) = ( = a = ab) = ( = aba = a-ab-ab) = 0. 0.9 0. 9
he Univeriy of Mancheer Join robabiliy of aeand feaure Eay mulily he emiion and raniion robabiliie ( L L ) 2 2 = = ( ) ( ) ( ) ( ) ( ) ( ) ( SOP ) 2 2 2 L ( ) ( ) ( ) + = Bu: he ae ah 2 i unknown i hidden 0
he Univeriy of Mancheer HMM inference We don know he hidden ae o he join robabiliy of ae and feaure in a ueful hing o comue We will conider wo more ueful ak: Claificaion: Modelling differen clae of daa e.g. ye and no Decoding: Finding he mo likely ae given a feaure vecor Comuing hee i harder and require he ue of clever algorihm
he Univeriy of Mancheer Claificaion Build a model for each cla of daa e.g. C = ye C 2 = no 0.95 0.99 0.95 SAR.0 il 0.05 ye 0.0 il 0.05 SOP Comue ( 2 C i ) for each cla C i Aly Baye rule ( C L ) 2 = ( ) 2 L C ( C ) ( ) 2 L Ci ( Ci ) i Aly a claificaion rule e.g. elec he mo likely cla 2
he Univeriy of Mancheer Claificaion Build a model for each cla of daa e.g. C = ye C 2 = no 0.95 0.99 0.95 SAR.0 il 0.05 ye 0.0 il 0.05 SOP Comue ( 2 C i ) for each cla C i Aly Baye rule ( C L ) 2 = ( ) 2 L C ( C ) ( ) 2 L Ci ( Ci ) i Aly a claificaion rule e.g. elec he mo likely cla 3
he Univeriy of Mancheer Claificaion Need a way o comue ( 2 ) for each model Require a um over all oible ah hrough he model ( L ) = L ( L ) 2 2 2 L S S S 2 Wor cae: S erm in hi um 4
he Univeriy of Mancheer Claificaion Need an efficienway o comue ( 2 ) for each model Ue a imilar recurion relaion a for Markov chain cae ( ) ( ) ( ) = Queion in Eamle hee 8 i a imilar idea: hi i called he Forward Algorihm ( ) ( ) ( ) ( ) ( ) ( ) ( ) S S SOP for 2 2 2 2 2 L L L L = = 5
he Univeriy of Mancheer Decoding Claificaion can deal wih a limied number of model Le ueful for hrae or enence An alernaive aroach i o decodehe daa: * 2 L ( L ) = arg ma 2 Decoding: find he mo likely ah hrough he hidden ae 6
he Univeriy of Mancheer Decoding 0.9 SAR a 0.9 0.9 b ab 0. 0. 0. SOP Daa decoding i ambiguou: abababaab? Mo likely ah i a b a b a b a2 a2b2 Le likely ah i a b2 a2 b2 a2 b2 a2 a2b2 7
he Univeriy of Mancheer * = Decoding arg ma 2 2 L ( L ) Require earchfor ah maimiing he quaniy on he righ here can be a many a S oible ah ehauive earch in oible he VierbiAlgorihm ue recurion o find he oimal ah efficienly hi i an eamle of an oimiaionroblem. 8
he Univeriy of Mancheer raining I eay if daa i labelled i.e. he ae ah of he raining daa i known Labelling i ime conuming difficul and error-rone he Baum-Welch algorihm allow raining wih unlabelled daa Hel if we know omehing abou he daa e.g. reading from a cri. hi maively reduce he earch ace. 9
he Univeriy of Mancheer Seech Recogniion Now we have mo of he ingredien for eech recogniion 20