Hidden Markov Models for Speech Recognition
|
|
- Darren Ward
- 5 years ago
- Views:
Transcription
1 Hdden Marv Mdel fr peech Recgnn Reference: Berln Chen Deparmen f Cmpuer cence & Infrman Engneerng anal awan rmal Unvery. Raner and Juang. Fundamenal f peech Recgnn. Chaper 6. Huang e. al. pen Language rceng. Chaper Raner. A ural n Hdden Marv Mdel and eleced Applcan n peech Recgnn. rceedng f he IEEE vl. 77. Feruary Gale and Yung. he Applcan f Hdden Marv Mdel n peech Recgnn Chaper Yung. HMM and Relaed peech Recgnn echnlge. Chaper 7 prnger Hand f peech rceng prnger J.A. Blme A Genle ural f he EM Algrhm and Applcan arameer Eman fr Gauan Mxure and Hdden Marv Mdel U.C. Bereley R-97-0
2 Hry Hdden Marv Mdel (HMM): A Bref vervew ulhed n paper f Baum n lae 960 and early 970 Inrduced peech prceng y Baer (CMU) and Jelne (IBM) n he 970 (dcree HMM) hen exended cnnuu HMM y Bell La Aumpn peech gnal can e characerzed a a paramerc randm (chac) prce arameer can e emaed n a prece well-defned manner hree fundamenal prlem Evaluan f praly (lelhd) f a equence f ervan gven a pecfc HMM Deermnan f a e equence f mdel ae Adumen f mdel parameer a e accun fr erved gnal (r dcrmnan purpe) - Berln Chen
3 chac rce A chac prce a mahemacal mdel f a pralc expermen ha evlve n me and generae a equence f numerc value Each numerc value n he equence mdeled y a randm varale A chac prce u a (fne/nfne) equence f randm varale Example (a) he equence f recrded value f a peech uerance () he equence f daly prce f a c (c) he equence f hurly raffc lad a a nde f a cmmuncan newr (d) he equence f radar meauremen f he pn f an arplane - Berln Chen 3
4 ervale Marv Mdel ervale Marv Mdel (Marv Chan) Fr-rder Marv chan f ae a rple (A) a e f ae A he marx f rann prale eween ae ( = - = - = ) ( = - =) A he vecr f nal ae prale =( =) he upu f he prce he e f ae a each nan f me when each ae crrepnd an ervale even he upu n any gven ae n randm (deermnc!) mple decre he peech gnal characerc Fr-rder and me-nvaran aumpn - Berln Chen 4
5 ervale Marv Mdel (cn.) (rev. ae Cur. ae) Fr-rder Marv chan f ae ecnd-rder Marv chan f ae - Berln Chen 5
6 ervale Marv Mdel (cn.) Example : A 3-ae Marv Chan ae generae yml A nly ae generae yml B nly and ae 3 generae yml C nly 0.6 A B C Gven a equence f erved yml ={CABBCABC} he nly ne crrepndng ae equence { } and he crrepndng praly A 0.5 ( ) =( 3 )( 3 )( )( )( 3 )( 3 )( )( 3 ) = = Berln Chen 6
7 ervale Marv Mdel (cn.) Example : A hree-ae Marv chan fr he Dw Jne Indural average he praly f 5 cnecuve up day 5 cnecuve up day a a a a π Berln Chen 7
8 ervale Marv Mdel (cn.) Example 3: Gven a Marv mdel wha he mean ccupancy duran f each ae d d = d = a praly ma funcn f d a a Expeced numer f d d d da a a a a d a duran n a ae a duran d a gemerc drun raly a n ae d d me (Duran) - Berln Chen 8
9 Hdden Marv Mdel - Berln Chen 9
10 Hdden Marv Mdel (cn.) HMM an exended vern f ervale Marv Mdel he ervan urned e a pralc funcn (dcree r cnnuu) f a ae nead f an ne--ne crrepndence f a ae he mdel a duly emedded chac prce wh an underlyng chac prce ha n drecly ervale (hdden) Wha hdden? he ae equence! Accrdng he ervan equence we are n ure whch ae equence generae! Elemen f an HMM (he ae-upu HMM) ={AB} a e f ae A he marx f rann prale eween ae B a e f praly funcn each decrng he ervan praly wh repec a ae he vecr f nal ae prale - Berln Chen 0
11 Hdden Marv Mdel (cn.) w mar aumpn Fr rder (Marv) aumpn he ae rann depend nly n he rgn and denan me-nvaran A upu-ndependen aumpn All ervan are dependen n he ae ha generaed hem n n neghrng ervan - Berln Chen
12 Hdden Marv Mdel (cn.) w mar ype f HMM accrdng he ervan Dcree and fne ervan: he ervan ha all dnc ae generae are fne n numer V={v v v 3 v M } v R L In h cae he e f ervan praly drun B={ (v )} defned a (v )=( =v =) M : ervan a me : ae a me fr ae (v ) cn f nly M praly value A lef--rgh HMM - Berln Chen
13 Hdden Marv Mdel (cn.) w mar ype f HMM accrdng he ervan Cnnuu and nfne ervan: he ervan ha all dnc ae generae are nfne and cnnuu ha V={v vr d } In h cae he e f ervan praly drun B={ (v)} defned a (v)=f ( =v =) (v) a cnnuu praly deny funcn (pdf) and fen a mxure f Mulvarae Gauan (rmal) Drun v M Mxure Wegh w π d Σ Cvarance Marx exp v μ Σ v μ Mean Vecr ervan Vecr - Berln Chen 3
14 Hdden Marv Mdel (cn.) Mulvarae Gauan Drun When X=(x x x d ) a d-dmennal randm vecr he mulvarae Gauan pdf ha he frm: f X x μ Σ x; μ Σ Σ he cverance marx Σ and he - elevmen f Σ π where μ he L - dmennal mean vecr h E Σ he he deermnan f d exp x μ Σ x μ Σ μ E x x μx μ Exx μμ Σ E x μ x μ Ex x μ μ If x x x d are ndependen he cvarance marx reduced dagnal cvarance Vewed a d ndependen calar Gauan drun Mdel cmplexy gnfcanly reduced - Berln Chen 4
15 Hdden Marv Mdel (cn.) Mulvarae Gauan Drun - Berln Chen 5
16 Hdden Marv Mdel (cn.) Cvarance marx f he crrelaed feaure vecr (Mel-frequency fler an upu) Cvarance marx f he parally de-crrelaed feaure vecr (MFCC whu C 0 ) MFCC: Mel-frequency cepral ceffcen - Berln Chen 6
17 Hdden Marv Mdel (cn.) Mulvarae Mxure Gauan Drun (cn.) Mre cmplex drun wh mulple lcal maxma can e apprxmaed y Gauan (a unmdal drun) mxure f M M x w x; μ Σ w Gauan mxure wh enugh mxure cmpnen can apprxmae any drun - Berln Chen 7
18 Hdden Marv Mdel (cn.) Example 4: a 3-ae dcree HMM E 0.6 A A 0.3 B 0. C A 0.7 B 0. C A B C Gven a equence f ervan ={ABC} here are 7 ple crrepndng ae equence and herefre he crrepndng praly {A:.7B:.C:.} 7 7 : ae. g. when 3 A B C 3 0.5*0.7 * equence Ergdc HMM {A:.3B:.C:.5} {A:.3B:.6C:.} 0.7 *0.* Berln Chen 8
19 Hdden Marv Mdel (cn.) an: ={ 3 }: he ervan (feaure) equence = { 3 } : he ae equence : mdel fr HMM ={AB} ( ) : he praly f ervng gven he mdel ( ) : he praly f ervng gven and a ae equence f ( ) : he praly f ervng and gven ( ) : he praly f ervng gven and Ueful frmula Baye Rule : A B AB B A B B A A B B AA A BB A B A B B A : mdel decrng he praly chan rule AB B - Berln Chen 9
20 Hdden Marv Mdel (cn.) Ueful frmula (Cn.): al raly herem A margnal praly all B B f A B A BB all B A BdB f A Bf B B f db B dree and dn f B cnnuu f x x...x n x x... x x x...x are ndependen n n B B 3 A B B 5 B 4 E z q z fz z z q z q z dz z z : dcree : cnnuu Venn Dagram Expecan - Berln Chen 0
21 hree Bac rlem fr HMM Gven an ervan equence =(.. ) and an HMM =(AB) rlem : Hw effcenly cmpue ( )? Evaluan prlem rlem : Hw che an pmal ae equence =( )? Decdng rlem rlem 3: Hw adu he mdel parameer =(AB) maxmze ( )? Learnng / ranng rlem - Berln Chen
22 - Berln Chen Bac rlem f HMM (cn.) Gven and fnd ( )= r[ervng gven ] Drec Evaluan Evaluang all ple ae equence f lengh ha generang ervan equence : he praly f each pah By Marv aumpn (Fr-rder HMM) all all a a a 3... By Marv aumpn By chan rule
23 - Berln Chen 3 Bac rlem f HMM (cn.) Drec Evaluan (cn.) : he n upu praly alng he pah By upu-ndependen aumpn he praly ha a parcular ervan yml/vecr emed a me depend nly n he ae and cndnally ndependen f he pa ervan By upu-ndependen aumpn
24 - Berln Chen 4 Bac rlem f HMM (cn.) Drec Evaluan (Cn.) Huge Cmpuan Requremen: ( ) Expnenal cmpuanal cmplexy A mre effcen algrhm can e ued evaluae Frward/Bacward rcedure/algrhm.. all all a a a a a ADD : - MUL - Cmplexy
25 Bac rlem f HMM (cn.) Drec Evaluan (Cn.) 3 ae-me rell Dagram ae me 3 - dene ha ( ) ha een cmpued a dene ha a ha een cmpued - Berln Chen 5
26 Bac rlem f HMM - he Frward rcedure Baed n he HMM aumpn he calculan f and nvlve nly and ple cmpue he lelhd wh recurn n Frward varale :... he praly ha he HMM n ae a me havng generang paral ervan - Berln Chen 6
27 Algrhm. Inalzan. Inducn α 3.ermnan Bac rlem f HMM - he Frward rcedure (cn.) α Cmplexy: ( ) MUL ADD : : π α a α Baed n he lace (rell) rucure Cmpued n a me-ynchrnu fahn frm lef--rgh where each cell fr me cmpleely cmpued efre prceedng me + All ae equence regardle hw lng prevuly merge nde (ae) a each me nance Berln Chen 7
28 - Berln Chen 8 Bac rlem f HMM - he Frward rcedure (cn.) a fr-rder Marv aumpn B A A B A B all B A A A B A B A upu ndependen aumpn
29 Bac rlem f HMM - he Frward rcedure (cn.) 3 (3)=( 3 3 =3 ) =[ ()*a 3 + ()*a 3 + (3)*a 33 ] 3 ( 3 ) ae me 3 - dene ha ( ) ha een cmpued a dene a ha een cmpued - Berln Chen 9
30 Bac rlem f HMM - he Frward rcedure (cn.) A hree-ae Hdden Marv Mdel fr he Dw Jne Indural average (0.6* * *0.009)*0.7 = Berln Chen 30
31 Bac rlem f HMM - he Bacward rcedure Bacward varale : ()=( = ). Inalzan :. Inducn: a - 3. ermnan : Cmplexy MUL: - ; ADD: Berln Chen 3
32 - Berln Chen 3 Bac rlem f HMM - Bacward rcedure (cn.) Why?
33 Bac rlem f HMM - he Bacward rcedure (cn.) (3)=( 3 4 =3) =a 3 * ( 3 )* 3 () +a 3 * ( 3 )* 3 ()+a 33 * ( 3 )* 3 (3) 3 ae me Berln Chen 33
34 HMM a Knd f Bayean ewr Berln Chen 34
35 Bac rlem f HMM Hw che an pmal ae equence =( )? he fr pmal crern: Che he ae are ndvdually m lely a each me Defne a perr praly varale m m m m m ae ccupan praly (cun) a f algnmen f HMM ae he ervan (feaure) lun : * = arg max [ ()] rlem: maxmzng he praly a each me ndvdually *= * * * may n e a vald equence (e.g. a * + * = 0) - Berln Chen 35
36 Bac rlem f HMM (cn.) ( 3 = 3 )= 3 (3)* 3 (3) 3 ae 3 (3) 3 (3) a 3 = me Berln Chen 36
37 Bac rlem f HMM - he Ver Algrhm he ecnd pmal crern: he Ver algrhm can e regarded a he dynamc prgrammng algrhm appled he HMM r a a mdfed frward algrhm Inead f ummng up prale frm dfferen pah cmng he ame denan ae he Ver algrhm pc and rememer he e pah Fnd a ngle pmal ae equence =( ) Hw fnd he ecnd hrd ec. pmal ae equence (dffcul?) he Ver algrhm al can e lluraed n a rell framewr mlar he ne fr he frward algrhm ae-me rell dagram. R. Bellman n he hery f Dynamc rgrammng rceedng f he anal Academy f cence 95. A.J. Ver "Errr und fr cnvlunal cde and an aympcally pmum decdng algrhm IEEE ranacn n Infrman hery 3 () Berln Chen 37
38 Algrhm By nducn Bac rlem f HMM - he Ver Algrhm (cn.) Fnd a e ae equence = ervan Defne a max.... We can acrace frm new varale.. max a arg max a... * arg max δ = he e cre alng a ngle pah a me whch accun....? fr he fr ervan and end n ae fr a gven Fr acracng Cmplexy: ( ) - Berln Chen 38
39 Bac rlem f HMM - he Ver Algrhm (cn.) 3 ae 3 (3) me Berln Chen 39
40 Bac rlem f HMM - he Ver Algrhm (cn.) A hree-ae Hdden Marv Mdel fr he Dw Jne Indural average (0.6*0.35)*0.7 = Berln Chen 40
41 Bac rlem f HMM - he Ver Algrhm (cn.) Algrhm n he lgarhmc frm Fnd a e ae equence = ervan Defne a By nducn max lg.... We can acracefrm.. fr hefr ervan and end n ae = he e crealng a ngle pah a me whch accun new varale.... max lga lg arg max lga * arg maxδ? fr a gven...fr acracng - Berln Chen 4
42 Hmewr A hree-ae Hdden Marv Mdel fr he Dw Jne Indural average Fnd he praly: (up up unchanged dwn unchanged dwn up ) Fnd he pmal ae equence f he mdel whch generae he ervan equence: (up up unchanged dwn unchanged dwn up) - Berln Chen 4
43 raly Addn n F-B Algrhm In Frward-acward algrhm peran uually mplemened n lgarhmc dman lg + Aume ha we wan add and lg lg( + ) f ele lg lg lg lg lg lg lg lg lg lg x he value f lg can e aved n n a ale peedup he peran - Berln Chen 43
44 raly Addn n F-B Algrhm (cn.) An example cde #defne LZER (-.0E0) // ~lg(0) #defne LMALL (-0.5E0) // lg value < LMALL are e LZER #defne mnlgexp -lg(-lzer) // ~=-3 dule LgAdd(dule x dule y) { dule empdffz; f (x<y) { emp = x; x = y; y = emp; } dff = y-x; //nce ha dff <= 0 f (dff<mnlgexp) // f y far maller han x reurn (x<lmall)? LZER:x; ele { z = exp(dff); reurn x+lg(.0+z); } } - Berln Chen 44
45 Bac rlem 3 f HMM Inuve Vew Hw adu (re-emae) he mdel parameer =(AB) maxmze ( L ) r lg( L )? Belngng a ypcal prlem f nferenal ac he m dffcul f he hree prlem ecaue here n nwn analycal mehd ha maxmze he n praly f he ranng daa n a cle frm L lg... L lg l L l lg R l lg l l l all - uppe ha we have L ranng uerance fr he HMM - :a ple ae equence f he HMM he lg f um frm dffcul deal wh he daa ncmplee ecaue f he hdden ae equence Well-lved y he Baum-Welch (nwn a frward-acward) algrhm and EM (Expecan-Maxmzan) algrhm Ierave updae and mprvemen Baed n Maxmum Lelhd (ML) crern - Berln Chen 45
46 Maxmum Lelhd (ML) Eman: A chemac Depcn (/) Hard Agnmen Gven he daa fllw a mulnmal drun ae (B )=/4=0.5 (W )=/4=0.5 - Berln Chen 46
47 Maxmum Lelhd (ML) Eman: A chemac Depcn (/) f Agnmen Gven he daa fllw a mulnmal drun Maxmze he lelhd f he daa gven he algnmen ae ae (B )=( )/ ( ) =.6/.5=0.64 (W )=( )/ ( ) =0.9/.5= (B )=(0.3+0.)/ ( ) =0.4/.5=0.7 (W )=( )/ ( ) =0./.5= Berln Chen 47
48 - Berln Chen 48 Bac rlem 3 f HMM Inuve Vew (cn.) Relanhp eween he frward and acward varale a... a...
49 - Berln Chen 49 Bac rlem 3 f HMM Inuve Vew (cn.) Defne a new varale: raly eng a ae a me and a ae a me + Recall he perr praly varale: m n n mn n m a a a ) (fr Ο m m m a d repreene can e al : e B B A p B A p +
50 Bac rlem 3 f HMM Inuve Vew (cn.) ( 3 = 3 4 = )= 3 (3)*a 3 * ( 4 )* (4) 3 ae me Berln Chen 50
51 Bac rlem 3 f HMM Inuve Vew (cn.) expeced numer f rann frm ae ae n expeced numer f rann frm ae n A e f reanale re-eman frmula fr {A} expeced freqency (numer f me) n ae expeced numer f ran n frm ae ae expeced numer f ran n frm ae a me a - Frmulae fr ngle ranng Uerance - ξ γ - Berln Chen 5
52 Bac rlem 3 f HMM Inuve Vew (cn.) A e f reanale re-eman frmula fr {B} Fr dcree and fne ervan (v )=( =v =) v v expeced numer f me n ae and ervng expeced numer f me n ae yml v uch ha v Fr cnnuu and nfne ervan (v)=f ( =v =) M v c v μ Σ M ; c exp μ L / Σ v μ v Mdeled a a mxure f mulvarae Gauan drun - Berln Chen 5
53 - Berln Chen 53 Bac rlem 3 f HMM Inuve Vew (cn.) Fr cnnuu and nfne ervan (Cn.) Defne a new varale he praly f eng n ae a me wh he -h mxure cmpnen accunng fr M m m m m c c p m p m p m p m p m p m m m ; ; (ervan - ndependen aumpn appled)... Σ μ Σ μ c c c 3 3 Drun fr ae M m m : e B B A p B A p
54 Bac rlem 3 f HMM Inuve Vew (cn.) Fr cnnuu and nfne ervan (Cn.) c expeced numer f me n ae and mxure expeced numer f me n ae M γ m γ m μ weghed average (mean) f erva n a ae and mxure Σ weghed cvarancef ervan a ae μ μ and mxure Frmulae fr ngle ranng Uerance - Berln Chen 54
55 Bac rlem 3 f HMM Inuve Vew (cn.) Mulple ranng Uerance F/B F/B F/B 台師大 3 - Berln Chen 55
56 Bac rlem 3 f HMM Inuve Vew (cn.) Fr cnnuu and nfne ervan (Cn.) expeced freqency (numer f me) n ae a me ( ) L L l l a expeced numer f rann frm ae ae expeced numer f rann frm ae L l - l ξ l L l - l γ l c expeced numer f me n ae and mxure expeced numer f me n ae L l l γ l L l M l γ l m m μ weghed average (mean) f ervana ae and mxure L l l l L l l l Σ weghed cvarance f ervan a ae and mxure L l l l μ μ L l l l Frmulae fr Mulple (L) ranng Uerance - Berln Chen 56
57 Bac rlem 3 f HMM Inuve Vew (cn.) Fr dcree and fne ervan (cn.) expeced freqency (numer f me) n ae a me ( ) L L l l a expeced numer f rann frm ae ae expeced numer f rann frm ae L - l ξ l L - l l l γ l v v expeced numer f me n ae and ervng ymlv expeced numer f me n ae L l l l uch ha v L l l l Frmulae fr Mulple (L) ranng Uerance - Berln Chen 57
58 emcnnuu HMM he HMM ae mxure deny funcn are ed geher acr all he mdel frm a e f hared ernel he emcnnuu r ed-mxure HMM ae upu raly f ae M M f v μ Σ -h mxure wegh -h mxure deny funcn r -h cdewrd f ae (hared acr HMM M very large) (dcree mdel-dependen) A cmnan f he dcree HMM and he cnnuu HMM A cmnan f dcree mdel-dependen wegh ceffcen and cnnuu mdel-ndependen cde praly deny funcn Becaue M large we can mply ue he L m gnfcan value f v Experence hwed ha L ~3% f M adequae aral yng f f fr dfferen phnec cla v - Berln Chen 58
59 - Berln Chen 59 emcnnuu HMM (cn.) 3 3 M.... M.... M μ Σ μ Σ M M Σ μ Σ μ
60 HMM plgy peech me-evlvng nn-anary gnal Each HMM ae ha he aly capure me qua-anary egmen n he nn-anary peech gnal A lef--rgh plgy a naural canddae mdel he peech gnal (al called he ead-n-a-rng mdel) I general repreen a phne ung 3~5 ae (Englh) and a yllale ung 6~8 ae (Mandarn Chnee) - Berln Chen 60
61 Inalzan f HMM A gd nalzan f HMM ranng : egmenal K-Mean egmenan n ae Aume ha we have a ranng e f ervan and an nal emae f all mdel parameer ep : he e f ranng ervan equence egmened n ae aed n he nal mdel (fndng he pmal ae equence y Ver Algrhm) ep : Fr dcree deny HMM (ung M-cdewrd cde) he numer f vecr wh cde ndex n ae he numer f vecr n ae Fr cnnuu deny HMM (M Gauan mxure per ae) cluer he ervan vecr w m m whn each ae numer f vecr clafed n cluer m f ae dvded y he numer f vecr n ae n a e f ample mean f he vecr clafed n cluer m f ae m ample cvarance marx f he vecr clafed n cluer m f ae ep 3: Evaluae he mdel cre If he dfference eween he prevu and curren mdel cre greaer han a hrehld g ac ep herwe p he nal mdel generaed M 3 cluer - Berln Chen 6
62 Inalzan f HMM (cn.) ranng Daa Mdel Reeman Emae parameer f ervan va egmenal K-mean aeequence egmeman Inal Mdel Mdel Cnvergence? YE Mdel arameer - Berln Chen 6
63 Inalzan f HMM (cn.) An example fr dcree HMM 3 ae and cdewrd 3 ae (v )=3/4 (v )=/4 (v )=/3 (v )=/3 3 (v )=/3 3 (v )=/3 v v - Berln Chen 63
64 Inalzan f HMM (cn.) An example fr Cnnuu HMM 3 ae and 4 Gauan mxure per ae 3 ae K-mean { } { } Glal mean Cluer mean Cluer mean { } { } - Berln Chen 64
65 Knwn Lman f HMM (/3) he aumpn f cnvennal HMM n peech rceng he ae duran fllw an expnenal drun Dn prvde adequae repreenan f he empral rucure f peech d a a Fr-rder (Marv) aumpn: he ae rann depend nly n he rgn and denan upu-ndependen aumpn: all ervan frame are dependen n he ae ha generaed hem n n neghrng ervan frame Reearcher have prped a numer f echnque addre hee lman ale hee lun have n gnfcanly mprved peech recgnn accuracy fr praccal applcan. - Berln Chen 65
66 Knwn Lman f HMM (/3) Duran mdelng gemerc/ expnenal drun emprcal drun Gamma drun Gauan drun - Berln Chen 66
67 Knwn Lman f HMM (3/3) he HMM parameer raned y he Baum-Welch algrhm (r EM algrhm) were nly lcally pmzed Lelhd Curren Mdel Cnfguran Mdel Cnfguran pace - Berln Chen 67
68 Hmewr- (/) 0.34 {A:.34B:.33C:.33} {A:.33B:.34C:.33} {A:.33B:.33C:.34} rane :. ABBCABCAABC. ABCABC 3. ABCA ABC 4. BBABCAB 5. BCAABCCAB 6. CACCABCA 7. CABCABCA 8. CABCA 9. CABCA rane :. BBBCCBC. CCBABB 3. AACCBBB 4. BBABBAC 5. CCA ABBAB 6. BBBCCBAA 7. ABBBBABA 8. CCCCC 9. BBAAA - Berln Chen 68
69 Hmewr- (/). leae pecfy he mdel parameer afer he fr and 50h eran f Baum-Welch ranng. leae hw he recgnn reul y ung he ave ranng equence a he eng daa (he -called nde eng). *Yu have perfrm he recgnn a wh he HMM raned frm he fr and 50h eran f Baum-Welch ranng repecvely 3. Whch cla d he fllwng eng equence elng? ABCABCCAB AABABCCCCBBB 4. Wha are he reul f ervale Marv Mdel were nead ued n and 3? - Berln Chen 69
70 Ilaed Wrd Recgnn Wrd Mdel M p X M Lelhd f M Wrd Mdel M peech gnal Feaure Exracn Feaure equence X p X M Wrd Mdel M V p X M V Lelhd f M Lelhd f M V M Le Wrd elecr Lael X arg max px M M ML Ver Apprxman Lael X arg max max px M Wrd Mdel M l p X M l Lelhd f M l - Berln Chen 70
71 Meaure f AR erfrmance (/8) Evaluang he perfrmance f aumac peech recgnn (AR) yem crcal and he Wrd Recgnn Errr Rae (WER) ne f he m mpran meaure here are ypcally hree ype f wrd recgnn errr uun An ncrrec wrd wa uued fr he crrec wrd Delen A crrec wrd wa med n he recgnzed enence Inern An exra wrd wa added n he recgnzed enence Hw deermne he mnmum errr rae? - Berln Chen 7
72 Meaure f AR erfrmance (/8) Calculae he WER y algnng he crrec wrd rng agan he recgnzed wrd rng A maxmum urng machng prlem Can e handled y dynamc prgrammng Example: deleed Crrec : he effec clear Recgnzed: effec n clear WER+ WAR =00% Errr analy: ne delen and ne nern Meaure: wrd errr rae (WER) wrd crrecn rae (WCR) wrd accuracy rae (WAR) Wrd Wrd Wrd mached nered mached Mgh e hgher han 00% u. Del. In. wrd Errr Rae 00% 50%. f wrd n he crrec enence 4 Mached wrd 3 Crrecn Rae 00% 75%. f wrd n he crrec enence 4 Mached - In. wrd 3 Accuracy Rae 00% 50%. f wrd n he crrec enence 4 Mgh e negave - Berln Chen 7
73 Meaure f AR erfrmance (3/8) A Dynamc rgrammng Algrhm (ex) //dene fr he wrd lengh f he crrec/reference enence //dene fr he wrd lengh f he recgnzed/e enence mnmum wrd errr algnmen a he a grd [] /h nd f algnmen /h e Ref - Berln Chen 73
74 Meaure f AR erfrmance (4/8) Algrhm (y Berln Chen) ep : Inalzan : Ref G[0][0] 0; fr... n { //e G[][0] G[ -][0] ; B[][0] ; //Inern e } (Hrznal Drecn) fr... m { //reference G[0][] G[0][ -] ; B[0][] ; // Delen } (Vercal Drecn) ep : Ieran : fr...n { //e fr...m { //reference G[ -][] (Inern) G[][-] (Delecn) G[][] mn G[ -][-] (f LR[]! L[] uun) G[ -][-] (f LR[] L[] Mach) ; //Inern (Hrznal Drecn) ; //Delen (Vercal Drecn) B[][] 3;//uun (Dagnal Drecn) 4; //mach (Dagnal Drecn) } //fr reference } //fr e ep 3 : Meaure pmal f ele f ele and Bacrace G[n][m] Wrd Errr Rae 00% m Wrd Accuracy Rae 00 % Wrd acrace B[][] B[][] : pah (B[n][m] prn " prn " LR[] prn " LR[] Errr... Rae B[0][0]) L[]" ; //Iner "; //Delen LR[] "; //H/Mac e: he penale fr uun delen and nern errr are all e e here n hen hen g lef g dwn h r uu n hen g dwn dagnally - Berln Chen 74
75 Crrec/Reference Wrd equence Meaure f AR erfrmance (5/8) A Dynamc rgrammng Algrhm Inalzan fr (=;<=m;++) { //reference grd[0][] = grd[0][-]; grd[0][].dr = VER; grd[0][].cre += Delen; grd[0][].del ++; } HK m m Del. 3 Del. Del. 0 grd[0][0].cre = grd[0][0].n = grd[0][0].del = 0; grd[0][0].u = grd[0][0].h = 0; grd[0][0].dr = IL; 0 Del. (-) (--) In. () Del. (-) In. (nm) n- n In. In. 3In. Recgnzed/e Wrd fr (=;<=n;++) { // e equence grd[][0] = grd[-][0]; grd[][0].dr = HR; grd[][0].cre +=Inen; grd[][0].n ++; } - Berln Chen 75 Del.
76 Meaure f AR erfrmance (6/8) HK rgram fr (=;<=n;++) //e { grd = grd[]; grd = grd[-]; fr (=;<=m;++) //reference { h = grd[].cre +nen; d = grd[-].cre; f (lref[]!= le[]) d += uen; v = grd[-].cre + delen; f (d<=h && d<=v) { /* DIAG = h r u */ grd[] = grd[-]; //rucure agnmen grd[].cre = d; grd[].dr = DIAG; f (lref[] == le[]) ++grd[].h; ele ++grd[].u; } ele f (h<v) { /* HR = n */ grd[] = grd[]; grd[].cre = h; grd[].dr = HR; ++ grd[].n; } ele { /* VER = del */ grd[] = grd[-]; grd[].cre = v; grd[].dr = VER; ++grd[].del; } } /* fr */ } /* fr */ //rucure agnmen //rucure agnmen Example Crrec (0500) (0400) (0300) (000) (000) Crrec: e: C C B C A (040) (030) (00) (00) (000) (InDeluH) (03) (0) (0) r (00) (0) r(000) (00) (0) r (30) H A e 0 0 In B B A B C (0000) (000) (000) (3000) (4000) In B (0) (0) H B (0) Del C (00) A C B C C B A B C H A Del C H B H C Del C (03) (03) H C (0) r (0) (00) (300) Delee C Algnmen : WER= 60% ll have an her pmal algnmen! - Berln Chen 76
77 Meaure f AR erfrmance (7/8) Example e: he penale fr uun delen and nern errr are all e e here (InDeluH) Algnmen : WER= 80% Crrec: e: Crrec: e: In B (0500) (0400) (0300) (000) (000) C C B C A (0000) 0 0 A C B C C B A A C H A Del C u B A C B C C B A A C u A u C u B H C Crrec H C Del C Del C (040) (030) (00) (00) (03) (0) (0) r (00) (0) r(000) (0) r (30) Del C () () u B (0) () () (0) r (0) (00) Delee C H C (000) (00) (00) (300) H A e In B B A A C (000) (000) (3000) (4000) Algnmen 3: WER=80% Algnmen : WER=80% Crrec: e: In B A C B C C B A A C H A u C Del B H C Del C - Berln Chen 77
78 Meaure f AR erfrmance (8/8) w cmmn eng f dfferen penale fr uun delen and nern errr /* HK errr penale */ uen = 0; delen = 7; nen = 7; /* I errr penale*/ ueni = 4; deleni = 3; neni = 3; - Berln Chen 78
79 Hmewr 3 Meaure f AR erfrmance Reference 桃 芝 颱 風 重 創 花 蓮 光 復 鄉 大 興 村 死 傷 慘 重 感 觸 最 多 AR upu 桃 芝 颱 風 重 創 花 蓮 光 復 鄉 打 新 村 次 傷 殘 周 感 觸 最 多. - Berln Chen 79
80 Hmewr B re f AR upu Repr he CER (characer errr rae) f he fr ne and 506 re he reul huld hw he numer f uun delen and nern errr verall Reul E: %Crrec=0.00 [H=0 = =] WRD: %Crr=8.5 Acc=8.5 [H=75 D=4 =3 I=0 =9] =================================================================== verall Reul E: %Crrec=0.00 [H=0 =00 =00] WRD: %Crr=87.66 Acc=86.83 [H=083 D=77 =348 I=0 =357] =================================================================== verall Reul E: %Crrec=0.00 [H=0 =00 =00] WRD: %Crr=87.9 Acc=87.8 [H=657 D=93 =84 I=86 =5774] =================================================================== verall Reul E: %Crrec=0.00 [H=0 =506 =506] WRD: %Crr=86.83 Acc=86.06 [H=5744 D=89 =7839 I=504 =658] =================================================================== - Berln Chen 80
81 yml fr Mahemacal peran - Berln Chen 8
82 he EM Algrhm (/7) {A:.3B:.C:.5} {A:.7B:.C:.} {A:.3B:.6C:.} p( ) p( )> p( ) A B erved daa : : all equence Laen daa : : le equence arameer e emaed maxmze lg( ) ={(A)(B)(B A)(A B)(R A)(G A)(R B)(G B)} - Berln Chen 8
83 he EM Algrhm (/7) Inrducn f EM (Expecan Maxmzan): Why EM? mple pmzan algrhm fr lelhd funcn rele n he nermedae varale called laen daa In ur cae here he ae equence he laen daa Drec acce he daa neceary emae he parameer mple r dffcul In ur cae here alm mple emae {AB } whu cnderan f he ae equence w Mar ep : E : expecan wh repec he laen daa ung he curren emae f he parameer and cndned n he E ervan M: prvde a new eman f he parameer accrdng Maxmum lelhd (ML) r Maxmum A err (MA) Crera - Berln Chen 83
84 ML and MA he EM Algrhm (3/7) Eman prncple aed n ervan: x x x... he Maxmum Lelhd (ML) rncple fnd he mdel parameer Φ ha he lelhd px Φ maxmum fr example f Φ μ Σ he parameer f a mulvarae nrmal drun and X..d. (ndependen dencally drued) hen he ML emae f Φ μ Σ μ ML n x x n Σ ML n n n x μ x μ he Maxmum A err (MA) rncple fnd he mdel parameer Φ ha he lelhd pφ x maxmum X ML X X... X n ML - Berln Chen 84
85 he EM Algrhm (4/7) he EM Algrhm mpran HMM and her learnng echnque Dcver new mdel parameer maxmze he lg-lelhd f ncmplee daa lg y eravely maxmzng he expecan f lg-lelhd frm cmplee daa lg Frly ung calar (dcree) randm varale nrduce he EM algrhm he ervale ranng daa We wan maxmze a parameer vecr he hdden (unervale) daa E.g. he cmpnen prale (r dene) f ervale daa r he underlyng ae equence n HMM - Berln Chen 85
86 he EM Algrhm (5/7) Aume we have and emae he praly ha each ccurred n he generan f reend we had n fac erved a cmplee daa par wh frequency prprnal he praly cmpued a new he maxmum lelhd emae f De he prce cnverge? Algrhm Lg-lelhd expren and expecan aen ver lg lg Baye rule cmplee daa lelhd ncmplee daa lelhd unnwn mdel eng lg lg ae expecan ver lg lg lg - Berln Chen 86
87 he EM Algrhm (6/7) Algrhm (Cn.) We can hu expre a fllw lg We wan lg lg Q H where Q H lg lg lg lg lg lg Q H Q H Q Q H H lg - Berln Chen 87
88 he EM Algrhm (7/7) ha he fllwng prpery H 0 H lg 0 H H herefre fr maxmzng lg we nly need maxmze he Q-funcn (auxlary funcn) Q H H lg Kulluac-Leler (KL) dance ( lg x x ) Jenen nequaly Expecan f he cmplee daa lg lelhd wh repec he laen ae equence - Berln Chen 88
89 - Berln Chen 89 EM Appled Dcree HMM ranng (/5) Apply EM algrhm eravely refne he HMM parameer vecr By maxmzng he auxlary funcn Where and can e expreed a ) ( π B A lg lg Q a a a lg lg lg lg lg lg lg lg
90 - Berln Chen 90 EM Appled Dcree HMM ranng (/5) Rewre he auxlary funcn a v Q a a Q Q Q Q Q Q all all all lg lg lg lg lg lg a a a π a π w y???
91 EM Appled Dcree HMM ranng (3/5) he auxlary funcn cnan hree ndependen erm a and Can e maxmzed ndvdually All f he ame frm F F y gy y... y y ha maxmum value when : w lg y y where w w y and y 0 - Berln Chen 9
92 - Berln Chen 9 EM Appled Dcree HMM ranng (4/5) rf: Apply Lagrange Mulpler w w y w w y y w 0 y w y F y y lg w y lg w F ha uppe Mulpler By applyng Lagrange Cnran Lagrange Mulpler: hp://
93 - Berln Chen 93 EM Appled Dcree HMM ranng (5/5) he new mdel parameer e can e expreed a: B A π = v v a....
94 EM Appled Cnnuu HMM ranng (/7) Cnnuu HMM: he ae ervan de n cme frm a fne e u frm a cnnuu pace he dfference eween he dcree and cnnuu HMM le n a dfferen frm f ae upu praly Dcree HMM requre he quanzan prcedure map ervan vecr frm he cnnuu pace he dcree pace Cnnuu Mxure HMM M c he ae ervan drun f HMM mdeled y mulvarae Gauan mxure deny funcn (M mxure) M M c; μ Σ c exp L / Σ μ M c μ Σ w w w 3 3 Drun fr ae - Berln Chen 94
95 - Berln Chen 95 EM Appled Cnnuu HMM ranng (/7) Expre wh repec each ngle mxure cmpnen M M M c a a p... equence ae wh he alng equence cmpnen mxure ple he f : ne K K c a p K K p p M M M M M M M a a a a a a a a a a a e:
96 EM Appled Cnnuu HMM ranng (3/7) herefre an auxlary funcn fr he EM algrhm can e wren a: Q K lg p K K K p K p lg p K K lg lga lg lg p lgc Q Q a Q Q c Q c π nal prale a ae rann prale Gauan deny funcn mxure cmpnen - Berln Chen 96
97 - Berln Chen 97 EM Appled Cnnuu HMM ranng (4/7) he nly dfference we have when cmpared wh Dcree HMM ranng M c M c Q Q lg lg Ο c Ο
98 - Berln Chen 98 EM Appled Cnnuu HMM ranng (5/7) M L M Q L 0 lg lg lg lg lg exp ; Le μ μ Σ μ μ μ Σ μ μ Σ μ Σ μ Σ μ Σ Σ μ Ο ymmerc here and ) ( Σ d d x C C x Cx x
99 - Berln Chen 99 EM Appled Cnnuu HMM ranng (6/7) M Q L 0 lg lg lg lg lg μ μ Σ Σ Σ μ μ Σ Σ Σ Σ Σ Σ μ μ Σ Σ Σ μ μ Σ Σ Σ Σ Σ μ μ Σ Σ Σ μ μ Σ Σ Σ Σ Σ μ Σ μ Σ d d X a X X X a ) ( ymmerc here and ) de( de Σ d d X X X X
100 - Berln Chen 00 EM Appled Cnnuu HMM ranng (7/7) he new mdel parameer e fr each mxure cmpnen and mxure wegh can e expreed a: p p p p μ p p p p μ μ μ μ Σ M c
Hidden Markov Models for Speech Recognition
Hdden Marv Mdel fr Speech Recgnn Reference: Berln Chen Deparmen f Cmpuer Scence & Infrman Engneerng anal awan rmal Unvery. Raner and Juang. Fundamenal f Speech Recgnn. Chaper 6. Huang e. al. Spen Language
More informationSpeech Recognition. Berlin Chen Department of Computer Science & Information Engineering National Taiwan Normal University
Reference: Hdden Marv Mdel fr Speech Recgnn Berln Chen Deparmen f Cmpuer Scence & Infrman Engneerng anal awan rmal Unvery. Rabner and Juang. Fundamenal f Speech Recgnn. Chaper 6. Huang e. al. Spen Language
More informationHidden Markov Models
Hdden Marv Mdel Berln Chen 004 Reference:. Rabner and Juang. Fundamenal f Speech Recgnn. Chaper 6. Huang e. al. Spen Language rceng. Chaper 4 8 3. Vaegh. Advanced Dgal Sgnal rceng and e Reducn. Chaper
More informationDishonest casino as an HMM
Dshnes casn as an HMM N = 2, ={F,L} M=2, O = {h,} A = F B= [. F L F L 0.95 0.0 0] h 0.5 0. L 0.05 0.90 0.5 0.9 c Deva ubramanan, 2009 63 A generave mdel fr CpG slands There are w hdden saes: CpG and nn-cpg.
More informationNatural Language Processing NLP Hidden Markov Models. Razvan C. Bunescu School of Electrical Engineering and Computer Science
Naural Language rcessng NL 6840 Hdden Markv Mdels Razvan C. Bunescu Schl f Elecrcal Engneerng and Cmpuer Scence bunescu@h.edu Srucured Daa Fr many applcans he..d. assumpn des n hld: pels n mages f real
More informationApplications of Sequence Classifiers. Learning Sequence Classifiers. Simple Model - Markov Chains. Markov models (Markov Chains)
Learnng Sequence Classfers pplcans f Sequence Classfers Oulne pplcans f sequence classfcan ag f wrds, n-grams, and relaed mdels Markv mdels Hdden Markv mdels Hgher rder Markv mdels Varans n Hdden Markv
More informationPhysics 20 Lesson 9H Rotational Kinematics
Phyc 0 Len 9H Ranal Knemac In Len 1 9 we learned abu lnear mn knemac and he relanhp beween dplacemen, velcy, acceleran and me. In h len we wll learn abu ranal knemac. The man derence beween he w ype mn
More informationOutline. Probabilistic Model Learning. Probabilistic Model Learning. Probabilistic Model for Time-series Data: Hidden Markov Model
Probablsc Model for Tme-seres Daa: Hdden Markov Model Hrosh Mamsuka Bonformacs Cener Kyoo Unversy Oulne Three Problems for probablsc models n machne learnng. Compung lkelhood 2. Learnng 3. Parsng (predcon
More informationRAMIFICATIONS of POSITION SERVO LOOP COMPENSATION
RAMIFICATIONS f POSITION SERO LOOP COMPENSATION Gerge W. Yunk, P.E. Lfe Fellw IEEE Indural Cnrl Cnulg, Inc. Fnd du Lac, Wcn Fr many year dural pg er dre dd n ue er cmpena he frward p lp. Th wa referred
More informationIntroduction ( Week 1-2) Course introduction A brief introduction to molecular biology A brief introduction to sequence comparison Part I: Algorithms
Course organzaon Inroducon Wee -2) Course nroducon A bref nroducon o molecular bology A bref nroducon o sequence comparson Par I: Algorhms for Sequence Analyss Wee 3-8) Chaper -3, Models and heores» Probably
More informationConvection and conduction and lumped models
MIT Hea ranfer Dynamc mdel 4.3./SG nvecn and cndcn and lmped mdel. Hea cnvecn If we have a rface wh he emperare and a rrndng fld wh he emperare a where a hgher han we have a hea flw a Φ h [W] () where
More information(,,, ) (,,, ). In addition, there are three other consumers, -2, -1, and 0. Consumer -2 has the utility function
MACROECONOMIC THEORY T J KEHOE ECON 87 SPRING 5 PROBLEM SET # Conder an overlappng generaon economy le ha n queon 5 on problem e n whch conumer lve for perod The uly funcon of he conumer born n perod,
More informationDiscrete Markov Process. Introduction. Example: Balls and Urns. Stochastic Automaton. INTRODUCTION TO Machine Learning 3rd Edition
EHEM ALPAYDI he MI Press, 04 Lecure Sldes for IRODUCIO O Machne Learnng 3rd Edon alpaydn@boun.edu.r hp://www.cmpe.boun.edu.r/~ehem/ml3e Sldes from exboo resource page. Slghly eded and wh addonal examples
More informationVariable Forgetting Factor Recursive Total Least Squares Algorithm for FIR Adaptive filtering
01 Inernanal Cnference n Elecrncs Engneerng and Infrmacs (ICEEI 01) IPCSI vl 49 (01) (01) IACSI Press Sngapre DOI: 107763/IPCSI01V4931 Varable Frgeng Facr Recursve al Leas Squares Algrhm fr FIR Adapve
More informationEXPLOITATION OF FEATURE VECTOR STRUCTURE FOR SPEAKER ADAPTATION
Erc Ch e al. Eplan f Feaure Vecr Srucure EXPLOIION OF FEUE VECO SUCUE FO SPEKE DPION Erc H.C. Ch, rym Hler, Julen Epps, run Gpalarshnan Mrla usralan esearch Cenre, Mrla Las {Erc.Ch, rym.hler, Julen.Epps,
More informationMicroelectronic Circuits. Feedback. Ching-Yuan Yang
Mcrelecrnc rcu Feedback hng-yuan Yang Nanal hung-hng Unvery Deparmen Elecrcal Engneerng Oulne The General Feedback Srucure Sme Prpere Negave Feedback The Fur Bac Feedback Tplge The Sere-Shun Feedback mpler
More informationLecture Slides for INTRODUCTION TO. Machine Learning. ETHEM ALPAYDIN The MIT Press,
Lecure Sldes for INTRDUCTIN T Machne Learnng ETHEM ALAYDIN The MIT ress, 2004 alpaydn@boun.edu.r hp://www.cmpe.boun.edu.r/~ehem/2ml CHATER 3: Hdden Marov Models Inroducon Modelng dependences n npu; no
More information( ) [ ] MAP Decision Rule
Announcemens Bayes Decson Theory wh Normal Dsrbuons HW0 due oday HW o be assgned soon Proec descrpon posed Bomercs CSE 90 Lecure 4 CSE90, Sprng 04 CSE90, Sprng 04 Key Probables 4 ω class label X feaure
More informationExperiment 6: STUDY OF A POSITION CONTROL SERVOMECHANISM
Expermen 6: STUDY OF A POSITION CONTROL SERVOMECHANISM 1. Objecves Ths expermen prvdes he suden wh hands-n experence f he peran f a small servmechansm. Ths sysem wll be used fr mre cmplex wrk laer n he
More informationLecture 11: Stereo and Surface Estimation
Lecure : Sereo and Surface Emaon When camera poon have been deermned, ung rucure from moon, we would lke o compue a dene urface model of he cene. In h lecure we wll udy he o called Sereo Problem, where
More informationMultiple Regressions and Correlation Analysis
Mulple Regreon and Correlaon Analy Chaper 4 McGraw-Hll/Irwn Copyrgh 2 y The McGraw-Hll Compane, Inc. All rgh reerved. GOALS. Decre he relaonhp eween everal ndependen varale and a dependen varale ung mulple
More informationControl Systems. Mathematical Modeling of Control Systems.
Conrol Syem Mahemacal Modelng of Conrol Syem chbum@eoulech.ac.kr Oulne Mahemacal model and model ype. Tranfer funcon model Syem pole and zero Chbum Lee -Seoulech Conrol Syem Mahemacal Model Model are key
More informationA Demand System for Input Factors when there are Technological Changes in Production
A Demand Syem for Inpu Facor when here are Technologcal Change n Producon Movaon Due o (e.g.) echnologcal change here mgh no be a aonary relaonhp for he co hare of each npu facor. When emang demand yem
More information10.7 Temperature-dependent Viscoelastic Materials
Secin.7.7 Temperaure-dependen Viscelasic Maerials Many maerials, fr example plymeric maerials, have a respnse which is srngly emperaure-dependen. Temperaure effecs can be incrpraed in he hery discussed
More information( ) () we define the interaction representation by the unitary transformation () = ()
Hgher Order Perurbaon Theory Mchael Fowler 3/7/6 The neracon Represenaon Recall ha n he frs par of hs course sequence, we dscussed he chrödnger and Hesenberg represenaons of quanum mechancs here n he chrödnger
More informationFundamentals of PLLs (I)
Phae-Locked Loop Fundamenal of PLL (I) Chng-Yuan Yang Naonal Chung-Hng Unvery Deparmen of Elecrcal Engneerng Why phae-lock? - Jer Supreon - Frequency Synhe T T + 1 - Skew Reducon T + 2 T + 3 PLL fou =
More informationAverage Small-Signal Modelling of the Power Stage of Power Factor Correctors with a Fast Output- Voltage Feedback Loop
erage SmallSgnal dellng f he wer Sage f wer Facr rrecr wh a Fa Oupu lage Feedbac p Jaer Sebaán, Deg. amar, ara aría Hernand, guel dríguez and rur Fernández Unerdad de Oed. rup de Sema Elecrónc de lmenacón
More informationBrace-Gatarek-Musiela model
Chaper 34 Brace-Gaarek-Musiela mdel 34. Review f HJM under risk-neural IP where f ( T Frward rae a ime fr brrwing a ime T df ( T ( T ( T d + ( T dw ( ( T The ineres rae is r( f (. The bnd prices saisfy
More informationMatrix reconstruction with the local max norm
Marx reconrucon wh he local max norm Rna oygel Deparmen of Sac Sanford Unvery rnafb@anfordedu Nahan Srebro Toyoa Technologcal Inue a Chcago na@cedu Rulan Salakhudnov Dep of Sac and Dep of Compuer Scence
More informationMachine Learning for Signal Processing Prediction and Estimation, Part II
Machine Learning fr Signal Prceing Predicin and Eimain, Par II Bhikha Raj Cla 24. 2 Nv 203 2 Nv 203-755/8797 Adminirivia HW cre u Sme uden wh g really pr mark given chance upgrade Make i all he way he
More informationRheological Models. In this section, a number of one-dimensional linear viscoelastic models are discussed.
helgcal Mdels In hs secn, a number f ne-dmensnal lnear vscelasc mdels are dscussed..3. Mechancal (rhelgcal) mdels The wrd vscelasc s derved frm he wrds "vscus" + "elasc"; a vscelasc maeral exhbs bh vscus
More informationOptimal Control. Lecture. Prof. Daniela Iacoviello
Opmal Cnrl Lecure Pr. Danela Iacvell Gradng Prjec + ral eam Eample prjec: Read a paper n an pmal cnrl prblem 1 Sudy: backgrund mvans mdel pmal cnrl slun resuls 2 Smulans Yu mus gve me al leas en days bere
More informationClustering (Bishop ch 9)
Cluserng (Bshop ch 9) Reference: Daa Mnng by Margare Dunham (a slde source) 1 Cluserng Cluserng s unsupervsed learnng, here are no class labels Wan o fnd groups of smlar nsances Ofen use a dsance measure
More informationR th is the Thevenin equivalent at the capacitor terminals.
Chaper 7, Slun. Applyng KV Fg. 7.. d 0 C - Takng he derae f each erm, d 0 C d d d r C Inegrang, () ln I 0 - () I 0 e - C C () () r - I 0 e - () V 0 e C C Chaper 7, Slun. h C where h s he Theenn equalen
More informationEfficient Built-In Self-Test Techniques for Sequential Fault Testing of Iterative Logic Arrays
Prceedngs f he 6h WSES Inernanal Cnference n ppled Infrmacs and Cmmuncans, Elunda, Greece, ugus 8-0, 006 (pp53-57) Effcen Bul-In Self-Tes Technques fr Sequenal Faul Tesng f Ierave Lgc rrays SHYUE-KUNG
More informationThe Components of Vector B. The Components of Vector B. Vector Components. Component Method of Vector Addition. Vector Components
Upcming eens in PY05 Due ASAP: PY05 prees n WebCT. Submiing i ges yu pin ward yur 5-pin Lecure grade. Please ake i seriusly, bu wha cuns is wheher r n yu submi i, n wheher yu ge hings righ r wrng. Due
More informationCS626 Speech, Web and natural Language Processing End Sem
CS626 Speech, Web and naural Language Proceng End Sem Dae: 14/11/14 Tme: 9.30AM-12.30PM (no book, lecure noe allowed, bu ONLY wo page of any nformaon you deem f; clary and precon are very mporan; read
More informationFX-IR Hybrids Modeling
FX-IR Hybr Moeln Yauum Oajma Mubh UFJ Secure Dervave Reearch Dep. Reearch & Developmen Dvon Senor Manaer oajma-yauum@c.mu.jp Oaka Unvery Workhop December 5 h preenaon repreen he vew o he auhor an oe no
More informationCooling of a hot metal forging. , dt dt
Tranen Conducon Uneady Analy - Lumped Thermal Capacy Model Performed when; Hea ranfer whn a yem produced a unform emperaure drbuon n he yem (mall emperaure graden). The emperaure change whn he yem condered
More informationCS434a/541a: Pattern Recognition Prof. Olga Veksler. Lecture 4
CS434a/54a: Paern Recognon Prof. Olga Veksler Lecure 4 Oulne Normal Random Varable Properes Dscrmnan funcons Why Normal Random Varables? Analycally racable Works well when observaon comes form a corruped
More informationGraduate Macroeconomics 2 Problem set 5. - Solutions
Graduae Macroeconomcs 2 Problem se. - Soluons Queson 1 To answer hs queson we need he frms frs order condons and he equaon ha deermnes he number of frms n equlbrum. The frms frs order condons are: F K
More informationA. Inventory model. Why are we interested in it? What do we really study in such cases.
Some general yem model.. Inenory model. Why are we nereed n? Wha do we really udy n uch cae. General raegy of machng wo dmlar procee, ay, machng a fa proce wh a low one. We need an nenory or a buffer or
More informationCHAPTER 10: LINEAR DISCRIMINATION
CHAPER : LINEAR DISCRIMINAION Dscrmnan-based Classfcaon 3 In classfcaon h K classes (C,C,, C k ) We defned dscrmnan funcon g j (), j=,,,k hen gven an es eample, e chose (predced) s class label as C f g
More information(V 1. (T i. )- FrC p. ))= 0 = FrC p (T 1. (T 1s. )+ UA(T os. (T is
. Yu are repnible fr a reacr in which an exhermic liqui-phae reacin ccur. The fee mu be preheae he hrehl acivain emperaure f he caaly, bu he pruc ream mu be cle. T reuce uiliy c, yu are cniering inalling
More informationIn the complete model, these slopes are ANALYSIS OF VARIANCE FOR THE COMPLETE TWO-WAY MODEL. (! i+1 -! i ) + [(!") i+1,q - [(!
ANALYSIS OF VARIANCE FOR THE COMPLETE TWO-WAY MODEL The frs hng o es n wo-way ANOVA: Is here neracon? "No neracon" means: The man effecs model would f. Ths n urn means: In he neracon plo (wh A on he horzonal
More informationMultiple Failures. Diverse Routing for Maximizing Survivability. Maximum Survivability Models. Minimum-Color (SRLG) Diverse Routing
Mulple Falure Dvere Roung for Maxmzng Survvably One-falure aumpon n prevou work Mulple falure Hard o provde 100% proecon Maxmum urvvably Maxmum Survvably Model Mnmum-Color (SRLG) Dvere Roung Each lnk ha
More informationLet s start from a first-order low pass filter we already discussed.
EEE0 Netrk Analy II Dr. harle Km Nte09: Actve Flter ---Part. gher-order Actve Flter The rt-rder lter d nt harply dvde the pa band and the tp band. One apprach t btan a harper trantn beteen the pa band
More informationOrdinary Differential Equations in Neuroscience with Matlab examples. Aim 1- Gain understanding of how to set up and solve ODE s
Ordnary Dfferenal Equaons n Neuroscence wh Malab eamples. Am - Gan undersandng of how o se up and solve ODE s Am Undersand how o se up an solve a smple eample of he Hebb rule n D Our goal a end of class
More informationVolatility Interpolation
Volaly Inerpolaon Prelmnary Verson March 00 Jesper Andreasen and Bran Huge Danse Mares, Copenhagen wan.daddy@danseban.com brno@danseban.com Elecronc copy avalable a: hp://ssrn.com/absrac=69497 Inro Local
More informationA Nonisolated Transformerless High Voltage Gain Buck Boost dc-dc Converter
OAE JOUNA OF EEA ENGNEENG, O 5, NO AUUN 5 9 A Nnlaed ranfrmerle Hgh lage Gan Buck B dcdc nverer hammad eza Banae, Hen Ajdar Faegh bnab * Abrac A nvel ranfrmer le hgh epup buck b dcdc cnverer prped n h
More informationOn One Analytic Method of. Constructing Program Controls
Appled Mahemacal Scences, Vol. 9, 05, no. 8, 409-407 HIKARI Ld, www.m-hkar.com hp://dx.do.org/0.988/ams.05.54349 On One Analyc Mehod of Consrucng Program Conrols A. N. Kvko, S. V. Chsyakov and Yu. E. Balyna
More informationGAMS Handout 2. Utah State University. Ethan Yang
Uah ae Universiy DigialCmmns@UU All ECAIC Maerials ECAIC Repsiry 2017 GAM Handu 2 Ehan Yang yey217@lehigh.edu Fllw his and addiinal wrs a: hps://digialcmmns.usu.edu/ecsaic_all Par f he Civil Engineering
More informationEnergy Storage Devices
Energy Srage Deces Objece f ecure Descrbe The cnsrucn f an nducr Hw energy s sred n an nducr The elecrcal prperes f an nducr Relanshp beween lage, curren, and nducance; pwer; and energy Equalen nducance
More informationTHE PREDICTION OF COMPETITIVE ENVIRONMENT IN BUSINESS
THE PREICTION OF COMPETITIVE ENVIRONMENT IN BUSINESS INTROUCTION The wo dmensonal paral dfferenal equaons of second order can be used for he smulaon of compeve envronmen n busness The arcle presens he
More informationLecture 11 SVM cont
Lecure SVM con. 0 008 Wha we have done so far We have esalshed ha we wan o fnd a lnear decson oundary whose margn s he larges We know how o measure he margn of a lnear decson oundary Tha s: he mnmum geomerc
More informationIntroduction to Congestion Games
Algorihmic Game Theory, Summer 2017 Inroducion o Congeion Game Lecure 1 (5 page) Inrucor: Thoma Keelheim In hi lecure, we ge o know congeion game, which will be our running example for many concep in game
More informationChapter 5 Signal-Space Analysis
Chaper 5 Sgnal-Space Analy Sgnal pace analy provde a mahemacally elegan and hghly nghful ool for he udy of daa ranmon. 5. Inroducon o Sacal model for a genec dgal communcaon yem n eage ource: A pror probable
More informationA New Structure of Buck-Boost Z-Source Converter Based on Z-H Converter
Jurnal f Operan and Auman n wer Engneerng l. 4, N., ec., ages: 7-3 hp://jape.uma.ac.r A New rucure f Buck-Bs Z-urce nverer Based n Z-H nverer E. Babae*,. Ahmadzadeh Faculy f Elecrcal and mpuer Engneerng,
More informationChapter 6: AC Circuits
Chaper 6: AC Crcus Chaper 6: Oulne Phasors and he AC Seady Sae AC Crcus A sable, lnear crcu operang n he seady sae wh snusodal excaon (.e., snusodal seady sae. Complee response forced response naural response.
More informationof Manchester The University COMP14112 Hidden Markov Models
COMP42 Lecure 8 Hidden Markov Model he Univeriy of Mancheer he Univeriy of Mancheer Hidden Markov Model a b2 0. 0. SAR 0.9 0.9 SOP b 0. a2 0. Imagine he and 2 are hidden o he daa roduced i a equence of
More informationEmergence time, curvature, space, causality, and complexity in encoding a discrete impulse information process Vladimir S.
Emergence me, curvaure, pace, caualy, and cmplexy n encdng a dcree mpule nfrman prce Vladmr S. Lerner, USA, lernerv@gmal.cm Abrac Ineracn buld rucure f Unvere, and mpule elemenary neracve dcree acn mdelng
More informationAdvanced Machine Learning & Perception
Advanced Machne Learnng & Percepon Insrucor: Tony Jebara SVM Feaure & Kernel Selecon SVM Eensons Feaure Selecon (Flerng and Wrappng) SVM Feaure Selecon SVM Kernel Selecon SVM Eensons Classfcaon Feaure/Kernel
More informationA Method of Insulator Detection from Aerial Images
Sensrs & Transducers Vl. 177 Issue 8 Augus 2014 pp. 7-13 Sensrs & Transducers 2014 by IFSA Publshng S. L. hp://www.sensrspral.cm A Mehd f Insular Deecn frm Aeral Images 1 Yngje Zha 1 Yang Wu 2 Hngka Chen
More informationL N O Q. l q l q. I. A General Case. l q RANDOM LAGRANGE MULTIPLIERS AND TRANSVERSALITY. Econ. 511b Spring 1998 C. Sims
Econ. 511b Sprng 1998 C. Sm RAD AGRAGE UPERS AD RASVERSAY agrange mulpler mehod are andard fare n elemenary calculu coure, and hey play a cenral role n economc applcaon of calculu becaue hey ofen urn ou
More informationThis document is downloaded from DR-NTU, Nanyang Technological University Library, Singapore.
Ths documen s downloaded from DR-NTU, Nanyang Technologcal Unversy Lbrary, Sngapore. Tle A smplfed verb machng algorhm for word paron n vsual speech processng( Acceped verson ) Auhor(s) Foo, Say We; Yong,
More informationConsider processes where state transitions are time independent, i.e., System of distinct states,
Dgal Speech Processng Lecure 0 he Hdden Marov Model (HMM) Lecure Oulne heory of Marov Models dscree Marov processes hdden Marov processes Soluons o he hree Basc Problems of HMM s compuaon of observaon
More informationNotes on Inductance and Circuit Transients Joe Wolfe, Physics UNSW. Circuits with R and C. τ = RC = time constant
Nes n Inducance and cu Tansens Je Wlfe, Physcs UNSW cus wh and - Wha happens when yu clse he swch? (clse swch a 0) - uen flws ff capac, s d Acss capac: Acss ess: c d s d d ln + cns. 0, ln cns. ln ln ln
More informationPower Decoupling Method for Isolated DC to Single-phase AC Converter using Matrix Converter
Pwer Decuplng Mehd fr Islaed DC Sngle-phase AC Cnverer usng Marx Cnverer Hrk Takahash, Nagsa Takaka, Raul Rber Rdrguez Guerrez and Jun-ch Ih Dep. f Elecrcal Engneerng Nagaka Unversy f Technlgy Nagaka,
More informationThe Buck Resonant Converter
EE646 Pwer Elecrnics Chaper 6 ecure Dr. Sam Abdel-Rahman The Buck Resnan Cnverer Replacg he swich by he resnan-ype swich, ba a quasi-resnan PWM buck cnverer can be shwn ha here are fur mdes f pera under
More informationHidden Markov Models
11-755 Machne Learnng for Sgnal Processng Hdden Markov Models Class 15. 12 Oc 2010 1 Admnsrva HW2 due Tuesday Is everyone on he projecs page? Where are your projec proposals? 2 Recap: Wha s an HMM Probablsc
More informationAn introduction to Support Vector Machine
An nroducon o Suppor Vecor Machne 報告者 : 黃立德 References: Smon Haykn, "Neural Neworks: a comprehensve foundaon, second edon, 999, Chaper 2,6 Nello Chrsann, John Shawe-Tayer, An Inroducon o Suppor Vecor Machnes,
More informationCS344: Introduction to Artificial Intelligence
C344: Iroduco o Arfcal Iellgece Puhpa Bhaacharyya CE Dep. IIT Bombay Lecure 3 3 32 33: Forward ad bacward; Baum elch 9 h ad 2 March ad 2 d Aprl 203 Lecure 27 28 29 were o EM; dae 2 h March o 8 h March
More informationRobustness Experiments with Two Variance Components
Naonal Insue of Sandards and Technology (NIST) Informaon Technology Laboraory (ITL) Sascal Engneerng Dvson (SED) Robusness Expermens wh Two Varance Componens by Ana Ivelsse Avlés avles@ns.gov Conference
More informationLecture 4 ( ) Some points of vertical motion: Here we assumed t 0 =0 and the y axis to be vertical.
Sme pins f erical min: Here we assumed and he y axis be erical. ( ) y g g y y y y g dwnwards 9.8 m/s g Lecure 4 Accelerain The aerage accelerain is defined by he change f elciy wih ime: a ; In analgy,
More informationIntroduction. If there are no physical guides, the motion is said to be unconstrained. Example 2. - Airplane, rocket
Kinemaic f Paricle Chaper Inrducin Kinemaic: i he branch f dynamic which decribe he min f bdie wihu reference he frce ha eiher caue he min r are generaed a a reul f he min. Kinemaic i fen referred a he
More informationSolution in semi infinite diffusion couples (error function analysis)
Soluon n sem nfne dffuson couples (error funcon analyss) Le us consder now he sem nfne dffuson couple of wo blocks wh concenraon of and I means ha, n a A- bnary sysem, s bondng beween wo blocks made of
More informationNON-HOMOGENEOUS SEMI-MARKOV REWARD PROCESS FOR THE MANAGEMENT OF HEALTH INSURANCE MODELS.
NON-HOOGENEOU EI-AKO EWA POCE FO THE ANAGEENT OF HEATH INUANCE OE. Jacque Janen CEIAF ld Paul Janon 84 e 9 6 Charlero EGIU Fax: 32735877 E-mal: ceaf@elgacom.ne and amondo anca Unverà a apenza parmeno d
More informationLecture 18: The Laplace Transform (See Sections and 14.7 in Boas)
Lecure 8: The Lalace Transform (See Secons 88- and 47 n Boas) Recall ha our bg-cure goal s he analyss of he dfferenal equaon, ax bx cx F, where we emloy varous exansons for he drvng funcon F deendng on
More informationComparison Of Single Stage And Two Stage Stage Grid-tie Inverters
Unvery f Cenral Flrda Elecrnc hee and Deran Maer he Open Acce Cmparn Of Sngle Sage And w Sage Sage Grd-e nverer 007 Keh Manfeld Unvery f Cenral Flrda Fnd mlar wrk a: hp://ar.lbrary.ucf.edu/ed Unvery f
More informationRandomized Perfect Bipartite Matching
Inenive Algorihm Lecure 24 Randomized Perfec Biparie Maching Lecurer: Daniel A. Spielman April 9, 208 24. Inroducion We explain a randomized algorihm by Ahih Goel, Michael Kapralov and Sanjeev Khanna for
More informationTechnical Note: Auto Regressive Model
Techncal Ne: Au Regressve Mdel We rgnall cmsed hese echncal nes afer sng n n a me seres analss class. Over he ears, we ve mananed hese nes and added new nsghs, emrcal bservans and nuns acqured. We fen
More information10.5 Linear Viscoelasticity and the Laplace Transform
Scn.5.5 Lnar Vclacy and h Lalac ranfrm h Lalac ranfrm vry uful n cnrucng and analyng lnar vclac mdl..5. h Lalac ranfrm h frmula fr h Lalac ranfrm f h drvav f a funcn : L f f L f f f f f c..5. whr h ranfrm
More informationEstimation of Gene Expression
AS-74.4330 Genmc Cnrl ewrs, Semnar Repr, Fall 005 Esman f Gene Expressn Ren Vrrans Helsn Unversy f Technlgy, Cnrl Engneerng Labrary Emal: ren.vrrans@hu.f Suden ID: 5406J Absrac The mcrarray echnlgy has
More informationAppendix H: Rarefaction and extrapolation of Hill numbers for incidence data
Anne Chao Ncholas J Goell C seh lzabeh L ander K Ma Rober K Colwell and Aaron M llson 03 Rarefacon and erapolaon wh ll numbers: a framewor for samplng and esmaon n speces dversy sudes cology Monographs
More informationNotes on the stability of dynamic systems and the use of Eigen Values.
Noes on he sabl of dnamc ssems and he use of Egen Values. Source: Macro II course noes, Dr. Davd Bessler s Tme Seres course noes, zarads (999) Ineremporal Macroeconomcs chaper 4 & Techncal ppend, and Hamlon
More informationComputing Relevance, Similarity: The Vector Space Model
Compung Relevance, Smlary: The Vecor Space Model Based on Larson and Hears s sldes a UC-Bereley hp://.sms.bereley.edu/courses/s0/f00/ aabase Managemen Sysems, R. Ramarshnan ocumen Vecors v ocumens are
More informationEP2200 Queuing theory and teletraffic systems. 3rd lecture Markov chains Birth-death process - Poisson process. Viktoria Fodor KTH EES
EP Queung heory and eleraffc sysems 3rd lecure Marov chans Brh-deah rocess - Posson rocess Vora Fodor KTH EES Oulne for oday Marov rocesses Connuous-me Marov-chans Grah and marx reresenaon Transen and
More informationMaximum Likelihood Estimation
Mau Lkelhood aon Beln Chen Depaen of Copue Scence & Infoaon ngneeng aonal Tawan oal Unvey Refeence:. he Alpaydn, Inoducon o Machne Leanng, Chape 4, MIT Pe, 4 Saple Sac and Populaon Paaee A Scheac Depcon
More informationJohn Geweke a and Gianni Amisano b a Departments of Economics and Statistics, University of Iowa, USA b European Central Bank, Frankfurt, Germany
Herarchcal Markov Normal Mxure models wh Applcaons o Fnancal Asse Reurns Appendx: Proofs of Theorems and Condonal Poseror Dsrbuons John Geweke a and Gann Amsano b a Deparmens of Economcs and Sascs, Unversy
More informationKinematics Review Outline
Kinemaics Review Ouline 1.1.0 Vecrs and Scalars 1.1 One Dimensinal Kinemaics Vecrs have magniude and direcin lacemen; velciy; accelerain sign indicaes direcin + is nrh; eas; up; he righ - is suh; wes;
More informationJanuary Examinations 2012
Page of 5 EC79 January Examnaons No. of Pages: 5 No. of Quesons: 8 Subjec ECONOMICS (POSTGRADUATE) Tle of Paper EC79 QUANTITATIVE METHODS FOR BUSINESS AND FINANCE Tme Allowed Two Hours ( hours) Insrucons
More information2015 Sectional Physics Exam Solution Set
. Crrec answer: D Ne: [quan] denes: uns quan WYSE cadec Challenge 05 Secnal Phscs Ea SOLUTION SET / / / / rce lengh lengh rce enu ass lengh e a) / ass ass b) energ c) wrk lengh e pwer energ e d) (crrec
More informationSpeech, NLP and the Web
peech NL ad he Web uhpak Bhaacharyya CE Dep. IIT Bombay Lecure 38: Uuperved learg HMM CFG; Baum Welch lecure 37 wa o cogve NL by Abh Mhra Baum Welch uhpak Bhaacharyya roblem HMM arg emac ar of peech Taggg
More informationDC-DC Switch-Mode Converters
- Swich-Mde nverers - cnverers are used : egulaed swich-mde pwer supplies, nrmally wih HF elecrical isla Mr drives, nrmally wihu an isla ransfrmer We will lk a he w basic dc-dc cnverer plgies: Sep-dwn
More informationDepartment of Economics University of Toronto
Deparmen of Economcs Unversy of Torono ECO408F M.A. Economercs Lecure Noes on Heeroskedascy Heeroskedascy o Ths lecure nvolves lookng a modfcaons we need o make o deal wh he regresson model when some of
More informationEnergy & Work
rk Dne by a Cntant Frce 6.-6.4 Energy & rk F N m jule () J rk Dne by a Cntant Frce Example Pullng a Sutcae-n-heel Fnd the wrk dne the rce 45.0-N, the angle 50.0 degree, and the dplacement 75.0 m. 3 ( F
More informationCHAPTER 5: MULTIVARIATE METHODS
CHAPER 5: MULIVARIAE MEHODS Mulvarae Daa 3 Mulple measuremens (sensors) npus/feaures/arbues: -varae N nsances/observaons/eamples Each row s an eample Each column represens a feaure X a b correspons o he
More informationMASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.265/15.070J Fall 2013 Lecture 15 10/30/2013. Ito integral for simple processes
MASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.65/15.7J Fall 13 Lecure 15 1/3/13 I inegral fr simple prcesses Cnen. 1. Simple prcesses. I ismery. Firs 3 seps in cnsrucing I inegral fr general prcesses 1 I inegral
More informationUNIVERSITAT AUTÒNOMA DE BARCELONA MARCH 2017 EXAMINATION
INTERNATIONAL TRADE T. J. KEHOE UNIVERSITAT AUTÒNOMA DE BARCELONA MARCH 27 EXAMINATION Please answer wo of he hree quesons. You can consul class noes, workng papers, and arcles whle you are workng on he
More informationSSRG International Journal of Thermal Engineering (SSRG-IJTE) Volume 4 Issue 1 January to April 2018
SSRG Inernaonal Journal of Thermal Engneerng (SSRG-IJTE) Volume 4 Iue 1 January o Aprl 18 Opmal Conrol for a Drbued Parameer Syem wh Tme-Delay, Non-Lnear Ung he Numercal Mehod. Applcaon o One- Sded Hea
More information