EE53 Audo Sgnals and Systes Statstcal Pattern Classfcaton Kevn D. Donohue Electrcal and Couter Engneerng Unversty of Kentucy
Interretaton of Audtory Scenes Huan erceton and cognton greatly eceeds any couter-based syste for abstractng sounds nto objects and creatng eanngful audtory scenes. hs erceton of objects not just detectng acoustc energy allows for nterretaton of stuatons leadng to an arorate resonse or further analyses. Sensory organs ears searate acoustc energy nto frequency bands and convert band energy nto neural frngs he audtory corte receves the neural resonses and abstracts an audtory scene.
Audtory Scene Perceton derves a useful reresentaton of realty fro sensory nut. Audtory Strea refers to a ercetual unt assocated wth a sngle haenng A.S. Bregan, 990. Acoustc to Neural Converson Organze nto Audtory Streas Reresentaton of Realty
Couter Interretaton In order for a couter algorth to nterret a scene Acoustc sgnals ust be converted to nubers usng eanngful odels. Sets of nubers or atterns are aed nto events ercetons. Events are analyzed wth other events n relaton to the goal of the algorth and aed nto a stuaton cognton or dervng eanng. Stuaton s aed nto an acton/resonse. Nubers etracted fro the acoustc sgnal for the urose of classfcaton deternaton of event are referred to as features. e -based features are etracted fro sgnal transfors such as: Enveloe Correlatons Frequency-based features are etracted fro sgnal transfors such as: Sectru Cestru Power Sectral Densty
Feature Selecton Eale Consder a roble of dscrnatng between the soen words yes and no based on features:. he estate of frst forant frequency g resonance of the sectral enveloe. he rato n db of the altude of the second forant frequency over the thrd forant frequency g. A fcttous eerent was erfored and these features were couted for 5 recordngs of eole sayng these words. he feature were lotted for each class to develo an algorth to classfy these sales correctly.
Feature Plot Defne a feature vector. G g g Plot G, gven a yes was soen, wth green o s, and gven a no was soen, be wht red s.
Mnu Dstance Aroach Create reresentatve vector for yes and no features 5 μ yes G n yes 5 n 5 μno G n no 5 n For a new sale wth estated features, use decson rule: G μ no no < yes G μ yes Results n 3 ncorrect decsons.
Noralzaton Wth SD he frequency features had larger values than the altude ratos, and therefore had ore nfluence n the decson rocess. Reove scale dfferences by noralzng each feature by ts standard devaton over all classes. σ 5 5 g n yes μ yes + g n no μ no 5 n n Now 4 errors result why would t change?
Mnu Dstance Classfer Consder feature vector wth the otental to be classfed as belongng to K eclusve classes. Classfcaton decson wll be based on the dstance of the feature vector to one of the telate vectors reresentng each of the K classes. he decson rule s for a gven observaton and set of telate vectors z for each class, decde on class such that: argn [ ] D z z
Mnu Dstance Classfer If soe features need to be weghted ore than others n the decson rocess, as well as elotng correlaton between the features, the dstance for each feature can be weghted to result n the weghted nu dstance classfer: argn [ ] D z W z where W s a square atr of weghts wth denson equal to length of. If W s a dagonal atr, t sly scales each of the features n the decson rocess. Off dagonal ters scale the correlaton between features. If W s the nverse of the covarance atr of the features n, and z s the ean feature vector for each class, then the above dstances are referred to as the Mahanalobs dstance. z E K [ ] W E z z K [ ]
Correlaton Recever It can be shown that selectng the class based on the nu dstance between the observaton vector and the telate vector s equvalent to fndng the au correlaton between the observaton vector and the telate: [ ] arg n D z z arga[ C z ] or [ ] D z W z [ C Wz ] arg n arga where the telate vectors have been noralzed such that z z P P s a constant for all
Defntons Rando varable RV s a functon that as events sets nto a dscrete set of real nubers for a dscrete RV, or a contnuous set of real nubers for a contnuous RV. Rando rocess RP s a seres of RVs ndeed by a countable set for a dscrete RP, or by a non-countable set for contnuous RP.
Defntons: PDF Frst Order he lelhood of RV values s descrbed through the robablty densty functon df. X Pr e [ < ] b X e d 0 and d b X X
Defntons: Jont PDF he robabltes descrbng ore than one RV s descrbed by a jont df. Pr [ < < ] b X e yb Y ye XY y y e b e b XY, y ddy, y 0, y and, y ddy XY
Defntons: Condtonal PDF he robabltes descrbng a RV gven that the another event has already occurred s descrbed by a condtonal df. Closely related to ths s Bayes rule:, y y y Y XY Y X, y y y y y y y X Y Y X X Y X X Y XY Y Y X
Eales: Gaussan PDF A frst order Gaussan RV df scalar wth ean µ and standard devaton σ s gven by: A hgher order jont Gaussan df colun vector wth ean vector and covarance atr s gven by: e σ μ πσ X [ ] [] [ ] n n E E,, e / / X L π
Eale Uncorrelated Prove that for an N th order sequence of uncorrelated Gaussan zero-ean RVs the jont PDF can be wrtten as: X N πσ e σ Note that for Gaussan RVs uncorrelated les statstcal ndeendence. Assue varances are equal for all eleents. What would the autocorrelaton of ths sequence loo le? How would the above analyss change f RVs were not zero ean?
Class PDFs When features are odeled as RVs, ther dfs can be used to derve dstance easures for the classfer, and an otal decson rule that nzes classfcaton error can be desgned. Consder K classes ndvdually denoted by. Feature values assocated wth each class can be descrbed by: a osteror robablty lelhood the class after observaton/data a ror robablty lelhood the class before observaton/data Lelhood functon lelhood observaton/data gven a class
Class PDFs he lelhood functon can be estated through ercal studes. Consder 3 seaers whose 3 rd forant frequency s dstrbuted by: Decson hresholds 3 Classfer robabltes can be obtaned fro Bayes rule
Mau a osteror Decson Rule For K classes and observed feature vector, the au a osteror MAP decson rule states: or by alyng Bayes rule: For the bnary case ths reduces to the log lelhood rato j j > f Decde j j j > f Decde > < > < ln ln ln j j j j j j
Eale Consder a class roble wth Gaussan dstrbuted feature vectors [,, ] L E E E [ ] E[ ] N [ ] [ ] Derve the log lelhood rato and descrbe how the classfer uses dstance nforaton to dscrnate between the classes.
Hoewor Consder a features for use n a bnary classfcaton roble. he features are Gaussan dstrbuted are for feature vector [, ]. Derve the log lelhood rato and corresondng classfer for the 3 dfferent cases lsted below: 0.5 0.5 [,] 0.6 0 0,. [, ] 0.8 0 0 0. 3 4 0.5 0. 0 [ 0,0] 0, 0. 0.5 0 0 0.5 [,] 0.6 0 0.5 0. [,] 0,. [, ] 0. 0.5 0. 0.8 [, ] 0.8 0 0 0. Coent how each classfer coutes dstance and uses t n the classfcaton rocess.
Classfcaton Error Classfcaton error s the ercentage of decson statstcs that occur on the wrong sde of the threshold, scaled by the ercentage of tes such an event occurs. λ λ λ λ λ λ 3 e λ λ dλ + λ λ dλ + λ λ dλ + 3 λ λ 3 dλ
Hoewor For the revous eale, wrte an eresson for robablty of a correct classfcaton by changng the ntegrals and lts.e. do not sly wrte c - e
Aroatng a Bayes Classfer If densty functons are not nown: Deterne telate vectors that nze dstances to feature vectors n each class for tranng data vector quantzaton. Assue for of densty functon and estate araeters drectly or teratvely fro the data araetrc or eectaton azaton. Learn osteror robabltes drectly fro tranng data and nterolate on test data neural networs.