The conditional density p(x s ) Bayes rule explained. Bayes rule for a classification problem INF

Transcription

1 INF Mulvarae clafcao Ae Solberg Baed o Chaper -6 Duda ad Har: Paer Clafcao Baye rule for a clafcao proble Suppoe we have J, =,J clae he cla label for a pel, ad he oberved feaure vecor We ca ue Baye rule o fd a epreo for he cla wh he hghe probably: p P P p pror probably poeror probably lkelhood oralzg facor P he pror probably for cla If we do' have pecal kowledge ha oe of he clae occur ore freque ha oher clae, we e he equal for all clae P =/J, =,,,J 04 INF 4300 INF 4300 Baye rule eplaed p P P p p he probably dey fuco ha odel he lkelhood for obervg feaure vecor f he pel belog o cla Typcally we aue a ype of drbuo, eg Gaua, ad he ea ad covarace of ha drbuo fed o oe daa ha we kow belog o ha cla Th fg called clafer rag P he poeror probably ha he pel acually belog o cla p u a calg facor ha aure ha he probable u o INF The codoal dey p Ay probably dey fuco ca be ued o odel p A coo odel he ulvarae Gaua dey The ulvarae Gaua dey: p ep / Σ / μ Σ μ If we have feaure, a vecor of legh ad ad a ar deped o cla μ S ΣS 3 he deera of he ar, ad - he vere INF Syerc ar he varace of feaure he covarace bewee feaure ad feaure

2 Mea vecor ad covarace arce deo If a -deoal feaure vecor for oe obec/pel, we ca forulae ea vecor ad covarace ar a: wh feaure, he ea vecor wll be of ze ad or ze INF Σ μ Ipecg p INF p μ Σ μ Σ / / ep Scalar vecor rapoed vecor rapoed ar Ivere of covarace ar Scalar probably The ea vecor for each cla The ea vecor for cla defed a he epeced value of : wh feaure, he ea vecor wll be of ze If we have M rag aple ha we kow belog o cla, we ca eae he ea vecor a: INF μ rag aple belogg o cla where he u over all, ˆ M M μ The covarace ar for each cla The covarace for cla defed a he epeced value of -- : wh feaure, he covarace ar wll be of ze If we have M rag aple ha we kow belog o cla, we ca eae he covarace ar The eae of a rado varable f deoed ach er copued a: INF Σ rag aple belogg o cla where he u over all ˆ ˆ ˆ M M for cla ad for he covarace bewee feaure ˆ ˆ,,, M M fˆ

3 More o he covarace ar The covarace ar wll alway be yerc ad pove edefe If all copoe of have o-zero varace, wll be pove defe he covarace bewee feaure ad If feaure ad are ucorrelaed, = 0 I he geeral cae, wll have +/ dffere value A D Gaua odel Paraeer μ ad Σ defe a dey a a bup The curve o he plo are coour of equal probably, u a he coour o a ap The ar Σ h cae ha hree dffere elee, varace each of he ae, ad covarace bewee he ae S he varace for feaure = he covarace bewee feaure ad he varace for feaure INF INF The covarace ar ad ellpe I D, he Gaua odel ca be hough of a approag he clae D feaure pace wh ellpe The ea vecor =[, ] defe he he ceer po of he ellpe, he covarace bewee he feaure defe he oreao of he ellpe ad defe he wdh of he ellpe he a ad b paraeer S The ellpe defe po where he probably dey equal qual he ee ha he dace o he ea a copued by he Mahalaob dace equal The Mahalaob dace bewee a po ad he cla ceer : T r The a ae of he ellpe deered by he egevecor of The egevalue of gve her legh INF 4300 ucldea dace v Mahalaob dace ucldea dace bewee po ad cla ceer : T Mahalaob dace bewee ad : r T INF 4300 Po wh equal dace o le o a crcle Po wh equal dace o le o a ellpe

4 Dcra fuco for he oral dey We aw la lecure ha he u-error-rae clafcao ca copued ug he dcra fuco g l p l P Wh a ulvarae Gaua we ge: g μ μ l l l P Le u look a h epreo for oe pecal cae: Cae : Σ =σ I Coder ha we aue he feaure ucorrelaed depede wh he ae varace σ The covarace σ =0 by defo f he feaure are ucorrelaed The dcra fuco wll be plfed o: μ g l P where μ μ μ We how h o he e lde Thu we odel he probable a -deoal phere becaue po ha have equal dcra fuco wll le o a crcle aroud he ea Σ - =I/σ Σ = σ INF INF Cae : Σ =σ I The dcra fuco plfe o lear fuco ug uch a hape o he probably drbuo T g μ l l μ P T T T μ l l μ μ P Lear algebra bac: Ier produc bewee wo vecor The er produc or do produc bewee wo vecor of legh N ad y or gve by N, y y y The agle bewee wo vecor A ad B defed a: T Coo for all clae, o eed o copue hee wo er Sce T coo for all clae, g reduce o a lear fuco of INF co A, B A B If he er produc of wo vecor zero, hey are oral o each oher 6

5 Cae : Σ =σ I A equvale forulao of he dcra fuco: g w w 0 where w μ ad 0 l w μ μ P The equao g =g cabe wrea If he feaure were depee Σ =σ I he dcra fuco wa plfed o: ' T g μ l μ P μ l P w 0 where w μ -μ 0 P ad 0 μ -μ l μ -μ μ -μ P w= - he vecor bewee he ea value Th equao defe a hyperplae hrough he po 0, ad orhogoal o w If P =P he hyperplae wll be locaed halfway bewee he ea value Th reul lear deco boudare Copug h dcra fuco o clafy paer volve copug he dace fro he po o he ea value for each cla INF INF A ple odel, Σ =σ I The dcra fuco whe Σ =σ I ha defe he border bewee cla ad he feaure pace a ragh le The dcra fuco erec he le coecg he wo cla ea a he po 0 = - / f we do o coder pror probable The dcra fuco wll alo be oral o he le coecg he ea 0 Deco boudary The drbuo are phercal deo The deco boudary a geeralzed hyperplae of - deo The deco boudary perpedcular o he le eparag he wo ea value Th kd of a clafer called a lear clafer, or a lear dcra fuco Becaue he deco fuco a lear fuco of If P = P, he deco boudary wll be half-way bewee ad 9 INF

6 Mu dace clafcao If all clae have equal pror probable, 0 wll be he po halfway bewee he ea vecor Clafcao wll co of agg feaure vecor o he ae cla a he cloe ea eaured by ucldea dace - A clafer baed o he ucldea dace called a u dace clafer Cae : Coo covarace, Σ = Σ If we aue ha all clae have he ae hape of daa cluer, a uve odel o aue ha her probably drbuo have he ae hape By h aupo we ca ue all he daa o eae he covarace ar Th eae coo for all clae, ad h ea ha alo h cae he dcra fuco becoe lear fuco T g μ Σ μ l Σ l P T T T Σ μ Σ μ Σ μ l Σ l P INF 4300 Coo for all clae, o eed o copue Sce T coo for all clae, g aga reduce o a lear fuco of INF 4300 Cae : Coo covarace, Σ = Σ Coo covarace, Σ = Σ A equvale forulao of he dcra fuco g w w0 where w Σ μ ad w0 μ Σ μ l P The deco boudare are aga hyperplae Becaue w = Σ - - o he dreco of -, he hyperpla wl o be orhogoal o he le bewee he ea The clae ca be decrbed by hyperellpode d deo All hyperellpod have he ae oreao The deco boudary wll aga be a hyperplae Becaue w= Σ - - geerally o he dreco of -, he hyperplae wll o be perpedcular o he le bewee he ea Coder a po 0 o he le - defed by he pror probable: If P = P, 0 wll be half way bewee he ea The eparag hyperplae wll erec he le a 0 INF INF

7 Cae 3:, Σ =arbrary Whe all clae are odeled a havg dffere hape, he dcra fuco cao be plfed Th ea ha he dcra fuco wll be quadrac fuco Deco boudare wll be hyperquadrc ad aue ay of he geeral for: hyperplae, par of hyperplae, hyperphere, hyperellode, hyperparabolod, hyperhyperbolod INF Cae 3:, Σ =arbrary The dcra fuco wll be quadrac: g W w w0 where W Σ, w Σ μ ad w0 μ Σ μ l Σ l P The deco urface are hyperquadrc ad ca aue ay of he geeral for: hyperplae hyperhpere par of hyperplae hyperellod, Hyperparabolod, The e lde how eaple of h I h geeral cae we cao uvely draw he deco boudare u by lookg a he ea ad covarace INF The full odel, Σ =arbrary - eaple The full odel, Σ=arbrary - eaple INF INF

8 The full odel, Σ =arbrary - eaple The full odel, Σ=arbrary - eaple INF INF The full odel, Σ=arbrary - eaple A ulcla eaple INF INF

9 I he Gaua clafer he oly choce? The Gaua clafer gve lear or quadrac dcra fuco Oher clafer ca gve arbrary cople deco urface ofe pecewe-lear Mure of Gaua Oher probably dey fuco -drbuo, epoeal drbuo Neural ework Suppor vecor ache INF 5300 eble of ple clafer ADAboo Rado fore/deco ree knn k-neare-neghbor clafcao e week INF Ug ak o ra ad e Trag ak: a ak where rego o ra each cla are arked ug dffere pel value, eg cla label= for cla, for cla ec Te ak: a lar ak a rag, bu o eae clafer accuracy oly INF Trag a clafer Oba a ay groud ruh aple for each cla a poble If vual peco relable, eper ca ark rag rego eracvely For reoe eg, go ou he feld ad collec feld aple or ue age fro a dffere eor For ybol recogo, ark a e of ybol aually For edcal applcao, ue eg ue aple or erpreao ade by eper Dvde he groud ruh o a rag e ad a e e Ue feaure eraco ad feaure eleco/evaluao o deere he be e of feaure Decde f a lear or quadrac clafer eeded ˆ ha elee ˆ ha -/ elee For each cla, copue ad ug he gve Mau Lkelhood eae INF Clafyg ew daa For each aple, copue he poeror probable for each cla P p P ep P / / P Clafy he aple o he cla wh he hghe poeror probably valuae he perforace of he clafer We ca alo produce age of he poeror probably for each cla INF

10 Valdag clafer perforace Clafcao perforace evaluaed o a dffere e of aple wh kow cla - he e e The rag e ad he e e u be depede! Norally, he e of groud ruh pel wh kow cla parooed o a e of rag pel ad a e of e pel of approaely he ae ze Th ca be repeaed everal e o copue ore robu eae a average e accuracy over everal dffere paro of e e ad rag e By elecg eg 0 rado paro of he e of aple o a rag e ad a e e Cofuo arce A ar wh he rue cla label veru he eaed cla label for each cla aed cla label True cla label Cla Cla Cla 3 Toal #apl e Cla Cla Cla Toal INF INF Cofuo ar - co Alerave: Repor of correcly clafed pel for each cla Repor he perceage of correcly clafed pel for each cla Repor he perceage of correcly clafed pel oal Why h o a good eaure f he uber of e pel fro each cla vare bewee clae? Cla Cla Cla 3 Toal #a ple Cla Cla Cla Toal INF A clafcao eaple Lada age wh 6 pecral bad The 6 bad wll be he feaure Trag area ad e area how ak Upper par: RGB-fale color age creaed fro bad 4,5 ad 6 wh rag ad e rego overlad Lower par: age of rag rego oly 4 INF

11 Vual peco of feaure Vual peco of feaure Cla fore ee o be well eparaed, Maybe alo cla urba Cla fore ee o be well eparaed 4 3 INF INF Vual peco of feaure 3 Vual peco of feaure 4 Cla fore ee o be well eparaed, Cla urba ee o be well eparaed Cla waer ee o be well eparaed, Maybe alo cla 4 agrculural INF INF

12 Vual peco of feaure 5 Vual peco of feaure 6 Waer ad fore appear lar - bu he varace gh be dffere See lar o feaure 5, bu wh beer cora Urba ad agrculural appear lar bu he varace gh be dffere INF INF Seleced caer plo gcaer Clafed age Scaerplo bewee feaure ad 4 Scaerplo bewee feaure 5 ad 6 The ere age clafed o he o probable cla INF INF

13 Dplay he poeror probable a age Cofuo ar for he rag e Dark value: Probable cloe o 0 True cla Aged o Cla Aged o Cla Aged o Cla 3 Cla Aged o Cla4 Poeror probably for cla urba Poeror probably for cla fore Brgh value: Probable cloe o Cla Cla Cla Poeror probably for cla waer Poeror probably for cla agrculural Accuracy per cla: Averaged over all clae: 97% Cla: 8% Cla: 96% Cla3: 00% Cla4: 90% INF INF Cofuo ar for he e e Learg goal fro h lecure True cla Aged o Cla Aged o Cla Aged o Cla 3 Cla Cla Cla Cla Accuracy per cla: Averaged over all clae: 875% Cla: 85% Cla: 8% Cla3: 98% Cla4: 86% Aged o Cla4 INF Be able o ue ad plee Baye rule wh a - deoal Gaua drbuo Kow how ad are eaed Uderad he -deoal cae where a covarace ar lluraed a a ellpe Be able o plfy he geeral dcra fuco for 3 cae Have a geoerc erpreao of clafcao wh feaure INF