Pattern Classification

Transcription

1 attern Classfcaton All materals n these sldes were taken from attern Classfcaton nd ed by R. O. Duda,. E. Hart and D. G. Stork, John Wley & Sons, 000 wth the ermsson of the authors and the ublsher

2 Chater art 3 Bayesan Decson Theory Sectons -6,-9 Dscrmnant Functons for the Normal Densty Bayes Decson Theory Dscrete Features

3 Dscrmnant Functons for the Normal Densty We saw that the mnmum error-rate classfcaton can be acheved by the dscrmnant functon g ln + ln Case of multvarate normal g t d ln π ln Σ + ln attern Classfcaton, Chater art 3

4 Case Σ σ I I stands for the dentty matr What does Σ σ I say about the dmensons? What about the varance of each dmenson? Note : both Σ and d/ lnπ are ndeendent of n g Thus we can smlfy to : χ g σ where denotes t + ln d the Eucldean norm ln π ln Σ 3 + ln attern Classfcaton, Chater art 3

5 4 We can further smlfy by recognzng that the quadratc term t mlct n the Eucldean norm s the same for all. g where : 0 w + t w 0 lnear dscrmnant functon t w ; w ln 0 + σ σ s called the threshold for the th category! attern Classfcaton, Chater art 3

6 5 A classfer that uses lnear dscrmnant functons s called a lnear machne The decson surfaces for a lnear machne are eces of hyerlanes defned by: g g The equaton can be wrtten as: w t attern Classfcaton, Chater art 3

7 attern Classfcaton, Chater art 3 6 The hyerlane searatng R and R always orthogonal to the lne lnkng the means! ln 0 + σ then 0 f +

8 7 attern Classfcaton, Chater art 3

9 8 attern Classfcaton, Chater art 3

10 9 attern Classfcaton, Chater art 3

11 attern Classfcaton, Chater art 3 0 Case Σ Σ covarance of all classes are dentcal but arbtrary! Hyerlane searatng R and R the hyerlane searatng R and R s generally not orthogonal to the lne between the means! [ ]. / ln and Where 0 the equaton Has 0 0 t t Σ Σ w w +

12 attern Classfcaton, Chater art 3

13 attern Classfcaton, Chater art 3

14 attern Classfcaton, Chater art 3 3 Case Σ arbtrary The covarance matrces are dfferent for each category The decson surfaces are hyerquadratcs Hyerquadrcs are: hyerlanes, ars of hyerlanes, hyersheres, hyerellsods, hyerarabolods, hyerhyerbolods ln ln w w W : where w w W g t 0 0 t t Σ Σ Σ Σ + +

15 4 attern Classfcaton, Chater art 3

16 5 attern Classfcaton, Chater art 3

17 attern Classfcaton, Chater art 3 6 Bayes Decson Theory Dscrete Features Comonents of are bnary or nteger valued, can take only one of m dscrete values v, v,, v m concerned wth robabltes rather than robablty denstes n Bayes Formula: c where

18 Bayes Decson Theory Dscrete Features 7 Condtonal rsk s defned as before: Rα Aroach s stll to mnmze rsk: α * arg mn R α attern Classfcaton, Chater art 3

19 Bayes Decson Theory Dscrete Features 8 Case of ndeendent bnary features n category roblem Let [,,, d ] t where each s ether 0 or, wth robabltes: q attern Classfcaton, Chater art 3

20 attern Classfcaton, Chater art 3 9 Bayes Decson Theory Dscrete Features Assumng condtonal ndeendence, can be wrtten as a roduct of comonent robabltes: d d d q q q q lkelhood rato gven by : yeldng a and

21 attern Classfcaton, Chater art 3 0 Bayes Decson Theory Dscrete Features Takng our lkelhood rato ln ln ln yelds: ln ln and luggng t nto Eq.3 q q g g q q d d + + +

22 The dscrmnant functon n ths case s: g and where w : w decde 0 : d ln q d f w + ln g w > 0 q q + ln 0 and,...,d f g 0 attern Classfcaton, Chater art 3