Overview. Introduction Building Classifiers (2) Introduction Building Classifiers. Introduction. Introduction to Pattern Recognition and Data Mining

Ovrv Iroduco o ar Rcogo ad Daa Mg Lcur 4: Lar Dcra Fuco Irucor: Dr. Gova Dpar of Copur Egrg aa Clara Uvry Iroduco Approach o uldg clafr Lar dcra fuco: dfo ad urfac Lar paral ca rcpro crra Ohr hod Lar Dcra Aaly LDA Rrcd Gaua clafr Lcur Lar Rgro -- Mu quard-error ME crra Fhr gorc v of LDA Logc Rgro G. Q/4 Iroduco Buldg Clafr Cla-codoal grav approach p,θ ar odld plcly; θˆ ar ad va ML Cod h a of p ar vrd va Bay rul o arrv a p Rgro approach p ar odld plcly.g., Logc rgro Dcrav approach Try o odl h dco oudary drcly.., a appg fro pu o o of h cla Au ko h for for h dcra fuco g Iroduco Buldg Clafr Clafcao a ar prol ha dy ao Vapk Why u dy ao a a rda p? Rr lklhood rao: oly d o ko f.., oly rao ar! p õ p > àõ õ àõ â p > p G. Q/4 3 G. Q/4 4

Iroduco Lar Dcra Fuco Iroduco Lar Dcra Fuco Dfo Dco rul - o-cla ca Ju a lar coao of h aur of r a g h gh vcor of h odl h a or hrhold gh Opal f udrlyg druo ar cooprav Dcd f g> ad f g<.., ag o f cd hrhold If g ag udfd.., ca go hr ay Dagra of odl Gaua h Σ û I or Σ Σ LDA - Lcur plcy ak h aracv for al, ral clafr Ca grald o lar o gv of fuco ϕ G. Q/4 5 G. Q/4 6 Iroduco Lar Dcra Fuco 3 Hoogou for Augd gh & faur vcor W r ga y d d g hr a M d y M d Iroduco Dco urfac Equao g df urfac ha para po agd o h cagory fro po agd o h cagory g lar urfac a hyprpla H Codr ad oh o h dco urfac: or oral o ay vcor lyg h hyprpla Orao of H drd y hr g< hr g> G. Q/4 7 G. Q/4 8

Iroduco Dco urfac g dac fro o H Epr a p r Iroduco Mulcla Ca O pr cla dcopoo lar ach.., C dcra fuco v. cau g p g g p r r g r g : g :, 3,, c, 3,, c arga d, H / g c : c Locao of H drd y,,, c- G. Q/4 9 G. Q/4 Iroduco Mulcla Ca Dco oudar Iroduco Mulcla Ca 3 Whou arga, aguou cla ag ca ar H dfd y g g Nur of H of fr ha cc-/ Dco rgo ar cov ad gly cocd Mo ual h p uodal May cpo! G. Q/4 G. Q/4

Lar paral Ca rcpro Lar paral Ca rcpro plfyg oralao Crro fuco Rplac apl y hr gav Fd a uch ha a > for all apl A calar fuco Ja ha d f a a oluo vcor Allo u of Grad Dc hod: a k a k η k J a or a k a k H J a No Ida : Ja # of clafd apl Ida : J p a o u of dac o dco oudary No ha a o uqu! J a a y p y Y hr Ya clafd G. Q/4 3 G. Q/4 4 Lar paral Ca rcpro 3 Fd-cr, gl-apl k do { k k f y k clafd y a { a a y k } } ul all par ar proprly clafd Covrgc Thor rcpro algorh guarad o fd a oluo f apl ar larly paral I oparal ca, rror-corrcg algorh produc a f quc ak ld applcaly G. Q/4 5 Lar Rgro Mu quard Error Crro fuco Y d augd daa ar dcaor rpo vcor.g., Raoal - g h of h rror vcor Ya No ha Y rcagular ad a ovrdrd G. Q/4 J a Ya a y Ya ordarly ha o ac oluo J a quadrac ca look for a gl gloal u J 6

Lar Rgro Mu quard Error Lar Rgro Mu quard Error 3 Clod-for oluo J J a y y Y Ya Y Ya Y a Y Y Y Y A or gral dfo of h pudovr alay : Y l Y Y εi Y ε W pc o oa a uful dcra oh h paral ad h oparal ca Wh c larg, v o akg prol Ha Eapl 4 3 3 4 I R: Y.p <-olvy %*% Y %*% Y 5/ 4 3/ 3/ 4 Y Y Y Y / / 6 / / 3 4 g a y 3 3 3 X 3 Y 3 3 7 / / 3 / 6 Y a 4 / 3 / 3 / 3 3 G. Q/4 7 G. Q/4 8 Fhr Lar Dcra Lo-Doal roco Gorc rprao of do produc Lgh of h proco of oo h u vcor coθ archg for h ha para h procd daa Fhr Lar Dcra Lo-Doal roco Crro fuco Ida : u h dac h procd apl a ~ ~ hr D Dpd o could ad arrarly larg Ida : a rao of -cla car a aov o h-cla car ~ ~ J F ~ hr ~ D ~ ~ Clarly, / a a of h varac of h poold daa G. Q/4 9 G. Q/4

G. Q/4 Fhr Lar Dcra Lo-Doal roco 3 ha op J F ca ho o Coco o LDA -- p Nµ, Σ For h c-cla prol, c fuco ar rqurd roco fro a d o a c doal pac d > c acrfc prforac for h advaag of lor-doal pac g g g µ µ µ c Σ Σ D hr G. Q/4 Logc Rgro Modlg oror Modl for: hr h logc fuco To-cla ca: Log of odd rao lar...3.4.5.6.7.8.9-6 -4-4 6 log dco oudar ar lar 3 G. Q/4 Logc Rgro Fg Modl gv y: Log-lklhood o-cla ca ; l ; l l ohr r r l / ] [ / X l 4 G. Q/4 Logc Rgro Fg Modl Dffrag aga o oa h Ha: No p : r r r r l / hr L M O M L H X WX H ] [ X X WX H k J k k

Logc Rgro Coparo o LDA W had g g g Σ ply o ha α α LR copud drcly o va µ, µ, Σ.., opg dffr crra µ µ g log c Σ µ Logc Rgro Coparo o LDA If ar o oral, h LDA ca uch or.g., r oulr µ µ LR ca dgra o paral daa Nurcal u h.8 LR hold alo for o o-oral d oly d h rao o of h logc yp If ar oral, h LDA 3% or ffc I gral, LR a afr, or rou, u of lar rul.6.4. -6-4 - 4 6 G. Q/4 5 G. Q/4 6