The Gaussian classifier. Nuno Vasconcelos ECE Department, UCSD

Transcription

1 he Gaussan classfer Nuno Vasconcelos ECE Department, UCSD

2 Bayesan decson theory recall that e have state of the orld X observatons decson functon L[,y] loss of predctn y th Bayes decson rule s the rule that mnmzes the rsk Rsk for the - loss E [ L X, ] X, L[, y],, y y

3 MA rule the optmal decson rule can be rtten as * ar ma X [ ] * ar ma X 3 [ l ] * ar ma X e have started to study the case of Gaussan classes X ep d π 3

4 he Gaussan classfer BDR can be rtten as [ d, α ] * ar mn th dscrmnant: X.5 d, y y y α π d the optmal rule s to assn to the closest class closest s measured th the Mahalanobs dstance d,y to hch the α constant s added to account for the class pror 4

5 he Gaussan classfer, If then dscrmnant: X.5 * ar ma th the BDR s a lnear functon or a lnear dscrmnant 5

6 Geometrc nterpretaton classes, share a boundary f there s a setofsuch that there s a set of such that or 6

7 Geometrc nterpretaton note that can be rtten as net e use net, e use 7

8 Geometrc nterpretaton hch can be rtten as usn ths n 8

9 Geometrc nterpretaton leads to b b ths s the equaton of the hyper-plane of parameters 9 hyper plane of parameters and b

10 Geometrc nterpretaton hch can also be rtten as or

11 Geometrc nterpretaton ths s the equaton of the hyper-plane of normal vector that passes throuh n 3 optmal decson boundary for Gaussan classes, equal covarance

12 Geometrc nterpretaton specal case I optmal boundary has I

13 Geometrc nterpretaton ths s vector alon the lne throuh and Gaussan classes, equal covarance I 3

14 Geometrc nterpretaton for equal pror probabltes optmal boundary: - plane throuh mdpont beteen and - orthoonal to the lne that ons and md-pont beteen and Gaussan classes, equal covarance I 4

15 Geometrc nterpretaton dfferent pror probabltes moves alon lne throuh and Gaussan classes, equal covarance I 5

16 Geometrc nterpretaton hat s the effect of the pror? moves aay from f > makn t more lkely to pck Gaussan classes, equal covarance I 6

17 Geometrc nterpretaton hat s the strenth of ths effect? nversely proportonal to the dstance beteen means n unts of standard devaton Gaussan classes, equal covarance I 7

18 Geometrc nterpretaton note the smlartes th scalar case, here hle here e have < hle here e have 8 hyper-plane s the hh-dmensonal verson of the threshold!

19 Geometrc nterpretaton boundary hyper-plane p n,, and 3D for varous pror confuratons 9

20 Geometrc nterpretaton specal case optmal boundary bascally the same, strenth of the pror nversely proportonal to Mahalanobs dstance beteen means s multpled by -, hch chanes ts drecton and the slope of the hyper-plane

21 Geometrc nterpretaton equal but arbtrary covarance Gaussan classes, equal covarance

22 Geometrc nterpretaton n the homeork you ll sho that the separatn plane s tanent to the pdf so-contours at Gaussan classes, equal covarance reflects the fact that the natural dstance s no Mahalanobs

23 Geometrc nterpretaton boundary hyperplane n,, and 3D for varous pror confuratons 3

24 Geometrc nterpretaton hat about the enerc case here covarances are dfferent? n ths case [ ] d α, ar mn *, y y y d there s not much to smplfy d π α 4

25 Geometrc nterpretaton and l l hch can be rtten as W W for classes the decson boundary s hyper-quadratc ths could mean hyper-plane par of hyper-planes hyper- 5 ths could mean hyper-plane, par of hyper-planes, hyperspheres, hyper-elpsods, hyper-hyperbolods, etc.

26 Geometrc nterpretaton n and 3D: 6

27 he smod e have derved all of ths from the -based BDR [ ] l l * h th l t l t l t t t [ ] ar ma X hen there are only to classes, t s also nterestn to look at the ornal defnton ar ma * th ar ma X X X X 7 X X X

28 he smod note that ths can be rtten as ar ma * ar ma X and, for Gaussan classes, the posteror probabltes are X { } ep α α d d here, as before,, y y y d 8 d π α

29 he smod the posteror ep d d α α s a smod and looks lke ths { } dscrmnant: C.5 9

30 he smod the smod d appears n neural netorks t s the true posteror for Gaussan problems here the covarances are the same Equal varances Snle boundary at halfay beteen means 3

31 he smod but not necessarly hen the covarances are dfferent Varances a are dfferent e o boundares 3

32 Bayesan decson theory advantaes: BDR s optmal and cannot be beaten Bayes keeps you honest models reflect causal nterpretaton of the problem, ths s ho e thnk natural decomposton nto hat e kne already pror and hat data tells us CCD no need for heurstcs to combne these to sources of nfo BDR s, almost nvarably, ntutve Bayes rule, chan rule, and marnalzaton enable modularty, and scalablty to very complcated models and problems problems: BDR s optmal only nsofar the models are correct. 3

33 33