CptS 570 Machine Learning School of EECS Washington State University. CptS Machine Learning 1

Transcription

1 ps 57 Machne Leann School of EES Washnon Sae Unves ps 57 - Machne Leann

2 Assume nsances of classes ae lneal sepaable Esmae paamees of lnea dscmnan If ( - -) > hen + Else - ps 57 - Machne Leann

3 lassfcaon (K classes) choose f ( ) K ma ( ) Lkelhood-based classfcaon Esmae pos ( ) and lkelhoods p( ) Defne () n ems of he poseos ( ) ( ) Reques knolede of pes of denses ps 57 - Machne Leann 3

4 Dscmnan-based classfcaon Lean model of boundaes beeen classes, nsead of denses of bounded eons ( ) Whee ae model paamees of he bounda ps 57 - Machne Leann 4

5 Smple Reques onl O(d) space o soe and O(d) me fo classfcaon Weh ndcaes mpoance of feaue lnea model befoe n moe complcaed model, d ps 57 - Machne Leann 5

6 If lnea model nsuffcen, could consde hhe-ode dscmnans Moe me and space equemens Ma ovef (bas/vaance dlemma) Alenave: lnea model of non-lnea feaues z, z, z3, z4, z5 Ne feaues z compse bass funcons k () ps 57 - Machne Leann 6

7 Weh veco defnes a hpeplane dvdn he nsance space no o eons f hoose ohese ps 57 - Machne Leann 7

8 K > classes Lean one hpeplane fo each class sepaan fom all ohe classes, hoose f K ma ps 57 - Machne Leann 8

9 Lean one hpeplane fo each pa of classes K(K-)/ hpeplanes, don' cae hoose, f mamzes, o f f ohese ps 57 - Machne Leann 9

10 If p( ) ~ N(, Σ), hen, hee, Σ, and ( ) can be esmaed fom he daa o class case: o-class case: 5 and / /.5 f choose Odds of Lo odds of, o ps 57 - Machne Leann ohese and,

11 Fo o nomal classes h common covaance ma, s lnea p p / / / / / ep / ep d d p hee ps 57 - Machne Leann

12 Invese of s he sc o smod funcon smod ep and can be esmaed fom he daa (see pevous slde) Smod ansfoms dscmnan value o a poseo pobabl ps 57 - Machne Leann

13 Losc (Smod) funcon an ehe alculae ()= +, choose f () > alculae =smod( + ), choose f >.5 ps 57 - Machne Leann 3

14 Dscmnan-based classfcaon seeks mnmzn eo E(X) on ann se X * a mn E ( X ) Use eave opmzaon mehod aden descen o fnd When E() s dffeenable, compue aden veco E E E E,,..., d ps 57 - Machne Leann 4

15 Sa h andom Updae n he oppose decon of he aden veco E, Dsance of updae deemned b sep sze (o leann faco) ɳ onnue updae unl aden s zeo Ma be a local mnmum ps 57 - Machne Leann 5

16 E ( ) E, E ( + ) + η ps 57 - Machne Leann 6

17 Deve a dffeenable eo funcon o-class: Assume lkelhood ao s lnea p o p p p p o hee o hee ˆ ps 57 - Machne Leann 7 ep

18 o-class case ( = o ): ~ Benoull, X ep X l E l, X oss Enop E, X ps 57 - Machne Leann 8

19 ann n -class case: Gaden descen d E, X E da d a smod If E E d,...,, ps 57 - Machne Leann 9

20 ann n -class case: Gaden descen Inal ehs andom and nea zeo Avods ovefn due o usn elevan feaues = ps 57 - Machne Leann

21 Evoluon of flne and smod a,, and eaons ps 57 - Machne Leann

22 K ) (, ~ Mul, X o K p p Le K be efeence class K K,...,, ep ep ˆ Sofma l, X E, X ps 57 - Machne Leann

23 ps 57 - Machne Leann 3

24 ps 57 - Machne Leann 4

25 === lassfe model (full ann se) === SmpleLosc: lass : [peallenh] * [pealdh] * -5.7 lass : [sepallenh] *.67 + [sepaldh] *.8 + [peallenh] * [pealdh] * % accuac ( msakes) lass : [sepallenh] * [sepaldh] * [peallenh] h] * [pealdh] *.89 ps 57 - Machne Leann 5

26 Assume classes can be sepaaed b a hpeplane Even f no lneal-sepaable, can sll fnd a ood lnea dscmnan Gaden descen used o lean ehs defnn hpeplanes Losc dscmnaon ps 57 - Machne Leann 6